Replay using photon seeds saved as results_detp.jdat fails

[g-buist]

Running a benchmark script with -F mc2, saving the photon seeds and then running a replay with results.mch executes without errors.
However running the same but with -F jnii and the replay with photon seed input from results_detp.jdat file gives an error for the replay. Checked and the .jdat format file with the photon seeds data exists and can be opened and read.
It seems there is a problem with extracting the photon seeds for the rerun from the results_dept.jdat file.

The .mch format case:

:\Windows>C:\Sources\mcx-win-x86_64-nightlybuild_2022_10_05\mcx\bin\mcx.exe --bench cube60  --saveseed -s D:\MonteCarlo\OutputTest\results
###############################################################################
#                      Monte Carlo eXtreme (MCX) -- CUDA                      #
#          Copyright (c) 2009-2022 Qianqian Fang           #
#                             http://mcx.space/                               #
#                                                                             #
# Computational Optics & Translational Imaging (COTI) Lab- http://fanglab.org #
#   Department of Bioengineering, Northeastern University, Boston, MA, USA    #
###############################################################################
#    The MCX Project is funded by the NIH/NIGMS under grant R01-GM114365      #
###############################################################################
$Rev::e281f3$ v2022 $Date::2022-09-07 10:23:12 -04$ by $Author::Qianqian Fang $
###############################################################################
- variant name: [Fermi] compiled by nvcc [7.5] with CUDA [7050]
- compiled with: RNG [xorshift128+] with Seed Length [4]

GPU=1 (GeForce RTX 2080 SUPER) threadph=10 extra=16960 np=1000000 nthread=98304 maxgate=1 repetition=1
initializing streams ...        init complete : 0 ms
requesting 2560 bytes of shared memory
launching MCX simulation for time window [0.00e+000ns 5.00e+000ns] ...
simulation run# 1 ...
kernel complete:        234 ms
retrieving fields ...   detected 3039 photons, total: 3039      transfer complete:      260 ms
normalizing raw data ...        source 1, normalization factor alpha=200.000000
data normalization complete : 266 ms
saving data to file ... saving data complete : 269 ms

simulated 1000000 photons (1000000) with 98304 threads (repeat x1)
MCX simulation speed: 4716.98 photon/ms
total simulated energy: 1000000.00      absorbed: 17.74692%
(loss due to initial specular reflection is excluded in the total)

C:\Windows>C:\Sources\mcx-win-x86_64-nightlybuild_2022_10_05\mcx\bin\mcx.exe --bench cube60 --outputtype j --seed D:\MonteCarlo\OutputTest\results.mch -s D:\MonteCarlo\OutputTest\results_replay
###############################################################################
#                      Monte Carlo eXtreme (MCX) -- CUDA                      #
#          Copyright (c) 2009-2022 Qianqian Fang           #
#                             http://mcx.space/                               #
#                                                                             #
# Computational Optics & Translational Imaging (COTI) Lab- http://fanglab.org #
#   Department of Bioengineering, Northeastern University, Boston, MA, USA    #
###############################################################################
#    The MCX Project is funded by the NIH/NIGMS under grant R01-GM114365      #
###############################################################################
$Rev::e281f3$ v2022 $Date::2022-09-07 10:23:12 -04$ by $Author::Qianqian Fang $
###############################################################################
- variant name: [Fermi] compiled by nvcc [7.5] with CUDA [7050]
- compiled with: RNG [xorshift128+] with Seed Length [4]

GPU=1 (GeForce RTX 2080 SUPER) threadph=0 extra=3039 np=3039 nthread=98304 maxgate=1 repetition=1
initializing streams ...        init complete : 0 ms
requesting 1536 bytes of shared memory
launching MCX simulation for time window [0.00e+000ns 5.00e+000ns] ...
simulation run# 1 ...
kernel complete:        8 ms
retrieving fields ...   detected 3039 photons, total: 3039      transfer complete:      19 ms
normalizing raw data ...        normalization factor for detector 0 alpha=0.000518
source 1, normalization factor alpha=0.000518
data normalization complete : 35 ms
saving data to file ... saving data complete : 38 ms

simulated 3039 photons (3039) with 98304 threads (repeat x1)
MCX simulation speed: 1013.00 photon/ms
total simulated energy: 3039.00 absorbed: 36.49826%
(loss due to initial specular reflection is excluded in the total)

=====================================================
path for .mch file: "D:\MonteCarlo\OutputTest\results.mch"

The .jdat format case:

====================================================

C:\Windows>C:\Sources\mcx-win-x86_64-nightlybuild_2022_10_05\mcx\bin\mcx.exe --bench cube60 -F jnii --saveseed -s D:\MonteCarlo\OutputTest\results
###############################################################################
#                      Monte Carlo eXtreme (MCX) -- CUDA                      #
#          Copyright (c) 2009-2022 Qianqian Fang           #
#                             http://mcx.space/                               #
#                                                                             #
# Computational Optics & Translational Imaging (COTI) Lab- http://fanglab.org #
#   Department of Bioengineering, Northeastern University, Boston, MA, USA    #
###############################################################################
#    The MCX Project is funded by the NIH/NIGMS under grant R01-GM114365      #
###############################################################################
$Rev::e281f3$ v2022 $Date::2022-09-07 10:23:12 -04$ by $Author::Qianqian Fang $
###############################################################################
- variant name: [Fermi] compiled by nvcc [7.5] with CUDA [7050]
- compiled with: RNG [xorshift128+] with Seed Length [4]

GPU=1 (GeForce RTX 2080 SUPER) threadph=10 extra=16960 np=1000000 nthread=98304 maxgate=1 repetition=1
initializing streams ...        init complete : 0 ms
requesting 2560 bytes of shared memory
launching MCX simulation for time window [0.00e+000ns 5.00e+000ns] ...
simulation run# 1 ...
kernel complete:        226 ms
retrieving fields ...   detected 3039 photons, total: 3039      transfer complete:      238 ms
normalizing raw data ...        source 1, normalization factor alpha=200.000000
data normalization complete : 245 ms
saving data to file ... compressing data [zlib] ...compression ratio: 92.5%     after encoding: 125.1%
saving data complete : 365 ms

compressing data [zlib] ...compression ratio: 11.0%     after encoding: 14.9%
compressing data [zlib] ...compression ratio: 52.4%     after encoding: 70.9%
compressing data [zlib] ...compression ratio: 100.0%    after encoding: 135.2%
simulated 1000000 photons (1000000) with 98304 threads (repeat x1)
MCX simulation speed: 4901.96 photon/ms
total simulated energy: 1000000.00      absorbed: 17.74692%
(loss due to initial specular reflection is excluded in the total)

C:\Windows>C:\Sources\mcx-win-x86_64-nightlybuild_2022_10_05\mcx\bin\mcx.exe --bench cube60 -F jnii --outputtype j --seed D:\MonteCarlo\OutputTest\results_detp.jdat -s D:\MonteCarlo\OutputTest\results_replay

MCX ERROR(-2):reading input file is terminated in unit mcx_utils.c:3526

================================================

The path to the .jdat file: "D:\MonteCarlo\OutputTest\results_detp.jdat"

[fangq]

@g-buist, unfortunately I am not able to reproduce this - can you try the latest nightly build and see if the issue still exist?

I tested your commands on my Linux machine and the replay runs fine, all 2980 photons in the baseline simulation were re-detected in the replay

fangq@taote:~/space/git/Project/github/mcx/src$ ../bin/mcx --bench cube60  --saveseed -s test  -F jnii 
###############################################################################
#                      Monte Carlo eXtreme (MCX) -- CUDA                      #
#          Copyright (c) 2009-2022 Qianqian Fang <q.fang at neu.edu>          #
#                             http://mcx.space/                               #
#                                                                             #
# Computational Optics & Translational Imaging (COTI) Lab- http://fanglab.org #
#   Department of Bioengineering, Northeastern University, Boston, MA, USA    #
###############################################################################
#    The MCX Project is funded by the NIH/NIGMS under grant R01-GM114365      #
###############################################################################
$Rev::31c0fa$v2022.10$Date::2023-05-04 16:12:10 -04$ by $Author::Qianqian Fang$
###############################################################################
- variant name: [Fermi] compiled by nvcc [9.0] with CUDA [9000]
- compiled with: RNG [xorshift128+] with Seed Length [4]

GPU=1 (NVIDIA TITAN V) threadph=6 extra=16960 np=1000000 nthread=163840 maxgate=1 repetition=1
initializing streams ...	init complete : 1 ms
requesting 2560 bytes of shared memory
launching MCX simulation for time window [0.00e+00ns 5.00e+00ns] ...
simulation run# 1 ... 
kernel complete:  	84 ms
retrieving fields ... 	detected 2980 photons, total: 2980	transfer complete:	97 ms
normalizing raw data ...	source 1, normalization factor alpha=200.000000
data normalization complete : 102 ms
saving data to file ...	compressing data [zlib] ...compression ratio: 92.5%	after encoding: 125.0%
saving data complete : 136 ms

compressing data [zlib] ...compression ratio: 11.0%	after encoding: 14.9%
compressing data [zlib] ...compression ratio: 51.8%	after encoding: 70.0%
compressing data [zlib] ...compression ratio: 99.8%	after encoding: 134.9%
simulated 1000000 photons (1000000) with 163840 threads (repeat x1)
MCX simulation speed: 22222.22 photon/ms
total simulated energy: 1000000.00	absorbed: 17.76674%
(loss due to initial specular reflection is excluded in the total)
fangq@taote:~/space/git/Project/github/mcx/src$ ../bin/mcx --bench cube60  --saveseed -s test  --seed test_detp.jdat
###############################################################################
#                      Monte Carlo eXtreme (MCX) -- CUDA                      #
#          Copyright (c) 2009-2022 Qianqian Fang <q.fang at neu.edu>          #
#                             http://mcx.space/                               #
#                                                                             #
# Computational Optics & Translational Imaging (COTI) Lab- http://fanglab.org #
#   Department of Bioengineering, Northeastern University, Boston, MA, USA    #
###############################################################################
#    The MCX Project is funded by the NIH/NIGMS under grant R01-GM114365      #
###############################################################################
$Rev::31c0fa$v2022.10$Date::2023-05-04 16:12:10 -04$ by $Author::Qianqian Fang$
###############################################################################
- variant name: [Fermi] compiled by nvcc [9.0] with CUDA [9000]
- compiled with: RNG [xorshift128+] with Seed Length [4]

GPU=1 (NVIDIA TITAN V) threadph=0 extra=2980 np=2980 nthread=163840 maxgate=1 repetition=1
initializing streams ...	init complete : 1 ms
requesting 2560 bytes of shared memory
launching MCX simulation for time window [0.00e+00ns 5.00e+00ns] ...
simulation run# 1 ... 
kernel complete:  	8 ms
retrieving fields ... 	detected 2980 photons, total: 2980	transfer complete:	21 ms
normalizing raw data ...	source 1, normalization factor alpha=67114.093750
data normalization complete : 26 ms
saving data to file ...	saving data complete : 26 ms

simulated 2980 photons (2980) with 163840 threads (repeat x1)
MCX simulation speed: 596.00 photon/ms
total simulated energy: 2980.00	absorbed: 34.91230%
(loss due to initial specular reflection is excluded in the total)

[g-buist]

Thanks it indeed works as expected in the nightlybuild. I might have made a mistake before.