Filling in trials automatically does not update meta data/json DB

[zbeekman]

[zbeekman@onyx01:/p/work/zbeekman/FDL3DI-2.1] $ du -h -d 0 .tau/CFD-KNL/Onyx-knl-FDL-sample-stall-cyc/4
18M	.tau/CFD-KNL/Onyx-knl-FDL-sample-stall-cyc/4
[zbeekman@onyx01:/p/work/zbeekman/FDL3DI-2.1] $ tau trial list 4
== Trial Configurations (/p/work/zbeekman/FDL3DI-2.1/.tau/project.json) =============================================================================================================================================================

+--------+-----------+---------------------------------------------------------------------------------------------------------------------+-------------+-----------+-----------------+
| Number | Data Size |                                                       Command                                                       | Description |  Status   | Elapsed Seconds |
+========+===========+=====================================================================================================================+=============+===========+=================+
|   4    |   0.0B    | aprun -n 64 -N 8 -d 16 -j 2 --cc=depth tau_exec -T papi,pthread,b50df500,intel,mpi -ebs ../FDLv2/fdl3di.exe.knl.kji |     N/A     | executing |       N/A       |
+--------+-----------+---------------------------------------------------------------------------------------------------------------------+-------------+-----------+-----------------+

Selected experiment: Onyx-knl-FDL-sample-stall-cyc

😠

[nchaimov]

What does "filling in trials automatically" mean?

[zbeekman]

tau trial del 4 when 5 and 6 exist, followed by tau trial create ... aka tau aprun -n ... a.out

[zbeekman]

Sorry, I sent that pre-maturely. To elaborate further, the meta data shown is from the deleted trial. This was not done on the latest unstable, so I've updated and I'm testing again. I suspect the version I was using is recent enough that this issue will persist, but I'll try to reproduce with the latest unstable.

[nchaimov]

The data size, command, etc. are all part of the Trial object... I don't see how it could be reusing metadata from the deleted trial unless the trial was not in fact deleted. Deleting the Trial should remove its entry from the database entirely (and, in the case of a Trial, also delete the corresponding directory).

[zbeekman]

Yes, I agree. This is in Onyx's $WORKDIR. Maybe it's lustre related? As you see in my first post, this is most certainly happening. TBH, my trust in TinyDB seems to be monotonically decreasing. However, since it's vendored, maybe updating it could help resolve some issues?

…

On Mon, Oct 29, 2018 at 9:33 PM Nicholas Chaimov ***@***.***> wrote: The data size, command, etc. are all part of the Trial object... I don't see how it could be reusing metadata from the deleted trial unless the trial was not in fact deleted. Deleting the file should remove its entry from the database entirely (and, in the case of a Trial, also delete the corresponding directory). — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#313 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAREPJSPXBSqkPkjYnv5RJPSsyXIEWAxks5up6xngaJpZM4YA_kM> .

[nchaimov]

Can this be reproduced consistently? If so, does examining the database file in a text editor (it's just a big JSON object) after the delete show that the deleted record is still in the database somehow?