No way to redirect frame logging output?

[Codym48]

We're using the frame logging feature to assess our simulation's realtime performance. We're using the -O option to redirect Trick simulation output files to a separate directory. I was really excited to discover the additional -OO and --read-only-sim options, the former present in @spfennell's initial commit and the latter added by @jdeans289 in #1420 described in the documentation as,

• The '-O <output_file_path>' option allows the user to specify the directory to which simulation data log files will be written. If this option is omitted, the RUN_ directory is used.
• The '-OO <output_file_path>' option allows the user to specify the directory to which ALL simulation output files will be written. If this option is omitted, the RUN_ directory is used.
• The '--read-only-sim' flag can be used to redirect all files written at simulation runtime into the output directory.

We noticed today that the DP_rt_*.xml files written at simulation runtime don't respond to any of these options, though.

• What's the intended difference between -O ("simulation data log files"), -OO ("ALL simulation output files"), and --read-only-sim ("all files written at simulation runtime")?
• Would it be straightforward to expand any of those to apply to the frame logging data, too? It seems like that could be as easy as inverting this conditional.
• Is there some other use case that inverting that conditional would break?

(Thank you for all your hard work building and maintaining Trick!)

[hchen99]

Thanks for your kind words and bringing this up with all the details. Looks to me that all frame logging data (log_frame.xxx, log_timeline, and log_timeline_init) is placed in the specified <output_file_path>. DP_rt_*.xml data product files (for plotting frame logging data) are expected to be in DP_Product folder for plotting app although can be manually opened if not shown by default. It might be the reason for DP_rt_*.xml staying in DP_Product folder. We'll discuss some options to have document and code in sync.

[hchen99]

@Codym48
Currently, looks like both -O <output_file_path> and -OO <output_file_path> are doing the same thing, all logged data plus some other files are saved in <output_file_path> if either used.

As for --read-only-sim flag, it is for making runtime generation of sie (S_sie.resource) into the output directory if -O or -OO used otherwise into the default directory. In order to use this flag, the sim input file needs to set trick.sie_append_runtime_objs() for the flag to be effective.

• For instance, once trick.sie_append_runtime_objs() is added to the input file, you should see S_sie.resource is saved in the <output_file_path> after you run the sim like:
  ./S_main_xxx.exe RUN_test/input.py -O <output_file_path> --read-only-sim

As for DP_rt_xxx.xml files being always saved into DP_Product regardless of -O or -OO, we could repurpose -OO so that all DP_rt_xxx.xml files are saved in the specified output folder. In order to work with plotting apps better, we might create the following folder hierarchy so once in the output folder, plotting apps can work as expected:

- <output_file_path>
   - RUN_test
      - log_xxx.xxx
   - DP_Product
      - DP_rt_xxx.xml

We'll update the document accordingly.

[alexlin0]

It would help my current project to move the DP_rt files to the output file path.

[sharmeye]

I agree that when the user specifies -OO, EVERYTHING gets put in the specified output directory. We can leave -O as-is, and for the first time in a LONG time -O and -OO will do different things.

[alexlin0]

Can everything include the sie file with the -OO option?

[Codym48]

Making -OO more complete sounds great to me!

[hchen99]

Sure, we can have everything saved in the specified output folder including the sie file with the -OO option

[hchen99]

Please check out the branch for this issue to see if -OO meets the need. Have not updated the document yet as the changes might need updating.

• -O <output_file_path>:
  • should be as is
• -OO <output_file_path>:
  • DP_ files are placed in DP_Product subdirectory of <output_file_path>
  • All other files are placed in RUN_xxx subdirectory of <output_file_path>
• --read-only-sim along with trick. trick.sie_append_runtime_objs() in input file, the S_sie.resource with runtime objects data would be in the <output_file_path>

[keithvetter]

If the DP_Product directory does not exist in the -OO output file path, will it be created?

[hchen99]

If the DP_Product directory does not exist in the -OO output file path, will it be created?

Yes, both DP_Product and RUN_xxx are created automatically