Update readoutapp_gen to use schema from DAQConf, introduce schema fo…

[eflumerf]

…r some app-specific options

[eflumerf]

readoutapp_gen may have grown some new features in this transition; I mostly re-synced it to daqconf's readout_gen.py. I think FELIX-card readout and FWTP mode are new additions.

[eflumerf]

Question for @plasorak...In the schema, I just had it inherit the "readout" block from daqconf, is this okay or should I explicitly put those options into the "readoutapp" block that is more specific to this module?

[plasorak]

I think that's fine as long as the parameters are the same? I guess the tricky situation would be if some parameters in daqconf's readout are not in the readoutlibs one. If it's "simple enough" I'd do that, because it creates an explicit link between the two configurations, and in theory they should be identical (right?).

So yeah, sorry for not being very clear, bottom line: if it's not wildly different, go for it, but make sure all the readoutlibs conf parameters that are in the CLI now are in the Json in a different section (with a smaller schema).

[hristovaivana]

Thanks for offering these changes. Excellent work, it is much better way to work with the configuration generation.

1. I tested it with most of the default configuration parameters, and this branch works out of the box.
2. Then, since this configuration script is part of the readout package, I use for a specific purpose - to test the firmware TPs with a fake source (instead of hardware, the FELIX card). I managed to run such a test with this branch, but I discovered the following:

• in addition to 'wib" it would be good to add 'raw_tp" or name containing "tp" to the list FRONTEND_TYPE because currently this is the way to "createSourceEmulator(...)" (not "createReadout(...)") for the fake fw TP source,
  see readoutmodules/src/CreateSourceEmulator.hpp
• with "raw_tp" added (in my local area), at least for the moment, I cannot get the tp_datahandler and tp_out_datahandler to work, getting this error at init
  2022-Sep-20 22:14:08,141 LOG [virtual void dunedaq::readoutmodules::DataLinkHandler::init(const data_t&) at /net/home/ppd/l mo86446/rawhit/v310/N22-09-20/sourcecode/readoutmodules/plugins/DataLinkHandler.cpp:46] tp_datahandler_103Initialize readou t implementation FAILED! Failed to find specialization for given queue setup!
  Most probably my local HardwareMap.txt file is incomplete, it contains these, I don't know where to find another example,

# DRO_SourceID DetLink DetSlot DetCrate DetID DRO_Host DRO_Card DRO_SLR DRO_Link 
101 0 4 6 3 localhost 0 0 0
102 1 4 6 3 localhost 0 0 1

• the solution to this error above is to not add tp_datahandler and tp_out_datahandler if "raw_tp" is the FRONTEND_TYPE
• also my local daqconf.json

{
  "readout": {
    "enable_firmware_tpg": true
  }
}

• The fake firmware TP source is fetched with
  curl https://cernbox.cern.ch/index.php/s/asyh9lHd8PhxMyI/download -o ./tp_frames.bin

If you understand what I mean and agree, you can make these changes in app_confgen.py, i.e. add "fw_tp" or "raw_tp" to the list of frontend types to trigger the creation of a fake source readout for the firmware TP. Otherwise I am very keen this branch to go to develop. The plan after that is before the v3.2.0 deadline to make a PR from my side with FW TP related changes - I want to remove one and add two configuration parameters.

It's much fun to play with this new configuration. It will be easier to keep "readoutapp_gen" synchronised with dunedaq configration and it seems promising to have the FW TPs added to the integration tests soon.

LGTM!

[hristovaivana]

Maybe a related comment. I've added two configuration parameters in readoutlibs, #33

s.field("fwtp_number_of_ticks", self.size, 64,
                doc="Number ticks over which a firmware TP (i.e. a hit) spans"),
s.field("fwtp_tick_length", self.size, 32,
                doc="Tick length as defined by DUNE WIB"),

In general, I want to/must add them here in readoutmodules, to app_confgen.py/ and readoutapp_gen, but would be nice if they can be picked up from the schema file in readoutlibs.

[plasorak]

The things I understand look correct

[ShyamB97]

readout_app confgen crashes when testing with real hardware. The issue seems to be the fact the tpset queues are not connected to anything so wrtiting trigger primitives to tpsets causes the system to crash. Testing with full scale system causes no issues so I think this is a configuration issue isolated to the confgen.