👨‍🌾 generate_ci_script,py crashes when creating a benchmark script.

[Blast545]

Bug Report

I'm not able to reproduce a benchmark job locally using the buildfarm scripts.

Steps to reproduce

pip install -U git+https://github.com/ros-infrastructure/ros_buildfarm#master
git clone https://github.com/ros-infrastructure/ros_buildfarm
cd ros_buildfarm/scripts/ci
./generate_ci_script.py https://raw.githubusercontent.com/ros2/ros_buildfarm_config/ros2/index.yaml rolling benchmark ubuntu jammy amd64

Additional info

It works with other jobs normally, apparently is failing only with the benchmark job, even when it's there in the index.yaml file. Apparently there's a problem with its template generation, and shows this error:

TypeError processing template 'ci/ci_job.xml.em'
Traceback (most recent call last):
  File "./generate_ci_script.py", line 170, in <module>
    sys.exit(main())
  File "./generate_ci_script.py", line 145, in main
    configure_ci_job(
  File "/home/misato2/.local/lib/python3.8/site-packages/ros_buildfarm/ci_job.py", line 206, in configure_ci_job
    job_config = _get_ci_job_config(
  File "/home/misato2/.local/lib/python3.8/site-packages/ros_buildfarm/ci_job.py", line 319, in _get_ci_job_config
    job_config = expand_template(template_name, job_data)
  File "/home/misato2/.local/lib/python3.8/site-packages/ros_buildfarm/templates/__init__.py", line 100, in expand_template
    interpreter.string(content, template_path, locals=data)
  File "/home/misato2/.local/lib/python3.8/site-packages/em.py", line 2391, in string
    self.safe(scanner, True, locals)
  File "/home/misato2/.local/lib/python3.8/site-packages/em.py", line 2401, in safe
    self.parse(scanner, locals)
  File "/home/misato2/.local/lib/python3.8/site-packages/ros_buildfarm/templates/__init__.py", line 67, in parse
    token.run(self, locals)
  File "/home/misato2/.local/lib/python3.8/site-packages/em.py", line 1527, in run
    self.subrun(subtokens, interpreter, locals)
  File "/home/misato2/.local/lib/python3.8/site-packages/em.py", line 1631, in subrun
    token.run(interpreter, locals)
  File "/home/misato2/.local/lib/python3.8/site-packages/em.py", line 1546, in run
    interpreter.assign(iterator, element, locals)
  File "/home/misato2/.local/lib/python3.8/site-packages/em.py", line 2503, in assign
    self.multi(left, value, locals)
  File "/home/misato2/.local/lib/python3.8/site-packages/em.py", line 2478, in multi
    self.invoke('beforeMulti', names=names, values=values, locals=locals)
  File "/home/misato2/.local/lib/python3.8/site-packages/em.py", line 2641, in invoke
    method(*(), **keywords)
TypeError: beforeMulti() got an unexpected keyword argument 'names'

I found this while investigating: ros-tooling/libstatistics_collector#132
Which is actually a problem in the benchmark job to find openssl.

[cottsay]

Intermediate cause is the underlay_from_ci_jobs entry in the build config. All of the jobs which chain from another job (nightly-cross-vendor-*, overlay, etc) will be affected by this bug.

This is clearly not the way that the code should act, but the fact that the job requires the artifact from another job makes the desired behavior less than straightforward.

Should we make the script download the job from Jenkins? Can it even do that? Or just shoot an error message? Maybe throw some instructions requiring that --underlay-source-path be specified?