The add-new-jobs method for firefox-android is broken

[alexandru-io]

The add-new-jobs method is broken for us which prevents us from doing backfill.
decision task

[task 2023-06-09T08:05:36.326Z] 2023-06-09 08:05:36,326 - INFO - No tasks removed during optimization
[task 2023-06-09T08:05:36.846Z] 2023-06-09 08:05:36,846 - INFO - Replaced 8 tasks by existing_tasks, 5 tasks by index-search (index-search) during optimization.
[task 2023-06-09T08:05:36.847Z] 2023-06-09 08:05:36,846 - INFO - writing artifact file `task-graph0.json`
[task 2023-06-09T08:05:36.847Z] 2023-06-09 08:05:36,847 - INFO - writing artifact file `label-to-taskid0.json`
[task 2023-06-09T08:05:36.848Z] 2023-06-09 08:05:36,848 - INFO - writing artifact file `to-run0.json`
[task 2023-06-09T08:05:36.849Z] 2023-06-09 08:05:36,849 - INFO - Creating task with taskId XATkp8s4TQenWwBIhBXupQ for browsertime-android-hw-a51-11-0-aarch64-shippable-qr/opt-allrecipes-fenix-nightly-simulation-armeabi-v7a-nofis
[task 2023-06-09T08:05:37.033Z] Traceback (most recent call last):
[task 2023-06-09T08:05:37.033Z]   File "/builds/worker/.local/lib/python3.8/site-packages/taskgraph/main.py", line 663, in action_callback
[task 2023-06-09T08:05:37.033Z]     return trigger_action_callback(
[task 2023-06-09T08:05:37.034Z]   File "/builds/worker/.local/lib/python3.8/site-packages/taskgraph/actions/registry.py", line 336, in trigger_action_callback
[task 2023-06-09T08:05:37.034Z]     cb(Parameters(**parameters), graph_config, input, task_group_id, task_id)
[task 2023-06-09T08:05:37.034Z]   File "/builds/worker/.local/lib/python3.8/site-packages/taskgraph/actions/add_new_jobs.py", line 64, in add_new_jobs_action
[task 2023-06-09T08:05:37.034Z]     combine_task_graph_files(list(range(times)))
[task 2023-06-09T08:05:37.034Z]   File "/builds/worker/.local/lib/python3.8/site-packages/taskgraph/actions/util.py", line 204, in combine_task_graph_files
[task 2023-06-09T08:05:37.034Z]     rename_artifact(f"{filename}-{suffixes[0]}.json", f"{filename}.json")
[task 2023-06-09T08:05:37.034Z]   File "/builds/worker/.local/lib/python3.8/site-packages/taskgraph/decision.py", line 379, in rename_artifact
[task 2023-06-09T08:05:37.034Z]     os.rename(ARTIFACTS_DIR / src, ARTIFACTS_DIR / dest)
[task 2023-06-09T08:05:37.034Z] FileNotFoundError: [Errno 2] No such file or directory: 'artifacts/task-graph-0.json' -> 'artifacts/task-graph.json'
[taskcluster 2023-06-09 08:05:37.598Z] === Task Finished ===
[taskcluster 2023-06-09 08:05:37.691Z] Artifact "public/docker-contexts" not found at "/builds/worker/checkouts/vcs/docker-contexts": (HTTP code 404) no such container - Could not find the file /builds/worker/checkouts/vcs/docker-contexts in container 96c471958a1c84f69945106a73b986e517cacdf4e5237efb1e7284ddfffe136b 
[taskcluster 2023-06-09 08:05:38.813Z] Unsuccessful task run with exit code: 1 completed in 49.123 seconds

[JohanLorenzo]

I don't know if taskgraph optimized away too many tasks. Although, for sure we shouldn't fail because of a missing generated file.

[jcristau]

[task 2023-06-09T08:05:36.847Z] 2023-06-09 08:05:36,846 - INFO - writing artifact file `task-graph0.json`

So we're writing to task-graph0.json, then we're trying to move task-graph-0.json, with an extra -.

[jcristau]

Regression from f21088f: suffix is 0, so if suffix is false and we don't add the dash in create_tasks, but we assume it's there in combine_task_graph_files.

[alexandru-io]

@jcristau still happening

[jcristau]

@alexandru-io this fix is in taskgraph 5.4.0, the job you link to uses 5.1.1.