api.JobSubmission only includes untyped -var and -var-file content

[the-nando]

Nomad version

Nomad v1.7.7
BuildDate 2024-04-16T19:26:43Z
Revision 0f34c85ee63f6472bd2db1e2487611f4b176c70

Issue

The Nomad API endpoint to retrieve a job submission assumes all HCL2 variables submitted with a job to be of type string: https://github.com/hashicorp/nomad/blob/release/1.7.7/api/jobs.go#L929

This causes problems in the UI when editing the Job Spec of a submitted job whose variables are of types other than string. A list is rendered in the UI as:

datacenters="["dc-1a","dc-1b","dc-1c"]"

And it will cause the following error when attempting to plan the edited job from the UI:

The E2E tests follow the same behaviour by expecting all types (string, int, bool) to be serialised as string:
https://github.com/hashicorp/nomad/blob/release/1.7.7/e2e/jobsubmissions/input/xyz.hcl#L9
https://github.com/hashicorp/nomad/blob/v1.7.7/e2e/jobsubmissions/jobsubapi_test.go#L85

If I edit the variables in the UI to the correct type and submit the job they are then returned by the API, with the correct type, as Variables instead of VariableFlags

Reproduction steps

Submit a job with a -var file containing a variable of type list(string). Open the UI, edit the HCL Job Spec and submit the job.

Expected Result

The job is submitted and the variable is returned by the API with the correct type.

Actual Result

The variable is casted to string and the submit fails.

[tgross]

Hi @the-nando! I think what's happening here is the content we're saving with the submission object are strings, because it's the literal strings you've typed as -var flags, or read from the environment, or read from -var-file file. Digging into the HCL2 parsing code a bit it looks like it might be possible for us to get the go-cty.Values that the variables have been parsed into, but I can see why whomever implemented this originally didn't want to go down the route of having to convert those to JSON-compatible values.

It might be possible to fix this entirely in the UI. I'm going to flag this for our UI expert to take a look at, and we'll discuss this internally for roadmapping and more investigation.

[the-nando]

Hey @tgross :) Thanks a lot for the quick reply, much appreciated as always!
I checked only the UI side of things and I see now that I should have perhaps checked how VariableFlags are actually stored on job submission as well. Hopefully it's an easy fix!