Attempting to parse Task identifier failed

[dohnto]

When running prometheus from hosts_and_queues 6aad879 (but master probably also), I see in logs:

ERRO[10304] Attempting to parse Task identifier failed: strconv.ParseInt: parsing "59,60": invalid syntax
ERRO[10304] Unable to marshal the XML cleanly into an object  error="strconv.ParseInt: parsing \"59,60\": invalid syntax"

Do you need the xml or this is enough?

[hjfbynara]

I have also seen this error and due to cause of the error, it is intermittent and depends a certain situation to happen. Here is a XML code block that shows an example of the situation:

<job_list state="pending"> <JB_job_number>42304214</JB_job_number> <JAT_prio>0.30043</JAT_prio> <JB_name>mrbayes_taskagent.sh</JB_name> <JB_owner>cmichael</JB_owner> <state>qw</state> <JB_submission_time>2024-02-18T13:03:40.715</JB_submission_time> <slots>8</slots> <tasks>2,3</tasks> </job_list>
I believe the issue is really with the gridengine go library used to process the XML and not specifically with the prometheus exporter. At the following function: https://github.com/metrumresearchgroup/gogridengine/blob/master/job.go#L33-L54

For me what I have seen is, if a job has more than one task AND if a task is scheduled for the same GridEngine execution node - then the code block will list in the <tasks> </tasks> XML a comma separated list of tasks numbers running on that node. Resulting in an error attempting to parse a set of comma separated integers as a single integer.

Once there are no longer multiple tasks from the same job running on a execution node - the error goes away.

Unfortunately the error stops the processing of the rest of the XML, which can leave large time spans of no metrics being captured if the tasks are long running tasks.

[tfhartmann]

@shairozan "at mention" to see if you've got any insight 😄