Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support for batch job ID for kubernetes #51

Closed
rptaylor opened this issue Aug 18, 2022 · 3 comments
Closed

support for batch job ID for kubernetes #51

rptaylor opened this issue Aug 18, 2022 · 3 comments

Comments

@rptaylor
Copy link

Addressing #50 will show us the right node names on k8s, but then we won't have the batch job IDs (which are currently reported as the node name).

Normally a job would display Batch ID under "key job parameters" and batchid under "all job parameters", for example:
https://bigpanda.cern.ch/job?pandaid=5564946734

However on kubernetes this is not detected:
https://bigpanda.cern.ch/job?pandaid=5565880899

Just as in #50 we can expose the batch ID as any environment variable.

I noticed https://github.com/PanDAWMS/pilot3/blob/master/pilot/util/auxiliary.py#L114 so I think we would hope to extend this so that the batch ID of a k8s job can be detected from an environment variable.
Note that pods already have a number of KUBERNETES_* env vars that are internally defined by kubernetes so I think we should choose something distinct from that, for example K8S_JOB_ID. I can work with Fernando to standardize this on all jobs submitted to k8s by Harvester. What do you think @PalNilsson ? Thanks.

@PalNilsson
Copy link
Collaborator

K8S_JOB_ID sounds good to me. I can add it to the dictionary you mention ('K8S_JOB_ID': 'Kubernetes').

@rptaylor
Copy link
Author

Thanks @PalNilsson , I have asked Fernando to set this up on the Harvester side, same for the other issue. I'll keep an eye out for the next pilot release, let me know if you need anything else.

@PalNilsson
Copy link
Collaborator

Added to v 3.3.4.76

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants