Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Model saving is calling on git which causes an issue when the subprocess library is from the system #723

Closed
sarthakpati opened this issue Nov 1, 2023 · 1 comment · Fixed by #729
Assignees

Comments

@sarthakpati
Copy link
Collaborator

Describe the bug
When the subprocess library is called from the system installation of python, git invocation does not work, since there is no git repo for the system python installation.

To Reproduce
Steps to reproduce the behavior:

  1. Start GaNDLF training
  2. See the following error:
$ > cat ${error_log}
[SNIP!]
  File "/geode2/home/u070/patis/BigRed200/projects/gandlf_mine/GANDLF/compute/training_loop.py", line 290, in training_loop
    save_model(
  File "/geode2/home/u070/patis/BigRed200/projects/gandlf_mine/GANDLF/utils/modelio.py", line 159, in save_model
    subprocess.check_output(["git", "rev-parse", "HEAD"])
  File "/N/soft/sles15/deeplearning/Python-3.10.10/Lib/subprocess.py", line 421, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/N/soft/sles15/deeplearning/Python-3.10.10/Lib/subprocess.py", line 503, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/N/soft/sles15/deeplearning/Python-3.10.10/Lib/subprocess.py", line 971, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/N/soft/sles15/deeplearning/Python-3.10.10/Lib/subprocess.py", line 1847, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)

Expected behavior
The model saving should work.

Screenshots
N.A.

GaNDLF Version

0.0.17-dev

Desktop (please complete the following information):
N.A.

Additional context
This is seen for the IU BigRed cluster.

@scap3yvt
Copy link
Collaborator

scap3yvt commented Nov 8, 2023

I have fixed this, and will put a PR soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants