Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FIX: Build cmdline from working directory #2521

Merged
merged 2 commits into from
Apr 3, 2018

Conversation

effigies
Copy link
Member

This fix supports an earlier ridiculous hack (#1857) that was broken by properly managing working directories (#2368).

Fixes #2518.

Changes proposed in this pull request

  • Construct cmdline within the working directory, for the probably one case in which the cmdline depends on the working directory.
  • Adds and tests an indirectory context manager, because I hate the os.getcwd(); os.chdir(); CONTENT; os.chdir() pattern.

There may be a more elegant way to get input traits into a file, but it's not clear to me what that is at this point, so I went for the cheap solution.

@codecov-io
Copy link

codecov-io commented Mar 30, 2018

Codecov Report

Merging #2521 into master will increase coverage by 0.04%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2521      +/-   ##
==========================================
+ Coverage   66.84%   66.88%   +0.04%     
==========================================
  Files         327      327              
  Lines       42479    42487       +8     
  Branches     5268     5268              
==========================================
+ Hits        28395    28419      +24     
+ Misses      13380    13366      -14     
+ Partials      704      702       -2
Flag Coverage Δ
#smoketests 50.85% <100%> (+0.32%) ⬆️
#unittests 64.24% <100%> (ø) ⬆️
Impacted Files Coverage Δ
nipype/pipeline/engine/nodes.py 83.94% <100%> (+0.35%) ⬆️
nipype/utils/filemanip.py 77.96% <100%> (+0.37%) ⬆️
nipype/interfaces/spm/base.py 86.79% <0%> (-0.67%) ⬇️
nipype/utils/provenance.py 84.71% <0%> (+1.27%) ⬆️
nipype/workflows/fmri/spm/preprocess.py 72.44% <0%> (+12.24%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 539930e...555a905. Read the comment docs.

@effigies effigies added this to the 1.0.3 milestone Mar 30, 2018
Copy link
Contributor

@oesteban oesteban left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The context manager is a great idea, left just one nit pick to check before merging.

@@ -627,7 +627,8 @@ def _run_command(self, execute, copyfiles=True):
self._interface.__class__.__name__)
if issubclass(self._interface.__class__, CommandLine):
try:
cmd = self._interface.cmdline
with indirectory(outdir):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does outdir always exist at this point?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We create the command.txt file in line 637, just below here, and copy files to working directory just above.

A quick scan didn't show me where the directory is created, but I'm reasonable confident it exists.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The directory is created here, inside run():

https://github.com/effigies/nipype/blob/555a905f623be290a16c5fede1ea3a1558ac1f57/nipype/pipeline/engine/nodes.py#L469

Therefore, it is not granted that it will exist at this point. We could add a makedirs call to the context manager.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

run() then calls _run_interface(), which calls _run_command(), which contains this code.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but you can always call cmdline manually before run.

inside a workflow this does not apply, but many people use the interfaces and call cmdline in their ipython interactive sessions.

@effigies
Copy link
Member Author

effigies commented Apr 1, 2018 via email

@satra
Copy link
Member

satra commented Apr 1, 2018

@effigies - that is indeed an important conceptual distinction.

interfaces do whatever the underlying command would do without trying to change the behavior of the command.

nodes have additional policies (fixed working directory), which is why interfaces have some additional metadata for nodes (copyfile in particular).

@effigies
Copy link
Member Author

effigies commented Apr 2, 2018

Not quite sure if we're all on the same page here. Just to clarify: It is my position that this doesn't need changing at this point. Node will create the directory prior to the indirectory() context, so makedirs is not required, and running the interface will not hit the indirectory() context manager code, nor should it, as interfaces do not change directories.

@satra @oesteban If you disagree or are expecting changes, please let me know.

@oesteban
Copy link
Contributor

oesteban commented Apr 2, 2018

Gotcha, no disagreement here

@effigies effigies merged commit f2aaeb9 into nipy:master Apr 3, 2018
@effigies effigies deleted the fix/cmdline_cwd branch April 3, 2018 12:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

expert.opts are being written to the root directory
4 participants