Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-7900: Implement --batch-type None to run the job in the current process #6

Merged
merged 2 commits into from Oct 6, 2016

Conversation

PaulPrice
Copy link
Contributor

No description provided.

Copy link
Contributor Author

@PaulPrice PaulPrice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a useful feature. I have a few comments about the implementation. In particular, the argv parsing has a bug and should be replaced with an existing feature in the BatchArgumentParser.

@@ -262,7 +262,9 @@ def submitCommand(self, scriptName):
return "exec %s" % scriptName


BATCH_TYPES = {'pbs': PbsBatch,
BATCH_TYPES = {'none' : None,
'None' : None,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of supporting both none and None, might we always convert the key to lower case?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could do that, but wouldn't I also have to implement case-insensitive argument parsing for the validate? I've done that elsewhere, but it's messy.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, don't worry about it then.

@@ -334,6 +336,10 @@ def makeBatch(self, args):
'submit': 'batchSubmit',
'options': 'batchOptions',
}

if not BATCH_TYPES[args.batchType]:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should compare against None.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a stylistic issue or a functional one?

Copy link
Contributor Author

@PaulPrice PaulPrice Oct 5, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it's not necessary. PEP8 says:

Comparisons to singletons like None should always be done with is or is not , never the equality operators.

But you're not using an equality operator. I guess I was just expressing my personal preference, so feel free to ignore me.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, PEP8 also says

Also, beware of writing if x when you really mean if x is not None -- e.g. when testing whether a variable or argument that defaults to None was set to some other value. The other value might have a type (such as a container) that could be false in a boolean context!

so I guess I'll change it


sys.argv = argv

cls.parseAndRun()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return cls.parseAndRun()

while i < len(sys.argv):
a = sys.argv[i]
if a.startswith("--batch-t"): # i.e. --batch-type and abbreviations thereof
i += 1 # skip the value
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if I specify --batch-type=none (as argparse supports)? Then the next, unrelated, argument gets dropped.

But you shouldn't be parsing the command-line yourself. I suggest you use:

sys.argv = shCommandFromArgs(batchArgs.leftover)

That will strip out all batch-related arguments.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Much better. I asked around for better solutions than my loop, but none were suggested.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good thing I checked; I think that that's

sys.argv = [sys.argv[0]] + batchArgs.leftover

We actually support None and none as all the other batch options are lower case
Not strictly on this ticket, but not worth one of its own
@RobertLuptonTheGood RobertLuptonTheGood merged commit 0953c1f into master Oct 6, 2016
@ktlim ktlim deleted the tickets/DM-7900 branch August 25, 2018 06:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants