Pass and use original sys.argv to/with workers

[blueyed]

This gets used e.g. by argparse for the "prog" part.

We could explicitly pass it through and/or set it on
config._parser.prog, but that is a bit tedious just for this use case,
and it looks like "simulating" the main prog here appears to not be that
bad of a hack after all.

Fixes #358, Fixes #384.

[blueyed]

Noticed this with pytest-dev/pytest#4523 - it uses "-c" otherwise.

[RonnyPfannschmidt]

hm, this hack is a double edged sword, we should reset that stuff after the config initialization is sorted out correctly

[nicoddemus]

This also should fix #384 (updated the description to reflect that).

[nicoddemus]

Changing sys.argv like this makes me a little uncomfortable, but it seems the only way to make this work if users want to peek at sys.argv in worker processes anyway.

[RonnyPfannschmidt]

again - this hack is going to be a problem ^^

[nicoddemus]

hm, this hack is a double edged sword, we should reset that stuff after the config initialization is sorted out correctly

Oh, to me this reads as "Well this is problematic, we should revert this when config is fixed", so I understood it was OK to merge this.

Well, then we might as well say that workers don't have access to the same sys.argv of the master process then. Seems like the correct approach given that workers are started with a different and undefined sys.argv and shouldn't be relied on. We could then add this to the FAQ.

[nicoddemus]

Hah, coincidentally just ran into this post by @nedbat! 😁

[nicoddemus]

actually, probably this patch would make his error appear with xdist as well.

[blueyed]

We could also make "mainargv" available somehow.

My use case was getting proper "prog" with the usage information in pytest, and that would at least require to pass through "prog" quite a bit - or set it on _parser directly, which also did not feel right.
Would you like that better, @RonnyPfannschmidt ?

[nicoddemus]

Would just document that master's sys.argv is available in config.workerinput["mainargv"] is enough? If yes, you only need to remove the sys.argv assignment.

[blueyed]

Yeah, that might be good enough.
And I assume the pytest plugin part could actually set something on _parser directly then from there? (otherwise it would also require a patch to pytest, but it might be good after all to handle prog there in a better way)

[nicoddemus]

And I assume the pytest plugin part could actually set something on _parser directly then from there?

Hmm I don't know about the _parser bit, can you elaborate?

[blueyed]

We would need to get it into here: https://github.com/pytest-dev/pytest/blob/abb0dfcfa36cf358196f1daa55fb45ab23d57b07/src/_pytest/config/argparsing.py#L321-L322
Via https://github.com/pytest-dev/pytest/blob/abb0dfcfa36cf358196f1daa55fb45ab23d57b07/src/_pytest/config/__init__.py#L587-L600 and https://github.com/pytest-dev/pytest/blob/abb0dfcfa36cf358196f1daa55fb45ab23d57b07/src/_pytest/config/argparsing.py#L14-L21.

[nicoddemus]

Ahh OK, that's the "use" bit in the title of the PR.

I think this can be postponed to another PR, as it seems quite tricky to get it right, while just making mainargv available is already useful for some users.

[blueyed]

@RonnyPfannschmidt
Do you agree?

How is workerinput used typically / after all?

[blueyed]

Done.
But could not find a place to document it, added a changelog though.

[blueyed]

Requires pytest-dev/pytest#4528.

[nicoddemus]

But could not find a place to document it, added a changelog though.

I suggest to add a new section before/after "Identifying the worker process during a test" in the README.

[nicoddemus]

I created a follow up for the docs: #391

Merging now as the failure is unrelated and has been fixed on master already. 👍

Thanks @blueyed!