Environment marker support fails API tests on PyPy

[ghost]

Originally reported by: jaraco (Bitbucket: jaraco, GitHub: jaraco)

---

In this Travis-CI run, we see setuptools fails two publish API tests in the environment marker support:

File "/home/travis/build/jaraco/setuptools/tests/api_tests.txt", line 344, in api_tests.txt
Failed example:
    print(im("sys_platform==")) # doctest: +ELLIPSIS
Expected:
    unexpected EOF while parsing (...line 1)
Got:
    invalid syntax (<string>, line 1)

and

File "/home/travis/build/jaraco/setuptools/tests/api_tests.txt", line 356, in api_tests.txt
Failed example:
    print(im("(extra")) # doctest: +ELLIPSIS
Expected:
    unexpected EOF while parsing (...line 1)
Got:
    parenthesis is never closed (<string>, lines 1-2)

It seems the differences in the parser yield different error messages.

Either the API expectations need to be relaxed or setuptools needs to wrap this call to translate the error messages when invoked on PyPy.

---

• Bitbucket: https://bitbucket.org/pypa/setuptools/issue/50

[ghost]

Original comment by iElectric (Bitbucket: iElectric, GitHub: Unknown):

---

+1 see http://hydra.nixos.org/build/5592031/nixlog/1/raw

[ghost]

Original comment by iElectric (Bitbucket: iElectric, GitHub: Unknown):

---

@jaraco would it be fine to port these two tests into python code? It's hard to handle non-deterministic tests in doctests format.

[ghost]

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):

---

It would be fine except that these tests not only test the behavior, but also document and advertise the API, on which a client should be able to rely. If the message is different on PyPy, we need to provide a less strict API (not guarantee the output, but only that an error message occurs) or update setuptools to meet the prescribed API even on PyPy.

[ghost]

Original comment by iElectric (Bitbucket: iElectric, GitHub: Unknown):

---

Do we really need to show how not to use API? And if so, we could use ellipsis to hide the output.

[ghost]

Original comment by pje (Bitbucket: pje, GitHub: pje):

---

That isn't showing how not to use the API, it's showing that the API correctly detects invalid environment markers. Part of its contract is being able to tell you whether a marker is valid.

(For the parse errors provided by Python, though, it doesn't especially matter what the error output is.)

[ghost]

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):

---

Replaced overly-specific error messages with more general ones for improved cross-implementation compatibility. Fixes #50.

[ghost]

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):

---

I created one implementation that normalizes the error messages returned on the various implementations. If the error messages are unimportant, perhaps the api tests should just report/guarantee the boolean output, and add some internal unit tests to exercise other guarantees.

[ghost]

Original comment by pje (Bitbucket: pje, GitHub: pje):

---

Looks good to me! Thanks.