JSON API gives different results depending on HTTP client

[jbarlow83]

After updating my package in production on legacy PyPI, the warehouse package shows the current version, but the PyPI JSON API seems to return stale results depending on which HTTP client queries it.

*When I use urlopen() (Py3) to access https://pypi.io/package/json, the JSON reports the results for the previous version rather than latest.
*When I use curl to access the JSON page, it also reports the old version.
*Unexpectedly, when I access https://pypi.io/package/json in my web browser, it reports the latest version. Therefore, it appears that pypi.io only responds with the latest version when more detailed HTTP headers are provided.

I have not found any documentation that what this header should be.

This behavior has broken some of my build scripts that worked about a month ago. (I can check exact dates if this is useful for bisection.) My Travis CI build process and my local box both exhibit the behavior, so it is quite unlikely that it is a local machine cache issue on my end.

To reproduce:
-use twine to upload a new version of a test package, targeting legacy PyPI
-wait for warehouse's rich HTML to reflect that the new package is available
-check the same JSON page in a web browser, and it shows the current package, as expected
-curl https://pypi.io/package/json and it incorrectly reports the old package

[jbarlow83]

About 24 hours later, curl reports the correct (new) package.

[dstufft]

I'm pretty sure that this is just going to be down to the Vary header and caching in Fastly. Uploading to legacy PyPI doesn't purge Warehouse so you have to wait for stuff to fall out of the cache "naturally" in that case. Uploading to Warehouse purges both Warehouse and Legacy.

I'm going to go ahead and close this, we're unlikely to solve this since it'll go away once Legacy upload goes away.

[jbarlow83]

@dstufft Do you recommend uploading to Warehouse at this point?

[dstufft]

@jbarlow83 Yes, and both twine and Python default to it in their latest version if you haven't overridden it in your ~/.pypirc. You can upload to https://upload.pypi.org/legacy/ for production Warehouse and https://test.pypi.org/legacy/ for TestPyPI.