Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stale data on https://pypi.org/pypi/{package}/json endpoint #11939

Closed
cswinter opened this issue Jul 24, 2022 · 12 comments
Closed

Stale data on https://pypi.org/pypi/{package}/json endpoint #11939

cswinter opened this issue Jul 24, 2022 · 12 comments
Labels

Comments

@cswinter
Copy link

Describe the bug

I released version 0.1.3 of https://pypi.org/project/entity-gym-rs/ yesterday.
However, https://pypi.org/pypi/entity-gym-rs/json shows 0.1.1 as the latest version:

Full json dump, click to expand
{
  "info": {
    "author": "Clemens Winter <clemenswinter1@gmail.com>",
    "author_email": "Clemens Winter <clemenswinter1@gmail.com>",
    "bugtrack_url": null,
    "classifiers": [],
    "description": "# EntityGym for Rust\n\n[![Crates.io](https://img.shields.io/crates/v/entity-gym-rs.svg)](https://crates.io/crates/entity-gym-rs)\n[![MIT/Apache 2.0](https://img.shields.io/badge/license-MIT%2FApache-blue.svg)](./LICENSE)\n[![Crates.io](https://img.shields.io/crates/d/entity-gym-rs.svg)](https://crates.io/crates/entity-gym-rs)\n[![Actions Status](https://github.com/entity-neural-network/entity-gym-rs/workflows/Test/badge.svg)](https://github.com/entity-neural-network/entity-gym-rs/actions)\n[![Discord](https://img.shields.io/discord/913497968701747270?style=flat-square)](https://discord.gg/SjVqhSW4Qf)\n\n\n[EntityGym](https://github.com/entity-neural-network/entity-gym) is a Python library that defines a novel entity-based abstraction for reinforcement learning environments which enables highly ergonomic and efficient training of deep reinforcement learning agents.\nThis crate provides bindings that allows Rust programs to be used as EntityGym training environments, and to load and run neural networks agents trained with [Entity Neural Network Trainer](https://github.com/entity-neural-network/enn-trainer) inside Rust.\n\n## Overview\n\nThe entity-gym-rs crate provides a high-level API that allows neural network agents to interact directly with Rust data structures.\n\n```rust\nuse entity_gym_rs::agent::{Agent, AgentOps, Obs, Action, Featurizable};\n\n// The `Action` trait can be automatically derived for any enum with only unit variants.\n#[derive(Action, Debug)]\nenum Move { Up, Down, Left, Right }\n\n// The `Featurizable` trait can be automatically derived for any struct that contains\n// only primitive number types, booleans, or other `Featurizable` types.\n#[derive(Featurizable)]\nstruct Player { x: i32, y: i32 }\n\n#[derive(Featurizable)]\nstruct Cake {\n    x: i32,\n    y: i32,\n    size: u32,\n}\n\nfn main() {\n    // Creates an agent that acts completely randomly.\n    let mut agent = Agent::random();\n    // To load an neural network agent from an enn-trainer checkpoint, you would use the `load` method instead.\n    // let mut agent = Agent::load(\"agent\");\n\n    // An observation can be constructed from any number of `Featurizable` objects.\n    let obs = Obs::new(0.0)\n        .entities([Player { x: 0, y: 0 }])\n        .entities([\n            Cake { x: 4, y: 0, size: 4 },\n            Cake { x: 10, y: 42, size: 12 },\n        ]);\n    \n    // The agent `act` method takes an observation and returns an action of the specified type.\n    let action = agent.act::<Move>(obs);\n    println!(\"{:?}\", action);\n}\n```\n\n## Docs\n\n- [bevy_snake](examples/bevy_snake): Example of how to use entity-gym-rs in a Bevy game.\n- [bevy_multisnake](examples/bevy_snake): Example of more advanced Bevy integration and adversarial training with multiple agents.\n- [EntityGym Rust API Docs](https://docs.rs/entity-gym-rs/0.1.0/entity_gym_rs/): Rust API reference.\n\n",
    "description_content_type": "text/markdown; charset=UTF-8; variant=GFM",
    "docs_url": null,
    "download_url": null,
    "downloads": {
      "last_day": -1,
      "last_month": -1,
      "last_week": -1
    },
    "home_page": null,
    "keywords": null,
    "license": "MIT OR Apache-2.0",
    "maintainer": null,
    "maintainer_email": null,
    "name": "entity-gym-rs",
    "package_url": "https://pypi.org/project/entity-gym-rs/",
    "platform": null,
    "project_url": "https://pypi.org/project/entity-gym-rs/",
    "project_urls": {
      "Source Code": "https://github.com/entity-neural-network/entity-gym-rs"
    },
    "release_url": "https://pypi.org/project/entity-gym-rs/0.1.1/",
    "requires_dist": [
      "numpy~=1.22.3",
      "ragged-buffer~=0.3.7",
      "entity-gym~=0.1.5"
    ],
    "requires_python": ">=3.6",
    "summary": "Rust bindings for the entity-gym library",
    "version": "0.1.1",
    "yanked": false,
    "yanked_reason": null
  },
  "last_serial": 14529226,
  "releases": {
    "0.1.1": [
      {
        "comment_text": null,
        "digests": {
          "md5": "59245f8b1594105457f41bd17ab123ab",
          "sha256": "ba281c910eac141d8e6cb7c3287792d16c6a9272e8eebf46fe2aa5f756b511c6"
        },
        "downloads": -1,
        "filename": "entity_gym_rs-0.1.1-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.whl",
        "has_sig": false,
        "md5_digest": "59245f8b1594105457f41bd17ab123ab",
        "packagetype": "bdist_wheel",
        "python_version": "cp36",
        "requires_python": ">=3.6",
        "size": 443830,
        "upload_time": "2022-07-24T00:24:16",
        "upload_time_iso_8601": "2022-07-24T00:24:16.142150Z",
        "url": "https://files.pythonhosted.org/packages/e3/f9/46614866574082708ffbc1eaabf96e87c592ffd28ac513bf20161ce55963/entity_gym_rs-0.1.1-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.whl",
        "yanked": false,
        "yanked_reason": null
      },
      {
        "comment_text": null,
        "digests": {
          "md5": "02942aec7e26f3a8bded125ac6c3a4e2",
          "sha256": "1573e275cac73d01b3cb19b288011350382bcf5255c4ae5368a6abf0a3190c36"
        },
        "downloads": -1,
        "filename": "entity_gym_rs-0.1.1.tar.gz",
        "has_sig": false,
        "md5_digest": "02942aec7e26f3a8bded125ac6c3a4e2",
        "packagetype": "sdist",
        "python_version": "source",
        "requires_python": ">=3.6",
        "size": 26096,
        "upload_time": "2022-07-24T00:24:17",
        "upload_time_iso_8601": "2022-07-24T00:24:17.686626Z",
        "url": "https://files.pythonhosted.org/packages/76/92/770c2cbf9b53abdab113adbf16a3d52c7ef4cbee94ae7d248ee1b08cf901/entity_gym_rs-0.1.1.tar.gz",
        "yanked": false,
        "yanked_reason": null
      }
    ]
  },
  "urls": [
    {
      "comment_text": null,
      "digests": {
        "md5": "59245f8b1594105457f41bd17ab123ab",
        "sha256": "ba281c910eac141d8e6cb7c3287792d16c6a9272e8eebf46fe2aa5f756b511c6"
      },
      "downloads": -1,
      "filename": "entity_gym_rs-0.1.1-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.whl",
      "has_sig": false,
      "md5_digest": "59245f8b1594105457f41bd17ab123ab",
      "packagetype": "bdist_wheel",
      "python_version": "cp36",
      "requires_python": ">=3.6",
      "size": 443830,
      "upload_time": "2022-07-24T00:24:16",
      "upload_time_iso_8601": "2022-07-24T00:24:16.142150Z",
      "url": "https://files.pythonhosted.org/packages/e3/f9/46614866574082708ffbc1eaabf96e87c592ffd28ac513bf20161ce55963/entity_gym_rs-0.1.1-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.whl",
      "yanked": false,
      "yanked_reason": null
    },
    {
      "comment_text": null,
      "digests": {
        "md5": "02942aec7e26f3a8bded125ac6c3a4e2",
        "sha256": "1573e275cac73d01b3cb19b288011350382bcf5255c4ae5368a6abf0a3190c36"
      },
      "downloads": -1,
      "filename": "entity_gym_rs-0.1.1.tar.gz",
      "has_sig": false,
      "md5_digest": "02942aec7e26f3a8bded125ac6c3a4e2",
      "packagetype": "sdist",
      "python_version": "source",
      "requires_python": ">=3.6",
      "size": 26096,
      "upload_time": "2022-07-24T00:24:17",
      "upload_time_iso_8601": "2022-07-24T00:24:17.686626Z",
      "url": "https://files.pythonhosted.org/packages/76/92/770c2cbf9b53abdab113adbf16a3d52c7ef4cbee94ae7d248ee1b08cf901/entity_gym_rs-0.1.1.tar.gz",
      "yanked": false,
      "yanked_reason": null
    }
  ],
  "vulnerabilities": []
}

The API does seem to be working as expected when explicitly requesting 0.1.3:

https://pypi.org/pypi/entity-gym-rs/0.1.3/json response
{
  "info": {
    "author": "Clemens Winter <clemenswinter1@gmail.com>",
    "author_email": "Clemens Winter <clemenswinter1@gmail.com>",
    "bugtrack_url": null,
    "classifiers": [],
    "description": "# EntityGym for Rust\n\n[![Crates.io](https://img.shields.io/crates/v/entity-gym-rs.svg?style=flat-square)](https://crates.io/crates/entity-gym-rs)\n[![PyPI](https://img.shields.io/pypi/v/entity-gym-rs.svg?style=flat-square)](https://pypi.org/project/entity-gym-rs/)\n[![MIT/Apache 2.0](https://img.shields.io/badge/license-MIT%2FApache-blue.svg?style=flat-square)](./LICENSE)\n[![Crates.io](https://img.shields.io/crates/d/entity-gym-rs.svg?style=flat-square)](https://crates.io/crates/entity-gym-rs?style=flat-square)\n[![Discord](https://img.shields.io/discord/913497968701747270?style=flat-square)](https://discord.gg/SjVqhSW4Qf)\n[![Actions Status](https://github.com/entity-neural-network/entity-gym-rs/workflows/Test/badge.svg)](https://github.com/entity-neural-network/entity-gym-rs/actions)\n\n\n[EntityGym](https://github.com/entity-neural-network/entity-gym) is a Python library that defines a novel entity-based abstraction for reinforcement learning environments which enables highly ergonomic and efficient training of deep reinforcement learning agents.\nThis crate provides bindings that allows Rust programs to be used as EntityGym training environments, and to load and run neural networks agents trained with [Entity Neural Network Trainer](https://github.com/entity-neural-network/enn-trainer) inside Rust.\n\n## Overview\n\nThe entity-gym-rs crate provides a high-level API that allows neural network agents to interact directly with Rust data structures.\n\n```rust\nuse entity_gym_rs::agent::{Agent, AgentOps, Obs, Action, Featurizable};\n\n// We can derive an `Action` trait on enums with only unit variants to allow it to be used as a categorical action.\n#[derive(Action, Debug)]\nenum Move { Up, Down, Left, Right }\n\n// The `Featurizable` trait converts data structures into a format that can be processed by neural networks.\n// It can be automatically derived for any struct that contains, only primitive number types, booleans, or\n// other `Featurizable` types.\n#[derive(Featurizable)]\nstruct Player { x: i32, y: i32 }\n\n#[derive(Featurizable)]\nstruct Cake {\n    x: i32,\n    y: i32,\n    size: u32,\n}\n\nfn main() {\n    // Creates an agent that acts completely randomly.\n    let mut agent = Agent::random();\n    // Alternatively, load a trained neural network agent from a checkpoint.\n    // let mut agent = Agent::load(\"agent\");\n\n    // The neural network agents supported by entity-gym can process observations consisting\n    // of any number of `Featurizable` objects.\n    let obs = Obs::new(0.0)\n        .entities([Player { x: 0, y: 0 }])\n        .entities([\n            Cake { x: 4, y: 0, size: 4 },\n            Cake { x: 10, y: 42, size: 12 },\n        ]);\n    \n    // To get an action from an agent, we simple call the `act` method with the observation we constructed.\n    let action = agent.act::<Move>(obs);\n    println!(\"{:?}\", action);\n}\n```\n\n## Docs\n\n- [bevy_snake](examples/bevy_snake): Example of how to use entity-gym-rs in a Bevy game.\n- [bevy_multisnake](examples/bevy_snake): Example of more advanced Bevy integration and adversarial training with multiple agents.\n- [EntityGym Rust API Docs](https://docs.rs/entity-gym-rs/0.1.0/entity_gym_rs/): Rust API reference.\n\n",
    "description_content_type": "text/markdown; charset=UTF-8; variant=GFM",
    "docs_url": null,
    "download_url": null,
    "downloads": {
      "last_day": -1,
      "last_month": -1,
      "last_week": -1
    },
    "home_page": null,
    "keywords": null,
    "license": "MIT OR Apache-2.0",
    "maintainer": null,
    "maintainer_email": null,
    "name": "entity-gym-rs",
    "package_url": "https://pypi.org/project/entity-gym-rs/",
    "platform": null,
    "project_url": "https://pypi.org/project/entity-gym-rs/",
    "project_urls": {
      "Source Code": "https://github.com/entity-neural-network/entity-gym-rs"
    },
    "release_url": "https://pypi.org/project/entity-gym-rs/0.1.3/",
    "requires_dist": [
      "entity-gym~=0.1.5",
      "ragged-buffer~=0.3.7",
      "numpy~=1.22.3"
    ],
    "requires_python": ">=3.7",
    "summary": "Rust bindings for the entity-gym library",
    "version": "0.1.3",
    "yanked": false,
    "yanked_reason": null
  },
  "last_serial": 14529926,
  "urls": [
    {
      "comment_text": null,
      "digests": {
        "md5": "81c131b098e01864e73c3fdc7d2a1611",
        "sha256": "b2cd2f032037ef7abfe160c4722d5416136a1f9f38608cb4da2c860d39074ed7"
      },
      "downloads": -1,
      "filename": "entity_gym_rs-0.1.3-cp37-cp37m-macosx_10_7_x86_64.whl",
      "has_sig": false,
      "md5_digest": "81c131b098e01864e73c3fdc7d2a1611",
      "packagetype": "bdist_wheel",
      "python_version": "cp37",
      "requires_python": ">=3.7",
      "size": 406093,
      "upload_time": "2022-07-24T04:00:30",
      "upload_time_iso_8601": "2022-07-24T04:00:30.305549Z",
      "url": "https://files.pythonhosted.org/packages/56/d1/ca3cc47a17424d7c171a78697d933b2d34d718bb0e2fe6370c2144cdba30/entity_gym_rs-0.1.3-cp37-cp37m-macosx_10_7_x86_64.whl",
      "yanked": false,
      "yanked_reason": null
    },
    {
      "comment_text": null,
      "digests": {
        "md5": "a148c3097d31261440fcd6b9bdcb84da",
        "sha256": "283b7ae82c3886bf75d6c760bc073f60fabfc44ac36cd242e55285a84d33b305"
      },
      "downloads": -1,
      "filename": "entity_gym_rs-0.1.3-cp38-cp38-macosx_10_7_x86_64.whl",
      "has_sig": false,
      "md5_digest": "a148c3097d31261440fcd6b9bdcb84da",
      "packagetype": "bdist_wheel",
      "python_version": "cp38",
      "requires_python": ">=3.7",
      "size": 406237,
      "upload_time": "2022-07-24T04:00:31",
      "upload_time_iso_8601": "2022-07-24T04:00:31.788941Z",
      "url": "https://files.pythonhosted.org/packages/7c/61/89176cbbef779c1ce7e61daeefdf35f68d0a963d7b2cb6660aec9ac5ffdc/entity_gym_rs-0.1.3-cp38-cp38-macosx_10_7_x86_64.whl",
      "yanked": false,
      "yanked_reason": null
    },
    {
      "comment_text": null,
      "digests": {
        "md5": "2648dad6c246053f6e31362a1b871e0c",
        "sha256": "6678b681461289daa4e7c5dd3ae3a4ca26026e45cdd7517436ad8bf18879df8e"
      },
      "downloads": -1,
      "filename": "entity_gym_rs-0.1.3-cp39-cp39-macosx_10_7_x86_64.whl",
      "has_sig": false,
      "md5_digest": "2648dad6c246053f6e31362a1b871e0c",
      "packagetype": "bdist_wheel",
      "python_version": "cp39",
      "requires_python": ">=3.7",
      "size": 406000,
      "upload_time": "2022-07-24T04:00:33",
      "upload_time_iso_8601": "2022-07-24T04:00:33.202552Z",
      "url": "https://files.pythonhosted.org/packages/aa/90/e617a7cd7424c0d4c33dcd47b4145542a38f218c7b12a889438cc73978b0/entity_gym_rs-0.1.3-cp39-cp39-macosx_10_7_x86_64.whl",
      "yanked": false,
      "yanked_reason": null
    },
    {
      "comment_text": null,
      "digests": {
        "md5": "ad2456cfe786833c4e50d759b65cccfa",
        "sha256": "d8065dd94e9b460918aac8308d05742619938d24aff372ff5544a4a37377cac3"
      },
      "downloads": -1,
      "filename": "entity_gym_rs-0.1.3.tar.gz",
      "has_sig": false,
      "md5_digest": "ad2456cfe786833c4e50d759b65cccfa",
      "packagetype": "sdist",
      "python_version": "source",
      "requires_python": ">=3.7",
      "size": 25958,
      "upload_time": "2022-07-24T04:00:34",
      "upload_time_iso_8601": "2022-07-24T04:00:34.458194Z",
      "url": "https://files.pythonhosted.org/packages/00/f2/9e38e0a6063ded29ddddd2b69ad4c5a2ed8bac080589984030c3ffa8eb06/entity_gym_rs-0.1.3.tar.gz",
      "yanked": false,
      "yanked_reason": null
    }
  ],
  "vulnerabilities": []
}

This appears to make it impossible to install the latest version of this package when using poetry. When using pip, version 0.1.3 installs just fine.

Expected behavior

I expect https://pypi.org/pypi/entity-gym-rs/json to return 0.1.3 as the latest version, rather than 0.1.1.

To Reproduce

Additional context

Possibly related to #11937 and #11935?

@cswinter cswinter added bug 🐛 requires triaging maintainers need to do initial inspection of issue labels Jul 24, 2022
@jedie
Copy link

jedie commented Jul 24, 2022

Maybe that cause python-poetry/poetry#3543 ?

I just release PyInventory v0.14.0 it's public:

grafik

But the JSON api still expose the older v0.13.1: https://pypi.org/pypi/pyinventory/json

grafik

@chrisbc
Copy link

chrisbc commented Jul 24, 2022

I have exactly the same problem with version 0.7.1 and 0.7.2 of https://pypi.org/project/deluge-card. 0.7.2 was released about 15 hours ago.

https://pypi.org/project/deluge-card/json returns >>>

r2 = requests.get('https://pypi.org/pypi/deluge-card/json', headers=headers)
>>> r2.json()['info']['release_url']
'https://pypi.org/project/deluge-card/0.7.0/'

and yet the HTML view is up to date at 0.7.2
Screen Shot 2022-07-25 at 8 33 37 AM

Poetry (even the preview version 1.20b3) cannot find/install 0.7.1 or 0.7.2, but pip3 can.

UPDATE: @ approx t+24hrs

  • the pypi/deluge-card/json API is now showing 0.7.2 for deluge-card, and
  • install using Poetry works (nb no cache updates needed)

To the warehouse team: first) thank you! and second) there were no notices or comms about this issue that I could find on pypi.org (or anywhere for that matter) - and no evidence that anyone was looking at it (no-one assigned, no on the team commenting). It'd be good to share with the community that something's wrong and you're aware of it. Also I'd like to know if it's now typical that package updates take up to 24hrs to propagate fully, or if something has been or is being fixed.

@mgorny
Copy link

mgorny commented Jul 25, 2022

Also happened e.g. to Sphinx 5.1.0.

@neersighted
Copy link

We're seeing a big wave of reports from Poetry users -- can anyone from the Warehouse team chime in with a likely root cause or timeline?

@di
Copy link
Member

di commented Jul 25, 2022

This is due to #11949, we are working on a fix.

@di di removed the requires triaging maintainers need to do initial inspection of issue label Jul 25, 2022
@willronchetti
Copy link

Is this released yet? Still cannot install updated versions. Any timeline on release? Looks like relevant PRs have been merged?

@dstufft
Copy link
Member

dstufft commented Jul 25, 2022

It should be live now, but it will take some time for the old cached objects to cycle through.

@di
Copy link
Member

di commented Jul 25, 2022

We just manually purged all projects that had stale JSON pages, this should work as expected now.

@di di closed this as completed Jul 25, 2022
@vwxyzjn
Copy link

vwxyzjn commented Jul 25, 2022

Hey @di , I am still having trouble with envpool==0.6.3. Its json API still returns the info on only one wheel when three wheels are available.

$ curl -s https://pypi.org/pypi/envpool/0.6.3/json | jq '.urls[].url'
"https://files.pythonhosted.org/packages/46/d0/0040229f6e10aa527035f4820b215553ad6824973a44d1d2a70bc8dfc7dc/envpool-0.6.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl"
$ curl -s https://pypi.org/pypi/envpool/0.6.2/json | jq '.urls[].url'
"https://files.pythonhosted.org/packages/9a/1e/285b91e7f13d58dc00cccc66543b0c71bf91a018ba853037833b159af51e/envpool-0.6.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl"
"https://files.pythonhosted.org/packages/8b/89/972a03db127d9dcb415019dd98d4f6ab36e687858e4bff2b9abb3eb4bef6/envpool-0.6.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl"
"https://files.pythonhosted.org/packages/6b/cf/3e5f2a8abcd38ba61c5799cba0e5b09de9bcde5cc2d886b08e5590c0324f/envpool-0.6.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl"

image

@ewdurbin
Copy link
Member

ewdurbin commented Jul 25, 2022

hi @vwxyzjn, we only initially purged the main JSON URLS, but are issuing purges for the versioned endpoints for all projects who had any releases or release file uploads during the impacted period.

you should see resolution for the URLS noted shortly.

@vwxyzjn
Copy link

vwxyzjn commented Jul 25, 2022

Thank you! I can confirm it works now :)

@claudiodsf
Copy link

Hi, not sure if this is the right place to report, or if you want me to open a new issue, but I'm currently experiencing the same problem with https://pypi.org/pypi/stockwell: latest version is 1.0.7, but https://pypi.org/pypi/stockwell/json is stuck to 1.0.6.

As a side problem, https://pypi.io/packages/source/s/stockwell/stockwell-1.0.7.tar.gz returns a 404.

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests