Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

opam does not wait for the Software Heritage fallback archives to be "cooked" #5721

Open
kit-ty-kate opened this issue Nov 8, 2023 · 3 comments
Open

opam does not wait for the Software Heritage fallback archives to be "cooked" #5721

kit-ty-kate opened this issue Nov 8, 2023 · 3 comments
Assignees
@rjbou
Labels
KIND: BUG
Projects
Opam 2.2.0

Comments

@kit-ty-kate
Copy link
Member

When using the software heritage fallback feature, it is understood that regenerating (as noted in #5720) the archive from a given SWHID (called "cooking" by Software Heritage) might take some time. The code currently denotes this by a 60 seconds timer:

opam/src/repository/opamDownload.ml

Line 299 in c4961ba

Unix.sleep 10;

However in practice, trying it locally, leads only a short wait then the following failure:

OpamSolution.Fetch_fail("\"/opt/homebrew/bin/wget -X POST --content-disposition -t 3 -O - -U opam/2.2.0~alpha2 -- https://archive.softwareheritage.org/api/1/vault/directory/6f61d745641e701d5cad73a9e918d5e888d5ab64/\" exited with code 8 \"2023-11-08 16:00:08 ERROR 404: Not Found.\"")

Calling the same command again when the archive has finished cooking, succeeds.
@kit-ty-kate kit-ty-kate added this to For RC in Opam 2.2.0 Nov 13, 2023
@kit-ty-kate kit-ty-kate removed this from For RC in Opam 2.2.0 Apr 2, 2024
@kit-ty-kate
Copy link
Member Author

The software heritage fallback was disabled by default in #5899 so moving this issue off the 2.2 milestone

@rjbou rjbou added this to For RC in Opam 2.2.0 Apr 9, 2024
@kit-ty-kate
Copy link
Member Author

After further investigation it looks like wget does not work to fetch the expected data, but curl does:

$ curl https://archive.softwareheritage.org/api/1/vault/directory/6f61d745641e701d5cad73a9e918d5e888d5ab64/
{"fetch_url": ...}

$ wget -O - https://archive.softwareheritage.org/api/1/vault/directory/6f61d745641e701d5cad73a9e918d5e888d5ab64/
<!DOCTYPE html>...

@rjbou rjbou self-assigned this May 27, 2024
@rjbou
Copy link
Collaborator

rjbou commented Jun 13, 2024

Some more output

Skipping 121 bytes of body: [{"exception":"NotFoundExc","reason":"Cooking of swh:1:dir:45b87e1ca127914b936884dc73c9f788df5b7abd was never requested."}] done.
2024-06-13 18:04:26 ERROR 404: Not Found.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rjbou rjbou

Labels
KIND: BUG
Projects
Opam 2.2.0
  
For RC
Milestone
No milestone
Development

No branches or pull requests
2 participants
@kit-ty-kate @rjbou