New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Problems creating a package from a SourceForge download URL #26
Comments
The problem stems that https://sourceforge.net/projects/scribus/files/scribus/1.6.0/scribus-1.6.0.tar.gz/download is not the actual direct download URL but is followed by several URL redirects that end up in a mirror. The final destination is something like where the first segment changes from mirror to mirror: The stable final URL would be https://master.dl.sourceforge.net/project/scribus/scribus/1.6.0/scribus-1.6.0.tar.gz None of these are practically visible and accessible. Therefore we should IMHO do these:
|
thanks @pombredanne your proposed solution looks good to me! |
Note that we have support for the We simply have to add support for this URL syntax: |
Signed-off-by: tdruez <tdruez@nexb.com>
@DennisClark I've added support for those type of URLs in the purl library, see package-url/packageurl-python#139 With those changes, we now generate a proper PURL and filename: |
Hi @tdruez I'm getting mixed results in Staging. My original scribus case went just fine, but I then tried another package from SourceForge, turbovnc-3.1.tar.gz , on staging with download URL of https://sourceforge.net/projects/turbovnc/files/3.1/turbovnc-3.1.tar.gz/download and it all went fine, including a scan, except that it did not assign any PURL values. See attached. |
Signed-off-by: tdruez <tdruez@nexb.com>
@DennisClark I've added support for the following URLs format:
You can give it another try. |
@tdruez I tested the 3 you identified in your comment, plus the scribus package, and they all look rather good, with one small issue. When I simply click on the download link for the ventoy package, it downloads a file name |
@tdruez one other observation, which is not directly related to this issue, but something that is somewhat perplexing. DejaCode found the existing scans that I created yesterday for the 4 packages (good) and apparently they did not get re-scanned (fine I think) but it did not perform any of the auto-updates to fields on the package (not so good), such as the license-expression, even though 3 of the 4 scans have a declared license. See attached. |
In the example above, the geoserver does not have a detected license anyway, so that's not a big deal, but the other 3 all have declared licenses. |
@tdruez Sorry I did not catch this one yesterday, but the results from creating a package with https://sourceforge.net/projects/spacesniffer/files/spacesniffer_1_3_0_2.zip/download do not look so great. See attached. |
It appears that there are an unknown number of (arbitrary) variations in the SourceForge download url's, suggesting we really do not have a satisfactory way to determine if we got them all. I'm sure you would like to finish this one, but it is possibly an unmanageable task. I'm ok if we go with "good enough" once we have fixed the ones we have actually discovered. |
Signed-off-by: tdruez <tdruez@nexb.com>
Signed-off-by: tdruez <tdruez@nexb.com>
Improve the support for SourgeForge download URLs #26
@DennisClark changes available for review:
Entered as #30 |
@tdruez The spacesniffer package creation works great now. The Ventoy package creation issue is fixed, although it was very slow to complete the Add Package step, with the cursor spinning for more than 2 minutes; I tested it with a different Ventoy version and had the same slow response. So it all appears to be working fine, but you might want to check on the performance problem. |
Perhaps this is a user "pilot" error, but when I create a Package in DejaCode from a SourceForge download URL, I get strange results. A recent Add Package using
https://sourceforge.net/projects/scribus/files/scribus/1.6.0/scribus-1.6.0.tar.gz/download
resulted in a Package with a filename of
download
rather thanscribus-1.6.0.tar.gz
.It also resulted in the rather verbose PURL value of
pkg:generic/download?download_url=https://sourceforge.net/projects/scribus/files/scribus/1.6.0/scribus-1.6.0.tar.gz/download
I scanned the package, using the same download URL, directly in SCIO v32.0.8, and it returned a PURL value of
pkg:autotools/scribus-1.6.0
in the key_files_packages section
So it appears that the rather eccentric download conventions of SourceForge are messing things up a bit.
The text was updated successfully, but these errors were encountered: