Add support "Source" indices and source level packages (*.orig.tar.gz, *.debian.tar.xz, *.dsc) #409
Labels
.feature
CHANGES/<issue_number>.feature
Comments
|
From: @quba42 (quba42) This is very much a valid feature request, but I cannot say if and when we will work on it. Contributions are of course welcome. If anyone is interested in working on this feel free to contact me for some starting help. See also: pulp_deb/pulp_deb/app/tasks/synchronizing.py Line 510 in a3ee34b |
|
From: markawr (markawr) Thanks @quba42. Definitely no expectation here to get the feature without doing the work. Myself and a possibly a few colleagues will have to learn how to get a development env setup and start to contribute code. Being new to Pulp it would be useful to have a bit of hand holding and patience as we navigate the process of contributing code and having it made available in a future update of the plugin. I will start by ensuring we are subscribed to the right lists etc., if there any other resources for Pulp n00bs please feel free to share. |
|
From: @quba42 (quba42) markawr wrote:
I have this idea kicking around my head of writing exactly such a guide (how to enter the project). I have a lot of local notes by myself for myself that I could rework and share in some form. The first thing to set up is indeed a development environment. The pulp_installer repo [1] includes a set of development VMs that can be run locally using vagrant + libvirt. I believe the README file in the repo includes some instructions on setting those up. The pulpcore documentation includes some general points on contributing [2] as well as a plugin writer's guide [3]. There are two mailing lists (pulp-list, and pulp-dev) referenced here [4]. Most in person help can be obtained on the freenode IRC channel, which also maintains a matrix bridge. (Though I believe that is currently in flux due to a hostile corporate takeover of freenode). [1] https://github.com/pulp/pulp_installer [2] https://docs.pulpproject.org/pulpcore/contributing/index.html [3] https://docs.pulpproject.org/pulpcore/plugins/index.html#plugin-writer-s-guide [4] https://pulpproject.org/get_involved/ You can email me at "quirinp at atix dot de", since this ticket is probably not the best place to continue this exchange. I am sure I can provide an interactive video conference/screen share session at some point as well, perhaps once you hit the first road block in setting up or using the development environment. |
|
From: pulpbot (pulpbot) PR: #295 |
masselstine
added a commit
to masselstine/pulp_deb
that referenced
this issue
Jan 25, 2022
fixes pulp#409 pulp#409 (was https://pulp.plan.io/issues/8775) Allows the addition of Debian Source Packages, DSC files and any associate source files (.orig.tar.gz, .debian.tar.xz, ...), as well as synchronizing with a remote's source indices (sync_sources=True). Publishing a distribution will make available DSC files, sources and source indices in the repository, compliant with the Debian rempository format. The source files referenced in a dsc_file must be uploaded as artifacts before the dsc_file so a typical workflow might be -- apt-get source bc http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1-2build1.debian.tar.xz http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1.orig.tar.gz http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1-2build1.dsc http --form post $BASE/pulp/api/v3/content/deb/source_packages/ \ artifact=/pulp/api/v3/artifacts/3094de7e-da61-4cf6-ae75-a35b74472d9a/ -- Attempting to create the source_packages content from a DSC file artifact before its source files are present as artifacts will result validation errors. Once source_packages content has been created it can be inspected via its pulp_href or using the source_packages endpoint http get $BASE/pulp/api/v3/content/deb/source_packages/ Synchronizing with a remote with "sync_sources=True" will synchronize the Source Indicies files inspecting each contained paragraph to download all associated DSC files and referced source files (note: source file download may be deferred if remote policy="on_demand"), performing a high level of data validation and creating all required associations. As with uploading inspecting contents after a sync is complete can be done with the same endpoints as above. To remain spec compliant with https://wiki.debian.org/DebianRepository "md5" must be present in the ALLOWED_CONTENT_CHECKSUMS. In all cases the use of md5 is supplemental so security concerns around this addition, at least for use with this feature, should be minimal.
masselstine
added a commit
to masselstine/pulp_deb
that referenced
this issue
Jan 26, 2022
fixes pulp#409 pulp#409 (was https://pulp.plan.io/issues/8775) Allows the addition of Debian Source Packages, DSC files and any associate source files (.orig.tar.gz, .debian.tar.xz, ...), as well as synchronizing with a remote's source indices (sync_sources=True). Publishing a distribution will make available DSC files, sources and source indices in the repository, compliant with the Debian rempository format. The source files referenced in a dsc_file must be uploaded as artifacts before the dsc_file so a typical workflow might be -- apt-get source bc http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1-2build1.debian.tar.xz http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1.orig.tar.gz http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1-2build1.dsc http --form post $BASE/pulp/api/v3/content/deb/source_packages/ \ artifact=/pulp/api/v3/artifacts/3094de7e-da61-4cf6-ae75-a35b74472d9a/ -- Attempting to create the source_packages content from a DSC file artifact before its source files are present as artifacts will result validation errors. Once source_packages content has been created it can be inspected via its pulp_href or using the source_packages endpoint http get $BASE/pulp/api/v3/content/deb/source_packages/ Synchronizing with a remote with "sync_sources=True" will synchronize the Source Indicies files inspecting each contained paragraph to download all associated DSC files and referced source files (note: source file download may be deferred if remote policy="on_demand"), performing a high level of data validation and creating all required associations. As with uploading inspecting contents after a sync is complete can be done with the same endpoints as above. To remain spec compliant with https://wiki.debian.org/DebianRepository "md5" must be present in the ALLOWED_CONTENT_CHECKSUMS. In all cases the use of md5 is supplemental so security concerns around this addition, at least for use with this feature, should be minimal.
masselstine
added a commit
to masselstine/pulp_deb
that referenced
this issue
Aug 4, 2022
fixes pulp#409 pulp#409 (was https://pulp.plan.io/issues/8775) Allows the addition of Debian Source Packages, DSC files and any associate source files (.orig.tar.gz, .debian.tar.xz, ...), as well as synchronizing with a remote's source indices (sync_sources=True). Publishing a distribution will make available DSC files, sources and source indices in the repository, compliant with the Debian rempository format. The source files referenced in a dsc_file must be uploaded as artifacts before the dsc_file so a typical workflow might be -- apt-get source bc http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1-2build1.debian.tar.xz" http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1.orig.tar.gz" http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1-2build1.dsc" http --form post $BASE/pulp/api/v3/content/deb/source_packages/ \ artifact=/pulp/api/v3/artifacts/3094de7e-da61-4cf6-ae75-a35b74472d9a/ -- Attempting to create the source_packages content from a DSC file artifact before its source files are present as artifacts will result validation errors. Once source_packages content has been created it can be inspected via its pulp_href or using the source_packages endpoint http get $BASE/pulp/api/v3/content/deb/source_packages/ Synchronizing with a remote with "sync_sources=True" will synchronize the Source Indicies files inspecting each contained paragraph to download all associated DSC files and referced source files (note: source file download may be deferred if remote policy="on_demand"), performing a high level of data validation and creating all required associations. As with uploading inspecting contents after a sync is complete can be done with the same endpoints as above. To remain spec compliant with https://wiki.debian.org/DebianRepository "md5" must be present in the ALLOWED_CONTENT_CHECKSUMS. In all cases the use of md5 is supplemental so security concerns around this addition, at least for use with this feature, should be minimal.
masselstine
added a commit
to masselstine/pulp_deb
that referenced
this issue
Aug 30, 2022
fixes pulp#409 pulp#409 (was https://pulp.plan.io/issues/8775) Allows the addition of Debian Source Packages, DSC files and any associate source files (.orig.tar.gz, .debian.tar.xz, ...), as well as synchronizing with a remote's source indices (sync_sources=True). Publishing a distribution will make available DSC files, sources and source indices in the repository, compliant with the Debian rempository format. The source files referenced in a dsc_file must be uploaded as artifacts before the dsc_file so a typical workflow might be -- apt-get source bc http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1-2build1.debian.tar.xz" http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1.orig.tar.gz" http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1-2build1.dsc" http --form post $BASE/pulp/api/v3/content/deb/source_packages/ \ artifact=/pulp/api/v3/artifacts/3094de7e-da61-4cf6-ae75-a35b74472d9a/ -- Attempting to create the source_packages content from a DSC file artifact before its source files are present as artifacts will result validation errors. Once source_packages content has been created it can be inspected via its pulp_href or using the source_packages endpoint http get $BASE/pulp/api/v3/content/deb/source_packages/ Synchronizing with a remote with "sync_sources=True" will synchronize the Source Indicies files inspecting each contained paragraph to download all associated DSC files and referced source files (note: source file download may be deferred if remote policy="on_demand"), performing a high level of data validation and creating all required associations. As with uploading inspecting contents after a sync is complete can be done with the same endpoints as above. To remain spec compliant with https://wiki.debian.org/DebianRepository "md5" must be present in the ALLOWED_CONTENT_CHECKSUMS. In all cases the use of md5 is supplemental so security concerns around this addition, at least for use with this feature, should be minimal.
masselstine
added a commit
to masselstine/pulp_deb
that referenced
this issue
Dec 16, 2022
fixes pulp#409 pulp#409 (was https://pulp.plan.io/issues/8775) Allows the addition of Debian Source Packages, DSC files and any associate source files (.orig.tar.gz, .debian.tar.xz, ...), as well as synchronizing with a remote's source indices (sync_sources=True). Publishing a distribution will make available DSC files, sources and source indices in the repository, compliant with the Debian rempository format. The source files referenced in a dsc_file must be uploaded as artifacts before the dsc_file so a typical workflow might be -- apt-get source bc http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1-2build1.debian.tar.xz" http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1.orig.tar.gz" http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1-2build1.dsc" http --form post $BASE/pulp/api/v3/content/deb/source_packages/ \ artifact=/pulp/api/v3/artifacts/3094de7e-da61-4cf6-ae75-a35b74472d9a/ -- Attempting to create the source_packages content from a DSC file artifact before its source files are present as artifacts will result validation errors. Once source_packages content has been created it can be inspected via its pulp_href or using the source_packages endpoint http get $BASE/pulp/api/v3/content/deb/source_packages/ Synchronizing with a remote with "sync_sources=True" will synchronize the Source Indicies files inspecting each contained paragraph to download all associated DSC files and referced source files (note: source file download may be deferred if remote policy="on_demand"), performing a high level of data validation and creating all required associations. As with uploading inspecting contents after a sync is complete can be done with the same endpoints as above. To remain spec compliant with https://wiki.debian.org/DebianRepository "md5" must be present in the ALLOWED_CONTENT_CHECKSUMS. In all cases the use of md5 is supplemental so security concerns around this addition, at least for use with this feature, should be minimal.
masselstine
added a commit
to masselstine/pulp_deb
that referenced
this issue
Mar 27, 2023
fixes pulp#409 pulp#409 (was https://pulp.plan.io/issues/8775) Allows the addition of Debian Source Packages, DSC files and any associate source files (.orig.tar.gz, .debian.tar.xz, ...), as well as synchronizing with a remote's source indices (sync_sources=True). Publishing a distribution will make available DSC files, sources and source indices in the repository, compliant with the Debian rempository format. The source files referenced in a dsc_file must be uploaded as artifacts before the dsc_file so a typical workflow might be -- apt-get source bc http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1-2build1.debian.tar.xz" http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1.orig.tar.gz" http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1-2build1.dsc" http --form post $BASE/pulp/api/v3/content/deb/source_packages/ \ artifact=/pulp/api/v3/artifacts/3094de7e-da61-4cf6-ae75-a35b74472d9a/ -- Attempting to create the source_packages content from a DSC file artifact before its source files are present as artifacts will result validation errors. Once source_packages content has been created it can be inspected via its pulp_href or using the source_packages endpoint http get $BASE/pulp/api/v3/content/deb/source_packages/ Synchronizing with a remote with "sync_sources=True" will synchronize the Source Indicies files inspecting each contained paragraph to download all associated DSC files and referced source files (note: source file download may be deferred if remote policy="on_demand"), performing a high level of data validation and creating all required associations. As with uploading inspecting contents after a sync is complete can be done with the same endpoints as above. To remain spec compliant with https://wiki.debian.org/DebianRepository "md5" must be present in the ALLOWED_CONTENT_CHECKSUMS. In all cases the use of md5 is supplemental so security concerns around this addition, at least for use with this feature, should be minimal.
Moustafa-Moustafa
pushed a commit
to Moustafa-Moustafa/pulp_deb
that referenced
this issue
May 24, 2023
fixes pulp#409 pulp#409 (was https://pulp.plan.io/issues/8775) Allows the addition of Debian Source Packages, DSC files and any associate source files (.orig.tar.gz, .debian.tar.xz, ...), as well as synchronizing with a remote's source indices (sync_sources=True). Publishing a distribution will make available DSC files, sources and source indices in the repository, compliant with the Debian rempository format. The source files referenced in a dsc_file must be uploaded as artifacts before the dsc_file so a typical workflow might be -- apt-get source bc http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1-2build1.debian.tar.xz" http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1.orig.tar.gz" http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1-2build1.dsc" http --form post $BASE/pulp/api/v3/content/deb/source_packages/ \ artifact=/pulp/api/v3/artifacts/3094de7e-da61-4cf6-ae75-a35b74472d9a/ -- Attempting to create the source_packages content from a DSC file artifact before its source files are present as artifacts will result validation errors. Once source_packages content has been created it can be inspected via its pulp_href or using the source_packages endpoint http get $BASE/pulp/api/v3/content/deb/source_packages/ Synchronizing with a remote with "sync_sources=True" will synchronize the Source Indicies files inspecting each contained paragraph to download all associated DSC files and referced source files (note: source file download may be deferred if remote policy="on_demand"), performing a high level of data validation and creating all required associations. As with uploading inspecting contents after a sync is complete can be done with the same endpoints as above. To remain spec compliant with https://wiki.debian.org/DebianRepository "md5" must be present in the ALLOWED_CONTENT_CHECKSUMS. In all cases the use of md5 is supplemental so security concerns around this addition, at least for use with this feature, should be minimal.
masselstine
added a commit
to masselstine/pulp_deb
that referenced
this issue
Jun 1, 2023
fixes pulp#409 pulp#409 (was https://pulp.plan.io/issues/8775) Allows the addition of Debian Source Packages, DSC files and any associate source files (.orig.tar.gz, .debian.tar.xz, ...), as well as synchronizing with a remote's source indices (sync_sources=True). Publishing a distribution will make available DSC files, sources and source indices in the repository, compliant with the Debian rempository format. The source files referenced in a dsc_file must be uploaded as artifacts before the dsc_file so a typical workflow might be -- apt-get source bc http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1-2build1.debian.tar.xz" http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1.orig.tar.gz" http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1-2build1.dsc" http --form post $BASE/pulp/api/v3/content/deb/source_packages/ \ artifact=/pulp/api/v3/artifacts/3094de7e-da61-4cf6-ae75-a35b74472d9a/ -- Attempting to create the source_packages content from a DSC file artifact before its source files are present as artifacts will result validation errors. Once source_packages content has been created it can be inspected via its pulp_href or using the source_packages endpoint http get $BASE/pulp/api/v3/content/deb/source_packages/ Synchronizing with a remote with "sync_sources=True" will synchronize the Source Indicies files inspecting each contained paragraph to download all associated DSC files and referced source files (note: source file download may be deferred if remote policy="on_demand"), performing a high level of data validation and creating all required associations. As with uploading inspecting contents after a sync is complete can be done with the same endpoints as above. To remain spec compliant with https://wiki.debian.org/DebianRepository "md5" must be present in the ALLOWED_CONTENT_CHECKSUMS. In all cases the use of md5 is supplemental so security concerns around this addition, at least for use with this feature, should be minimal.
masselstine
added a commit
to masselstine/pulp_deb
that referenced
this issue
Jun 1, 2023
related to: pulp#409 (was https://pulp.plan.io/issues/8775) Extend the tests fixture to include source indicies and an example (hello) source package composed of an orig.tar.gz, DSC file and debian.tar.xz. The 'test_sync' test which already does sync tests on a remote repo with and without installer (udeb) packages is extended to perform a new round of testing which will include source packages. This is a minimum set of tests for the new source package support. Future commits will expand the testing of source package support. Signed-off-by: Mark Asselstine <mark.asselstine@windriver.com>
masselstine
added a commit
to masselstine/pulp_deb
that referenced
this issue
Jun 2, 2023
fixes pulp#409 pulp#409 (was https://pulp.plan.io/issues/8775) Allows the addition of Debian Source Packages, DSC files and any associate source files (.orig.tar.gz, .debian.tar.xz, ...), as well as synchronizing with a remote's source indices (sync_sources=True). Publishing a distribution will make available DSC files, sources and source indices in the repository, compliant with the Debian rempository format. The source files referenced in a dsc_file must be uploaded as artifacts before the dsc_file so a typical workflow might be -- apt-get source bc http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1-2build1.debian.tar.xz" http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1.orig.tar.gz" http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1-2build1.dsc" http --form post $BASE/pulp/api/v3/content/deb/source_packages/ \ artifact=/pulp/api/v3/artifacts/3094de7e-da61-4cf6-ae75-a35b74472d9a/ -- Attempting to create the source_packages content from a DSC file artifact before its source files are present as artifacts will result validation errors. Once source_packages content has been created it can be inspected via its pulp_href or using the source_packages endpoint http get $BASE/pulp/api/v3/content/deb/source_packages/ Synchronizing with a remote with "sync_sources=True" will synchronize the Source Indicies files inspecting each contained paragraph to download all associated DSC files and referced source files (note: source file download may be deferred if remote policy="on_demand"), performing a high level of data validation and creating all required associations. As with uploading inspecting contents after a sync is complete can be done with the same endpoints as above. To remain spec compliant with https://wiki.debian.org/DebianRepository "md5" must be present in the ALLOWED_CONTENT_CHECKSUMS. In all cases the use of md5 is supplemental so security concerns around this addition, at least for use with this feature, should be minimal.
masselstine
added a commit
to masselstine/pulp_deb
that referenced
this issue
Jun 2, 2023
related to: pulp#409 (was https://pulp.plan.io/issues/8775) Extend the tests fixture to include source indicies and an example (hello) source package composed of an orig.tar.gz, DSC file and debian.tar.xz. The 'test_sync' test which already does sync tests on a remote repo with and without installer (udeb) packages is extended to perform a new round of testing which will include source packages. This is a minimum set of tests for the new source package support. Future commits will expand the testing of source package support. Signed-off-by: Mark Asselstine <mark.asselstine@windriver.com>
masselstine
added a commit
to masselstine/pulp_deb
that referenced
this issue
Jun 5, 2023
fixes pulp#409 pulp#409 (was https://pulp.plan.io/issues/8775) Allows the addition of Debian Source Packages, DSC files and any associate source files (.orig.tar.gz, .debian.tar.xz, ...), as well as synchronizing with a remote's source indices (sync_sources=True). Publishing a distribution will make available DSC files, sources and source indices in the repository, compliant with the Debian rempository format. The source files referenced in a dsc_file must be uploaded as artifacts before the dsc_file so a typical workflow might be -- apt-get source bc http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1-2build1.debian.tar.xz" http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1.orig.tar.gz" http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1-2build1.dsc" http --form post $BASE/pulp/api/v3/content/deb/source_packages/ \ artifact=/pulp/api/v3/artifacts/3094de7e-da61-4cf6-ae75-a35b74472d9a/ -- Attempting to create the source_packages content from a DSC file artifact before its source files are present as artifacts will result validation errors. Once source_packages content has been created it can be inspected via its pulp_href or using the source_packages endpoint http get $BASE/pulp/api/v3/content/deb/source_packages/ Synchronizing with a remote with "sync_sources=True" will synchronize the Source Indicies files inspecting each contained paragraph to download all associated DSC files and referced source files (note: source file download may be deferred if remote policy="on_demand"), performing a high level of data validation and creating all required associations. As with uploading inspecting contents after a sync is complete can be done with the same endpoints as above. To remain spec compliant with https://wiki.debian.org/DebianRepository "md5" must be present in the ALLOWED_CONTENT_CHECKSUMS. In all cases the use of md5 is supplemental so security concerns around this addition, at least for use with this feature, should be minimal.
masselstine
added a commit
to masselstine/pulp_deb
that referenced
this issue
Jun 5, 2023
[noissue] related to: pulp#409 (was https://pulp.plan.io/issues/8775) Extend the tests fixture to include source indicies and an example (hello) source package composed of an orig.tar.gz, DSC file and debian.tar.xz. The 'test_sync' test which already does sync tests on a remote repo with and without installer (udeb) packages is extended to perform a new round of testing which will include source packages. This is a minimum set of tests for the new source package support. Future commits will expand the testing of source package support. Signed-off-by: Mark Asselstine <mark.asselstine@windriver.com>
masselstine
added a commit
to masselstine/pulp_deb
that referenced
this issue
Jun 24, 2023
fixes pulp#409 pulp#409 (was https://pulp.plan.io/issues/8775) Allows the addition of Debian Source Packages, DSC files and any associate source files (.orig.tar.gz, .debian.tar.xz, ...), as well as synchronizing with a remote's source indices (sync_sources=True). Publishing a distribution will make available DSC files, sources and source indices in the repository, compliant with the Debian rempository format. The source files referenced in a dsc_file must be uploaded as artifacts before the dsc_file so a typical workflow might be -- apt-get source bc http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1-2build1.debian.tar.xz" http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1.orig.tar.gz" http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1-2build1.dsc" http --form post $BASE/pulp/api/v3/content/deb/source_packages/ \ artifact=/pulp/api/v3/artifacts/3094de7e-da61-4cf6-ae75-a35b74472d9a/ -- Attempting to create the source_packages content from a DSC file artifact before its source files are present as artifacts will result validation errors. Once source_packages content has been created it can be inspected via its pulp_href or using the source_packages endpoint http get $BASE/pulp/api/v3/content/deb/source_packages/ Synchronizing with a remote with "sync_sources=True" will synchronize the Source Indicies files inspecting each contained paragraph to download all associated DSC files and referced source files (note: source file download may be deferred if remote policy="on_demand"), performing a high level of data validation and creating all required associations. As with uploading inspecting contents after a sync is complete can be done with the same endpoints as above. To remain spec compliant with https://wiki.debian.org/DebianRepository "md5" must be present in the ALLOWED_CONTENT_CHECKSUMS. In all cases the use of md5 is supplemental so security concerns around this addition, at least for use with this feature, should be minimal.
masselstine
added a commit
to masselstine/pulp_deb
that referenced
this issue
Jun 24, 2023
[noissue] related to: pulp#409 (was https://pulp.plan.io/issues/8775) Extend the tests fixture to include source indicies and an example (hello) source package composed of an orig.tar.gz, DSC file and debian.tar.xz. The 'test_sync' test which already does sync tests on a remote repo with and without installer (udeb) packages is extended to perform a new round of testing which will include source packages. This is a minimum set of tests for the new source package support. Future commits will expand the testing of source package support. Signed-off-by: Mark Asselstine <mark.asselstine@windriver.com>
masselstine
added a commit
to masselstine/pulp_deb
that referenced
this issue
Jun 24, 2023
fixes pulp#409 pulp#409 (was https://pulp.plan.io/issues/8775) Allows the addition of Debian Source Packages, DSC files and any associate source files (.orig.tar.gz, .debian.tar.xz, ...), as well as synchronizing with a remote's source indices (sync_sources=True). Publishing a distribution will make available DSC files, sources and source indices in the repository, compliant with the Debian rempository format. The source files referenced in a dsc_file must be uploaded as artifacts before the dsc_file so a typical workflow might be -- apt-get source bc http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1-2build1.debian.tar.xz" http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1.orig.tar.gz" http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1-2build1.dsc" http --form post $BASE/pulp/api/v3/content/deb/source_packages/ \ artifact=/pulp/api/v3/artifacts/3094de7e-da61-4cf6-ae75-a35b74472d9a/ -- Attempting to create the source_packages content from a DSC file artifact before its source files are present as artifacts will result validation errors. Once source_packages content has been created it can be inspected via its pulp_href or using the source_packages endpoint http get $BASE/pulp/api/v3/content/deb/source_packages/ Synchronizing with a remote with "sync_sources=True" will synchronize the Source Indicies files inspecting each contained paragraph to download all associated DSC files and referced source files (note: source file download may be deferred if remote policy="on_demand"), performing a high level of data validation and creating all required associations. As with uploading inspecting contents after a sync is complete can be done with the same endpoints as above. To remain spec compliant with https://wiki.debian.org/DebianRepository "md5" must be present in the ALLOWED_CONTENT_CHECKSUMS. In all cases the use of md5 is supplemental so security concerns around this addition, at least for use with this feature, should be minimal.
masselstine
added a commit
to masselstine/pulp_deb
that referenced
this issue
Jun 24, 2023
[noissue] related to: pulp#409 (was https://pulp.plan.io/issues/8775) Extend the tests fixture to include source indicies and an example (hello) source package composed of an orig.tar.gz, DSC file and debian.tar.xz. The 'test_sync' test which already does sync tests on a remote repo with and without installer (udeb) packages is extended to perform a new round of testing which will include source packages. This is a minimum set of tests for the new source package support. Future commits will expand the testing of source package support. Signed-off-by: Mark Asselstine <mark.asselstine@windriver.com>
masselstine
added a commit
to masselstine/pulp_deb
that referenced
this issue
Oct 17, 2023
fixes pulp#409 pulp#409 (was https://pulp.plan.io/issues/8775) Allows the addition of Debian Source Packages, DSC files and any associate source files (.orig.tar.gz, .debian.tar.xz, ...), as well as synchronizing with a remote's source indices (sync_sources=True). Publishing a distribution will make available DSC files, sources and source indices in the repository, compliant with the Debian rempository format. The source files referenced in a dsc_file must be uploaded as artifacts before the dsc_file so a typical workflow might be -- apt-get source bc http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1-2build1.debian.tar.xz" http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1.orig.tar.gz" http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1-2build1.dsc" http --form post $BASE/pulp/api/v3/content/deb/source_packages/ \ artifact=/pulp/api/v3/artifacts/3094de7e-da61-4cf6-ae75-a35b74472d9a/ -- Attempting to create the source_packages content from a DSC file artifact before its source files are present as artifacts will result validation errors. Once source_packages content has been created it can be inspected via its pulp_href or using the source_packages endpoint http get $BASE/pulp/api/v3/content/deb/source_packages/ Synchronizing with a remote with "sync_sources=True" will synchronize the Source Indicies files inspecting each contained paragraph to download all associated DSC files and referced source files (note: source file download may be deferred if remote policy="on_demand"), performing a high level of data validation and creating all required associations. As with uploading inspecting contents after a sync is complete can be done with the same endpoints as above. To remain spec compliant with https://wiki.debian.org/DebianRepository "md5" must be present in the ALLOWED_CONTENT_CHECKSUMS. In all cases the use of md5 is supplemental so security concerns around this addition, at least for use with this feature, should be minimal.
masselstine
added a commit
to masselstine/pulp_deb
that referenced
this issue
Oct 17, 2023
[noissue] related to: pulp#409 (was https://pulp.plan.io/issues/8775) The 'test_sync' test which already does sync tests on a remote repo with and without installer (udeb) packages is extended to perform a new round of testing which will include source packages. This is a minimum set of tests for the new source package support. Future commits will expand the testing of source package support. Signed-off-by: Mark Asselstine <mark.asselstine@windriver.com>
daviddavis
pushed a commit
to daviddavis/pulp_deb
that referenced
this issue
Nov 20, 2023
fixes pulp#409 pulp#409 (was https://pulp.plan.io/issues/8775) Allows the addition of Debian Source Packages, DSC files and any associate source files (.orig.tar.gz, .debian.tar.xz, ...), as well as synchronizing with a remote's source indices (sync_sources=True). Publishing a distribution will make available DSC files, sources and source indices in the repository, compliant with the Debian rempository format. The source files referenced in a dsc_file must be uploaded as artifacts before the dsc_file so a typical workflow might be -- apt-get source bc http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1-2build1.debian.tar.xz" http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1.orig.tar.gz" http --form post $BASE/pulp/api/v3/artifacts/ \ file@"/tmp/bc_1.07.1-2build1.dsc" http --form post $BASE/pulp/api/v3/content/deb/source_packages/ \ artifact=/pulp/api/v3/artifacts/3094de7e-da61-4cf6-ae75-a35b74472d9a/ -- Attempting to create the source_packages content from a DSC file artifact before its source files are present as artifacts will result validation errors. Once source_packages content has been created it can be inspected via its pulp_href or using the source_packages endpoint http get $BASE/pulp/api/v3/content/deb/source_packages/ Synchronizing with a remote with "sync_sources=True" will synchronize the Source Indicies files inspecting each contained paragraph to download all associated DSC files and referced source files (note: source file download may be deferred if remote policy="on_demand"), performing a high level of data validation and creating all required associations. As with uploading inspecting contents after a sync is complete can be done with the same endpoints as above. To remain spec compliant with https://wiki.debian.org/DebianRepository "md5" must be present in the ALLOWED_CONTENT_CHECKSUMS. In all cases the use of md5 is supplemental so security concerns around this addition, at least for use with this feature, should be minimal.
daviddavis
pushed a commit
to daviddavis/pulp_deb
that referenced
this issue
Nov 20, 2023
[noissue] related to: pulp#409 (was https://pulp.plan.io/issues/8775) The 'test_sync' test which already does sync tests on a remote repo with and without installer (udeb) packages is extended to perform a new round of testing which will include source packages. This is a minimum set of tests for the new source package support. Future commits will expand the testing of source package support. Signed-off-by: Mark Asselstine <mark.asselstine@windriver.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Author: markawr (markawr)
Date: 2021-05-18
Redmine Issue: 8775, https://pulp.plan.io/issues/8775
The Debian Repository Format (https://wiki.debian.org/DebianRepository/Format) allows for "Sources" indices allowing source level packages to exist alongside deb packages. This allows for 'deb-src' lines to be included in a system's sources.list or equivalent files and allows a user to use 'apt-get source <package_name>' to then inspect a package's source, make modifications and use tools such as pbuilder to build a custom version of a package.
The current Pulp Deb plugin does not handle "Source" indices and using generic_content is not really viable.
Having the Pulp Debian plugin support "sources" indices and source level packages would be a welcome addition to the plugin.
(I have reviewed past work items and didn't see previous requests or discussions on this topic, hopefully I didn't miss anything. I am willing to contribute as needed along with some colleagues but want to gauge first if this support would be accepted and desired by folks maintaining this plugin)
The text was updated successfully, but these errors were encountered: