Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add nFPM backend for building system packages (only the nfpm core bits) #19308

Merged
merged 200 commits into from
Aug 14, 2024

Conversation

cognifloyd
Copy link
Member

@cognifloyd cognifloyd commented Jun 14, 2023

Builds an nfpm backend for nFPM according to this plan:
https://docs.google.com/document/d/1b2ZvP_Ol6bOURHBaZYOD8gxixxHpV22S_zKiHKZw9iY/edit?usp=sharing

In summary, these are the new targets in this backend (along with many fields that these targets use):

  • nfpm_content_files
  • nfpm_content_file
  • nfpm_content_dirs
  • nfpm_content_dir
  • nfpm_content_symlinks
  • nfpm_content_symlink

These targets (and their fields) are prepared for follow-up PRs:

  • nfpm_deb_package
  • nfpm_apk_package
  • nfpm_archlinux_package
  • nfpm_rpm_package

There are several fields that I did not implement (as mentioned in the doc linked above):

  • fields related to package signing (I want to add this, but it'll wait for a follow-up PR):
    • signature_key_file will need dep inference when added
    • signature_key_id
    • signature_key_name (apk only)
    • signature_method (deb only)
    • signature_type (deb only)
    • signature_signer (deb only)
  • packager-specific changelog generation fields
    • changelog a yaml config file that nFPM transforms into packager-specific changelog files (only for deb, rpm)
      Supporting this might require additional fields for the pants-plugin to generate the file somehow.

In the future, someone could also add support for IPK packages. nFPM just barely gained support for that in v2.37.1. Writing the documentation for all of the other fields took me a long time, so I don't plan on adding nfpm_ipk_package any time soon. If someone wants this, they just need to add the target, ipk-specific fields, and wire the target into the various rules (via UnionRule or tiny wrapper rules), which should be fairly straight forward. I'm happy to help guide anyone that chooses to implement this.

Relationship with other PRs:

Also, add type hints for some of the field.alias properties
because mypy was getting confused.
This is needed for the field_set calculations
A common field that all nfpm packages will share,
making it easier to build a common FieldSet base.
Almost all fields are "required" in that, if present, they are
used in the generation of the package.

Users do not have to supply them, but the fields have to exist
on the Nfpm*Package targets.
@cognifloyd cognifloyd requested a review from tdyas August 13, 2024 17:27
@cognifloyd
Copy link
Member Author

I extracted TupleSequenceField into a new PR: #21301

Copy link
Contributor

@tdyas tdyas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice. Thank you for splitting up the original PR into separate PRs.

Approved with some minor style comments.

@@ -63,6 +63,41 @@ The `output_path` field present on targets which can be packaged by `pants packa

### Backends

#### NEW: nFPM
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you plan on having an example-nfpm repository users could look at? No need to develop it now or even reference it if you have one, but might be good to keep in mind as a follow-on.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hadn't thought about that. I think a section in the documentation might be useful as well, but I don't know when I can tackle something like that.

src/python/pants/backend/nfpm/subsystem.py Outdated Show resolved Hide resolved
src/python/pants/backend/nfpm/util_rules/sandbox_test.py Outdated Show resolved Hide resolved
@cognifloyd cognifloyd enabled auto-merge (squash) August 13, 2024 23:51
@cognifloyd cognifloyd merged commit 6b6579e into main Aug 14, 2024
25 checks passed
@cognifloyd cognifloyd deleted the cognifloyd/nfpm branch August 14, 2024 00:08
cognifloyd added a commit that referenced this pull request Aug 14, 2024
This adds deb packaging to the nfpm backend introduced in #19308. It
also adds the integration tests that require an `nfpm_*_package` target.

New target (and related fields):

- `nfpm_deb_package`
cognifloyd added a commit that referenced this pull request Aug 14, 2024
This adds rpm packaging to the nfpm backend introduced in #19308.

New target (and related fields):
- `nfpm_rpm_package`
cognifloyd added a commit that referenced this pull request Aug 14, 2024
This adds apk packaging to the nfpm backend introduced in #19308.

New target (and related fields):

- `nfpm_apk_package`
cognifloyd added a commit that referenced this pull request Aug 14, 2024
This adds archlinux packaging to the nfpm backend introduced in #19308.

New target (and related fields):

- `nfpm_archlinux_package`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants