Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add doc/distribution.md to help reduce some of the confusion. #6856

Merged
merged 7 commits into from Jan 21, 2023

Conversation

ioquatix
Copy link
Member

@ioquatix ioquatix commented Dec 4, 2022

https://bugs.ruby-lang.org/issues/19178

To clarify, I think this document should be an overview of the entire process, from source code to package installed on the user computer. However, we should emphasize and clarify the process for downstream maintainers.

doc/distribution.md Outdated Show resolved Hide resolved
doc/distribution.md Outdated Show resolved Hide resolved
ioquatix and others added 4 commits December 4, 2022 17:42
Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
```bash
$ ./autogen.sh
$ ./configure -C
$ make
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Building ruby from the source itself is not necessary, if ruby is available already.

Simply ruby ./tool/make-snapshot tmp can work.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But most distributions build ruby in an environment where ruby is not yet available.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why needs to make tarballs in such environment, instead of installing?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First of all, working ruby (currently >= 2.2) is required to build ruby from a git-cloned source tree, and tool/make-snapshot works only on a git-cloned source tree.
Although this sounds like chicken-and-egg, make dist is not intended for all Ruby developers, but for only the Ruby release managers.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use https://cache.ruby-lang.org/pub/ruby/3.0/ruby-3.0.4.tar.xz to build ruby from and it doesn't like like we need ruby for that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, this section is useful for building snapshot packages. And I am using tool/make-snapshot for that.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ioquatix I agree that it is a good idea, but it probably is not "for operating system distributions".

@voxik You're creating tarballs by yourself and then building packages?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, we don't. For official Fedora releases, we always use official tarballs. I don't remember any exception for past decade.

However, I am using self prepared tarballs for test builds, especially closer to the major upstream releases. I could use a nightly snapshots, but if there is some fast turnaround with fixes, the nightly snapshots are not flexible enough. And since I already have script which prepares the tarball as well as adjusts patches we are carrying around and updates part of the .spec file, I am using them always.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was very confusing, the first section of this document was basically something package managers should rarely if ever do (they should use release tarballs for packaging). I fixed it in 826067b.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

doc/distribution.md Outdated Show resolved Hide resolved
doc/distribution.md Outdated Show resolved Hide resolved
doc/distribution.md Outdated Show resolved Hide resolved
### Releasing an Updated Ruby Gem

Normally, the Ruby gem maintainer will release an updated gem. This gem can be installed alongside the default gem. This allows the user to update the gem without having to update Ruby.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, this suggestion would not work in practice. If distribution wanted to fix some issue, then it would be fixed by applying patch. Because as it is stated above "The standard way to build a package for distribution is to build a tarball." If there was a way to modify the tarball or override parts of the expanded tarball, then this could be different discussion.

Copy link
Member Author

@ioquatix ioquatix Dec 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should try to get clarity on this because IIUC, if a vulnerability is found in a gem, this is the current process. If the process does not work well, let's propose a better way.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From distro POV, the vulnerable code needs to be fixed. And the fix does not mean shipping the fixed gem but keeping the vulnerable version around. Distro also promises that security fixes won't break the user code. This is hard to achieve with dependencies locked in Gemfile (or possibly not locked, because who would specify e.g. dependency on cgi gem, when it is part of StdLib and this causes various other issues). All in all, patching is the way to go. The vulnerability is fixed, the vulnerable code is not shipped anymore, the Gemfile.lock files are still valid.

Now, of course I could imagine some improvements. E.g. one problem is, that the default gems are not kept as a gems in the StdLib. This basically disallows to remove the vulnerable code. Quite some time ago, I have proposed this change:

https://bugs.ruby-lang.org/issues/14737


### Releasing a New Ruby Version

If the update is critical, then the Ruby maintainers may decide to release a new version of Ruby. This new version will include the updated standard library.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Who is the target audience of this paragraph? The Ruby maintainers od the distribution package maintainers? This kind of speaks to the former, while it should provide explanations to latter.

Copy link
Contributor

@voxik voxik Dec 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe the question about the target audience should be generalized to the whole document, because it is not clear anymore, after re-reading this document

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The target audience is downstream maintainers, but it's explaining the process by which they would receive or need to make changes.

ioquatix and others added 3 commits December 5, 2022 00:35
Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
@ioquatix ioquatix merged commit 5cdf312 into ruby:master Jan 21, 2023
@ioquatix ioquatix deleted the doc-distribution branch January 21, 2023 11:12
@ioquatix
Copy link
Member Author

This document is by no means perfect, but it's a good step in the right direction. Please feel free to create more PRs to improve it.

@ioquatix
Copy link
Member Author

ioquatix commented Feb 7, 2023

@voxik @Segaja I wrote my thoughts here: https://bugs.ruby-lang.org/issues/19421

It would be great to have you, and other package maintainers, involved in that discussion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
6 participants