Consolidate Core Atom Packages
Atom's official distribution is comprised of 92 core packages which provide its built-in functionality. These packages currently live in their own independent repositories in the Atom organization, all with their own separate issues, PRs, releases, and CI configurations. This RFC proposes that by consolidating most, if not all, of these core packages back into the
atom/atom repo, we will see the following benefits:
- Less confusion for new contributors
- Simpler core package contribution experience
- Greatly reduced burden for maintainers
Let's cover each of the bullet points mentioned above:
Less confusion for contributors
Imagine that a new contributor wants to add a small new feature to the
tree-view package. The first place they are likely to look is the
atom/atom repository. Scanning through the folders will lead to a dead end as nothing that looks like
tree-view code can be found. They might take one of the following steps next:
- By reading README.md, maybe they will decide to click the link to the Atom Flight Manual and maybe find the Contributing to Official Atom Packages page there
- They could read the CONTRIBUTING.md file which has a section that explains where to find the repos for core packages and how to contribute, but we don't really have a clear pointer to that in our README.md
- If they don't happen to find that page, they might use Google to search for "atom tree view" and find the atom/tree-view repo and maybe read the CONTRIBUTING.md file which sends them to Atom's overall contribution documentation
- They might go to the Atom Forum or Slack community to ask how to contribute to a particular part of Atom and hopefully get a helpful response that points them in the right direction
Having all of the core Atom packages represented in a top-level
packages folder, even if they all don't actually live in the repo, will go a long way to making the core package code more discoverable.
Simpler core package contribution experience
Separating core Atom features out into individual repositories and delivering them to Atom builds via
apm is a great idea in theory because it validates the Atom package ecosystem and gives developers many examples of how to develop an Atom package. It also gives Atom developers real-world experience working with Atom's APIs so that we ensure community package authors have the same hackability that Atom developers enjoy.
On the other hand, having these packages live in separate repositories and released "independently" introduces a great deal of overhead when adding new features. Here is a comparison of the current package development workflow contrasted to what we could achieve with consolidated packages:
Current Package Development Workflow
For example, to add a single feature to the
tree-view package, one must:
- Fork and clone the
tree-viewrepository to their computer (making sure to pull the commit relevant to the version of Atom they are working with)
apm linkinside of the repo folder
- Make their desired changes to the code
- Open a PR to the
tree-viewrepo and wait for CI to pass and a maintainer to review it
- Work with maintainers to get the PR approved and merged
After this is finished, an Atom maintainer must take the following steps:
- Clone the
apm publishto publish a new release of the package
package.jsonin the Atom repo to reflect the new version of
- Commit and push the changes to the relevant branch where the change belongs (
Simplified Package Development
If we were to move
tree-view (or any other core Atom package) back into
atom/atom, the development workflow would look more like this:
- Fork and clone
atom/atomand switch to a release branch if necessary
- Build Atom and launch it in dev mode
- Make desired changes to the code in
- Open a PR on
atom/atomand wait for CI to pass and a maintainer to review it
- Work with maintainers to get the PR approved and merged
At this point, the change is merged into Atom and ready for inclusion in the next release.
Greatly reduced burden for maintainers
Since packages all have their own repositories, this means that we have to watch 91 different repos for issues and pull requests. This also means that we have to redirect issues filed on
atom/atom to the appropriate repository when a user doesn't know where it belongs. Even more importantly, there's not an easy way to prioritize and track issues across the Atom organization without using GitHub Projects.
Also, as mentioned above, there's the added duty of doing the package "version dance" when we merge any new PRs to a package repository: publish the package update, update
package.json in Atom. It's very easy to forget to do this and not have community contributions included in the next Atom release!
The more core packages live in
atom/atom, the less work Atom maintainers have to do overall.
Many of Atom's core packages now live in the core
atom/atom repository. To the Atom user, this change will be imperceptible as these packages still show up in the list of Core Packages in the Settings View. Users can still optionally disable these packages.
For maintainers and contributors, there will be less juggling of repositories and no more publishing of updates to these packages with
Contributors now clone and build
atom/atom to work on improvements to core packages. They will no longer have to use
apm link in dev mode to test changes they make to packages in the repo's
packages folder. Core packages that aren't consolidated still have folders under
packages with README.md files that point to the home repository for that package.
When a contributor sends a PR to
atom/atom that only affects files in a folder under
packages, only the specs for the relevant package folders will be executed using Atom's CI scripts. This means that a full Atom build will not be required when no Atom Core code is changed in a PR. Package specs are also now run against all 3 OSes on Atom
master and release builds.
Atom maintainers no longer have to publish new versions to consolidated core packages and then edit
package.json to bump the package version in a particular Atom release branch (Stable, Beta, or
master). When a PR against a consolidated core package in
atom/atom is merged, no version number change is required and the changes will immediately be a part of the next release from that branch.
One possible drawback of this approach is that there might be some initial confusion where core Atom packages live, especially if some are consolidated into
atom/atom and others still live in their own repositories. We will manage this confusion by doing the following:
README.mdfile in the
packagesfolder which lists core packages that are not consolidated in the Atom repo. This will enable users to find the home repositories of those packages.
Archive the repositories for consolidated core packages, but only after migrating existing issues, merging or closing existing PRs, and updating the README.md to point to the new home of the package code.
Also, contributors will now have to fork, clone, and build
atom/atom to contribute to core packages where they would previously just need to clone the package repository. This might put added burden on them such as installing necessary build dependencies on their machine that they wouldn't otherwise need. It is very likely we could simplify this process for them, though.
One final drawback is that it will now be harder to have single-package maintainers. We currently have 7 core packages where there is a maintainer who isn't a part of the core Atom maintainers team. These maintainers generally are able to merge community PRs and make commits to those packages with their own judgement. If we get rid of individual package repositories, do we now make those maintainers full Atom maintainers?
Rationale and alternatives
The Motivation section explains most of the rationale, so this section will focus on the process of consolidating packages back into
atom/atom. The set of packages we've chosen to consolidate were evaluated based on a few factors:
- Number of open issues and PRs (exclude any with > 10 open PRs)
- Time since last update (longer duration since last update is prioritized)
- Number of package-only maintainers on the repo (exclude any with package maintainers for now)
Using this criteria, all 91 packages have been evaluated and categorized to determine whether they are good candidates for consolidation:
Initial Consolidation Candidates
|Package||Open Issues||Open PRs||Outside Maintainers||Last Updated|
Packages to be Consolidated Later
The following packages will not be consolidated until the stated reasons can be resolved or we decide on a consolidation strategy for them:
|Package||Open Issues||Open PRs||Outside Maintainers||Last Updated||Reason|
|find-and-replace||219||17||0||6/4/18||Too many open PRs|
|fuzzy-finder||89||22||0||5/17/18||Too many open PRs|
|language-c||53||15||0||7/10/18||Too many open PRs|
|language-go||12||2||1||6/18/18||Package maintainer, possibly inactive?|
|language-perl||17||1||1||10/30/17||Package maintainer, possibly inactive?|
|markdown-preview||139||12||0||1/8/18||Too many open PRs|
|settings-view||137||18||0||5/17/18||Too many open PRs|
|spell-check||68||14||1||5/25/18||Too many open PRs, package maintainer|
|symbols-view||86||13||0||12/10/17||Too many open PRs|
|tree-view||210||36||0||3/21/18||Too many open PRs|
Packages to Never Consolidate
These packages will not be consolidated because they would inhibit contributions from our friends in the Nuclide team at Facebook:
To consolidate a single core package repository back into
atom/atom, the following steps will be taken:
- All open pull requests on the package's repository must either be closed or merged before consolidation can proceed
- The package repository's code in
masterwill be copied over to Atom's
packagesfolder in a subfolder bearing that package's name.
package.jsonfile will be updated to change the package's
packageDependenciesentry to reference its local path with the following syntax:
- A test build will be created locally to manually verify that the package loads and works correctly at first glance
- The package specs for the newly-consolidated package will be run against the local Atom build
- A PR will be sent to
atom/atomto verify that CI passes with the introduction of the consolidated package
- Once CI is clean and the PR is approved, the PR will be merged
- The package's original repository will have all of its existing issues moved over to
atom/atomusing a bulk issue mover tool, assigning a label to those issues relative to the package name, like
- The package's original repository will have its README.md updated to point contributors to the code's new home in
- The package's original repository will now be archived on GitHub
One alternative approach would be to break this core Atom functionality out of packages and put it directly in the Atom codebase without treating them as packages. This would simplify the development process even further but with the following drawbacks:
- The Atom team would have less regular exposure to Atom package development
- Users would no longer be able to disable core packages to replace their behavior with other packages (different tree views, etc)
Is there a good reason to not move the
One concern here is that there exist projects which depend directly on these repositories for the TextMate syntax grammars they contain. Moving the code into
atom/atomwould require that we notify the consumers of the grammars so that they can redirect their requests to the
Should we use
git subtreeto migrate the entire commit history of these packages over or just depend on the history from a package's original repository?
For now, we won't use
git subtreedue to the possibility that bringing over thousands of commits could cause unknown problems in the Atom repo. We may try this for newly consolidated packages in the future if we decide that not having the package commit history is a sufficient impediment to problem investigations.
What are the criteria we might use to eventually decide to move larger packages like
Will we be losing any useful data about these packages if we don't have standalone repositories anymore?
Should we use this as an opportunity to remove any unnecessary packages from the core Atom distribution?