Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Incubators Proposal #9786
This issue is to propose a project-governance protocol for Qooxdoo projects, based loosely on Apache “Incubator” projects, where new additions to the core framework can be developed with a rapid code/test/review/release cycle and at the same time have the oversight of the core team, following our core principals of quality control and longevity.
When proposing large additions to the core framework, using PRs is not ideal because integrating a PR into master is an all-or-nothing decision - the bigger the addition, the more it would benefit from real-world testing before integrating. OTOH it’s difficult to get real world testing because testers have to switch to using the PR’s branch (which is really hard if testers also need to use their own fork of the framework, or more than one feature under development), plus logistical difficulties such as the branch needs to be constantly rebased.
This puts pressure on the core-team to accept a PR before it is fully tested, and subsequent changes for improvements or bug fixes require a round trip of submission and approval by the whole core team each time; the newer a feature is, the more likely there are to be a need for “hotfixes” to solve urgent problems, which again is more pressure to accept changes without a proper review, and possibly a sign that the original review process wasn’t good enough.
New features could be presented as an “Incubator” project, which would be a repo inside the https://github.com/qooxdoo organisation and which would follow our normal rules for quality control (i.e. a PR/review/merge cycle, concerns for backwards compatibility, policy on documentation, etc) and longevity. The project becomes an official sub project, so we’re putting commitment as a group into some kind of stewardship that may outlive the original developer’s involvement (which is not to say that the project cannot be deprecated and retired, just that the life cycle goes beyond one person).
As an official project (especially one which is being considered for inclusion into the core framework) it would be normally sponsored by at least one core-team member (or at least, someone who is a strong candidate to become a core team member). And if it will use the
As an official Incubator, the project would follow Qooxdoo guidelines for compatibility - EG freezing the API when we get to beta, or deprecating features in one version before removing them in the next, but this compatibility would be separate from that of the core framework.
For example, an Incubator could be compatible with Qooxdoo v6 and yet also take advantage of ES6 and therefore require the compiler (and therefore be incompatible with the generator).
Potentially, an Incubator could include an ES6->ES5 transpile as part of it’s release process to produce a version especially for generator compatibility; but some transpiled code is very hard to read: async/await is translated into a state machine inside the function where it occurs. The decision to provide this or not could be left the Incubator
It would be useful to be able to identify repos with special “Incubator” status, the easiest way to do that could be to have a naming convention eg append “.incubator” to the repo name. We already use “qxl.” as a prefix for libraries like qxl.apiviewer and qxl.demobrowser - although a repo name like “qxl.apiviewer” implies that there is a namespace called “qxl.apiviewer”.
For Incubators which are proposed as an addition to the core framework, and which use the
Using the Incubator
Because the compiler supports dependencies it would be easy for anyone to incorporate Incubators into their development - if the Incubator is ES5 then the same can be achieved with the generator, albeit with some additional manual work to get it up and running.
Example Use Cases
In recent issues #9780 and #9781, @cboulanger and I have been working on a sub project which is in the
The code in the latest alpha version is getting fairly complex, and relies heavily on ES6 - not just nice-to-have syntactic sugar like arrow functions, but async/await handling is a huge improvement for asynchronous work, and in this case async is a very big part of the code. Translating it into ES5 would be a lot of grind with the only value being backwards compatibility for the generator.
If incubators are accepted, I would propose dividing the work so far into two separate Incubator projects under the https://github.com/qooxdoo organisation:
qx.persistence.incubator - provides persistence of Qooxdoo objects, allowing arbitrary depth of objects to be stored in a flat object storage, including support for complex data features such as recursive references.
qx.remote.incubator - provides the remote data binding discussed in #9781, building on the code in qx.persistence.incubator to synchronise objects across network boundaries (ie client/server and window-to-window) and add remote code execution.
I fully support this proposal. I wouldn't be concerned with generator compatibility, though, as this would put an unnecessary burden on the process with very little benefit. I think we should establish a release workflow that would allow to create qooxdoo variants that are based on the latest, backward-compatible main branch and include the incubator project code. We should then schedule the full inclusion of these features for version 7, which removes the generator and backward compatibility. With the release of version 6, we can document how to use the incubator variant for people that want to use the attractive features that you outline.
I completely agree; that's not my reason for proposing Incubators, but it was one of the things that started me thinking about it. The async/await is just so useful that ES5 compatibility is impractical by comparison
We could, and the compiler's recursive dependency resolution makes that trivial to try it out on the fly before committing to providing an official distro.
I would much prefer that approach to something creating a "Qooxdoo v7" branch or "Qooxdoo v8" branch that is coded on, because "the next great version" is a huge distraction - it's very tempting to work on the latest and greatest features, which leads to the BC version becoming neglected. We have enough difficulty as it is, knowing what to back port to v5 without compounding the problem.
Incubators would be a big staging area for changes, and major releases would be the ideal time to decide on "taking the plunge" and integrating an incubator into the core release.
Yes, what about a "qooxdoo.next"
I like the general goal introduced here, but I'm not sure I'm fully on board with this approach. The problems I see are that we have an already-overworked team, and this approach adds more (eventually) to qooxdoo core (or at least, required to be supported by the qooxdoo team). I really like the library scheme that we've implemented. We have the qxl namespace for libraries that are highly related to core functionality. I'm curious why what's being discussed here doesn't fit directly into that architecture? Why does it require a different architecture. And what projects fit into incubator vs qxl library, and why?
@derrell I'd say:
One major benefit of the incubator approach would be that we can enforce quality checks that are not necessarily enforceable with a
The core team would not be concerned with the individual incubator project except when 1) it is first accepted as such and 2) it is ready to be reviewed. At least that is how I would envision it - maybe @johnspackman has a different idea for implementation...
My inclination would be to keep them as
@cboulanger I think we should stay away from a branch, because I think that it's important to focus on the current release of qooxdoo core framework; the qooxdoo/next was a disappointing diversion, and everyone's curious about the "what's 'next'?" A rollup package would be OK, but if users have to use the compiler that would be an optimisation-of-distribution.
@derrell I think that there is a problem in how we do things now when someone submits a PR which is a substantial change and with dozens of files - I've done it myself, and it's always a dilemma trying to balance the need for change (driven ultimately by my own private needs) vs time constraints & how long to hold off. It's really difficult to share code in practice if it is only available in a PR, and I hope this is something that will make it easier and less work for the team to review because a whole section of "core" new feature can be matured before final agreement to join the core.
My thoughts started from the point of view of having a
A big part of the incubator idea is that it allows substantial additions to be trialed without requiring the core team to approve via the core framework repo. Incubators would be a mechanism for saying "we think this is a good solution, but it needs to prove itself before we integrate it into the core".
It's like saying that the integration status of feature "X" is in alpha or beta, even if feature "X" is itself considered stable in its own right by a number of people.
When v6 is released and we're working towards v7, that could be a good point to consider whether an incubator is stable enough to incorporate into core. I think this is @cboulanger's point, that we should have a road map for when the proposed new libraries are integrated into the core, ie they are not allowed to languish at github, they have to have a specific destiny - of course this implies that if they don't make it, they get deprecated and then retired.
IMHO it is important to prevent github.com/qooxdoo from becoming a dumping ground of half finished repos, and most importantly prevent
I do have a vested interest in this - I'm finding that more of my work lately is leading me to take Qooxdoo more and more into the server, and where there is an opportunity to collaborate this is a good two-way thing. A case in point is @cboulanger 's issue about remote binding, happened to be something I was already working on (in part) and our joint need has accelerated things somewhat; given that it was interesting to others too, it seemed logical to progress it.
@johnspackman I mistakenly said "branch" when I really meant "repo" (I have edited my comment since then). I fully agree a branch is not the right solution. What I meant is some completely automated way of distributing an "experimental" version of qooxdoo which also does the Quality Assurance. That's why suggested using the new GitHub Actions feature to assemble this version. The repo itself would not get any manual update / change, all work is done either in the framework, the compiler, or the incubator repos. The difference between incubators and
Ok. I'm coming around to it. I still have reservations and concerns, but I'm getting there.
What do we do, if the namespace includes
ah, sorry - an npm module called
I agree with @hkollmann I was thinking that the repo has the
With the dependency management of compiler, the incubator could be disabled at the same time that it is integrated - this could be a tag on the repo or something in
@derrell The point for me is that such a repo/package would be a functional equivalent of a "development" branch, which we don't have, and a much better implementation of it at that. It would mean that the master branch would only receive smaller, manageable updates, and the "big" new features would only be merged once they have proven to be totally stable. As @johnspackman has argued, the current problem is that currently, the temptation is to merge new features once they appear stable, which is often not the case, and then the core team has to be involved when fixing them. With the proposed setup, master will be kept free from such experiments.