Bring revolution home?

[yarikoptic]

What is the problem?

What is your plan @mih about the https://github.com/datalad/datalad-revolution ? To keep it as a separate extension for eternity or bring pieces (or entirety) into datalad codebase for perspective future possibly API breaking release?
As I expressed myself, reviewing those changes is not easy and thus largely is not done. As I stated before, IMHO having them even in master branch (aiming e.g. for 0.11 release), while possibly still carrying out bugfixing in 0.10.x branch would be IMHO more preferable long term.

edits to outline steps to see this issue resolved:

• establish rf-revolution1 branch to contain initial changes
• remove direct mode (WiP: annihilation of direct mode support for the sake of the revolution #3036) - merge into rf-revolution1. WiP
  • go through leftover '# TODO: RM DIRECT' to see if anything else could/should be removed/refactored
  • add dedicated tests for handling situations when a repository in direct mode is encountered (directly by AnnexRepo or as a submodule) and assure a consistent/informative blow up
• Workflow for changes propagation
  • (rf-revolution1-3rd branch) bring in revolution code base into the core under 3rd/datalad_revolution (TODO: tools/injest_revolution_code to be reused to suck in the changes)
  • (rf-revolution1 branch, to keep merging rf-revolution1-3rd) git mv 3rd/datalad_revolution/datalad_revolution datalad/revolution (verify that tests pass, needs relative imports)
  • move out pieces from datalad/revolution/ around the core, where needed prefixing with rev_
    • move our *repo.py to core_*repo.py, while having revolution ones replace ours, while importing the core_ ones
• establish rel-0.11 branch to absorb relevant fixes (if possible - submit fixes against that branch when applicable)
• finalize Update rf-revolution1 #3108 and merge into rf-revolution1
• merge rf-revolution1 into master
• extend core *Repo classes with the API from revolution
• investigate mechanism to by default disable "experimental" revolution API within core datalad (config setting datalad.experimental.revolution to be enabled by extensions before importing API etc?)
• compare intentions/usecases/convergence for the API google doc API Use-cases convergence: 1st Revolution
• ...
• make sure no rev_ files, and no Rev classes are left

[mih]

The aim is to achieve a self-contained suite of commands that could replace core commands, with the goal of having them in -core.

ATM I see no benefit from doing that outside an extension, as -core has too much baggage that is/becomes irrelevant in the scope of this work. The more I work in this framework to more I appreciate the ability to draw a clear line wrt what kind of functionality I want to build on.

Moreover, there should be nothing in -revolution that could be cherry-picked short of the whole thing (which is not ready). Small non-intrusive changes made their way into -core anyways and already. I do not plan to change this (i.e. fix in -core what needs fixing, but limit to just fixes).

Another point that makes me prefer to status quo is the speed. I need to move fast ATM and with this setup I can have 2-3 times the changeset iterations that I can possibly achieve in -core in any given day, just due to the test suite runtime alone (not bringing up the direct mode issue again...). At the same time I can co-install this development functionality with a stable datalad installation (without having to switch), and the stuff that is written already gets a lot more real-world usage that it would get, if it would live in some long-lived branch (with the need for repeated merges and conflict resolution).

That being said, I am very much open for alternative workflows that have the same benefit, but less of the disadvantages that a separate code base implies. The current approach is simply the best of the possibilities that I am aware of.

[yarikoptic]

Where such an alternative falls short?
A branch with only a subset of tests ran, until it is time to test it in full for the merge into master.
Switching between environments is done by switching to that branch.
master doesn't have to be merged "until ready"

[mih]

FTR: when datalad/datalad-revolution#29 is merged I would consider the revolution done. I see no path how to incorporate this into core without major friction. I consider rev_status to be very useful, and rev_save replaces save and add (which what I believe is a saner, more comprehensible approach) and shows how one can get away without path annotation.

Anyways.... the revolution usually eats its children...

[yarikoptic]

so revolution will be done and then what? i.e. what is the long term plan for it -- to just to be used as an extension (and thus just by a selected/enlightened/forced-to group of users)?

What friction do you foresee?

Anyways.... the revolution usually eats its children...

or parents?

[mih]

I dont have plans. If core absorbs it, I am good. If it stays an extension, I am good too. The extensions that need this functionality are good too, either way.

Major friction point: direct mode -- there is not even a trace of support.

[mih]

List of issues affecting -core that are fixed in -revolution

[yarikoptic]

ping

[mih]

This is a process that is now ongoing, and likely to keep going for a while. But not an issue anymore!