RF: Implement new API (add)

[bpoldrack]

No description provided.

[coveralls]

Coverage decreased (-0.3%) to 85.196% when pulling d4672f6 on bpoldrack:rf-add into a3f0bb7 on datalad:master.

[codecov-io]

Codecov Report

Merging #612 into master will decrease coverage by 0.02%.
The diff coverage is 91.76%.

@@            Coverage Diff             @@
##           master     #612      +/-   ##
==========================================
- Coverage   86.79%   86.76%   -0.03%     
==========================================
  Files         198      200       +2     
  Lines       17903    18124     +221     
==========================================
+ Hits        15539    15726     +187     
- Misses       2364     2398      +34

Impacted Files	Coverage Δ
datalad/distribution/tests/test_publish.py	100% <ø> (ø)	⬆️
datalad/distribution/create.py	92.18% <ø> (ø)	⬆️
datalad/distribution/tests/test_add_sibling.py	100% <ø> (ø)	⬆️
datalad/interface/__init__.py	100% <ø> (ø)	⬆️
datalad/distribution/tests/test_update.py	100% <ø> (ø)	⬆️
datalad/distribution/tests/test_target_ssh.py	100% <ø> (ø)	⬆️
datalad/distribution/dataset.py	95.43% <100%> (+0.66%)	⬆️
datalad/distribution/tests/test_create.py	100% <100%> (ø)	⬆️
datalad/distribution/publish.py	73.61% <100%> (-0.19%)	⬇️
datalad/distribution/tests/test_install.py	100% <100%> (ø)	⬆️
... and 11 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4755adb...5ba54ae. Read the comment docs.

[coveralls]

Coverage decreased (-0.07%) to 85.509% when pulling 777cea3 on bpoldrack:rf-add into 8a40728 on datalad:master.

[coveralls]

Coverage decreased (-0.02%) to 85.554% when pulling 50ac59e on bpoldrack:rf-add into 8a40728 on datalad:master.

[coveralls]

Coverage increased (+0.03%) to 86.401% when pulling b3822b6 on bpoldrack:rf-add into e6858b5 on datalad:master.

[mih]

Note: this command should query a datasets configuration to decide how to add files. For example, I might want to configure that any files in code/ are always added to git directly.

[coveralls]

Coverage decreased (-0.01%) to 86.589% when pulling e574053 on bpoldrack:rf-add into 64730e6 on datalad:master.

[mih]

API has changed: see #740

[mih]

Observation:

% touch one two three
% datalad add one two
Added one
Added two
% datalad add one two
Nothing was added

# expected so far, but
% datalad add one two --to-git
Added one
Added two
% datalad add one two --to-git
Added one
Added two

[mih]

Recursive does not work as expected (continued scenario from above):

% datalad create deep --add-to-super
Created dataset at /tmp/some/deep.
% datalad save
% datalad add -r .
Added three
% datalad ls -r
.     [annex]  master  - 2016-09-01/09:14:03  X
deep  [annex]  master  - 2016-09-01/09:13:52  X
% git -C deep status
On branch master
Untracked files:
  (use "git add <file>..." to include in what will be committed)

    some

nothing added to commit but untracked files present (use "git add" to track)

I guess it doesn't realize that . matches (or should match IHMO) the submodule too.

[mih]

Otherwise this is looking good, I think. I'd appreciate quick fixes and a swift merge (that I'd myself ;-)

[mih]

See bpoldrack#30 for some trivial fixes.

[bpoldrack]

Regarding recursion: Currently unclear to me, whether this is what we want. Need to talk, I guess. Note, that datalad add * -r works as expected.

[bpoldrack]

Regarding your observation of reporting success twice in case of --to-git:
This is due to the way I currently generate a return value for git add. Won't fix in this PR. It's a more general thing, that is to solve within GitRepo.

[mih]

FTR: --recursive means "recurse into subdatasets". Otherwise recursion into subdirectories is always implied (just like git annex does it).

IMHO: any argument needs to be resolved against the root of the respective dataset (that had its add() called). If any of the given paths is a directory, any subdataset below this directory needs to receive '.' as the sole path argument to its add() call.

[bpoldrack]

Agree. That's exactly, what I'm currently implementing.

[mih]

Tests fail, but otherwise looks good.

[bpoldrack]

Strange. Apparently, Travis ran on a master, that I didn't merge yet, while the buildbots used an older one and passed.

[coveralls]

Coverage decreased (-0.03%) to 86.769% when pulling 5ba54ae on bpoldrack:rf-add into 4755adb on datalad:master.

[mih]

OK, let's merge this now and unify the behavior re save in one go, once the required utilities are in.

[yarikoptic]

FYI merge caused major havoc conflict in #746 which I hopefully resolve correctly now

[yarikoptic]

actually I am wrong -- this one alone couldn't do it because of only a tiny change (force arg)... heh -- smth else then