Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

non-annex superdataset causes saving in subdataset to use git add not git-annex add #7351

Closed
yarikoptic opened this issue Mar 23, 2023 · 1 comment · Fixed by #7355
Closed
Assignees
Labels
severity-important major effect on the usability of a package, without rendering it completely unusable to everyone

Comments

@yarikoptic
Copy link
Member

I will not report amount of time it left me and @afni-rickr confused on why .nii.gz and other monsters get committed to git instead of git-annex, and I was surprised to realize that we have such a bug

this script reproduces the problem
#!/bin/bash
set -ex

cd "$(mktemp -d ${TMPDIR:-/tmp}/dl-XXXXXXX)"

# if you remove --no-annex -- all is good

datalad create --no-annex super
cd super
datalad create  -d . -c text2git subds

dd if=/dev/random of=subds/data.dat  count=1 bs=1024

cp subds/data.dat subds/data2.dat

# lets save in subds first the same content
datalad save -d subds -m 'saved one in subds directly' subds/data.dat

# now save the same content but from the super
datalad save -m "Super with data in subdataset" -r -d . -r subds

# you will see that the one saved while referring to non-annex super
# is committed directly to git because we call  "git add" instead of
# "git annex"
ls -l subds/data*.dat

which at the end would result in

+ ls -l subds/data2.dat subds/data.dat
-rw------- 1 yoh yoh 1024 Mar 23 18:51 subds/data2.dat
lrwxrwxrwx 1 yoh yoh  122 Mar 23 18:51 subds/data.dat -> .git/annex/objects/zj/54/MD5E-s1024--867e9587850130c42bdf311b8e15493d.dat/MD5E-s1024--867e9587850130c42bdf311b8e15493d.dat

showing that data2.dat is committed (while saving from super dataset) from identical in content data.dat.

@yarikoptic yarikoptic added the severity-important major effect on the usability of a package, without rendering it completely unusable to everyone label Mar 23, 2023
@yarikoptic yarikoptic self-assigned this Mar 24, 2023
yarikoptic added a commit to yarikoptic/datalad that referenced this issue Mar 24, 2023
Although the file is about .repo.save and not ds.save, there is already a good
number of other tests which pretty much test ds.save.  It is also the only
place which already uses that convoluted scenario so I decided it would be good
place for an addition of another testing thus not introducing much of extra run
time as if I was to create an independent new test for this problem.
yarikoptic added a commit to yarikoptic/datalad that referenced this issue Mar 24, 2023
@yarikoptic-gitmate
Copy link
Collaborator

Issue fixed in 0.18.3

yarikoptic added a commit to yarikoptic/datalad that referenced this issue Mar 28, 2023
* origin/maint: (47 commits)
  Set minimal version of typing_extensions to be 3.10.0.0
  Fix Providers caching of config files
  [skip ci] Update docs/source/changelog.rst
  [skip ci] Update CHANGELOG
  Fix spelling
  pythonic object comparison
  Fix git identity test
  [release-action] Autogenerate changelog snippet for PR 7301
  Reload global ConfigManager via any instance
  Stop returning a custom ConfigManager with Dataset[uninstalled].config
  Document datalad#7299
  [release-action] Autogenerate changelog snippet for PR 7353
  [release-action] Autogenerate changelog snippet for PR 7355
  BF: save - force git based on particular repo having uuid, not the reference ds
  [release-action] Autogenerate changelog snippet for PR 7342
  TST(BK): extend test to demonstrate datalad#7351
  Ensure that GitRepo initiated repo remains without git-annex upon Dataset.create
  Fix test for incompatible annex and rsync versions
  Do use/pass repocls into get_convoluted_situation
  Refine type of `content` argument to `SignalingThread.signal()`
  ...

 Conflicts:
	setup.py -- tqdm dependency was versioned in master due to deprecations
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
severity-important major effect on the usability of a package, without rendering it completely unusable to everyone
Projects
None yet
2 participants