Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: duecredit support for citing datalad itself AND datasets it opens #3184

Merged
merged 22 commits into from Mar 16, 2019

Conversation

@yarikoptic
Copy link
Member

@yarikoptic yarikoptic commented Mar 1, 2019

This pull request fixes #2065 - those eyes didn't let me sleep! I will blame @tyarkoni for anything to come.

  • includes support for actually DueCrediting any dataset we "instantiate"
  • ATM just ad-hoc extraction of metadata fields from bids and datacite (for CRCNS) records. Proper solution will need integration with extractors or some other harmonization which we do not have ATM
  • Relies on changes yet to be merged/released in DueCredit: duecredit/duecredit#144
    • basic support for free text citations (so we could at least point to those datasets etc)
    • access to due.active

With those changes and this PR, we are getting smth like:

> rm .duecredit.p; DUECREDIT_ENABLE=1 datalad ls ~/datalad/openfmri/ds000241 /home/yoh/datalad/crcns/aa-1                                                                        
/home/yoh/datalad/openfmri/ds000241   [annex]  master  1.0.0+1-2-g6bb6937 2018-06-08/07:44:21  ✓
/home/yoh/datalad/crcns/aa-1   [annex]  master  ✗ 2018-06-08/10:23:47  ✓

DueCredit Report:
- DataLad - Data management and distribution platform / datalad (v 0.11.3.dev22) [1]
  - Single-unit recordings from two auditory areas in male zebra finches / datalad:77271f9e (v 1.0) [2]
  - AK6: Animal Kingdom; 6 Species / datalad:f7ccf78c (v 1.0.0+1-2-g6bb6937) [3]

1 package cited
0 modules cited
2 functions cited

References
----------

[1] Visconti di Oleggio Castello, M. et al., 2019. datalad/datalad 0.11.3.
[2] Theunissen, F.E. et al., 2009. Single-unit recordings from two auditory areas in male zebra finches.
[3] Please cite: 'Connolly, A. C., J. S. Guntupalli, J. Gors, M. Hanke, Y. O. Halchenko, Y.-C. Wu, H. Abdi, and J. V. Haxby. The Representation of Biological Classes in the Human Brain. Journal of Neuroscience 32.8 (2012): 2608-618.' and include the following message: 'This data was obtained from the OpenfMRI database. Its accession number is ds000241'
$> duecredit summary --format bibtex              
@misc{https://doi.org/10.5281/zenodo.808846,
  doi = {10.5281/zenodo.808846},
  url = {https://zenodo.org/record/808846},
  author = {Visconti di Oleggio Castello, Matteo and Halchenko, Yaroslav O. and Hanke, Michael and Poldrack, Benjamin and Meyer, Kyle and Solanky, Debanjum Singh and Alteva, Gergana and Gors, Jason and MacFarlane, Dave and Olaf Häusler, Christian and Olson, Taylor and Waite, Alex and De La Vega, Alejandro and Sochat, Vanessa and Keshavan, Anisha and Ma, Feilong and Christian, Horea and Poelen, Jorrit and Skytén, Kusti and Visconti di Oleggio Castello, Matteo and Hardcastle, Nell and Stoeter, Torsten and C Lau, Vicky},
  keywords = {data version control, data distribution, execution provenance tracking},
  title = {datalad/datalad 0.11.3},
  publisher = {Zenodo},
  year = {2019}
}
@misc{https://doi.org/10.6080/k0f769gp,
  doi = {10.6080/k0f769gp},
  url = {http://crcns.org/data-sets/aa/aa-1},
  author = {Theunissen, Frederic E. and Hauber, ME and Woolley, Sarah M. N. and Gill, Patrick and Shaevitz, SS and Amin, Noopur and Hsu, A and Singh, NC and Grace, GA and Fremouw, Thane and Zhang, Junli and Cassey, P and Doupe, AJ and David, SV and Vinje, WE},
  keywords = {Neuroscience, Electrophysiology, auditory area, avian (zebra finch)},
  language = {eng},
  title = {Single-unit recordings from two auditory areas in male zebra finches},
  publisher = {CRCNS.org},
  year = {2009}
}
2019-02-28 23:40:55,436 [WARNING] Failed to generate bibtex for Text(u"Please cite: 'Connolly, A. C., J. S. Guntupalli, J. Gors, M. Hanke, Y. O. Halchenko, Y.-C. Wu, H. Abdi, and J. V. Haxby. The Representation of Biological Classes in the Human Brain. Journal of Neuroscience 32.8 (2012): 2608-618.' and include the following message: 'This data was obtained from the OpenfMRI database. Its accession number is ds000241'", key=u"please cite: 'connolly, a. c., j. s. guntupalli, j. gors, m. hanke, y. o. halchenko, y.-c. wu, h. abdi, and j. v. haxby. the representation of biological classes in the human brain. journal of neuroscience 32.8 (2012): 2608-618.' and include the following message: 'this data was obtained from the openfmri database. its accession number is ds000241'") (io.py:383)

Would be fun to see it output for a real use case with Python analysis script which uses DataLad to get the data -- have any in mind? ;)

TODOs

  • Basic unit- or integration/smoke test

Please have a look @datalad/developers

@yarikoptic yarikoptic added this to the 0.11.x milestone Mar 1, 2019
@codecov
Copy link

@codecov codecov bot commented Mar 4, 2019

Codecov Report

Merging #3184 into 0.11.x will decrease coverage by 8.12%.
The diff coverage is 37.36%.

Impacted file tree graph

@@            Coverage Diff             @@
##           0.11.x    #3184      +/-   ##
==========================================
- Coverage   57.09%   48.97%   -8.13%     
==========================================
  Files          92      251     +159     
  Lines       14793    32897   +18104     
==========================================
+ Hits         8446    16111    +7665     
- Misses       6347    16786   +10439
Impacted Files Coverage Δ
datalad/support/gitrepo.py 66.87% <100%> (+1.66%) ⬆️
datalad/support/due_utils.py 13.95% <13.95%> (ø)
datalad/support/tests/test_gitrepo.py 17.53% <18.75%> (ø)
datalad/support/due.py 76% <76%> (ø)
datalad/distribution/dataset.py 81.33% <80%> (+5.87%) ⬆️
datalad/downloaders/tests/utils.py 38.46% <0%> (ø)
datalad/customremotes/tests/__init__.py 33.33% <0%> (ø)
datalad/cmdline/tests/test_helpers.py 32.69% <0%> (ø)
datalad/support/tests/test_stats.py 13.11% <0%> (ø)
datalad/tests/test_utils.py 19.57% <0%> (ø)
... and 193 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 524f4d9...aafa892. Read the comment docs.

@codecov
Copy link

@codecov codecov bot commented Mar 4, 2019

Codecov Report

Merging #3184 into 0.11.x will increase coverage by 0.04%.
The diff coverage is 89.8%.

Impacted file tree graph

@@            Coverage Diff             @@
##           0.11.x    #3184      +/-   ##
==========================================
+ Coverage   90.77%   90.82%   +0.04%     
==========================================
  Files         249      252       +3     
  Lines       32848    33006     +158     
==========================================
+ Hits        29819    29977     +158     
  Misses       3029     3029
Impacted Files Coverage Δ
datalad/support/tests/test_gitrepo.py 99.87% <100%> (-0.13%) ⬇️
datalad/support/gitrepo.py 89.15% <100%> (+0.02%) ⬆️
datalad/tests/test_utils.py 96.39% <100%> (+0.06%) ⬆️
datalad/distribution/dataset.py 94.22% <80%> (-0.33%) ⬇️
datalad/support/due.py 80% <80%> (ø)
datalad/support/tests/test_due_utils.py 87.8% <87.8%> (ø)
datalad/utils.py 87.52% <90%> (+0.12%) ⬆️
datalad/support/due_utils.py 90.69% <90.69%> (ø)
datalad/support/digests.py 100% <0%> (ø) ⬆️
... and 6 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c1fb9c6...2035899. Read the comment docs.

yarikoptic added a commit to yarikoptic/datalad that referenced this issue Mar 11, 2019
We do take care about tuning up PWD while chdir etc.  But when we run
external process and set its cwd, we left PWD unchanged if was provided
in (likely) inherited and tuned environment.  This fix would change
PWD to match provided cwd, if it is known to the environment and cwd
is set.

I was not bold enough to always get an env (if not provided) and change it
there, but may be that is what we will need to end up doing at some
point.

This should resolve the failing test in
datalad#3184
caused by underlying datalad process, which is started from a temp directory
while we provide it with a now non-matching PWD:
https://travis-ci.org/datalad/datalad/jobs/501525590#L4325
* origin/0.11.x:
  Do not test against revolution in this (0.11.x) branch
  BF: install p7zip-full for testing correct extraction of .gz e.g. by crawler extension
  RF: dedicated exception PathKnownToRepositoryError for this case
  BF(TST): do not test in direct mode for inability to create a new ds inplace of another
  BF: workaround for the check in direct mode
  BF(workaround): make _CRON job green by not caring to test upstream build of git annex
  BF(TST): we must not be able to create a new subdataset in a path of submodule
  Do not decode fetched content when retrieving status
  TST: travis: Widen selector for extension runs
  BF(TST): adjusted test_custom_runner_protocol to reflect us running the "ls-files" first
  ENH: prevent users to create a repo in a subdir with files known to git above
  Use assure_unicode
  BF: Fix LORIS token authenticator type with Python3
  BF: Add switch to avoid pointless warnings (fixes dataladgh-3185)
  ENH: Removed duplicate (and leading! ;)) Matteo from the past (Dartmouth)
@yarikoptic
Copy link
Member Author

@yarikoptic yarikoptic commented Mar 12, 2019

this could cause some disturbance, so I will not rush to merge it without feedback/review

@yarikoptic
Copy link
Member Author

@yarikoptic yarikoptic commented Mar 14, 2019

but feedback would be welcome so I do not just keep dealing with conflicts (attn: @mih @kyleam )

* origin/0.11.x:
  ENH: run: Add 'tmpdir' placeholder
  RF: run: Reflow a function call
  ENH: search: Avoid showing an empty log message
  BF: specify correctly dependency on pylibzma as prior python 3.3
  TST: run: Swallow more test output
  BF: actually we do not do analysis for shell value so might be True later - set PWD regardless
  BF(TST): fix PWD only for non-shell invocations and fixup test for that
  Revert "RF(TST): travis - do not call  pip install -e .  with sudo"
  RF(TST): travis - do not call  pip install -e .  with sudo
  BF(TST): also set encoding (to None) in the monkey patched StringIO
  BF: Fix PWD to match provided cwd in Runner
  ENH: do not use -s -v in travis runs
  RF: make ignore_nose_capturing_stdout do nothing and just warn that it is used
  RF(TST): do not use @ignore_nose_capturing_stdout
  BF(workaround): monkey patch nose to use monkey patched StringIO with .fileno

 Conflicts:
	setup.py - minor conflict in comments
@yarikoptic yarikoptic mentioned this pull request Mar 15, 2019
4 tasks
@yarikoptic
Copy link
Member Author

@yarikoptic yarikoptic commented Mar 15, 2019

Note sure how/if

appveyor failure
Versions: appdirs=1.4.3 boto=2.49.0 cmd:annex=7.20190219-g728228d5d cmd:git=2.17.1.windows.2 cmd:system-git=2.17.1.windows.2 cmd:system-ssh=7.7p1 git=2.1.11 gitdb=2.0.5 humanize=0.5.1 iso8601=0.1.12 keyring=18.0.0 keyrings.alt=3.1.1 msgpack=0.6.1 requests=2.21.0 six=1.12.0 tqdm=4.31.1 wrapt=1.11.1
Obscure filename: str=b' ;abc' repr=' ;abc'
Encodings: default='utf-8' filesystem='mbcs' locale.prefered='cp1252'
Environment: PATH='C:\\Miniconda35\\envs\\test-environment\\Library\\bin;C:\\Miniconda35\\envs\\test-environment;C:\\Miniconda35\\envs\\test-environment\\Library\\mingw-w64\\bin;C:\\Miniconda35\\envs\\test-environment\\Library\\usr\\bin;C:\\Miniconda35\\envs\\test-environment\\Library\\bin;C:\\Miniconda35\\envs\\test-environment\\Scripts;C:\\Miniconda35;C:\\Miniconda35\\Scripts;C:\\Program Files (x86)\\Microsoft SDKs\\Azure\\CLI2\\wbin;C:\\Perl\\site\\bin;C:\\Perl\\bin;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\Program Files\\7-Zip;C:\\Program Files\\Microsoft\\Web Platform Installer\\;C:\\Tools\\GitVersion;C:\\Tools\\PsTools;C:\\Program Files\\Git LFS;C:\\Program Files (x86)\\Subversion\\bin;C:\\Program Files\\Microsoft SQL Server\\120\\Tools\\Binn\\;C:\\Program Files\\Microsoft SQL Server\\Client SDK\\ODBC\\110\\Tools\\Binn\\;C:\\Program Files (x86)\\Microsoft SQL Server\\120\\Tools\\Binn\\;C:\\Program Files\\Microsoft SQL Server\\120\\DTS\\Binn\\;C:\\Program Files (x86)\\Microsoft SQL Server\\120\\Tools\\Binn\\ManagementStudio\\;C:\\Tools\\WebDriver;C:\\Program Files (x86)\\Microsoft SDKs\\TypeScript\\1.4\\;C:\\Program Files (x86)\\Microsoft Visual Studio 12.0\\Common7\\IDE\\PrivateAssemblies\\;C:\\Program Files (x86)\\Microsoft SDKs\\Azure\\CLI\\wbin;C:\\Ruby193\\bin;C:\\Tools\\NUnit\\bin;C:\\Tools\\xUnit;C:\\Tools\\MSpec;C:\\Tools\\Coverity\\bin;C:\\Program Files (x86)\\CMake\\bin;C:\\go\\bin;C:\\Program Files\\Java\\jdk1.8.0\\bin;C:\\Python27;C:\\Program Files\\nodejs;C:\\Program Files (x86)\\iojs;C:\\Program Files\\iojs;C:\\Program Files\\Microsoft SQL Server\\130\\Tools\\Binn\\;C:\\Program Files (x86)\\MSBuild\\14.0\\Bin;C:\\Tools\\NuGet;C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\Common7\\IDE\\CommonExtensions\\Microsoft\\TestWindow;C:\\Program Files\\Microsoft DNX\\Dnvm;C:\\Program Files\\Microsoft SQL Server\\Client SDK\\ODBC\\130\\Tools\\Binn\\;C:\\Program Files (x86)\\Microsoft SQL Server\\130\\Tools\\Binn\\;C:\\Program Files (x86)\\Microsoft SQL Server\\130\\DTS\\Binn\\;C:\\Program Files\\Microsoft SQL Server\\130\\DTS\\Binn\\;C:\\Program Files (x86)\\Microsoft SQL Server\\110\\DTS\\Binn\\;C:\\Program Files (x86)\\Microsoft SQL Server\\120\\DTS\\Binn\\;C:\\Program Files (x86)\\Apache\\Maven\\bin;C:\\Python27\\Scripts;C:\\Tools\\NUnit3;C:\\Program Files\\Mercurial\\;C:\\Program Files\\dotnet\\;C:\\Tools\\curl\\bin;C:\\Program Files\\Amazon\\AWSCLI\\;C:\\Program Files (x86)\\Microsoft SQL Server\\140\\DTS\\Binn\\;C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\Common7\\IDE\\Extensions\\Microsoft\\SQLDB\\DAC\\140;C:\\Tools\\vcpkg;C:\\Program Files (x86)\\Microsoft SQL Server\\140\\Tools\\Binn\\;C:\\Program Files\\Microsoft SQL Server\\140\\Tools\\Binn\\;C:\\Program Files\\Microsoft SQL Server\\140\\DTS\\Binn\\;C:\\Program Files\\erl9.2\\bin;C:\\Program Files (x86)\\NSIS;C:\\Tools\\Octopus;C:\\Program Files\\Git\\cmd;C:\\Program Files\\Git\\usr\\bin;C:\\Program Files\\Microsoft Service Fabric\\bin\\Fabric\\Fabric.Code;C:\\Program Files\\Microsoft SDKs\\Service Fabric\\Tools\\ServiceFabricLocalClusterManager;C:\\Program Files\\LLVM\\bin;C:\\Program Files (x86)\\Yarn\\bin\\;C:\\Users\\appveyor\\AppData\\Roaming\\npm;C:\\Program Files\\PowerShell\\6\\;C:\\Program Files (x86)\\nodejs\\;C:\\ProgramData\\chocolatey\\bin;C:\\Users\\appveyor\\.dotnet\\tools;C:\\Users\\appveyor\\AppData\\Local\\Yarn\\bin;C:\\Users\\appveyor\\AppData\\Roaming\\npm;C:\\Program Files\\AppVeyor\\BuildAgent\\' PYTHON_VERSION='3.5.1' PYTHON_ARCH='32' PYTHON='C:\\Python35' GIT_LFS_PATH='C:\\Program Files\\Git LFS'
SKIP: Skipping on Windows
======================================================================
FAIL: datalad.distribution.tests.test_clone.test_clone_crcns
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Miniconda35\envs\test-environment\lib\site-packages\datalad\tests\utils.py", line 1244, in assert_status
    assert_in(r['status'], label)
  File "C:\Miniconda35\envs\test-environment\lib\unittest\case.py", line 1079, in assertIn
    self.fail(self._formatMessage(msg, standardMsg))
  File "C:\Miniconda35\envs\test-environment\lib\unittest\case.py", line 665, in fail
    raise self.failureException(msg)
AssertionError: 'error' not found in ['ok']
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "C:\Miniconda35\envs\test-environment\lib\site-packages\nose\case.py", line 198, in runTest
    self.test(*self.arg)
  File "C:\Miniconda35\envs\test-environment\lib\site-packages\datalad\tests\utils.py", line 849, in newfunc
    return func(*args, **kwargs)
  File "C:\Miniconda35\envs\test-environment\lib\site-packages\vcr\cassette.py", line 109, in __call__
    function, args, kwargs
  File "C:\Miniconda35\envs\test-environment\lib\site-packages\vcr\cassette.py", line 124, in _execute_function
    return self._handle_function(fn=handle_function)
  File "C:\Miniconda35\envs\test-environment\lib\site-packages\vcr\cassette.py", line 148, in _handle_function
    return fn(cassette)
  File "C:\Miniconda35\envs\test-environment\lib\site-packages\vcr\cassette.py", line 117, in handle_function
    return function(*args, **kwargs)
  File "C:\Miniconda35\envs\test-environment\lib\site-packages\datalad\tests\utils.py", line 615, in newfunc
    return t(*(arg + (filename,)), **kw)
  File "C:\Miniconda35\envs\test-environment\lib\site-packages\datalad\tests\utils.py", line 615, in newfunc
    return t(*(arg + (filename,)), **kw)
  File "C:\Miniconda35\envs\test-environment\lib\site-packages\datalad\distribution\tests\test_clone.py", line 90, in test_clone_crcns
    assert_status('ok', res)
  File "C:\Miniconda35\envs\test-environment\lib\site-packages\datalad\tests\utils.py", line 1250, in assert_status
    dumps(results, indent=1, default=lambda x: str(x))))
AssertionError: Test 1/1: expected status ['ok'] not found in:
[
 {
  "message": [
   "Failed to clone from any candidate source URL. Encountered errors per each url were: %s",
   [
    {
     "http://datasets-tests.datalad.org/": "Cmd('git') failed due to: exit code(128)\n  cmdline: git clone --progress -v http://datasets-tests.datalad.org/ C:\\Users\\appveyor\\AppData\\Local\\Temp\\1\\datalad_temp_hxu7v5t2\\all-nonrecursive [cmd.py:wait:415]",
     "http://datasets-tests.datalad.org/.git": "Cmd('git') failed due to: exit code(128)\n  cmdline: git clone --progress -v http://datasets-tests.datalad.org/.git C:\\Users\\appveyor\\AppData\\Local\\Temp\\1\\datalad_temp_hxu7v5t2\\all-nonrecursive [cmd.py:wait:415]"
    }
   ]
  ],
  "source_url": "///",
  "path": "C:\\Users\\appveyor\\AppData\\Local\\Temp\\1\\datalad_temp_hxu7v5t2\\all-nonrecursive",
  "action": "install",
  "type": "dataset",
  "status": "error"
 }
]
is related

@kyleam
Copy link
Contributor

@kyleam kyleam commented Mar 15, 2019

this could cause some disturbance

Could you elaborate?

I know nothing about duecredit, but it looks like you guard against failing with duecredit-related erros. So perhaps you're talking about performance concerns, but I'd guess that should be negligible when duecredit isn't enabled.

I'm surprised that this much code is needed inside datalad itself, but again I know nothing about duecredit. I don't have any real objection to this change.

@yarikoptic
Copy link
Member Author

@yarikoptic yarikoptic commented Mar 15, 2019

this could cause some disturbance

Could you elaborate?

Largely it is just my personal paranoia. duecredit import is indeed guarded tightly in the due.py stub, since the idea of duecredit to not cause the entire 'parent' project to blow if something fails in duecredit. So far such cases were really rare (but not that many projects use durecredit yet). I do not think there should be any measurable performance concern unless duecredit is enabled (set DUECREDIT_ENABLE=1 env var) - then it will impact performance notably due to all the import introspection injections etc. Actually for the fun of it (may be even for CRON_ run) I should run a single matrix run with DUECREDIT_ENABLE.

Disturbance might come also at the packaging/installation stage now that I am thinking about it.
duecredit has few a bit heavy/unpleasant dependencies which is not a problem on debian, but might cause some irritation when installing via pip... so may be I should move it into a section of its own, so typically it wouldn't get installed when installing datalad via pip or conda... will do that shortly

I'm surprised that this much code is needed inside datalad itself, but again I know nothing about duecredit. I don't have any real objection to this change.

Well, to just take advantage of duecredit to get datalad cited if someone uses datalad and instantiates at least a single GitRepo - it is just the due.py copied from duecredit and a single call to due.cite. But in this PR I went further to realize the long lasting dream (and somewhat of an initial excuse for duecredit) - to cite DataLad datasets used in the analysis pipelines. That required that additional code. Whenever there would be some "harmonized" metadata that code could use - portions of code could go away.

Sure thing there could probably be a datalad-duecredit extension which would overload Dataset class to provide this single call out.

…l_requires

duecredit would bring additional dependencies which are generally not needed
for normal datalad operation. Thus we better make it very optional
@yarikoptic
Copy link
Member Author

@yarikoptic yarikoptic commented Mar 16, 2019

Ok, I guess I will just proceed forward with this one

@yarikoptic yarikoptic merged commit 01821d1 into datalad:0.11.x Mar 16, 2019
4 of 5 checks passed
@yarikoptic yarikoptic deleted the enh_duecredit branch Mar 19, 2019
yarikoptic added a commit that referenced this issue Apr 6, 2019
    ## 0.11.4 (Mar 18, 2019) -- get-ready

    Largely a bug fix release with a few enhancements

    ### Important

    - 0.11.x series will be the last one with support for direct mode of [git-annex][]
      which is used on crippled (no symlinks and no locking) filesystems.
      v7 repositories should be used instead.

    ### Fixes

    - Extraction of .gz files is broken without p7zip installed.  We now
      abort with an informative error in this situation.  ([#3176][])

    - Committing failed in some cases because we didn't ensure that the
      path passed to `git read-tree --index-output=...` resided on the
      same filesystem as the repository.  ([#3181][])

    - Some pointless warnings during metadata aggregation have been
      eliminated.  ([#3186][])

    - With Python 3 the LORIS token authenticator did not properly decode
      a response ([#3205][]).

    - With Python 3 downloaders unnecessarily decoded the response when
      getting the status, leading to an encoding error.  ([#3210][])

    - In some cases, our internal command Runner did not adjust the
      environment's `PWD` to match the current working directory specified
      with the `cwd` parameter.  ([#3215][])

    - The specification of the pyliblzma dependency was broken.  ([#3220][])

    - [search] displayed an uninformative blank log message in some
      cases.  ([#3222][])

    - The logic for finding the location of the aggregate metadata DB
      anchored the search path incorrectly, leading to a spurious warning.
      ([#3241][])

    - Some progress bars were still displayed when stdout and stderr were
      not attached to a tty.  ([#3281][])

    - Check for stdin/out/err to not be closed before checking for `.isatty`.
      ([#3268][])

    ### Enhancements and new features

    - Creating a new repository now aborts if any of the files in the
      directory are tracked by a repository in a parent directory.
      ([#3211][])

    - [run] learned to replace the `{tmpdir}` placeholder in commands with
      a temporary directory.  ([#3223][])

    - [duecredit][] support has been added for citing DataLad itself as
      well as datasets that an analysis uses.  ([#3184][])

    - The `eval_results` interface helper unintentionally modified one of
      its arguments.  ([#3249][])

    - A few DataLad constants have been added, changed, or renamed ([#3250][]):
      - `HANDLE_META_DIR` is now `DATALAD_DOTDIR`.  The old name should be
         considered deprecated.
      - `METADATA_DIR` now refers to `DATALAD_DOTDIR/metadata` rather than
        `DATALAD_DOTDIR/meta` (which is still available as
        `OLDMETADATA_DIR`).
      - The new `DATASET_METADATA_FILE` refers to `METADATA_DIR/dataset.json`.
      - The new `DATASET_CONFIG_FILE` refers to `DATALAD_DOTDIR/config`.
      - `METADATA_FILENAME` has been renamed to `OLDMETADATA_FILENAME`.

* tag '0.11.4': (82 commits)
  Updated CHANGELOG.md for having merged check for not being closed
  [DATALAD RUNCMD] CHANGELOG: Re-linkify 0.11.4 entries
  CHANGELOG.md: Update 0.11.4 entries
  CHANGELOG.md: Adjust the format of a link
  BF: split lines on spaces and commas befoe doing sed capture of #issue
  RF: adjust tools/link_issues_CHANGELOG to have MD links as [#issue][]
  BF+DOC: Fix all links to mark them valid markdown
  BF: Fix markup bug that prevents sphinx-build from succeeding
  DOC: extend coumentation of is_interactive
  BF: guard .istty with try/except
  ENH: ui: Drop progress bars if not attached to tty
  ENH: ui: Add another name for SilentConsoleLog
  BF: check for stdin/out/err to not be closed before checking for .isatty
  RF: HANDLE_META_DIR -> DATALAD_DOTDIR (but keeping an alias for compatibility)
  RF: define/use consts DATASET_CONFIG_FILE, DATASET_METADATA_FILE, METADATA_DIR
  RF: METADATA_DIR/FILE -> OLDMETADATA_DIR/FILE
  BF: do not cause a side-effect on kwargs in @eval_results
  BF: join with ds.path when trying to see if any other metadata file is available
  RF: Move duecredit into its own section so it is not a part of install_requires
  [DATALAD RUNCMD] CHANGELOG: Re-linkify 0.11.4 entries
  ...
yarikoptic added a commit to yarikoptic/datalad that referenced this issue Apr 6, 2019
    ## 0.11.4 (Mar 18, 2019) -- get-ready

    Largely a bug fix release with a few enhancements

    ### Important

    - 0.11.x series will be the last one with support for direct mode of [git-annex][]
      which is used on crippled (no symlinks and no locking) filesystems.
      v7 repositories should be used instead.

    ### Fixes

    - Extraction of .gz files is broken without p7zip installed.  We now
      abort with an informative error in this situation.  ([datalad#3176][])

    - Committing failed in some cases because we didn't ensure that the
      path passed to `git read-tree --index-output=...` resided on the
      same filesystem as the repository.  ([datalad#3181][])

    - Some pointless warnings during metadata aggregation have been
      eliminated.  ([datalad#3186][])

    - With Python 3 the LORIS token authenticator did not properly decode
      a response ([datalad#3205][]).

    - With Python 3 downloaders unnecessarily decoded the response when
      getting the status, leading to an encoding error.  ([datalad#3210][])

    - In some cases, our internal command Runner did not adjust the
      environment's `PWD` to match the current working directory specified
      with the `cwd` parameter.  ([datalad#3215][])

    - The specification of the pyliblzma dependency was broken.  ([datalad#3220][])

    - [search] displayed an uninformative blank log message in some
      cases.  ([datalad#3222][])

    - The logic for finding the location of the aggregate metadata DB
      anchored the search path incorrectly, leading to a spurious warning.
      ([datalad#3241][])

    - Some progress bars were still displayed when stdout and stderr were
      not attached to a tty.  ([datalad#3281][])

    - Check for stdin/out/err to not be closed before checking for `.isatty`.
      ([datalad#3268][])

    ### Enhancements and new features

    - Creating a new repository now aborts if any of the files in the
      directory are tracked by a repository in a parent directory.
      ([datalad#3211][])

    - [run] learned to replace the `{tmpdir}` placeholder in commands with
      a temporary directory.  ([datalad#3223][])

    - [duecredit][] support has been added for citing DataLad itself as
      well as datasets that an analysis uses.  ([datalad#3184][])

    - The `eval_results` interface helper unintentionally modified one of
      its arguments.  ([datalad#3249][])

    - A few DataLad constants have been added, changed, or renamed ([datalad#3250][]):
      - `HANDLE_META_DIR` is now `DATALAD_DOTDIR`.  The old name should be
         considered deprecated.
      - `METADATA_DIR` now refers to `DATALAD_DOTDIR/metadata` rather than
        `DATALAD_DOTDIR/meta` (which is still available as
        `OLDMETADATA_DIR`).
      - The new `DATASET_METADATA_FILE` refers to `METADATA_DIR/dataset.json`.
      - The new `DATASET_CONFIG_FILE` refers to `DATALAD_DOTDIR/config`.
      - `METADATA_FILENAME` has been renamed to `OLDMETADATA_FILENAME`.

* tag '0.11.4':
  Updated CHANGELOG.md for having merged check for not being closed
  [DATALAD RUNCMD] CHANGELOG: Re-linkify 0.11.4 entries
  CHANGELOG.md: Update 0.11.4 entries
  CHANGELOG.md: Adjust the format of a link
  BF: split lines on spaces and commas befoe doing sed capture of #issue
  RF: adjust tools/link_issues_CHANGELOG to have MD links as [#issue][]
  BF+DOC: Fix all links to mark them valid markdown
  BF: Fix markup bug that prevents sphinx-build from succeeding
  DOC: extend coumentation of is_interactive
  BF: guard .istty with try/except
  BF: check for stdin/out/err to not be closed before checking for .isatty
  [DATALAD RUNCMD] CHANGELOG: Re-linkify 0.11.4 entries
  CHANGELOG: Add entries for 0.11.4
  ENH: version boost and initial changes to CHANGELOG
@mih mih mentioned this pull request May 30, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants