ENH: Bump Dockerfile #2409

mgxd · 2021-05-13T15:10:55Z

This PR bumps up the Docker image base to Ubuntu Focal LTS (20.04).

As a result, a number of packages were updated, including:

AFNI
FSL
Connectome Workbench
Graphviz
Pandoc

Furthermore, Python was bumped to 3.8.x, and many of its packages were "modernized" accordingly.

This builds the foundation for #2392 , as it depends on updated scipy. Also closes #2382

effigies · 2021-05-13T15:16:31Z

Should we just use neurodocker for FSL?

mgxd · 2021-05-13T15:21:07Z

Good point, whatever we can do to get up to 20.04

Dockerfile

This PR bumps up the Docker image base to Ubuntu Focal LTS (20.04). As a result, a number of packages were updated, including: - AFNI - FSL - Connectome Workbench - Graphviz - Pandoc Furthermore, Python was bumped to 3.8.x, and many of its packages were "modernized" accordingly.

Co-authored-by: Chris Markiewicz <effigies@gmail.com>

effigies · 2021-05-14T14:50:35Z

Unclear what's up with the validator... Should we roll back for now?

mgxd · 2021-05-14T15:37:31Z

perhaps, but an assertion within the wrapper is also failing

log

Python version: Python 2.7.18
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Collecting pip<21
  Downloading https://files.pythonhosted.org/packages/27/79/8a850fe3496446ff0d584327ae44e7500daf6764ca1a382d2d02789accf7/pip-20.3.4-py2.py3-none-any.whl (1.5MB)
     |████████████████████████████████| 1.5MB 6.9MB/s eta 0:00:01
Collecting setuptools<45
  Downloading https://files.pythonhosted.org/packages/e1/b7/182161210a13158cd3ccc41ee19aadef54496b74f2817cc147006ec932b4/setuptools-44.1.1-py2.py3-none-any.whl (583kB)
     |████████████████████████████████| 583kB 39.4MB/s eta 0:00:01
Installing collected packages: pip, setuptools
  Found existing installation: pip 19.2.3
    Uninstalling pip-19.2.3:
      Successfully uninstalled pip-19.2.3
  Found existing installation: setuptools 41.2.0
    Uninstalling setuptools-41.2.0:
      Successfully uninstalled setuptools-41.2.0
Successfully installed pip-20.3.4 setuptools-44.1.1
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Processing ./wrapper
Requirement already satisfied, skipping upgrade: setuptools<45 in /opt/circleci/.pyenv/versions/2.7.18/lib/python2.7/site-packages (from fmriprep-docker==20.2.1+37.g485abc06) (44.1.1)
Using legacy 'setup.py install' for fmriprep-docker, since package 'wheel' is not installed.
Installing collected packages: fmriprep-docker
    Running setup.py install for fmriprep-docker ... -� �done
Successfully installed fmriprep-docker-20.2.1+37.g485abc06
/opt/circleci/.pyenv/shims/fmriprep-docker
Traceback (most recent call last):
  File "/opt/circleci/.pyenv/versions/2.7.18/bin/fmriprep-docker", line 11, in <module>
    load_entry_point('fmriprep-docker==20.2.1+37.g485abc06', 'console_scripts', 'fmriprep-docker')()
  File "/opt/circleci/.pyenv/versions/2.7.18/lib/python2.7/site-packages/fmriprep_docker.py", line 509, in main
    print(merge_help(parser.format_help(), targethelp))
  File "/opt/circleci/.pyenv/versions/2.7.18/lib/python2.7/site-packages/fmriprep_docker.py", line 163, in merge_help
    assert w_posargs.replace(']', '').replace('[', '') == t_posargs
AssertionError

effigies · 2021-05-14T15:44:11Z

Yeah, I saw that. I can try to look at it at some point before a release, but I would be fine suppressing that for now.

mgxd · 2021-05-14T17:32:45Z

I think the problem was the help formatting changed between python versions.

Haven't looked into the bids-validator error yet, pinging @rwblair in case he's already come across this.

log

BIDSValidator https://github.com/nipreps/fmriprep/blob/674124ee80b3e2a8affddf005e910e4ca1c97cc0/fmriprep/utils/bids.py#L73

Unhandled rejection (
  reason: TypeError: Cannot read property 'split' of undefined
    at Minimatch.match (/usr/lib/node_modules/bids-validator/node_modules/minimatch/minimatch.js:717:9)
    at minimatch (/usr/lib/node_modules/bids-validator/node_modules/minimatch/minimatch.js:107:42)
    at Object.ignoredFile (/usr/lib/node_modules/bids-validator/utils/config.js:11:13)
    at Function.format (/usr/lib/node_modules/bids-validator/utils/issues/index.js:65:16)
    at Function.exceptionHandler (/usr/lib/node_modules/bids-validator/utils/issues/index.js:191:25)
    at /usr/lib/node_modules/bids-validator/validators/bids/fullTest.js:217:35
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
).

effigies · 2021-05-17T12:11:56Z

Restarted build to pull in validator 1.7.2, released on Friday.

Dockerfile

mgxd · 2021-05-17T13:05:00Z

I have to investigate a bit futher to see why ds210 is not producing the HTML citation.
I'm a bit worried of the segfault ds054 seems to consistently hit...

ds054

...
210514-23:40:24,248 nipype.workflow IMPORTANT:
	 fMRIPrep finished successfully!
210514-23:40:24,258 nipype.workflow IMPORTANT:
	 Works derived from this fMRIPrep execution should include the boilerplate text found in <OUTPUT_PATH>/fmriprep/logs/CITATION.md.
Fatal Python error: Segmentation fault

Thread 0x00007f9f0be67700 (most recent call first):
  File "/usr/local/miniconda/lib/python3.8/threading.py", line 302 in wait
  File "/usr/local/miniconda/lib/python3.8/site-packages/sentry_sdk/_queue.py", line 180 in get
  File "/usr/local/miniconda/lib/python3.8/site-packages/sentry_sdk/worker.py", line 126 in _target
  File "/usr/local/miniconda/lib/python3.8/threading.py", line 870 in run
  File "/usr/local/miniconda/lib/python3.8/site-packages/sentry_sdk/integrations/threading.py", line 67 in run
  File "/usr/local/miniconda/lib/python3.8/threading.py", line 932 in _bootstrap_inner
  File "/usr/local/miniconda/lib/python3.8/threading.py", line 890 in _bootstrap

Current thread 0x00007f9f31cd4740 (most recent call first):
<no Python frame>
fMRIPrep: Please report errors to https://github.com/nipreps/fmriprep/issues

effigies · 2021-05-18T14:40:13Z

The ds005 failure may be related to a schema mismatch. I can't upload an updated tarball to OSF because apparently we've hit a file limit. Just to have something quick, I opened #2415. If that fixes ds005, feel free to cherry-pick.

effigies · 2021-05-18T17:59:57Z

Nope, doesn't seem to be a problem with the dataset being invalid...

Co-authored-by: Chris Markiewicz <effigies@gmail.com>

.circleci/config.yml

effigies · 2021-05-20T19:35:51Z

Not better...

rwblair · 2021-06-09T20:05:52Z

Couple notes from playing around in circle, doing a fresh npm install of the validator allows ds005 legacy multiproc it to valdiate, as does removing --no-network from the call. Have not figured out why those two things would fix it, or why anat-only works with no-network.

mgxd · 2021-06-09T20:20:15Z

doing a fresh npm install of the validator allows ds005 legacy multiproc it to valdiate, as does removing --no-network from the call

Interesting... if the validator itself does not require an internet connection to run, I wonder whether one of its dependencies does since the last version we've been using (1.5.4)

Have not figured out why those two things would fix it, or why anat-only works with no-network.

The test is being skipped, I bet it would also fail if we weren't fast tracking the anatomicals.

effigies · 2021-06-10T18:18:02Z

@mgxd The ds054 issue seems to be related to threading integration (https://docs.sentry.io/platforms/python/configuration/integrations/default-integrations/#threading) which is new as of 0.7.3. Have we recently upgraded sentry_sdk?

A couple thoughts I have are:

Flush the logs pre-shutdown. If DBG: Add debug logs to track down segfault #2423 shows that the failure is at sys.exit(), separating the steps might resolve it.
Try disabling threading integration.

effigies · 2021-06-12T12:09:22Z

setup.cfg

@@ -64,7 +64,7 @@ docs =
    %(doc)s
 duecredit = duecredit
 resmon =
-sentry = sentry-sdk >=0.6.9
+sentry = sentry-sdk >=0.6.9, <0.20


Last ditch: Could be a weird conda/pip issue. sentry_sdk is on conda-forge.

effigies · 2021-06-15T20:28:13Z

@mgxd FWIW: getsentry/sentry-python#1131

Rely on Docker to handle file ownership/permissions

I cannot reproduce this locally or by ssh-ing into the circle job (wtf) [skip ds005][skip ds210][skip ds054]

mgxd · 2021-06-17T20:21:11Z

Upping the circle machine images (and thus the docker version within them) seemed to fix the ds054 segfault - although I'm not sure why 🤷 .
However, I think the game of dependency whack a mole is complete.

effigies · 2021-06-17T21:09:33Z

JFC. Thank you for finally fixing this. crossing fingers that ds054 doesn't segfault again

effigies

LGTM. One small fix in Circle config.

.circleci/config.yml

Co-authored-by: Chris Markiewicz <effigies@gmail.com>

mgxd force-pushed the enh/update-dockerfile branch from 9bd6be5 to 000c270 Compare May 13, 2021 19:23

effigies reviewed May 13, 2021

View reviewed changes

Dockerfile Show resolved Hide resolved

effigies reviewed May 13, 2021

View reviewed changes

Dockerfile Outdated Show resolved Hide resolved

mgxd and others added 3 commits May 13, 2021 19:56

ENH: Bump bids-validator, pin svgo to minor

cab1f20

Update Dockerfile

7f0bcd0

Co-authored-by: Chris Markiewicz <effigies@gmail.com>

FIX: Robustness across argparse formatting change

d06ef8b

mgxd force-pushed the enh/update-dockerfile branch from 485abc0 to d06ef8b Compare May 14, 2021 17:29

FIX: Add pandoc citeproc

7aa4826

effigies reviewed May 17, 2021

View reviewed changes

Dockerfile Outdated Show resolved Hide resolved

effigies mentioned this pull request May 18, 2021

[WIP] CI: Update dataset_description.json in cache #2415

Closed

mgxd and others added 2 commits May 20, 2021 13:04

Update Dockerfile

6babc76

Co-authored-by: Chris Markiewicz <effigies@gmail.com>

TST: Refresh data cache

a7f16a0

mgxd force-pushed the enh/update-dockerfile branch from 315ea6e to a7f16a0 Compare May 20, 2021 17:05

effigies reviewed May 20, 2021

View reviewed changes

.circleci/config.yml Outdated Show resolved Hide resolved

This was referenced May 21, 2021

Upgrade Dockerfile base / python env nipreps/nibabies#67

Closed

Validator error with config file, fixed with ignoreSubjectConsistency bids-standard/legacy-validator#1233

Closed

ENH: Small improvements to bids validity checks

3caef2b

mgxd mentioned this pull request Jun 10, 2021

Fails with internal error Error: Could not load HED schema version "Latest" from remote repository bids-standard/legacy-validator#1205

Closed

TST: Downgrade validator to avoid network requirement

d5f03a5

effigies mentioned this pull request Jun 10, 2021

DBG: Add debug logs to track down segfault #2423

Closed

PIN: Older sentry-sdk [skip ds005][skip ds210]

890cf5f

effigies reviewed Jun 12, 2021

View reviewed changes

effigies mentioned this pull request Jun 15, 2021

Segmentation fault at exit for Python 3.8+ getsentry/sentry-python#1131

Closed

effigies mentioned this pull request Jun 16, 2021

WIP: Revert to Python 3.7 #2425

Closed

DBG: Bump circle Docker version

288a2bb

mgxd force-pushed the enh/update-dockerfile branch from 365ac5a to 8a7cb23 Compare June 16, 2021 22:48

mgxd added 2 commits June 16, 2021 21:08

DBG: Remove setfacl calls

32865c8

Rely on Docker to handle file ownership/permissions

FIX: Remove TTY flag when inspecting Docker stdout

08a48ff

mgxd force-pushed the enh/update-dockerfile branch from 8a7cb23 to 08a48ff Compare June 17, 2021 01:09

mgxd added 2 commits June 17, 2021 09:48

DBG: Print usage strings

adeb69f

I cannot reproduce this locally or by ssh-ing into the circle job (wtf) [skip ds005][skip ds210][skip ds054]

DBG: Improve positional arguments detection's robustness

828515b

DBG: Allow latest sentry_sdk

32d5cc7

mgxd requested review from oesteban and effigies June 18, 2021 13:31

FIX: Update wrapper bindings path [skip ci]

6f6e97f

effigies approved these changes Jun 19, 2021

View reviewed changes

.circleci/config.yml Outdated Show resolved Hide resolved

Update .circleci/config.yml

72e1a73

Co-authored-by: Chris Markiewicz <effigies@gmail.com>

effigies merged commit 237523c into nipreps:master Jun 20, 2021

mgxd mentioned this pull request Jul 12, 2021

MNT: Update Docker pins for Python packages #2372

Closed

effigies mentioned this pull request Jul 14, 2021

MNT: Update BIDS validator to 1.8.0 #2443

Merged

mgxd deleted the enh/update-dockerfile branch July 30, 2021 21:24

eilidhmacnicol mentioned this pull request Sep 16, 2021

RF: remove neurodebian nipreps/fmriprep-rodents#37

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ENH: Bump Dockerfile #2409

ENH: Bump Dockerfile #2409

mgxd commented May 13, 2021 •

edited

Loading

effigies commented May 13, 2021

mgxd commented May 13, 2021

effigies commented May 14, 2021

mgxd commented May 14, 2021

effigies commented May 14, 2021 •

edited

Loading

mgxd commented May 14, 2021

effigies commented May 17, 2021

mgxd commented May 17, 2021

effigies commented May 18, 2021

effigies commented May 18, 2021

effigies commented May 20, 2021

rwblair commented Jun 9, 2021

mgxd commented Jun 9, 2021

effigies commented Jun 10, 2021

effigies Jun 12, 2021

effigies commented Jun 15, 2021

mgxd commented Jun 17, 2021

effigies commented Jun 17, 2021

effigies left a comment

ENH: Bump Dockerfile #2409

ENH: Bump Dockerfile #2409

Conversation

mgxd commented May 13, 2021 • edited Loading

effigies commented May 13, 2021

mgxd commented May 13, 2021

effigies commented May 14, 2021

mgxd commented May 14, 2021

effigies commented May 14, 2021 • edited Loading

mgxd commented May 14, 2021

effigies commented May 17, 2021

mgxd commented May 17, 2021

effigies commented May 18, 2021

effigies commented May 18, 2021

effigies commented May 20, 2021

rwblair commented Jun 9, 2021

mgxd commented Jun 9, 2021

effigies commented Jun 10, 2021

effigies Jun 12, 2021

Choose a reason for hiding this comment

effigies commented Jun 15, 2021

mgxd commented Jun 17, 2021

effigies commented Jun 17, 2021

effigies left a comment

Choose a reason for hiding this comment

mgxd commented May 13, 2021 •

edited

Loading

effigies commented May 14, 2021 •

edited

Loading