ENH: use neurodocker 0.4.1 + apt install afni #2707

kaczmarj · 2018-09-19T17:52:13Z

Summary

This PR proposes updates to Dockerfile generation.

References #2706 (comment)

List of changes proposed in this PR (pull-request)

upgrade to neurodocker 0.4.1
install afni from neurodebian
minor refactoring in neurodocker generate commands.

Acknowledgment

(Mandatory) I acknowledge that this contribution will be available under the Apache 2 license.

effigies · 2018-09-20T15:09:57Z

This seems to be broken by #2691...

effigies · 2018-09-20T20:17:03Z

@kaczmarj Is there a trivial way to trigger a new docker build? I want to make sure that things aren't only passing on master because we're reusing the old images.

effigies · 2018-09-20T21:32:47Z

Built from scratch on rel/1.1.3 locally and no issues. Will try your branch now.

effigies · 2018-09-20T22:02:36Z

Yeah, that fails. Looks like you're changing the Matlab and/or SPM installs, and our doc parsing is failing for these versions. How about we push this off to 1.1.4? Or possibly restore the old Matlab/SPM.

kaczmarj · 2018-09-20T22:04:10Z

@effigies - i think the issue is with the matlab compiler runtime installation. i'm fine pushing this to the next release, but i'll fix it soon.

codecov-io · 2018-10-10T13:10:17Z

Codecov Report

Merging #2707 into master will increase coverage by 0.42%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #2707      +/-   ##
==========================================
+ Coverage   67.44%   67.87%   +0.42%     
==========================================
  Files         340      340              
  Lines       43170    44002     +832     
  Branches     5352     5430      +78     
==========================================
+ Hits        29116    29865     +749     
- Misses      13356    13433      +77     
- Partials      698      704       +6

Flag	Coverage Δ
#smoketests	`50.55% <ø> (-0.01%)`	⬇️
#unittests	`65.31% <ø> (+0.49%)`	⬆️

Impacted Files	Coverage Δ
nipype/testing/utils.py	`89.65% <0%> (-1.73%)`	⬇️
nipype/interfaces/afni/base.py	`66.93% <0%> (-1.62%)`	⬇️
nipype/interfaces/bru2nii.py	`73.68% <0%> (-0.39%)`	⬇️
nipype/interfaces/io.py	`54.44% <0%> (-0.16%)`	⬇️
nipype/utils/filemanip.py	`79.95% <0%> (+0.44%)`	⬆️
nipype/interfaces/afni/preprocess.py	`83.36% <0%> (+1.87%)`	⬆️
nipype/interfaces/afni/utils.py	`86.09% <0%> (+4.34%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d83b06d...23e6357. Read the comment docs.

effigies · 2018-10-10T17:51:51Z

Want to merge master? Circle seems to have done okay on Oscar's coverage update.

kaczmarj · 2018-10-10T17:53:33Z

The following command is failing.

flameo \
--copefile=/work/output/fmri_fsl_reuse/Linear/level1flow/fixedfx/_subject_id_s1/_fwhm_5.0/copemerge/mapflow/_copemerge0/cope1_merged.nii.gz \
--covsplitfile=/work/output/fmri_fsl_reuse/Linear/level1flow/fixedfx/_subject_id_s1/_fwhm_5.0/l2model/design.grp \
--designfile=/work/output/fmri_fsl_reuse/Linear/level1flow/fixedfx/_subject_id_s1/_fwhm_5.0/l2model/design.mat \
--dofvarcopefile=/work/output/fmri_fsl_reuse/Linear/level1flow/fixedfx/_subject_id_s1/_fwhm_5.0/gendofvolume/dof_file.nii.gz \
--ld=stats \
--maskfile=/work/output/fmri_fsl_reuse/Linear/level1flow/featpreproc/_subject_id_s1/_fwhm_5.0/dilatemask/mapflow/_dilatemask0/f3_dtype_mcf_bet_thresh_dil.nii.gz \
--runmode=fe \
--tcontrastsfile=/work/output/fmri_fsl_reuse/Linear/level1flow/fixedfx/_subject_id_s1/_fwhm_5.0/l2model/design.con \
--varcopefile=/work/output/fmri_fsl_reuse/Linear/level1flow/fixedfx/_subject_id_s1/_fwhm_5.0/varcopemerge/mapflow/_varcopemerge0/varcope1_merged.nii.gz

One can run the following to reproduce

/usr/bin/run_examples.sh fmri_fsl_reuse Linear /data/examples/ level1_workflow

kaczmarj · 2018-10-10T18:04:16Z

This is the specific error:

flameo --copefile=/work/output/fmri_fsl_reuse/Linear/level1flow/fixedfx/_subject_id_s1/_fwhm_5.0/copemerge/mapflow/_copemerge0/cope1_merged.nii.gz --covsplitfile=/work/output/fmri_fsl_reuse/Linear/level1flow/fixedfx/_subject_id_s1/_fwhm_5.0/l2model/design.grp --designfile=/work/output/fmri_fsl_reuse/Linear/level1flow/fixedfx/_subject_id_s1/_fwhm_5.0/l2model/design.mat --dofvarcopefile=/work/output/fmri_fsl_reuse/Linear/level1flow/fixedfx/_subject_id_s1/_fwhm_5.0/gendofvolume/dof_file.nii.gz --ld=stats --maskfile=/work/output/fmri_fsl_reuse/Linear/level1flow/featpreproc/_subject_id_s1/_fwhm_5.0/dilatemask/mapflow/_dilatemask0/f3_dtype_mcf_bet_thresh_dil.nii.gz --runmode=fe --tcontrastsfile=/work/output/fmri_fsl_reuse/Linear/level1flow/fixedfx/_subject_id_s1/_fwhm_5.0/l2model/design.con --varcopefile=/work/output/fmri_fsl_reuse/Linear/level1flow/fixedfx/_subject_id_s1/_fwhm_5.0/varcopemerge/mapflow/_varcopemerge0/varcope1_merged.nii.gz
Standard output:
Log directory is: stats
Setting up:
ntptsing=4.000000 

evs_group=1.000000 

Standard error:
terminate called after throwing an instance of 'RBD_COMMON::BaseException'
Aborted (core dumped)
Return code: 134

effigies · 2018-10-11T12:58:23Z

Neurodebian's FSL doesn't look to have been updated (correct @yarikoptic?). I wonder if there's a library bug that the new builds are hitting.

effigies · 2018-10-11T21:16:09Z

Looks like the build is fixed on master. Unclear why.

effigies · 2018-10-12T14:54:47Z

Ah, I see. Master is able to reuse the old base images, because of the compare_dockerfiles check. PRs from most users won't be able to do so. So fresh builds are still failing, we're just not doing them on master.

effigies · 2018-10-12T19:37:18Z

I reproduced the issue. Adding --verbose --debug=2 gives practically no more output. I see a couple options:

Try non-Neurodebian FSL to verify that the issue persists. (Possibly upgrade versions?)
Ping the FSL mailing list.

Any other thoughts?

kaczmarj · 2018-10-12T19:44:01Z

@effigies - i will see if using a neurodebian snapshot from a few months ago will solve the issue http://snapshot.debian.org/

effigies · 2018-10-17T00:15:12Z

@kaczmarj Any luck here? Is there something I can do to help this along?

effigies · 2018-10-23T16:40:34Z

@kaczmarj Any chance we can figure this one out this week? Tests are still failing for all new PRs, and it would be nice to have this fixed before the next release.

kaczmarj · 2018-10-26T14:16:52Z

@effigies - i'm working on this now

effigies · 2018-10-26T14:18:34Z

Thanks. Sorry to pester.

effigies · 2018-10-29T18:56:27Z

Did some of my own tests on #2763. Even installing directly from FSL, flameo crashes for both 5.0.9 and 5.0.11. Perhaps it's a libc issue?

IDK. Is there any chance of just fixing the broken aspects of the current Dockerfile? Or is this a minimal update already?

kaczmarj · 2018-10-29T19:00:28Z

@effigies - this is a minimal update already. i tried to update the original (neurodocker 0.3.2) dockerfile to use the non-neurodebian fsl to no avail.

effigies · 2018-10-29T20:38:40Z

docker/generate_dockerfiles.sh

+# neurodocker version 0.4.1-22-g7c44e01
+NEURODOCKER_IMAGE="kaczmarj/neurodocker:master@sha256:858632a7533cac100f70932749b4cfc77fc40f667f41fca208f406215cff8a27"
+# neurodebian:stretch-non-free pulled on September 19, 2018
+BASE_IMAGE="neurodebian:stretch-non-free@sha256:7cd978427d7ad215834fee221d0536ed7825b3cddebc481eba2d792dfc2f7332"


@kaczmarj What if we revert the BASE_IMAGE? If that helps, I think that would be decent evidence that it's a library version issue.

@effigies - i don't think that will help. the apt-get update calls will update to the latest neurodebian stretch sources. i tried using neurodebian snapshots from september 17, 2017 and from january 1, 2017, but both ran into the same flameo error. i'm going to try to use the nipype base image on dockerhub with the current nipype source code now.

@effigies - i used nipype's base image on dockerhub (i.e., nipype/nipype:base) and still got the flameo error. is it possible that nipype's code is causing this? could one of the inputs to flameo be corrupted / malformed?

But it works on master, still, with the same inputs. There's something new in the build.

@kaczmarj Is there any chance of using a Debian snapshot from earlier, as well? If it's a libc update or similar that's causing the issue, that wouldn't be resolved by using a neurodebian snapshot.

true, i'll try that now. i'm experimenting with updating packages in the current nipype py36 docker image and see if that will reproduce the error.

kaczmarj · 2018-10-30T20:57:07Z

@effigies - i've narrowed down the issue to the SPM12 matlab r2010a installation in neurodocker 0.4.1. unfortunately i don't have time to dig into it right this minute, but i will this evening / tomorrow morning.

matlab has a lot of old shared libraries, so they might be interfering in some way...

miykael · 2018-10-30T21:52:37Z

@kaczmarj, I remember that I had some library issues when I switched the tutorial from spm-dev to spm-r7219. I think you showed me this, but the solution in my case was to first look through the miniconda libraries before going into the ones from MATLAB, see here.

The libraries for Matlab compiler runtime 2010a are being found before the system libraries, and flameo was linking to one of matlab's libraries. This commit prepends the system libraries to LD_LIBRARY_PATH so that flameo (and other binaries) do not link to Matlab's libraries.

kaczmarj · 2018-10-31T14:29:03Z

thanks @miykael - this indeed was the issue. the most recent circleci build should pass now (it did on my computer).

effigies · 2018-10-31T15:45:06Z

Sweet! Here we go.

kaczmarj · 2018-10-31T15:48:51Z

Wahoo!

And as an added bonus, it takes about 9 minutes to build both the base docker image and the main (py36) image. according to @miykael, the matlab compiler runtime file for matlab 2010a is much smaller than that for 2018b, so using 2010a decreases the spm/mcr installation significantly. that and afni being installed from neurodebian.

miykael · 2018-10-31T15:59:16Z

Exactly. Switching to SPM-r7219 decreases MATLAB by a few GB and even SPM by a fe 100MB. And afni (and ants) from neuredebian has a similar reduction effect. Like this I was able to get the tutorial from 12GB to 6GB.

Jakub Kaczmarzyk added 3 commits September 19, 2018 13:49

use neurodocker 0.4.1 + apt install afni

2be7bf4

fix: put license in /opt/freesurfer-6.0.0-min/

32c2f42

use most recent neurodocker + activate conda env

da2fab0

effigies added this to the 1.1.4 milestone Sep 20, 2018

use newer neurodocker and neurodebian stretch

4c5e76f

Jakub Kaczmarzyk added 2 commits October 10, 2018 09:10

Merge remote-tracking branch 'upstream/master' into enh/neurodocker

c5ff5e8

upgrade pybids to 0.6.5 from 0.5.1

4a29a2a

miykael mentioned this pull request Oct 10, 2018

FIX: corrects readthedocs build error #2723

Merged

effigies mentioned this pull request Oct 10, 2018

ENH: enable/disable resource monitor in the fixture per test #2725

Merged

Merge remote-tracking branch 'upstream/master' into enh/neurodocker

2648de6

This was referenced Oct 24, 2018

FIX: Refine FSL.split output identification #2746

Merged

REL: 1.1.4 #2760

Merged

Merge remote-tracking branch 'upstream/master' into enh/neurodocker

fcaafaf

effigies mentioned this pull request Oct 29, 2018

TEST: Install FSL without NeuroDebian #2763

Closed

effigies reviewed Oct 29, 2018

View reviewed changes

effigies merged commit 988a648 into nipy:master Oct 31, 2018

kaczmarj deleted the enh/neurodocker branch October 31, 2018 15:45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ENH: use neurodocker 0.4.1 + apt install afni #2707

ENH: use neurodocker 0.4.1 + apt install afni #2707

kaczmarj commented Sep 19, 2018

effigies commented Sep 20, 2018

effigies commented Sep 20, 2018

effigies commented Sep 20, 2018

effigies commented Sep 20, 2018

kaczmarj commented Sep 20, 2018

codecov-io commented Oct 10, 2018 •

edited

Loading

effigies commented Oct 10, 2018

kaczmarj commented Oct 10, 2018

kaczmarj commented Oct 10, 2018

effigies commented Oct 11, 2018

effigies commented Oct 11, 2018

effigies commented Oct 12, 2018

effigies commented Oct 12, 2018

kaczmarj commented Oct 12, 2018

effigies commented Oct 17, 2018

effigies commented Oct 23, 2018

kaczmarj commented Oct 26, 2018

effigies commented Oct 26, 2018

effigies commented Oct 29, 2018

kaczmarj commented Oct 29, 2018

effigies Oct 29, 2018

kaczmarj Oct 29, 2018

kaczmarj Oct 29, 2018

effigies Oct 29, 2018

effigies Oct 30, 2018

kaczmarj Oct 30, 2018

kaczmarj commented Oct 30, 2018 •

edited

Loading

miykael commented Oct 30, 2018

kaczmarj commented Oct 31, 2018

effigies commented Oct 31, 2018

kaczmarj commented Oct 31, 2018

miykael commented Oct 31, 2018

ENH: use neurodocker 0.4.1 + apt install afni #2707

ENH: use neurodocker 0.4.1 + apt install afni #2707

Conversation

kaczmarj commented Sep 19, 2018

Summary

List of changes proposed in this PR (pull-request)

Acknowledgment

effigies commented Sep 20, 2018

effigies commented Sep 20, 2018

effigies commented Sep 20, 2018

effigies commented Sep 20, 2018

kaczmarj commented Sep 20, 2018

codecov-io commented Oct 10, 2018 • edited Loading

Codecov Report

effigies commented Oct 10, 2018

kaczmarj commented Oct 10, 2018

kaczmarj commented Oct 10, 2018

effigies commented Oct 11, 2018

effigies commented Oct 11, 2018

effigies commented Oct 12, 2018

effigies commented Oct 12, 2018

kaczmarj commented Oct 12, 2018

effigies commented Oct 17, 2018

effigies commented Oct 23, 2018

kaczmarj commented Oct 26, 2018

effigies commented Oct 26, 2018

effigies commented Oct 29, 2018

kaczmarj commented Oct 29, 2018

effigies Oct 29, 2018

Choose a reason for hiding this comment

kaczmarj Oct 29, 2018

Choose a reason for hiding this comment

kaczmarj Oct 29, 2018

Choose a reason for hiding this comment

effigies Oct 29, 2018

Choose a reason for hiding this comment

effigies Oct 30, 2018

Choose a reason for hiding this comment

kaczmarj Oct 30, 2018

Choose a reason for hiding this comment

kaczmarj commented Oct 30, 2018 • edited Loading

miykael commented Oct 30, 2018

kaczmarj commented Oct 31, 2018

effigies commented Oct 31, 2018

kaczmarj commented Oct 31, 2018

miykael commented Oct 31, 2018

codecov-io commented Oct 10, 2018 •

edited

Loading

kaczmarj commented Oct 30, 2018 •

edited

Loading