Skip to content

ENH: use neurodocker 0.4.1 + apt install afni #2707

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Oct 31, 2018
Merged

ENH: use neurodocker 0.4.1 + apt install afni #2707

Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
2be7bf4
use neurodocker 0.4.1 + apt install afni
Sep 19, 2018
32c2f42
fix: put license in /opt/freesurfer-6.0.0-min/
Sep 19, 2018
da2fab0
use most recent neurodocker + activate conda env
Sep 19, 2018
4c5e76f
use newer neurodocker and neurodebian stretch
Oct 10, 2018
c5ff5e8
Merge remote-tracking branch 'upstream/master' into enh/neurodocker
Oct 10, 2018
4a29a2a
upgrade pybids to 0.6.5 from 0.5.1
Oct 10, 2018
2648de6
Merge remote-tracking branch 'upstream/master' into enh/neurodocker
Oct 10, 2018
fcaafaf
Merge remote-tracking branch 'upstream/master' into enh/neurodocker
Oct 26, 2018
23e6357
fix: prepend system libraries to LD_LIBRARY_PATH
Oct 31, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
116 changes: 55 additions & 61 deletions docker/generate_dockerfiles.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,77 +52,71 @@ do
esac
done


# neurodocker version 0.3.1-22-gb0ee069
NEURODOCKER_IMAGE="kaczmarj/neurodocker@sha256:f15ca90803f4b89acfca55cd1c7269bf2ec2dfd95b3de1118b08afa34b87d9d1"

# neurodebian:stretch-non-free pulled on November 3, 2017
BASE_IMAGE="neurodebian@sha256:7590552afd0e7a481a33314724ae27f76ccedd05ffd7ac06ec38638872427b9b"
# neurodocker version 0.4.1-22-g7c44e01
NEURODOCKER_IMAGE="kaczmarj/neurodocker:master@sha256:858632a7533cac100f70932749b4cfc77fc40f667f41fca208f406215cff8a27"
# neurodebian:stretch-non-free pulled on September 19, 2018
BASE_IMAGE="neurodebian:stretch-non-free@sha256:7cd978427d7ad215834fee221d0536ed7825b3cddebc481eba2d792dfc2f7332"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kaczmarj What if we revert the BASE_IMAGE? If that helps, I think that would be decent evidence that it's a library version issue.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@effigies - i don't think that will help. the apt-get update calls will update to the latest neurodebian stretch sources. i tried using neurodebian snapshots from september 17, 2017 and from january 1, 2017, but both ran into the same flameo error. i'm going to try to use the nipype base image on dockerhub with the current nipype source code now.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@effigies - i used nipype's base image on dockerhub (i.e., nipype/nipype:base) and still got the flameo error. is it possible that nipype's code is causing this? could one of the inputs to flameo be corrupted / malformed?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But it works on master, still, with the same inputs. There's something new in the build.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kaczmarj Is there any chance of using a Debian snapshot from earlier, as well? If it's a libc update or similar that's causing the issue, that wouldn't be resolved by using a neurodebian snapshot.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

true, i'll try that now. i'm experimenting with updating packages in the current nipype py36 docker image and see if that will reproduce the error.


NIPYPE_BASE_IMAGE="nipype/nipype:base"
PKG_MANAGER="apt"
DIR="$(dirname "$0")"

function generate_base_dockerfile() {
docker run --rm "$NEURODOCKER_IMAGE" generate \
--base "$BASE_IMAGE" --pkg-manager "$PKG_MANAGER" \
--label maintainer="The nipype developers https://github.com/nipy/nipype" \
--spm version=12 matlab_version=R2017a \
--afni version=latest install_python2=true \
--freesurfer version=6.0.0 min=true \
--run 'echo "cHJpbnRmICJrcnp5c3p0b2YuZ29yZ29sZXdza2lAZ21haWwuY29tXG41MTcyXG4gKkN2dW12RVYzelRmZ1xuRlM1Si8yYzFhZ2c0RVxuIiA+IC9vcHQvZnJlZXN1cmZlci9saWNlbnNlLnR4dAo=" | base64 -d | sh' \
--install ants apt-utils bzip2 convert3d file fsl-core fsl-mni152-templates \
fusefat g++ git graphviz make ruby unzip xvfb \
--add-to-entrypoint "source /etc/fsl/fsl.sh" \
--env ANTSPATH='/usr/lib/ants' PATH='/usr/lib/ants:$PATH' \
--run "gem install fakes3" \
--no-check-urls > "$DIR/Dockerfile.base"
docker run --rm "$NEURODOCKER_IMAGE" generate docker \
--base "$BASE_IMAGE" --pkg-manager "$PKG_MANAGER" \
--label maintainer="The nipype developers https://github.com/nipy/nipype" \
--spm12 version=r7219 \
--env 'LD_LIBRARY_PATH=/lib/x86_64-linux-gnu:$LD_LIBRARY_PATH' \
--freesurfer version=6.0.0-min \
--run 'echo "cHJpbnRmICJrcnp5c3p0b2YuZ29yZ29sZXdza2lAZ21haWwuY29tCjUxNzIKICpDdnVtdkVWM3pUZmcKRlM1Si8yYzFhZ2c0RQoiID4gL29wdC9mcmVlc3VyZmVyLTYuMC4wLW1pbi9saWNlbnNlLnR4dA==" | base64 -d | sh' \
--install afni ants apt-utils bzip2 convert3d file fsl-core \
fsl-mni152-templates fusefat g++ git graphviz make python ruby \
unzip xvfb \
--add-to-entrypoint "source /etc/fsl/fsl.sh && source /etc/afni/afni.sh" \
--env ANTSPATH='/usr/lib/ants' \
PATH='/usr/lib/ants:$PATH' \
--run "gem install fakes3" \
> "$DIR/Dockerfile.base"
}


function generate_main_dockerfile() {
docker run --rm "$NEURODOCKER_IMAGE" generate \
--base "$NIPYPE_BASE_IMAGE" --pkg-manager "$PKG_MANAGER" \
--label maintainer="The nipype developers https://github.com/nipy/nipype" \
--env MKL_NUM_THREADS=1 OMP_NUM_THREADS=1 \
--user neuro \
--miniconda env_name=neuro \
activate=true \
--copy docker/files/run_builddocs.sh docker/files/run_examples.sh \
docker/files/run_pytests.sh nipype/external/fsl_imglob.py /usr/bin/ \
--copy . /src/nipype \
--user root \
--run 'chown -R neuro /src
&& chmod +x /usr/bin/fsl_imglob.py /usr/bin/run_*.sh
&& . /etc/fsl/fsl.sh
&& ln -sf /usr/bin/fsl_imglob.py ${FSLDIR}/bin/imglob
&& mkdir /work
&& chown neuro /work' \
--user neuro \
--arg PYTHON_VERSION_MAJOR=3 PYTHON_VERSION_MINOR=6 BUILD_DATE VCS_REF VERSION \
--miniconda env_name=neuro \
conda_install='python=${PYTHON_VERSION_MAJOR}.${PYTHON_VERSION_MINOR}
icu=58.1 libxml2 libxslt matplotlib mkl numpy paramiko
pandas psutil scikit-learn scipy traits=4.6.0' \
pip_opts="-e" \
pip_install="/src/nipype[all]" \
--miniconda env_name=neuro \
pip_install="grabbit==0.1.2" \
--run-bash "mkdir -p /src/pybids
&& curl -sSL --retry 5 https://github.com/INCF/pybids/tarball/0.5.1
| tar -xz -C /src/pybids --strip-components 1
&& source activate neuro
&& pip install --no-cache-dir -e /src/pybids" \
--workdir /work \
--label org.label-schema.build-date='$BUILD_DATE' \
org.label-schema.name="NIPYPE" \
org.label-schema.description="NIPYPE - Neuroimaging in Python: Pipelines and Interfaces" \
org.label-schema.url="http://nipype.readthedocs.io" \
org.label-schema.vcs-ref='$VCS_REF' \
org.label-schema.vcs-url="https://github.com/nipy/nipype" \
org.label-schema.version='$VERSION' \
org.label-schema.schema-version="1.0" \
--no-check-urls
docker run --rm "$NEURODOCKER_IMAGE" generate docker \
--base "$NIPYPE_BASE_IMAGE" --pkg-manager "$PKG_MANAGER" \
--label maintainer="The nipype developers https://github.com/nipy/nipype" \
--env MKL_NUM_THREADS=1 \
OMP_NUM_THREADS=1 \
--arg PYTHON_VERSION_MAJOR=3 PYTHON_VERSION_MINOR=6 BUILD_DATE VCS_REF VERSION \
--user neuro \
--miniconda create_env=neuro \
conda_install='python=${PYTHON_VERSION_MAJOR}.${PYTHON_VERSION_MINOR}
icu=58.1 libxml2 libxslt matplotlib mkl numpy paramiko
pandas psutil scikit-learn scipy traits=4.6.0' \
pip_install="grabbit==0.1.2 https://github.com/INCF/pybids/tarball/0.6.5" \
activate=true \
--copy docker/files/run_builddocs.sh docker/files/run_examples.sh \
docker/files/run_pytests.sh nipype/external/fsl_imglob.py /usr/bin/ \
--copy . /src/nipype \
--user root \
--run 'chown -R neuro /src
&& chmod +x /usr/bin/fsl_imglob.py /usr/bin/run_*.sh
&& . /etc/fsl/fsl.sh
&& ln -sf /usr/bin/fsl_imglob.py ${FSLDIR}/bin/imglob
&& mkdir /work
&& chown neuro /work' \
--user neuro \
--miniconda use_env=neuro \
pip_opts="-e" \
pip_install="/src/nipype[all]" \
--workdir /work \
--label org.label-schema.build-date='$BUILD_DATE' \
org.label-schema.name="NIPYPE" \
org.label-schema.description="NIPYPE - Neuroimaging in Python: Pipelines and Interfaces" \
org.label-schema.url="http://nipype.readthedocs.io" \
org.label-schema.vcs-ref='$VCS_REF' \
org.label-schema.vcs-url="https://github.com/nipy/nipype" \
org.label-schema.version='$VERSION' \
org.label-schema.schema-version="1.0"
}


Expand Down