Skip to content

FIX: Revise multi-echo reference generation, permitting using SBRefs too #1803

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jun 9, 2020

Conversation

tsalo
Copy link
Collaborator

@tsalo tsalo commented Oct 2, 2019

References #1745, nipreps/niworkflows#399, and nipreps/niworkflows#408.

Changes proposed in this pull request

  • Currently, just tags a specific version of niworkflows to ensure backward compatibility, by request of @effigies.

Documentation that should be reviewed

None

@effigies
Copy link
Member

effigies commented Oct 2, 2019

Depends on: nipreps/niworkflows#408

@tsalo
Copy link
Collaborator Author

tsalo commented Oct 3, 2019

@effigies test_pytest failed due to what looks like a random connection error, unless I'm missing something and the failure is actually meaningful. Would you mind retriggering that test?

@effigies
Copy link
Member

effigies commented Oct 3, 2019

Done. Thanks for the heads up.

@effigies
Copy link
Member

effigies commented Oct 4, 2019

@tsalo I merged master, so you'll want to pull before doing anything else. Feel free to rebase, if you'd rather not have the pin and merge in the history,

@pvelasco
Copy link
Contributor

Hi @tsalo,
Do you have a working Docker image of your multi-input-reference branch? I'm trying to test it on some ME-EPI data and I need a Singularity image.
I downloaded your branch, built a Docker image, tagged it as cbinyu/tsalo_fmriprep, uploaded it to DockerHub and finally did a singularity pull of that image.
When I ran it on some data, I got the following error:

Traceback (most recent call last):
  File "/usr/local/miniconda/bin/fmriprep", line 10, in <module>
    sys.exit(main())
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/cli/run.py", line 312, in main
    opts = get_parser().parse_args()
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/cli/run.py", line 50, in get_parser
    currentv = Version(__version__)
  File "/usr/local/miniconda/lib/python3.7/site-packages/packaging/version.py", line 277, in __init__
    raise InvalidVersion("Invalid version: '{0}'".format(version))
packaging.version.InvalidVersion: Invalid version: ''

So I was wondering if you had a working Docker image that I can singularity pull.
Thanks!

@tsalo
Copy link
Collaborator Author

tsalo commented Mar 18, 2020

@pvelasco I don't have a Docker image for it since I haven't touched this branch or the corresponding niworkflows branch in a while. However, I just resolved the current merge conflicts in nipreps/niworkflows#408 so if you switch the pinned niworkflows version in a current fMRIPrep version's (e.g., master's) setup.cfg to niworkflows @ git+https://github.com/tsalo/niworkflows.git@7a000cc7ebebffac50f1069ba0ecf8d005cced88 and build from that you should be good.

I'm reticent to do that here because I don't want to spam folks with a dummy commit.

@pvelasco
Copy link
Contributor

Hi @tsalo,
I managed to get my a Docker image of your branch. I didn't notice that I had to use the VERSION argument in docker build.
Now the dataset is being processed. I'll report if the boldref image for multi-echo improves.

@tsalo
Copy link
Collaborator Author

tsalo commented Mar 18, 2020

@pvelasco Nothing should improve yet. I was waiting on nipreps/niworkflows#408 to be merged before trying to implement things in fMRIPrep, so the only change in this PR is the pinned version of nipreps in the setup file. That said, I can probably work on getting those changes implemented in the next few days.

@oesteban oesteban added this to the 20.2.0 LTS milestone Jun 4, 2020
oesteban added a commit to tsalo/niworkflows that referenced this pull request Jun 6, 2020
oesteban added a commit to tsalo/niworkflows that referenced this pull request Jun 6, 2020
@oesteban oesteban force-pushed the multi-input-reference branch from 4c54ca4 to 3860254 Compare June 6, 2020 06:21
oesteban added a commit to tsalo/niworkflows that referenced this pull request Jun 6, 2020
@oesteban oesteban force-pushed the multi-input-reference branch from 3860254 to 4076fcd Compare June 6, 2020 07:36
@oesteban oesteban force-pushed the multi-input-reference branch 3 times, most recently from 5d4a5b6 to 3eabda9 Compare June 6, 2020 18:06
@oesteban oesteban force-pushed the multi-input-reference branch 2 times, most recently from 2e2b154 to 1f1d6ac Compare June 7, 2020 02:19
@oesteban oesteban mentioned this pull request Jun 7, 2020
2 tasks
@oesteban oesteban changed the title WIP, ENH: Use SBRefs with ME-EPI FIX: Revise multi-echo reference generation, permitting using SBRefs too Jun 7, 2020
@oesteban
Copy link
Member

oesteban commented Jun 8, 2020

Okay, a side-by-side comparison of ROI plots seems reassuring:

  • Unnoticeable changes for ds005
  • Minimal changes for ds054 (none problematic). I attribute these to the fact that the SBRef is actually used now. Then, because subsequent runs of the workflow do not use the SBRef (Revise SBRef handling #1511), the final ROI plot does not really change.
  • Finally, ds210 shows some changes but those are also minimal.

I think we can push this one in and keep working on #1511, #2182, #2174, and #2162.

tsalo and others added 7 commits June 8, 2020 17:40
Confirms that the

```
fMRIPrep failed: name 'is_container' is not defined
```
is gone.
taking into account comments from @tsalo.

the single-echo case (when you set ``--echo-idx`` or ``--bids-filter``)
is now more nicely handled (as single-echo).

Partially addresses: nipreps#2182.
@oesteban oesteban force-pushed the multi-input-reference branch from 6c02a18 to b4f9ed3 Compare June 9, 2020 00:40
@oesteban oesteban merged commit 278580c into nipreps:master Jun 9, 2020
oesteban added a commit to oesteban/fmriprep that referenced this pull request Jun 9, 2020
Builds on the new echo list extraction of nipreps#1803 to determine whether the
dataset is single-echo, multi-echo (and how many echoes are processed),
or multi-echo but processing only one of the echoes (e.g., using
``--bids-filter``).

Resolves: nipreps#2174.
HippocampusGirl added a commit to HippocampusGirl/fmriprep that referenced this pull request Sep 16, 2020
20.2.0 RC0 (September 04, 2020)

With this third minor release series of 2020,
the first *fMRIPrep LTS* (*long-term support*) is finally here!

This release contains a number of bug-fixes and enhancements mostly
related to easing the maintenance, anticipating patch-release breaking
changes to ensure a longstanding LTS, and addressing some run-to-run
repeatability problems of the CompCor implementation.

Thank you for using *fMRIPrep*!
If you encounter any issues with this release, please let us know
by posting an issue on our GitHub page!

A full list of changes can be found below.

* FIX: Revise the reproducibility of CompCor masks (nipreps#2130)
* FIX: Simplify transform aggregation in resampling, pass identity transforms for multi-echo cases (nipreps#2239)
* FIX: Skip the T1w check if ``--anat-derivatives`` is provided. (nipreps#2201)
* FIX: Storing ``--bids-filters`` within config file (nipreps#2177)
* FIX: Revise multi-echo reference generation, permitting using SBRefs too (nipreps#1803)
* FIX: FreeSurfer license manipulation & canary
* ENH: Output CompCor masks if ``--debug compcor`` is passed (nipreps#2248)
* ENH: Conform to BIDS Derivatives as of BIDS 1.4.0 (nipreps#2223)
* ENH: Reuse config (nipreps#2240)
* ENH: Save BOLD-anatomical transforms to derivatives folder (nipreps#2233)
* ENH: Leverage BIDSLayout's ``database_path`` (nipreps#2203)
* ENH: Add ``--no-tty`` option to ``fmriprep-docker.py`` (nipreps#2204)
* ENH: Report number of echoes in BOLD summary. (nipreps#2184)
* ENH: Ensure NiPype telemetry is just pinged once (nipreps#2168)
* DOC: Update reference in "Refinement of Brain Mask" description (nipreps#2215)
* DOC: List *TemplateFlow* templates that need to be prefetched (nipreps#2196)
* DOC: Update references to https://github.com/nipreps (nipreps#2191)
* DOC: Pin NiPype with new Sphinx extension syntax (nipreps#2092)
* MAINT: Track nipreps#2252 from 20.1.x series (nipreps#2253)
* MAINT: Silence PyBIDS warning by setting extension mode (nipreps#2250)
* MAINT: Drop CircleCI docs build (nipreps#2247)
* MAINT: Pin latest *NiPreps* (nipreps#2244)
* MAINT: Update ``setup.cfg`` (flake8 and pytest) (nipreps#2183)
* MAINT: Delete release-drafter (nipreps#2169)
* MAINT: Track bug-fix release on the 20.1.x series (nipreps#2165)
* MAINT: Remove auto-comment bot (nipreps#2166)
* MAINT: Improve the questions on the bug-report template (nipreps#2158)
HippocampusGirl added a commit to HippocampusGirl/fmriprep that referenced this pull request Sep 16, 2020
20.2.0rc1

With this third minor release series of 2020,
the first *fMRIPrep LTS* (*long-term support*) is finally here!

This release contains a number of bug-fixes and enhancements mostly
related to easing the maintenance, anticipating patch-release breaking
changes to ensure a longstanding LTS, and addressing some run-to-run
repeatability problems of the CompCor implementation.

.. admonition:: Long-Term Support (LTS)

    *fMRIPrep* 20.2 LTS introduces the `long-term support program
    <https://www.nipreps.org/devs/releases/#long-term-support-series>`__.
    This LTS version will be kindly steered and maintained by
    the group of Dr. Basile Pinsard and Prof. Pierre Bellec at
    `CRIUGM <http://www.criugm.qc.ca/>`__, (Université de Montréal).
    The LTS is planned for a window of 4 years of support (i.e., until
    September 2024).

.. caution::

    As with all minor version increments, working directories
    from previous versions **should not be reused**.

Thank you for using *fMRIPrep*!
If you encounter any issues with this release, please let us know
by posting an issue on our GitHub page!

A full list of changes can be found below.

* FIX: Get missing ``probseg`` file from MNI152NLin2009cAsym (nipreps#2271)
* FIX: Restore ``--ignore t2w/flair`` options (nipreps#2260)
* FIX: Revise the reproducibility of *CompCor* masks (nipreps#2130)
* FIX: Simplify transform aggregation in resampling, pass identity transforms for multi-echo cases (nipreps#2239)
* FIX: Skip the T1w check if ``--anat-derivatives`` is provided. (nipreps#2201)
* FIX: Storing ``--bids-filters`` within config file (nipreps#2177)
* FIX: Revise multi-echo reference generation, permitting using SBRefs too (nipreps#1803)
* FIX: *FreeSurfer* license manipulation & canary
* ENH: Output CompCor masks if ``--debug compcor`` is passed (nipreps#2248)
* ENH: Conform to BIDS Derivatives as of BIDS 1.4.0 (nipreps#2223)
* ENH: Reuse config (nipreps#2240)
* ENH: Save BOLD-anatomical transforms to derivatives folder (nipreps#2233)
* ENH: Leverage BIDSLayout's ``database_path`` (nipreps#2203)
* ENH: Add ``--no-tty`` option to ``fmriprep-docker.py`` (nipreps#2204)
* ENH: Report number of echoes in BOLD summary. (nipreps#2184)
* ENH: Ensure *NiPype* telemetry is just pinged once (nipreps#2168)
* DOC: Add FAQ entry for using pre-indexed layouts (nipreps#2256)
* DOC: Update reference in "Refinement of Brain Mask" description (nipreps#2215)
* DOC: List *TemplateFlow* templates that need to be prefetched (nipreps#2196)
* DOC: Update references to https://github.com/nipreps (nipreps#2191)
* DOC: Pin *NiPype* with new Sphinx extension syntax (nipreps#2092)
* MAINT: Track nipreps#2269 and nipreps#2269, bug-fixes on the 20.1.x series
* MAINT: Remove derivatives from layout index ignores (nipreps#2258)
* MAINT: Track nipreps#2252 from 20.1.x series (nipreps#2253)
* MAINT: Silence *PyBIDS* warning by setting extension mode (nipreps#2250)
* MAINT: Drop CircleCI docs build (nipreps#2247)
* MAINT: Pin latest *NiPreps* (nipreps#2244)
* MAINT: Update ``setup.cfg`` (flake8 and pytest) (nipreps#2183)
* MAINT: Delete release-drafter (nipreps#2169)
* MAINT: Track bug-fix release on the 20.1.x series (nipreps#2165)
* MAINT: Remove auto-comment bot (nipreps#2166)
* MAINT: Improve the questions on the bug-report template (nipreps#2158)

.. admonition:: Author list for papers based on *fMRIPrep* 20.2 LTS series

    As described in the `Contributor Guidelines
    <https://www.nipreps.org/community/CONTRIBUTING/#recognizing-contributions>`__,
    anyone listed as developer or contributor may write and submit manuscripts
    about *fMRIPrep*.
    To do so, please move the author(s) name(s) to the front of the following list:

    Markiewicz, Christopher J. \ :sup:`1`\ ; Goncalves, Mathias \ :sup:`1`\ ; DuPre, Elizabeth \ :sup:`2`\ ; Kent, James D. \ :sup:`3`\ ; Salo, Taylor \ :sup:`4`\ ; Ciric, Rastko \ :sup:`1`\ ; Pinsard, Basile \ :sup:`5`\ ; Finc, Karolina \ :sup:`6`\ ; de la Vega, Alejandro \ :sup:`7`\ ; Feingold, Franklin \ :sup:`1`\ ; Tooley, Ursula A. \ :sup:`8`\ ; Benson, Noah C. \ :sup:`9`\ ; Urchs, Sebastian \ :sup:`2`\ ; Blair, Ross W. \ :sup:`1`\ ; Erramuzpe, Asier \ :sup:`10`\ ; Lurie, Daniel J. \ :sup:`11`\ ; Heinsfeld, Anibal S. \ :sup:`12`\ ; Jacoby, Nir \ :sup:`13`\ ; Jamison, Keith W. \ :sup:`14`\ ; Frederick, Blaise B. \ :sup:`15, 16`\ ; Valabregue, Romain \ :sup:`17`\ ; Sneve, Markus H. \ :sup:`18`\ ; Liem, Franz \ :sup:`19`\ ; Adebimpe, Azeez \ :sup:`20`\ ; Velasco, Pablo \ :sup:`21`\ ; Wexler, Joseph B. \ :sup:`1`\ ; Groen, Iris I. A. \ :sup:`22`\ ; Ma, Feilong \ :sup:`23`\ ; Amlien, Inge K. \ :sup:`18`\ ; Bellec, Pierre \ :sup:`5`\ ; Cieslak, Matthew \ :sup:`20`\ ; Devenyi, Grabriel A. \ :sup:`24`\ ; Ghosh, Satrajit S. \ :sup:`25, 26`\ ; Gomez, Daniel E. P. \ :sup:`27`\ ; Halchenko, Yaroslav O. \ :sup:`23`\ ; Isik, Ayse Ilkay \ :sup:`28`\ ; Moodie, Craig A. \ :sup:`1`\ ; Naveau, Mikaël \ :sup:`29`\ ; Rivera-Dompenciel, Adriana \ :sup:`3`\ ; Satterthwaite, Theodore D. \ :sup:`20`\ ; Sitek, Kevin R. \ :sup:`30`\ ; Stojić, Hrvoje \ :sup:`31`\ ; Thompson, William H. \ :sup:`1`\ ; Wright, Jessey \ :sup:`1`\ ; Ye, Zhifang \ :sup:`32`\ ; Gorgolewski, Krzysztof J. \ :sup:`1`\ ; Poldrack, Russell A. \ :sup:`1`\ ; Esteban, Oscar \ :sup:`33`\ .

    Affiliations:

      1. Department of Psychology, Stanford University
      2. Montreal Neurological Institute, McGill University
      3. Neuroscience Program, University of Iowa
      4. Department of Psychology, Florida International University
      5. SIMEXP Lab, CRIUGM, University of Montréal, Montréal, Canada
      6. Centre for Modern Interdisciplinary Technologies, Nicolaus Copernicus University in Toruń
      7. University of Texas at Austin
      8. Department of Neuroscience, University of Pennsylvania, PA, USA
      9. Department of Psychology, New York University
      10. Computational Neuroimaging Lab, BioCruces Health Research Institute
      11. Department of Psychology, University of California, Berkeley
      12. Child Mind Institute
      13. Department of Psychology, Columbia University
      14. Department of Radiology, Weill Cornell Medicine
      15. McLean Hospital Brain Imaging Center, MA, USA
      16. Consolidated Department of Psychiatry, Harvard Medical School, MA, USA
      17. CENIR, INSERM U1127, CNRS UMR 7225, UPMC Univ Paris 06 UMR S 1127, Institut du Cerveau et de la Moelle épinière, ICM, F-75013, Paris, France
      18. Center for Lifespan Changes in Brain and Cognition, University of Oslo
      19. URPP Dynamics of Healthy Aging, University of Zurich
      20. Perelman School of Medicine, University of Pennsylvania, PA, USA
      21. Center for Brain Imaging, New York University
      22. Department of Psychology, New York University, NY, USA
      23. Dartmouth College: Hanover, NH, United States
      24. Department of Psychiatry, McGill University
      25. McGovern Institute for Brain Research, MIT, MA, USA
      26. Department of Otolaryngology, Harvard Medical School, MA, USA
      27. Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen
      28. Max Planck Institute for Empirical Aesthetics
      29. Cyceron, UMS 3408 (CNRS - UCBN), France
      30. Speech & Hearing Bioscience & Technology Program, Harvard University
      31. Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London
      32. State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University
      33. Department of Radiology, CHUV, Université de Lausanne
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Smarter test of multiecho datasets Fmriprep handling of multi-echo data with single-band reference scans
4 participants