E-COBIDAS: a webapp to improve neuroimaging methods and results reporting
Poster No:
1098
Submission Type:
Abstract Submission
Authors:
remi gau1, Cassandra Gould van Praag2, Sanu Ann Abraham3, David Moreau4, Tim van Mourik5, Zsuzsika Sjoerds6, Kristina Wiebels4, Satrajit Ghosh7, Thomas Nichols8
Institutions:
1Institute of Psychology, Université Catholique de Louvain, Louvain la neuve, Wallonie, 2University of Oxford, Oxford, Oxfordshire, 3MIT, WEST ROXBURY, MA, 4University of Auckland, Auckland, Auckland, 5Radboud University Nijmegen, Nijmegen, Other, 6Leiden University, Leiden, Netherlands, 7MIT, Cambridge, MA, 8University of Oxford, Oxford, United Kingdom
First Author:
Co-Author(s):
Introduction:
In any scientific study, a complete and precise method section is necessary to understand and evaluate the results, plan replications, seed new research and compare outcomes across studies. However, a large number of neuroimaging studies fail to report important details necessary for independent investigators to achieve these goals [1].
In order to improve this situation, the Committee on Best Practices in Data Analysis and Sharing (COBIDAS) of OHBM released a report to establish a set of best practices for methods and results reporting in f/MRI research [4]. This has been recently followed by a similar initiative for EEG and MEG [5].
The COBIDAS reports are accompanied by checklist tables meant to help authors ensure that their methods and results description complies with the Committee's recommendations. These checklists are a valuable resource, but the static PDF tables do not lend themselves to an actionable format, and users may be required to scroll through pages of items that do not concern their specific use-case. Moreover, while these checklists help generate a human readable method section, they do not provide ways to create a machine-readable equivalent that would encapsulate a large part of the metadata in a study.
The goal of the eCOBIDAS webapp is to provide a user-friendly solution for researchers to fill out the COBIDAS checklists, while generating a machine-readable summary of a methods section. This summary can then be used to automate methods section writing for authors, or to facilitate the assessment of a study quality during peer-review.
In order to improve this situation, the Committee on Best Practices in Data Analysis and Sharing (COBIDAS) of OHBM released a report to establish a set of best practices for methods and results reporting in f/MRI research [4]. This has been recently followed by a similar initiative for EEG and MEG [5].
The COBIDAS reports are accompanied by checklist tables meant to help authors ensure that their methods and results description complies with the Committee's recommendations. These checklists are a valuable resource, but the static PDF tables do not lend themselves to an actionable format, and users may be required to scroll through pages of items that do not concern their specific use-case. Moreover, while these checklists help generate a human readable method section, they do not provide ways to create a machine-readable equivalent that would encapsulate a large part of the metadata in a study.
The goal of the eCOBIDAS webapp is to provide a user-friendly solution for researchers to fill out the COBIDAS checklists, while generating a machine-readable summary of a methods section. This summary can then be used to automate methods section writing for authors, or to facilitate the assessment of a study quality during peer-review.
Methods:
The tables of the COBIDAS reports were reduced into 600 individual items. A subset of items was selected and converted into a hierarchical schema representation in accordance with the repronim schema (https://github.com/ReproNim/reproschema). Using this schema representation offers several advantages, as it allows:
-Efficient linking between the metadata about each item and the data provided by the user;
-Enforcing of output format consistency and of input validation directly at the data acquisition stage (i.e while the user is filling out the checklist);
-Specifying user interface details for certain use-cases (e.g., to let users be presented only with items or sections that concern them).
We then adapted the JavaScript-based webapp used for the visualization of the repronim schema to allow users to browse and complete each section of the checklist.
-Efficient linking between the metadata about each item and the data provided by the user;
-Enforcing of output format consistency and of input validation directly at the data acquisition stage (i.e while the user is filling out the checklist);
-Specifying user interface details for certain use-cases (e.g., to let users be presented only with items or sections that concern them).
We then adapted the JavaScript-based webapp used for the visualization of the repronim schema to allow users to browse and complete each section of the checklist.
Results:
The development of the webapp started at the OHBM Hackathon 2019 and aims to be a project developed by the community for the community. All information is centralized on an open science framework project [2]. The current prototype of the webapp can be accessed at https://cobidas-checklist.herokuapp.com/#/. It is based on the 88 metadata items available to characterise a collection of neuroimaging results uploaded on the neurovault website [3]. This only constitutes a subset of the full f/MRI COBIDAS checklist and will be expanded in the near future.
Users can fill out the different sections of the checklist (study design, participants, preprocessing, etc.) and are only presented with items that concern their use-case. Finally, the output can be exported in a set of machine-readable JSON files.
Users can fill out the different sections of the checklist (study design, participants, preprocessing, etc.) and are only presented with items that concern their use-case. Finally, the output can be exported in a set of machine-readable JSON files.
Conclusions:
Future developments of the webapp involve:
-Expansion of the list of items covered to include first, all the items from Carp 2012, then the entirety of the COBIDAS f/MRI checklist
-Establish a list of recommended MEEG items to create a version for electrophysiology studies
-Automatically generate a PDF summary and a method section after the checklist has been filled
-Automatically fill the checklist by allowing the application to query the content of a BIDS dataset or an SPM or FSL analysis pipeline (matlabbatch.mat or design.fsf)
eCOBIDAS is part of a broader trend towards open, reproducible science, to facilitate valid and robust inferences in neuroscience. We encourage any member of the community to contribute to this project.
-Expansion of the list of items covered to include first, all the items from Carp 2012, then the entirety of the COBIDAS f/MRI checklist
-Establish a list of recommended MEEG items to create a version for electrophysiology studies
-Automatically generate a PDF summary and a method section after the checklist has been filled
-Automatically fill the checklist by allowing the application to query the content of a BIDS dataset or an SPM or FSL analysis pipeline (matlabbatch.mat or design.fsf)
eCOBIDAS is part of a broader trend towards open, reproducible science, to facilitate valid and robust inferences in neuroscience. We encourage any member of the community to contribute to this project.
Modeling and Analysis Methods:
Other Methods 1
Neuroinformatics and Data Sharing:
Workflows 2
Informatics Other
Keywords:
Statistical Methods
Workflows
Other - checklist, reproducibility, standardization, replicability, transparency,
1|2Indicates the priority used for review
My abstract is being submitted as a Software Demonstration.
Please indicate below if your study was a "resting state" or "task-activation” study.
Healthy subjects only or patients (note that patient studies may also involve healthy subjects):
Was any human subjects research approved by the relevant Institutional Review Board or ethics panel? NOTE: Any human subjects studies without IRB approval will be automatically rejected.
Was any animal research approved by the relevant IACUC or other animal research panel? NOTE: Any animal studies without IACUC approval will be automatically rejected.
Please indicate which methods were used in your research:
For human MRI, what field strength scanner do you use?
Which processing packages did you use for your study?
Provide references using author date format
2 - Gau, Remi (2019), ‘COBIDAS Checklist’, OSF, doi: 10.17605/OSF.IO/ANVQY
3 - Gorgolewski K. J. (2015) ‘NeuroVault.org: a web-based repository for collecting and sharing unthresholded statistical maps of the brain’, Frontiers in Neuroinformatics. vol. 9, no. 8 doi: 10.3389/fninf.2015.00008
4 - Nichols T. E. (2017), ‘Best Practices in Data Analysis and Sharing in Neuroimaging using MRI’, Nature Neuroscience, vol. 20, no 3, pp. 299-303, doi: 10.1038/nn.4500.
5 - Pernet, C. R. (2018), Best Practices in Data Analysis and Sharing in Neuroimaging Using MEEG, OSF Preprints, August 9. doi:10.31219/osf.io/a8dhx