Review of neuroimaging meta-analyses: Topics, methods, and authors
Poster No:
2243
Submission Type:
Abstract Submission
Authors:
Kendra Oudyk1, Jérôme Dockès2, Alejandro De La Vega3, Angela Laird4, Brent McPherson5, Niusha Mirhakimi6, Mohammad Torabi1, Michelle Wang1
Institutions:
1McGill University, Montreal, Quebec, 2INRIA, Paris, Paris Region, 3University of Texas at AUstin, Austin, TX, 4Department of Physics, Florida International University, , Miami, FL, 5Mcgill, Montreal, Quebec, 6McGill University, Montreal, QC
First Author:
Co-Author(s):
Introduction:
Researchers and clinicians increasingly rely on meta-analyses to make sense of the rapidly-growing literature [0]. It is unclear whether we, like other fields [2], have over- or under-analyzed different topics. Further, in neuroimaging, researchers have invented novel methods for meta-analyzing image data [3], but we have not yet had a comprehensive review of the use of these methods. Finally, we do not know whether this field of meta-analysis is subject to the same gender biases as the larger neuroscience field [4]. In this review, we aim to address these gaps in our knowledge about the topics, methods, and authors of neuroimaging meta-analyses.
Methods:
For this project, we used the LitMining ecosystem of tools for meta-research [4]. We used pubget [4] to query PubMed Central and PubMed in order to find fMRI meta-analyses (n=849), as well as participant-level fMRI studies (n=4962). We further collected a set of papers associated with data on NeuroVault [5] (n=1091).
To extract topics, we applied non-negative matrix factorization on the matrix of term frequencies for fMRI papers. These terms were a subset of the terms from NeuroQuery [6], excluding anatomical terms. We then applied the topic weights to the term frequency matrices for a) the meta-analyses, and b) the papers associated with images on NeuroVault, in order to a) determine which topics have been more or less covered by previous meta-analyses, and b) to point to topics that have more image-based data on NeuroVault for potential future meta-analyses.
Further, we used labelbuddy [4] to manually label methodological details in meta-analyses, and present descriptive results to illustrate potential issues with methods. We store the annotations in a git repository hosted on Github: litmining/annotations.
Finally, we used pubextract [4] to automatically derive author genders from their names and author locations from their affiliations, for which we compare the subfield of meta-analyses and the larger fMRI field.
To extract topics, we applied non-negative matrix factorization on the matrix of term frequencies for fMRI papers. These terms were a subset of the terms from NeuroQuery [6], excluding anatomical terms. We then applied the topic weights to the term frequency matrices for a) the meta-analyses, and b) the papers associated with images on NeuroVault, in order to a) determine which topics have been more or less covered by previous meta-analyses, and b) to point to topics that have more image-based data on NeuroVault for potential future meta-analyses.
Further, we used labelbuddy [4] to manually label methodological details in meta-analyses, and present descriptive results to illustrate potential issues with methods. We store the annotations in a git repository hosted on Github: litmining/annotations.
Finally, we used pubextract [4] to automatically derive author genders from their names and author locations from their affiliations, for which we compare the subfield of meta-analyses and the larger fMRI field.
Results:
We see an uneven coverage of topics by meta-analyses, with some topics receiving more attention than others (see Figure 1). Further, we point to topics, such as 'risky decision making', that potentially have enough image data on NeuroVault [5] for future image-based meta-analyses. Image-based meta-analyses are the gold standard for meta-analyses, but are not common due to difficulty getting data.
Regarding methods, we see that the most common methods are activation likelihood estimation [7], and Seed-based d Mapping [8], with very few image-based meta-analyses (see Figure 2C). The number of papers included in each meta-analysis approximately follows a power law, with many meta-analyses each examining relatively few papers. Indeed, many meta-analyses have fewer papers than the recommended minimums (17 [9] or 30 [10]; see Figure 2D).
Regarding authors, our results suggest that there is a higher proportion of meta-analyses with women as the first and last author, compared to participant-level fMRI studies (see Figure 2A). Further, there are not many meta-analyses done in low-income countries; this is a potential area for growth, since meta-analyses have relatively low cost compared to imaging studies (see Figure 2B).
Regarding methods, we see that the most common methods are activation likelihood estimation [7], and Seed-based d Mapping [8], with very few image-based meta-analyses (see Figure 2C). The number of papers included in each meta-analysis approximately follows a power law, with many meta-analyses each examining relatively few papers. Indeed, many meta-analyses have fewer papers than the recommended minimums (17 [9] or 30 [10]; see Figure 2D).
Regarding authors, our results suggest that there is a higher proportion of meta-analyses with women as the first and last author, compared to participant-level fMRI studies (see Figure 2A). Further, there are not many meta-analyses done in low-income countries; this is a potential area for growth, since meta-analyses have relatively low cost compared to imaging studies (see Figure 2B).
Conclusions:
This analysis of the neuroimaging meta-analyses is one example of the LitMining ecosystem, which comprises a set of tools and recommendations enabling faster, open, collaborative and more reproducible studies of the literature. These results have implications across many areas of neuroimaging, since meta-analyses are widely used across topics.
Modeling and Analysis Methods:
Other Methods 2
Neuroinformatics and Data Sharing:
Informatics Other 1
Novel Imaging Acquisition Methods:
BOLD fMRI
Keywords:
Meta- Analysis
1|2Indicates the priority used for review
Provide references using author date format
[1] Ioannidis, J. P. (2016). The mass production of redundant, misleading, and conflicted systematic reviews and meta‐analyses. The Milbank Quarterly, 94(3), 485-514.
[2] Wager, T., et al. (2007). Meta-analysis of functional neuroimaging data: current and future directions. SCAN, 2(2), 150-158.
[3] Dworkin, J. (2020). The extent and drivers of gender imbalance in neuroscience reference lists. Nat. Neuro. 23, no. 8: 918-926.
[4] Oudyk, K.*, Dockès, J.*, et al.. (2023). Mining the neuroimaging literature. BioRxiv, 564783. *co-first authors
[5] Gorgolewski, K. J. et al. (2015). NeuroVault. org: a web-based repository for collecting and sharing unthresholded statistical maps of the human brain. Frontiers in neuroinformatics, 9, 8.
[6] Dockès, J., et al. (2020). NeuroQuery, comprehensive meta-analysis of human brain mapping. Elife, 9, e53385.
[7] Eickhoff, S. (2012). Activation likelihood estimation meta-analysis revisited. NeuroImage 59, no. 3:2349-2361.
[8] Albajes-Eizagirre A. (2019) Voxel-based meta-analysis via permutation of subject images (PSI): Theory and implementation for SDM. NeuroImage 2019; 186:174-184
[9] Eickhoff, S. et al.,(2016). Implementation errors in the GingerALE Software: Description and recommendations. HBM, 1(38), 7-11.
[10] Acar, F., et al. (2018). Assessing robustness against potential publication bias in Activation Likelihood Estimation (ALE) meta-analyses for fMRI. PloS one, 13(11), e0208177.