Spatial Confidence Sets - Beyond Null Hypothesis Testing of Cluster Size.
Presented During:
Tuesday, June 27, 2017: 11:08 AM - 11:20 AM
Vancouver Convention Centre
Room:
Ballroom AB
Submission No:
4171
Submission Type:
Abstract Submission
On Display:
Wednesday, June 28 & Thursday, June 29
Authors:
Alexander Bowring1, Armin Schwartzman2, Max Sommerfeld3, Thomas Nichols1
Institutions:
1University of Warwick, Coventry, United Kingdom, 2North Carolina State University, Raleigh, NC, 3University of Göttingen, Göttingen, Germany
First Author:
Introduction:
Null hypothesis testing lies at the foundation of human brain mapping as the core method for fMRI inference. However, recent studies have shown that under optimal conditions the null hypothesis is never true [1]. As ambitious, large-sample studies have become available (e.g. Human Connectome Project, N=1,200; UK Biobank final N=100,000), this we have high-quality, high-power data for which the null hypothesis test essentially shows universal activation even with stringent correction.
To overcome this, we apply recent work [2] to develop confidence sets (CSs) on clusters found in thresholded maps. Whereas traditional inferences indicate where the null, i.e. an effect size of 0, is rejected, the CSs are statements about non-zero effect sizes analogous to confidence intervals. For a cluster constructed with cluster-forming threshold c, the CSs comprise two sets of voxels: The upper CS is smaller, giving the voxels we infer to be truly larger than c; the larger lower CS is best described by its complement -- all voxels outside this set we infer to be truly smaller than c.
Here we describe the method, evaluate it with simulations and apply it to HCP data. We focus on inference on the percentage BOLD change map.
To overcome this, we apply recent work [2] to develop confidence sets (CSs) on clusters found in thresholded maps. Whereas traditional inferences indicate where the null, i.e. an effect size of 0, is rejected, the CSs are statements about non-zero effect sizes analogous to confidence intervals. For a cluster constructed with cluster-forming threshold c, the CSs comprise two sets of voxels: The upper CS is smaller, giving the voxels we infer to be truly larger than c; the larger lower CS is best described by its complement -- all voxels outside this set we infer to be truly smaller than c.
Here we describe the method, evaluate it with simulations and apply it to HCP data. We focus on inference on the percentage BOLD change map.
Methods:
While this method works with a general linear model, we describe it here in terms of a one-sample model. For subject i the BOLD response at voxel s is modelled
Yi(s) = μ(s) + εi(s)
where μ(s) is the unknown true mean BOLD activation and εi(s) the error term. Operating on raw units, the (hypothetical, noise-free) cluster found by thresholding μ(s) at level c is Ac := {s: μ(s) > c}; e.g., c = 1% BOLD change. We seek a lower set (Ac-) and upper set (Ac+) that surround Ac with a confidence level 1-α. These take the form
Ac- = {s: x(s) > c + a σ(s) / √n}
Ac+ = {s: x(s) > c - a σ(s) / √n},
where x(s) is the sample mean, σ(s) is the estimated standard deviation, and a is a critical value that is to be estimated. Specifically a is the (1-α)%ile of the maximum distribution of the absolute error process |εi(s)| over the boundary {s: x(s) = c}, calculated with a wild Bootstrap resampling scheme. Use of the maximum ensures that the confidence statement is simultaneous over the whole brain.
We evaluated the method by simulating 2D images (250 x 250 voxels) with a circular effect in the center, with maximum intensity of 3. We considered sample sizes of 60, 120 & 240 and smoothness of 10, 15 and 20 voxel FWHM. For a threshold of 1.3, we computed the "coverage", the percentage of times that the true thresholded signal was completely contained between the upper and lower sets. We also considered 3D images (112 x 112 x 16 voxels) with an ellipsoidal effect in the center, with maximum intensity 3. Similar to the 2D simulations, we computed the coverage for a threshold of 1.3 using the same sample sizes, but this time we used a lower smoothness of 1.5, 3, and 6 voxel FWHM. For both of the simulations, the boundary {s: x(s) = c} required as part of the method was estimated by dilating and eroding the set {s: x(s) > c} by 1 voxel, and using the set difference between these dilated and eroded sets.
Yi(s) = μ(s) + εi(s)
where μ(s) is the unknown true mean BOLD activation and εi(s) the error term. Operating on raw units, the (hypothetical, noise-free) cluster found by thresholding μ(s) at level c is Ac := {s: μ(s) > c}; e.g., c = 1% BOLD change. We seek a lower set (Ac-) and upper set (Ac+) that surround Ac with a confidence level 1-α. These take the form
Ac- = {s: x(s) > c + a σ(s) / √n}
Ac+ = {s: x(s) > c - a σ(s) / √n},
where x(s) is the sample mean, σ(s) is the estimated standard deviation, and a is a critical value that is to be estimated. Specifically a is the (1-α)%ile of the maximum distribution of the absolute error process |εi(s)| over the boundary {s: x(s) = c}, calculated with a wild Bootstrap resampling scheme. Use of the maximum ensures that the confidence statement is simultaneous over the whole brain.
We evaluated the method by simulating 2D images (250 x 250 voxels) with a circular effect in the center, with maximum intensity of 3. We considered sample sizes of 60, 120 & 240 and smoothness of 10, 15 and 20 voxel FWHM. For a threshold of 1.3, we computed the "coverage", the percentage of times that the true thresholded signal was completely contained between the upper and lower sets. We also considered 3D images (112 x 112 x 16 voxels) with an ellipsoidal effect in the center, with maximum intensity 3. Similar to the 2D simulations, we computed the coverage for a threshold of 1.3 using the same sample sizes, but this time we used a lower smoothness of 1.5, 3, and 6 voxel FWHM. For both of the simulations, the boundary {s: x(s) = c} required as part of the method was estimated by dilating and eroding the set {s: x(s) > c} by 1 voxel, and using the set difference between these dilated and eroded sets.
Results:
See Figures 1 and 2 for results.
Conclusions:
We have presented a method of Confidence Sets to obtain simultaneous confidence statements on where signal in the brain can be inferred to be near a given value c. It is not a hypothesis testing procedure, and explicitly makes statements about non-zero effects. With the growth of 'population neuroimaging' studies with arbitrarily large power, tools like this for interpreting (all significant) effects will become increasingly important.
Imaging Methods:
BOLD fMRI 2
Modeling and Analysis Methods:
Methods Development 1
Keywords:
Computational Neuroscience
Design and Analysis
FUNCTIONAL MRI
Meta- Analysis
Multivariate
Statistical Methods
1|2Indicates the priority used for review
Would you accept an oral presentation if your abstract is selected for an oral session?
I would be willing to discuss my abstract with members of the press should my abstract be marked newsworthy:
Please indicate below if your study was a "resting state" or "task-activation” study.
By submitting your proposal, you grant permission for the Organization for Human Brain Mapping (OHBM) to distribute the presentation in any format, including video, audio print and electronic text through OHBM OnDemand, social media channels or other electronic media and on the OHBM website.
Healthy subjects only or patients (note that patient studies may also involve healthy subjects):
Internal Review Board (IRB) or Animal Use and Care Committee (AUCC) Approval. Please indicate approval below. Please note: Failure to have IRB or AUCC approval, if applicable will lead to automatic rejection of abstract.
Please indicate which methods were used in your research:
For human MRI, what field strength scanner do you use?
Which processing packages did you use for your study?
Provide references in author date format
[2] Sommerfeld, M. (2015). Confidence regions for excursion sets in asymptotically Gaussian random fields, with an application to climate, 1–19. arXiv:1501.07000