Mapping task measures to latent constructs: An expert survey of the NIMH RDoC cognitive domain
Poster No:
931
Submission Type:
Abstract Submission
Authors:
Patrick Bissett1, Logan Bennett1, Jaime Ali Rios1, Sunjae Shim1, Paul McKee2, Christopher Iyer1, Nilam Ram1, Russell Poldrack1
Institutions:
1Stanford University, Stanford, CA, 2Duke University, Durham, NC
First Author:
Co-Author(s):
Introduction:
The National Institute of Mental Health (NIMH) Research Domain Criteria (RDoC; Insel et al., 2010) reconceptualizes mental health research by shifting focus from symptom-based diagnostic categories to a set of cross-disorder dimensional domains.
The cognitive systems domain of the RDoC matrix includes three constructs: attention, cognitive control, and working memory. As shown in Figure 1, these constructs are indicated by more specific subconstructs (third row of Figure 1, subsequently we use "construct" to indicate construct or subconstruct), which are measured by specific tasks. However, because tasks are not observed manifest variables on their own, an additional layer of information is needed to engage with and evaluate the proposed relations among cognitive constructs. In particular, the constructs must be mapped to specific measurements that quantify individuals' behavior in the tasks. To do so, we enlisted a group of experts to stipulate links between RDoC constructs and specific measures obtained from common cognitive tasks.
The cognitive systems domain of the RDoC matrix includes three constructs: attention, cognitive control, and working memory. As shown in Figure 1, these constructs are indicated by more specific subconstructs (third row of Figure 1, subsequently we use "construct" to indicate construct or subconstruct), which are measured by specific tasks. However, because tasks are not observed manifest variables on their own, an additional layer of information is needed to engage with and evaluate the proposed relations among cognitive constructs. In particular, the constructs must be mapped to specific measurements that quantify individuals' behavior in the tasks. To do so, we enlisted a group of experts to stipulate links between RDoC constructs and specific measures obtained from common cognitive tasks.
Methods:
We used a two-step survey to identify researchers with expertise in cognitive tasks and obtain their expert opinions on the construct–measure mappings. First, we distributed an online screener to experts identified through PubMed searches for relevant keywords, focusing on corresponding authors of publications with the highest citation counts. Then, we shared the screener link on Twitter/X, leveraging our online social networks to reach a broader audience. The screener requested age, academic training, contact information, and which tasks they published two or more scientific articles on.
If respondents had published two or more articles on a given task, provided an academic email address, and had begun a PhD in a related field, they received a second survey for that task. The second survey included a brief description of each task and a list of primary task measures that we identified. Respondents stipulated which RDoC constructs are operationalized by each measure, selecting none, one, or more than one construct as a multi-choice response.
If respondents had published two or more articles on a given task, provided an academic email address, and had begun a PhD in a related field, they received a second survey for that task. The second survey included a brief description of each task and a list of primary task measures that we identified. Respondents stipulated which RDoC constructs are operationalized by each measure, selecting none, one, or more than one construct as a multi-choice response.
Results:
From the 689 responses received to the initial screener, we identified and obtained survey responses from 34 respondents who were experts in and completed surveys about one or more tasks. For each task, we obtained, on average, 10.7 experts' mappings between the task measures and constructs.
To operationalize consensus, we binarized the data using a threshold that >=50% of the experts stipulated a construct-to-measure link. Figure 2 displays these binarized results, with constructs as columns and measures as rows. Here we highlight some key results.
Different measures within a given task were often mapped onto different cognitive constructs, confirming the need to stipulate the relationship between measures and constructs (and not just tasks and constructs, as in the current RDoC matrix). Attention was stipulated in an overwhelming majority of all measures across tasks (81%), consistent with attention being a key construct across various measures of cognitive control (Posner & Boies, 1971). Conversely, some constructs were very sparsely linked to measures (e.g., only two measures were linked to the interference control), suggesting narrow operationalization and/or experts' disagreement.
To operationalize consensus, we binarized the data using a threshold that >=50% of the experts stipulated a construct-to-measure link. Figure 2 displays these binarized results, with constructs as columns and measures as rows. Here we highlight some key results.
Different measures within a given task were often mapped onto different cognitive constructs, confirming the need to stipulate the relationship between measures and constructs (and not just tasks and constructs, as in the current RDoC matrix). Attention was stipulated in an overwhelming majority of all measures across tasks (81%), consistent with attention being a key construct across various measures of cognitive control (Posner & Boies, 1971). Conversely, some constructs were very sparsely linked to measures (e.g., only two measures were linked to the interference control), suggesting narrow operationalization and/or experts' disagreement.
Conclusions:
In this work, we surveyed domain experts to obtain consensus opinions on how the RDoC cognitive constructs are linked to observable task measures. This is the first step in an NIH-funded study in which our group aims to validate the neural circuits underlying key cognitive constructs. These expert survey results provide the basis for a model of the current state-of-the-field, to be related to neural circuits in a new dense neuroimaging sample (N=65, 15-hours of MRI each) based upon these same tasks. This MRI acquisition will test if expert beliefs are validated in relationships between the tasks' underlying neural circuits.
Higher Cognitive Functions:
Executive Function, Cognitive Control and Decision Making 1
Learning and Memory:
Working Memory 2
Keywords:
Cognition
Other - Cognitive control; working memory; attention; inhibition; task switching; taxonomy; ontology; expert survey; RDoC
1|2Indicates the priority used for review
Provide references using author date format
Poldrack, R. A., Kittur, A., Kalar, D., Miller, E., Seppa, C., Gil, Y., Parker, D. S., Sabb, F. W., & Bilder, R. M. (2011). The cognitive atlas: toward a knowledge foundation for cognitive neuroscience. Frontiers in Neuroinformatics, 5, 17.
Posner, M. I., & Boies, S. J. (1971). Components of attention. Psychological Review, 78(5), 391-408.