fMRI data of mixed gambles from the Neuroimaging Analysis Replication and Prediction Study

Abstract

There is an ongoing debate about the replicability of neuroimaging research. It was suggested that one of the main reasons for the high rate of false positive results is the many degrees of freedom researchers have during data analysis. In the Neuroimaging Analysis Replication and Prediction Study (NARPS), we aim to provide the first scientific evidence on the variability of results across analysis teams in neuroscience. We collected fMRI data from 108 participants during two versions of the mixed gambles task, which is often used to study decision-making under risk. For each participant, the dataset includes an anatomical (T1 weighted) scan and fMRI as well as behavioral data from four runs of the task. The dataset is shared through OpenNeuro and is formatted according to the Brain Imaging Data Structure (BIDS) standard. Data pre-processed with fMRIprep and quality control reports are also publicly shared. This dataset can be used to study decision-making under risk and to test replicability and interpretability of previous results in the field.

Variability in the analysis of a single neuroimaging dataset by many teams

“…For Hypotheses #1 and #3, there was also a subset of seven teams whose unthresholded maps were anticorrelated with those of the main cluster of teams. A comparison of the average map for the anticorrelated cluster for Hypotheses #1 and #3 confirmed that this map was highly correlated (r = 0.87) with the overall task activation map (averaged across the relevant group for these hypotheses) as reported in the NARPS Data Descriptor 11 . Further analysis of the model specifications for the six teams with available modeling details showed that four of them appeared to use models that did not properly separate the parametric effect of gain from overall task activation; because of the general anticorrelation of value system activations with task activations 18 , this model mis-specification led to an anticorrelation with the parametric effects of gain.…”

“…The two versions of the task were designed to address an ongoing debate in the literature regarding the impact of distributions of potential gains/losses on neural activity in this task 9,10 . A full description of the experimental procedures, validations and the dataset is available in a Data Descriptor 11 ; the dataset is openly available via OpenNeuro at DOI:10.18112/openneuro.ds001734.v1.0.4. Fully reproducible code for all analyses of the data reported here are available at DOI:10.5281/zenodo.3528171.…”

Variability in the analysis of a single neuroimaging dataset by many teams

“…The two versions were designed to address a debate on the effect of gain and loss distributions on neural activity in this task [10][11][12] . A full description of the dataset is available in a Data Descriptor 1 ; the dataset is openly available at https://doi.org/10.18112/openneuro. ds001734.v1.0.4.…”

“…Data analysis workflows in many scientific domains have become increasingly complex and flexible. Here we assess the effect of this flexibility on the results of functional magnetic resonance imaging by asking 70 independent teams to analyse the same dataset, testing the same 9 ex-ante hypotheses 1 . The flexibility of analytical approaches is exemplified by the fact that no two teams chose identical workflows to analyse the data.…”

Improving Practices and Inferences in Developmental Cognitive Neuroscience

“…Such a mega-analysis will yield biased results if it does not include data that is not available just because it did not yield a significant result. Although such data may be uploaded to a repository like openneuro.org, it benefits the researcher and the community to publish a citable data paper with comprehensive details on the protocol and what is available in the data set (Chavan & Penev, 2011;Gorgolewski et al, 2013; for an example data paper, see Botvinik-Nezer et al, 2019).…”