Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015

Camerer, Colin F.; Dreber, Anna; Holzmeister, Felix; Ho, Teck H.; Huber, Jürgen; Johannesson, Magnus; Kirchler, Michael; Nave, Gideon; Nosek, Brian A.

doi:10.31235/osf.io/4hmb6

Cited by 283 publications

(455 citation statements)

References 20 publications

Supporting

Mentioning

434

Contrasting

Unclassified

Order By: Relevance

“…Specifically, probability information tends to be neglected in medical decisions. Such a replication of previous findings is especially valuable in light of the recent replication crisis in the social sciences (Camerer et al, ; Open Science Collaboration, ). As a novel contribution, our results showed that the gap between medical and monetary decisions generalizes to choices for others: Choice behavior and information search in medical decisions differ from monetary decisions irrespective of who is affected by the outcome.…”

Section: Discussionmentioning

confidence: 96%

The gap between medical and monetary choices under risk persists in decisions for others

Popovic

Pachur

Gaissmaier

2019

Behavioral Decision Making

View full text Add to dashboard Cite

Decisions under risk in the medical domain have been found to systematically diverge from decisions in the monetary domain. When making choices between monetary options, people commonly rely on a decision strategy that trades off outcomes with their probabilities; when making choices between medical options, people tend to neglect probability information. In two experimental studies, we tested to what extent differences between medical and monetary decisions also emerge when the decision outcomes affect another person. Using a risky choice paradigm for medical and monetary decisions, we compared hypothetical decisions that participants made for themselves to decisions for a socially distant other (Study 1) and to recommendations as financial advisor or doctor (Study 2). In addition, we examined people's information search in a condition in which information about payoff distributions had to be learned from experiential sampling. Formal modeling and analyses of search behavior revealed a similarly pronounced gap between medical and monetary decisions in decisions for others as in decisions for oneself. Our results suggest that when making medical decisions, people try to avoid the worst outcome while neglecting its probability—even when the outcomes affect others rather than themselves.

show abstract

Section: Discussionmentioning

confidence: 96%

The gap between medical and monetary choices under risk persists in decisions for others

Popovic

Pachur

Gaissmaier

2019

Behavioral Decision Making

View full text Add to dashboard Cite

show abstract

“…Another noteworthy replication initiative is the Social Sciences Replication Project, whose collaborators aimed to replicate 21 experimental studies in the social sciences published in the prestigious journals Nature and Science between 2010 and 2015. They find a significant effect in the same direction as the original study for 13 (62%) studies, and the effect size of the replications is on average about 50% of the original effect size (Camerer et al ). Finally, while both of the above‐mentioned projects focus on results published in top journals, Maniadis, Tufano, and List () analyze replication attempts from 150 economic journals, and find a “success rate” of 42.3% among the 85 experimental replication studies in their sample.…”

Section: Dozen Thingsmentioning

confidence: 89%

The Dozen Things Experimental Economists Should Do (More of)

Czibor¹,

Jimenez-Gomez²,

List

2019

Southern Economic Journal

View full text Add to dashboard Cite

What was once broadly viewed as an impossibility—learning from experimental data in economics—has now become commonplace. Governmental bodies, think tanks, and corporations around the world employ teams of experimental researchers to answer their most pressing questions. For their part, in the past two decades academics have begun to more actively partner with organizations to generate data via field experimentation. Although this revolution in evidence‐based approaches has served to deepen the economic science, recently a credibility crisis has caused even the most ardent experimental proponents to pause. This study takes a step back from the burgeoning experimental literature and introduces 12 actions that might help to alleviate this credibility crisis and raise experimental economics to an even higher level. In this way, we view our “12 action wish list” as discussion points to enrich the field.

show abstract

“…Particularly, future studies should carefully examine the mechanisms behind the effect. Dawson et al () stemmed from a metaphoric priming approach; however, such priming research should be approached with caution, as it has generated substantial scepticism in the social psychology field due to failures to replicate (Bower, ; Camerer et al , ; Verschuere et al , ; Yong, ). For example, in an effort to replicate Dawson et al ’s () findings and other well‐known priming measures, Dianiska, Swanner, Brimbal, and Meissner () examined the influence of lexical (i.e., word scrambles related to openness concept), contextual (e.g., room decorative posters depicting open settings) and embodiment primes (e.g., interviewers’ open or closed off body postures) on information disclosure, failing to find convincing evidence of their influence.…”

Section: Discussionmentioning

confidence: 99%

The influence of room spaciousness on investigative interviews

Hoogesteyn

Meijer

Vrij

2019

Legal Criminol Psychol

View full text Add to dashboard Cite

Purpose The quality of information obtained from investigative interviews largely relies on the quality of communication between the interviewee and interviewer. One aspect of the communication process that has yet to be well examined is the environment in which the interviews take place. The present study examined the influence of physical spaciousness, manipulated as room size and interpersonal sitting distance between interviewer and interviewee on the disclosure of crime‐related information, as well as perceptions of rapport and overall interview experience. Methods Participants engaged in a virtual reality scenario depicting a crime and were interviewed as suspects in either a larger or smaller room, at a closer or larger distance. Results Results showed no links between room size and sitting distance on disclosure rates. However, an exploratory analysis did reveal that participants interviewed in the larger room reported more positive interview experience in terms of spaciousness, and consequently higher perceptions of rapport, compared to those interviewed in the small room. Conclusions We found evidence against an influence of room size and interpersonal distance on disclosure. Still, our study does provide initial evidence that manipulating room size in an interview context could positively impact rapport‐building.

show abstract

Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015

Abstract: These first ten authors contributed equally to this work.

Cited by 283 publications

References 20 publications

The gap between medical and monetary choices under risk persists in decisions for others

The gap between medical and monetary choices under risk persists in decisions for others

The Dozen Things Experimental Economists Should Do (More of)

The influence of room spaciousness on investigative interviews

Contact Info

Product

Resources

About