Max Planck Fellow

Gert G. Wagner

How People Know Their Risk Preference
Fact Boxes May Contribute to a More Positive Evaluation of COVID-19 Vaccinations

Research Team 2020–2022/23

Ruben C. Arslan (as of 04/2021: Leipzig University, Germany)

Predoctoral Fellow

Sarah Swanke (LIFE)

Gert G. Wagner, Professor of Economics, is a Max Planck Fellow at the MPI for Human Development. He collaborates with researchers from the Center for Adaptive Rationality, the Center for Lifespan Psychology, the Lise Meitner Group for Environmental Neuroscience, and the Max Planck Research Group Biosocial—Biology, Social Disparities, and Development. Wagner is part of the faculty of the International Max Planck Research School on the Life Course (LIFE) and, together with Ulman Lindenberger, is co-principal investigator of the Berlin Aging Study II (BASE-II) at the Institute. In 2023 Wagner was elected a fellow of the Association for Psychological Science.

Research in the years 2020 to 2022 was significantly dominated by analysis of the COVID-19 pandemic. But basic research still played a role, of course, and is reported on here before one COVID-19 project is presented.

How People Know Their Risk Preference

Two very different measurement traditions are used to investigate risk preferences in humans. The revealed preference approach, common in economics, studies choices under risk in the field and in the laboratory. The paradigmatic research designs in this tradition are observational studies of real behaviors (e.g., consumption and saving) and controlled choices between monetary lotteries. At the same time, psychologists as well as some economists use a different approach, in which people are simply asked to indicate their risk appetite in response to either general questions or hypothetical scenarios (the stated preference approach).

Economists are skeptical about the validity of stated preferences, because the statements involve no opportunity costs (“cheap talk”). However, inferring preferences from real-life behavior is fraught with assumptions, such as temporal stability and adequate control of confounding factors. Moreover, when researchers compared revealed and stated risk preference measures systematically, they found that the behavioral measures used in the revealed preference approach generally underperformed relative to the stated preference measures in terms of reliability, retest stability, and criterion validity. The behavioral measures used in the revealed preference approach do not correlate strongly across measures, meaning that they do not capture a clear latent preference that drives behavior across different choice situations. In contrast, stated risk preferences do correlate across measures, suggesting the existence of a general risk preference. Therefore, an interesting research question is: How do respondents manage to give such good answers when they are asked, “How do you see yourself: Are you generally a person who is fully prepared to take risks or do you try to avoid taking risks?” Answered on a scale of 0 (unwilling to take risks) to 10 (fully prepared to take risks), this single question, the general risk question (GRQ), has been used in several large and widely analyzed surveys.

To unpack the process of self-perception, how people translate their memories and intuitions into an answer to the GRQ was investigated. Stated risk preferences were collected as part of two large age-heterogeneous survey studies in Germany: the 2017 survey of the Berlin Aging Study II (BASE-II) and the 2017/2018 German Socio-Economic Panel Innovation Sample (SOEP-IS). Across both studies, 3,493 respondents answered the GRQ. After doing so, they were asked to explain their response in free-text questions about the topics and events they had thought about when answering.

Drawing on the extant literature and the free-text responses in the study, a coding scheme was developed with a list of broad risk domains and individual hazards. A set of coders then read the free-text responses. Nine coders read approximately 1,000 free-text answers each, so that each answer was coded in triplicate. Coders noted the presence of risk domains, such as investments or health, as well as more specific hazards, such as skydiving or divorce. Finally, each coder estimated—solely on the basis of the available text—the respondent’s stated risk preference (GRQ answer).

The data show that when respondents answered the open-ended question about what they had in mind when answering the GRQ, most focused on high-stakes risks in finance, relationships, career, and traffic. In other words, respondents recalled their revealed preferences in areas relevant to their lives. They did not think about low-stakes risks (which are common in experiments) and they rarely mentioned smoking, drinking, and gambling, even though the GRQ has prognostic validity for these behaviors (which is the reason the GRQ is used for many studies in different behavioral domains). Furthermore, third-party readers (coders) of respondents’ brief memories and explanations reached similar inferences about respondents’ preferences as stated on the GRQ, indicating the intersubjective validity of this information.

Ongoing research shows that the rating of children’s risk preferences by parents relies on the same process: When answering questions about the risk preferences of their children, parents bring to their minds the revealed preferences shown by their children in everyday life (e.g., at the playground, in school).

So, what economists perceive as a weakness of stated preferences (cheap talk) may actually be a strength. The study shows that the fairly vague, almost projective nature of a comprehensive single-item question allows people to refer back to their diagnostic memories and behaviors using a well-honed human capacity for social perception. That people can have different risk perceptions and conceptions could be problematic for the intersubjective comparability of their answers, but coders have been shown to generally agree on what risky behaviors imply about a person’s risk preference, irrespective of age and gender. The shared social perception of risks fosters agreement and comparability, as well as the prognostic validity of stated risk preferences. This does not imply that self-reports are always suitable. For instance, applicants for a financial manager position could foil an attempt to screen for risk-seekers by simply dissembling—just as they could in typical laboratory tasks, where stakes are generally low.

The overall result of this research is a kind of irony: The revealed preference approach delivers the best results, not by measuring observable real behavior (which is the traditional method applied by economists) but through the introspection of people who are asked to state the degree of their risk appetite (the stated preference approach, which is common in psychology).

Key Reference

Fact Boxes May Contribute to a More Positive Evaluation of COVID-19 Vaccinations

For the effective control of the COVID-19 pandemic with vaccines, most people in a population need to be vaccinated. It is thus important to know how to inform the public in a way that recognizes individual preferences—while also acknowledging the societal preference of encouraging vaccinations. According to the health care standard of informed decision making, a comparison of the benefits and harms of (not) being vaccinated would be required to inform undecided and skeptical people. In order to test evidence-based fact boxes—an established risk communication format—and to inform their development, their contribution to knowledge and to evaluations of COVID-19 vaccines was investigated with the help of the online panel survey COMPASS.

COMPASS was an ad hoc survey established by the research company infratest dimap (Berlin). The MPI for Human Development and the Harding Center for Risk Literacy (Potsdam) had the opportunity to place questions in this survey, which ran from the very beginning of the COVID-19 pandemic in March 2020 until summer 2022. Different waves of the survey were analyzed, with N = 1,942 to N = 6,056.

COMPASS showed that vaccination knowledge and vaccination intentions increased between November 2020 and February 2021. COMPASS also revealed objective information requirements and subjective information needs. Finally, COMPASS showed (fieldwork happened in January 2021) that simple fact boxes (see, for example, Figure 1) increased vaccination knowledge and positive evaluations in skeptics and undecideds. These fact boxes were developed at the Harding Center for Risk Literacy, with the help of the Robert Koch Institute (RKI). The RKI is the government’s central scientific institution in the field of biomedicine. It is one of the most important bodies for the safeguarding of public health in Germany.

It can be concluded that in Germany, the implementation of fact boxes supports evidence-based communication and, thus, empowerment at the population level. The mRNA fact boxes were disseminated via RKI’s media channels, including their website (with about 130 million visits in 2020), an RKI Twitter account, and the Permanent Vaccination Commission in Germany’s smartphone app. The results of our investigation suggest that presenting the around 10 million undecided and skeptical adults under 60 years of age in Germany with a simple fact box for about 90 seconds would familiarize more than 600,000 people with vaccine efficacy. A majority of them would evaluate vaccinations more positively. Additionally, by abstaining from persuasion, reluctance and distrust concerning the sender can be prevented or alleviated. Loss of trustworthiness is a relevant risk of alternative interventions that are nontransparent (e.g., nudges) or persuasive (e.g., advertisement), or that exert pressure (e.g., requiring authorized access to public activities).

From Rebitschek et al. (2022)
Original image licensed under CC BY 4.0

Key Reference