Dr. Betsy Levy Paluck's Research on Prejudice Reduction: A Critical Analysis

Dr. Betsy Levy Paluck, a professor of psychology and public affairs at Princeton University, is known for her groundbreaking work in areas such as violence, media effects, norms, prejudice, peacebuilding, and conflict in post-conflict settings. Her research, utilizing both qualitative and quantitative approaches in lab and field experiments, aims to enhance social outcomes. She is highly regarded for her innovative contributions over the past 15 years.

Dr. Betsy Levy Paluck expresses gratitude to her colleagues, including Roni Parat, Chelsea Clark, and John Green, for their invaluable contributions to her research. Their collaborative efforts have been instrumental in the publication of a significant paper in the Annual Review of Psychology in 2021.

Inspired by a request from a professor to identify effective interventions for reducing prejudice, discrimination, and hate crimes, Dr. Betsy Levy Paluck embarked on a comprehensive literature review. The review aimed to identify evidence-based strategies that could address these critical societal issues, leading to the publication of a seminal paper in the field.

The speaker emphasized the need for more experimental and causally identified work in prejudice reduction studies. They called for studies conducted in the field with the actual targets of change involved. This call was made in 2009 and was revisited to assess the literature's response.

The paper aimed to investigate the developments in prejudice reduction research over the last twelve years. It focused on the average effects of interventions, specifically experiments from the past twelve years, and addressed the public's call to reduce prejudice.

There has been an increase in prejudice reduction research over the years, with some studies becoming potential classics in psychology. However, the majority of research differs significantly from these landmark studies, raising concerns about their trustworthiness and the lack of support for popular prejudice reduction ideas.

The most rigorous research from the past twelve years indicates only small reductions in prejudice. This finding challenges the effectiveness of current prejudice reduction strategies and highlights the need for more robust interventions supported by strong evidence.

The speaker plans to delve into the details of how they arrived at their conclusions, discuss the next generation of prejudice reduction research, and question the current model of change. They aim to provide insights on the direction prejudice reduction research should take in the future.

Field experiments from the 2015 era onwards have been increasing at a rate of about one to three per year. The meta-analysis conducted followed biomedical standards, ensuring transparency in the analysis process. Over 16,000 non-unique results were obtained from five separate databases, narrowed down to about 1800 for inclusion in the study.

The study defined prejudice broadly as animus and excluded sexism to align with previous meta-analyses. Partisan prejudice and collegiate bias were also excluded. The final sample consisted of 418 experimental studies that were coded quantitatively and qualitatively to analyze intervention effectiveness and motivating theories.

Certain studies stood out for their robust interventions, detailed descriptions, and realistic implementation methods. They used large sample sizes, measured behavior, and employed correct econometric methods to ensure accuracy. These studies, led mostly by doctoral students, showed promising outcomes despite limited observations.

Many studies, including those led by doctoral students like Matt Lowe in India and Sama Musa in Northern Iraq, showcased innovative interventions. These studies, often focusing on sports leagues as context interventions, highlighted the younger generation's active role in research and experimentation.

Researchers conducted a study in Northern Iraq involving Muslims and Christians to test the contact hypothesis, exploring if cooperation during contact is necessary for positive outcomes. Surprisingly, previous studies had not thoroughly tested this aspect, leading to a gap in understanding the effects of contact between different groups.

Kevin Munger from Columbia University utilized bots on Twitter to confront individuals using racist language, specifically the n-word. By assigning bots with varying statuses and identities to address racist behavior, Munger aimed to analyze the effectiveness of confrontation in reducing recidivism and changing behavior over time.

Edward Chong collaborated on a field experiment with a global corporation to assess the impact of diversity training. The study focused on the effects of a one-off online training on women and people of color within the organization, revealing unexpected outcomes among groups not traditionally targeted by such initiatives. The study also emphasized measuring behavioral changes post-training through initiatives like diversity coffee hours and mentorship programs.

The research presented a diverse range of international and domestic studies, highlighting various intervention approaches. Contact hypothesis testing, confronting racism on social media, and diversity training were among the key areas explored. The data illustrated a significant emphasis on studying contact but noted the variation in approaches and outcomes across different interventions.

A significant portion of prejudice reduction studies in the last twelve years focus on contact interventions. The literature suggests that face-to-face contact is no longer deemed necessary, with extended and imagined contact methods being utilized more frequently.

Intervention studies primarily fall into three categories: contact interventions, cognitive and emotional training, and social categorization. Contact interventions involve imagining conversations with out-group members, while cognitive and emotional training focuses on reshaping thoughts and feelings about groups. Social categorization aims to redefine group boundaries.

Modal research predominantly centers on prejudice reduction through mentalizing. These interventions aim to change individuals' mental and emotional responses towards prejudice, emphasizing internal transformation over external factors like context, interactions, or institutions.

Light touch interventions, lasting 15 minutes or less, are prevalent in studies, comprising 76% of interventions. These brief interventions are cost-effective and easy to implement, often focusing on quick and simple methods for prejudice reduction.

Modal research faces methodological challenges such as small sample sizes, high attrition rates, inadequate analysis accounting, and lack of transparency. These issues hinder the accurate calculation of effect sizes, posing difficulties for meta-analyses and overall research reliability.

Studies predominantly measure outcomes related to explicit attitudes, beliefs, and self-reported behaviors. The focus remains on race and ethnicity as key indicators of prejudice, with an emphasis on intentions rather than actual behaviors.

An emergent category of prejudice identified in the discussion focuses on immigrants, asylum seekers, and refugees. This category is observed to align with current societal trends.

The average effect size discussed is d equals 0.357 with a standard error of 0.02. This effect size is explained in the context of a feeling thermometer, where a minor prejudice below neutral at 40 would increase to 48. The discussion highlights the positive direction and relatively small interventions resulting in a noticeable directional change.

The analysis delves into publication bias concerning sample sizes in studies. It is noted that smaller sample sizes, particularly in the bottom quintile of 25 people or less, exhibit an average effect size of 0.6, almost twice the overall average. Conversely, larger sample sizes of 78 people or more show a significantly lower effect size of 0.187. This linear relationship between sample size and effect size indicates potential publication bias favoring larger effects.

The discussion addresses the empirical claim regarding light touch manipulations leading to substantial effects. It is suggested that further testing of this claim can be conducted within the dataset, particularly focusing on longitudinal studies and other study characteristics to explore the potential for larger effects with more forceful manipulations.

The top quintile average effect size for prejudice reduction is d equals 0.187, which would elevate moderate prejudice from 40 to 44. This effect can be achieved with a light touch, indicating a modest improvement.

The small effect size for prejudice reduction remains consistent across different types of studies, including field studies, lab studies, and online studies. Even when categorized by intervention type, the effect size remains relatively low, with many studies showing results close to zero.

Restricting the sample to studies with 70 or more participants reveals a scarcity of research on diversity trainings. Only two studies in the past twelve years have sample sizes exceeding 78, highlighting a lack of robust evidence in this area.

There is no discernible trend in effect sizes for prejudice reduction interventions over the last decade, despite significant reforms in the field. This lack of variation suggests a consistent pattern of modest effects across studies.

Many popular prejudice reduction strategies, including diversity training and implicit bias interventions, lack robust empirical support. The limited number of studies with substantial sample sizes and rigorous methodologies casts doubt on the effectiveness of these interventions in changing behavior.

Only two studies in the entire database have successfully measured both implicit bias and behavior change outcomes, highlighting the difficulty in establishing a clear link between prejudice reduction interventions and actual behavioral changes. This lack of comprehensive data hinders the ability to draw definitive conclusions about the effectiveness of such interventions.

Contrary to the belief in recursive effects leading to larger impacts over time, the current evidence does not support the idea that small initial changes in attitudes or behaviors will necessarily grow or self-reinforce over time. The notion of cumulative effects from small changes remains unsubstantiated in the psychological literature.

The study aimed to test the effectiveness of prejudice reduction interventions by analyzing five outcomes from various studies. Authors were asked to identify their most significant outcomes, focusing on longitudinal data when available. However, due to limited studies collecting longitudinal data, there was insufficient data for meaningful analysis. The findings do not support claims of the effectiveness of diversity training or the cumulative impact of small changes, highlighting the need for more empirical evidence in social psychology interventions.

The research revealed very small average effects, with an average effect size of 0.187. This finding raised doubts about the self-reinforcing and cumulative nature of small changes over time. The lack of evidence for significant effects challenges assumptions about the effectiveness of diversity training, emphasizing the empirical nature of the question and the need for more data to draw conclusive claims.

The analysis suggests that improving methodological rigor and transparency alone may not lead to more impactful results in prejudice reduction interventions. While advocating for better methods and larger sample sizes is important, the regression analysis indicates that solely focusing on these aspects may diminish effect sizes, potentially rendering interventions ineffective over time. The collective impact of current interventions may not be positive, urging a reevaluation of research approaches for prejudice reduction.

The discussion shifts towards outlining the characteristics of the next generation of prejudice reduction research. Suggestions include enhancing fieldwork and laboratory studies, emphasizing the importance of both approaches in advancing the field. The paper provides recommendations for researchers interested in conducting fieldwork or laboratory experiments to improve the quality and impact of future studies in prejudice reduction.

The current model of change, as characterized in the discussion, focuses on individualized psychological interventions to address issues like racism, religious inequity, and biases within institutional structures. Researchers commonly view the problem as a psychological one, with solutions aimed at creating individual and societal change through psychological interventions.

An alternative model of change proposed in the discussion suggests shifting focus towards structural interventions to address psychological problems. This approach aims to create individual psychological change alongside societal change by targeting institutional structures, rules, leaders, and collective experiences. The idea is to move away from individualized psychological interventions and explore the impact of broader structural changes on societal issues.

The discussion delves into the concept of social structures as levers for changing mass collective experiences. It highlights the importance of unofficial organizations, mass media, large gatherings, and online platforms in shaping collective experiences. Examples like 'Black Twitter' are cited to illustrate how unofficial organizations can influence societal perceptions and norms, showcasing the potential impact of social structures on broader cultural shifts.

The conversation acknowledges the challenges in integrating structural interventions into psychological and behavioral theories. While some theories like social norms theory offer insights into how leaders and institutions can signal new social norms, there is a need to expand theoretical frameworks to encompass structural interventions more comprehensively. The discussion emphasizes the importance of moving beyond individualized approaches towards understanding and evaluating the effects of broader structural interventions on societal issues.

Perspective taking theory, exemplified by the prejudice reduction space, predicts that understanding group members' perspectives can reduce prejudice. An example is the Brockman and Cala canvassing intervention, where canvassers engage in deep conversations about discrimination based on identity, sharing stories related to issues like transgender rights and abortion rights.

The Brockman and Cala canvassing intervention is considered a structural intervention as it involves engaging with many individuals in a neighborhood, impacting collective change. Economists, particularly economic historians, have studied how long-standing institutions like slavery and reservations influence psychology and implicit bias, showcasing the impact of structures on individual psychologies.

Psychologists and behavioral scientists need to enhance their skills in understanding the structural aspects of theories and designing structural interventions. Collaboration is essential due to resource constraints. Research on backlash is crucial for understanding how different individuals perceive top-down changes in structural equity reform efforts.

The speaker discusses various justifications for structural changes, including values like justice and equity. Another justification mentioned is the need to study how these changes impact psychology, beyond immediate outcomes like demographic composition and salaries.

The speaker emphasizes the importance of using different causal identification strategies, such as regression discontinuity, instrumental variable analysis, and natural experiments. They highlight the need for psychologists to be involved in understanding the implementation of changes for prejudice reduction research.

The speaker suggests that future research on prejudice reduction should involve qualitative process training to understand the impact of changes. They stress the importance of having psychologists at the table to analyze the effects of significant interventions.

A participant expresses appreciation for the speaker's work and raises concerns about potential biases in research outcomes. They question whether the focus should be on eliminating bias or on addressing real-world disparities, such as the racial achievement gap.

The speaker emphasizes the importance of psychology in developing algorithms, highlighting that humans, particularly computational and machine learning scientists, play a crucial role in creating algorithms. Despite advancements in algorithmic suggestions, human interaction with machines remains significant, indicating the continued relevance of human involvement in algorithm development.

The discussion delves into the impact of human decisions on accepting algorithmic suggestions. While some decisions are non-negotiable, the speaker stresses the importance of engaging critical decision-makers to commission work that can lead to changes in algorithms and institutional structures. This approach aims to promote equity outcomes and address bias in decision-making processes.

The speaker suggests a strategic approach to targeting prejudice reduction, emphasizing the need to involve critical decision-makers in promoting equity outcomes. By focusing on bias reduction and engaging with the public through popular referenda and policy efforts, the speaker advocates for a comprehensive approach to achieving racial justice and equity.

Addressing ethical concerns in interventions, the speaker acknowledges the lack of evidence for certain theories and interventions. Despite the absence of convincing evidence, the speaker emphasizes the importance of ensuring interventions do not cause harm. The broad nature of interventions complicates general comments, highlighting the need for careful consideration of potential ethical ramifications.

Discussing the scale of institutional interventions, the speaker acknowledges the time-consuming nature of changing institutions, as suggested by economic theories. With the potential for interventions to take centuries or millennia, the speaker emphasizes the importance of identifying access points for interventions at institutional levels. This approach aims to address long-term systemic changes while navigating the complexities of institutional transformation.

More research is needed in the context of places where interventions are conducted to measure outcomes accurately. Despite legal threats, corporations, universities, and governments require interventions to address legal issues. Directing people towards empirically supported interventions is crucial, but significant work is necessary in this area.

Psychologists should not focus on designing interventions but rather contribute to the design process. They can provide valuable research to enhance intervention design. Instead of quick fixes, psychologists should participate in designing substantial interventions like creating safe spaces in public infrastructure based on psychological insights.

The duration of intervention effects is a key question. Studies are needed to understand why some interventions have lasting effects while others do not. Possible explanations include active processing of information (system two processing) or the novelty of experiences leading to better retention of information.

There are two perspectives on the durability of interventions discussed in the conversation. One perspective is less optimistic, suggesting that interventions may not have a lasting impact due to the lack of focused communication on issues like xenophobia or anti-immigrant sentiments. The other perspective is more pessimistic, focusing on external events like press coverage of a salient event, which could displace the effects of interventions. The discussion highlights the importance of considering the broader national political context and the potential displacement of intervention effects by external events.

The conversation delves into the concept of stress testing change and the endurance of interventions. It references a study where people were shown attack ads, revealing initial faltering but long-term endurance of change. This aspect of enduring change is considered impressive and convincing, showcasing the effectiveness of interventions over time. The discussion emphasizes the value of listening to practitioners for innovative ideas and the importance of theoretical testing grounded in practical experiences.

The conversation shifts to measuring prejudice and policy-related conversations, drawing parallels to discussions on morality and morally complicated issues. It distinguishes between basic and policy-related measurements of prejudice, highlighting the impact of conversations and interventions on shaping attitudes towards policies like immigration. The dialogue explores the idea of changing moral views as a path towards supporting more inclusive policies, blending pessimistic and optimistic views on addressing ingrained prejudices through cultural toolboxes and constructive dialogues.

Discussions can lead to lasting effects by providing individuals with a path to resolution on complex issues like transgender rights. This resolution goes beyond relying solely on 'feeling thermometers' and can contribute to enduring changes in attitudes and beliefs. More long-term measurement of such effects is needed to understand their impact.

There is a need to reconceptualize prejudice and structural change in psychology. Instead of focusing solely on individual beliefs and emotions, it is crucial to examine how societal structures either support or push against lasting effects. Incorporating socio-historical context in research can ensure that previous findings remain relevant and contribute to generative theory-making for future generations.

Psychologists need to understand racial inequity, discrimination, and violence. This involves exploring various psychologies beyond feeling thermometers. Research has delved into perceptions of community, acceptance of hate crimes, and prejudice as motivators for committing hate crimes.

Describing the context of an intervention shifts the focus to identifying what extinguishes or facilitates change. It prompts a closer examination of structures to determine what suppresses or promotes equitable interactions and behaviors.

Economists found that implicit bias towards women and women leaders changed after quotas for local women leaders were implemented in Indian villages. This shift in bias occurred without explicit bias training, highlighting the influence of contextual factors on attitudes and behaviors.

The discussion resonates with the speaker, who appreciates the insights shared. There is mutual agreement on the importance of exploring how psychologies and contexts facilitate change and influence behaviors. The speaker expresses admiration for the conversation and looks forward to future interactions.

The speaker acknowledges the valuable contribution of Betsy Levy Paluck to the discussion. They express gratitude for her participation and the enriching insights she provided during the conversation.