Diversity-Innovation Paradox
I’ve already noted the large experimental research literature on the role of diversity in creative problem solving and collective intelligence. In fact, within science itself, simply adding new collaborators increases originality, innovation, and multidisciplinary impact, in part by restoring the allostatic tension between trust and novelty. Research like this extends these findings well outside of lab-controlled experimental conditions, and this phenomenon extends beyond scientific innovation. Many years ago, I was on an advisory board for Credit Suisse, the Swiss banking giant, when we released a report showing that corporate boards with at least 1 woman had 3% greater return to shareholders, and that boards with 3 or more women returned an average of 5% more47. Research tracing the cultural diversity of thousands of London companies found that diverse management teams, in terms of both ethnicity and immigration status, produced more product innovations than their competitors. Even in the unsexy world of fishery ecosystems, diverse groups better understood “complex feedbacks and interdependencies” than homogenous groups. Just as with our matchmaker framework, however, the diverse inputs of fishery stakeholders needed a proper aggregation process–simply throwing every idea and stakeholder together in a room produced no advantage.
From global corporate boardrooms to London entrepreneurs, from fishery ecosystems to scientific innovation, increased diversity brings increased value creation. Still skeptical? What if we looked at all 1.2 million doctoral dissertations published in the last 50 years? This is exactly what was done in one of my favorite papers of 2020. Using text analysis and machine learning, the authors analyzed the career trajectories of every US doctoral recipient since 197748. They found that “demographically underrepresented students innovate at higher rates”. Unfortunately, this wasn’t the paper’s only finding. Those novel contributions from underrepresented scientists were less cited than their peers’. Despite contributing greater innovation, underrepresented scientists were less recognized and had worse career outcomes, robbing us all of their future impact. This phenomenon is known as the diversity-innovation paradox and it represents a major drag on global innovation.
Untrusted
While this paradox has deep roots in history, institutions, and personalities, it is fundamentally about trust49. As I’ve repeatedly argued, trust and diversity play fundamental and opposing roles in innovation and distributed cognition. Our well documented preference for trust over diversity drives organizations into the paradox. For example, cognitively diverse teams are more effective, and yet coworkers tend to sort themselves into cognitively homogeneous groups over time50. Even when an individual has a clear incentive to share information with a collaborator, they share less with individuals that are more culturally dissimilar. Left unchecked, these biases foment cultural barriers that suppress cognitive and broader diversity.
As cultural norms begin to favor homogeneity, individuals begin to conform to the norms of their ingroup “even when they understand that the norms…are arbitrary.” Deeper still, their emotional responses shift to align with their perceived ingroup. Although there is significant variation in how individual brains process social signals around trust51, a pattern emerges in which ingroup trust is associated with reward circuits while outgroup trust requires effortful top-down control. Additional stressors, such as time or financial pressures, increase the top-down effort needed for outgroup trust while having no impact on the reward circuits for ingroup trust. This creates another bias driving us towards homophily and undermining the prosocial behaviors necessary for success in diverse teams (and societies).
These barriers can be overcome, however, with engagement. Increased interpersonal experience shifts interpersonal trust from effortful medial prefrontal control to more reflexive (i.e., “automatized”) brain circuits. Simple engagement with people of different races and genders reduces belief in stereotypes and other outgroup associations. It is only when trust and diversity are brought into balance that teams can avoid the Paradox and gain the full innovation potential of their community.
Without that balance, lack of trust dominates and robs us of capacity. For example, in many academic fields, including economics and the sciences, seminar audiences are systematically less trusting of women, asking more total questions and exhibiting more patronizing or hostile attitudes, even after controlling for differences in quality of work or field of research. In the world of entrepreneurship, investors of all genders strongly prefer pitches from attractive male entrepreneurs over female entrepreneurs, even when the pitches are otherwise identical.
Analysis of meetings between venture capitalists (VCs) and female startup founders reveal that VCs undermine female founders with questions about losses while supporting male founders with questions about gains, and later negatively describe qualities in female founders that receive praise in male founders (e.g., “lacks experience” vs. “full of potential”). This difference in trust persists despite the finding that female founders return more than twice as much per dollar invested as male founders.
The potential anonymity of remote work suggests one possible solution to these known biases: we could all hide our differences within digital personas. Behind avatars and chat handles we could all discard our identities and hide our differences. This has appeal for many, but rather than removing difference, all of the research above suggests that it would actually produce the opposite, exaggerating even subtle differences and worsening ingroup-outgroup effects. Trust grows with engagement, not anonymity.
As noted way back in “Measuring Remote Success”, employees with more facetime with managers receive more recognition and more promotions. For most, though, working remotely reduces social network centrality, meaning that the average remote worker is less present in the minds of their coworkers52. This increases the need to manage one’s impression amongst coworkers and leaders within the company. While everyone experiences this same increased need for “job-focused impression management”, managers treat this behavior more negatively in employees of a different gender. Without the reflexive trust given to those similar to you, career management looks conniving and deceitful. This even plays out in elite levels of American politics–male senators are seen more positively the more they talk, while female senators…never know when to stop persisting.
“Distributed Innovation” explored how majority influence and herding effects reduce collective intelligence and innovation, and revealed the powerful impact of minority opinion incentives. This same effect is further exaggerated in situations demanding ingroup-outgroup trust. For example, when a deliberative process uses majority rule and few women are present, women experience more interruptions when speaking and their contributions are less trusted, reminiscent of the female academics described above53. When groups are required to respect minority opinion (e.g., unanimous rule), women are actually interrupted less than men, as the incentives of the system demand an effortful allocation of outgroup trust54.
In “Controlling the Beyond-Control” I described how the attribution bias causes us to arbitrarily associate situational challenges to perceived failings of specific individuals. In the context of trust, it is clear that negative attributions become even more pernicious across group divides. Even location plays a role in exaggerating attribution bias as more distant collaborators receive more blame for failures, often hiding the real source of a failure and allowing it to persist. And for all our generic complaints of “Zoom fatigue”, challenges with gaze and space in video meetings disproportionately affect those in outgroups. Cultural differences affect how we contextualize and interpret eye contact and other displays of emotion, influencing the way people interact. For example, East Asians tend to perceive direct gaze as evidence of anger or sadness compared to Western Europeans. Even visual cues like the perceived distance or relative position of a face in an image affect perceptions of gender and emotion. And those perceptions in turn alter the neural circuitry of trust.
Even within America, factors such as differences in family size or self-expression can easily fuel attribution bias. For example, census data shows that Black families have 20% more people in a household than the average US family; Mexican-American families are 80% larger. This creates more competition of bandwidth and less private space for video meetings. That little extra background noise or choppiness in video quality can easily lead to false assumptions about competence. All of this while that same video feed intrudes into a private home of those in outgroups, exposing “formerly safe, private spaces” to scrutiny and ingroup judgement.
Simple access to remote work is a form of privilege many don’t fully appreciate. Even before the pandemic began, there were racial biases by income and industry in who was able to work remotely. Remote work also exacerbates broader socioeconomic and regional differences. Despite significant variation, remote “jobs typically pay more than jobs that cannot be done at home and account for 46% of all US wages.” Of the highest earners in the US, 71% report being able to work from home, compared to only 41% from the bottom quintile. All of these differences in access further reduce the connections across socioeconomic, racial, and regional divides. In the end, you are more likely to encounter certain kinds of people working remotely, reinforcing in our minds that elite work is largely white-ish, male-ish, highly educated, and culturally homogeneous. The very tools we use to stay connected tend to exaggerate ingroup-outgroup effects and bolster the Diversity-Innovation Paradox.
Even the received wisdom of those with remote work experience reveals deep assumptions about the nature of trust and collaboration. “Set the meeting to 15 minutes by default, and only make it longer if absolutely necessary (the shorter the meeting, the more succinct you will have to be, and the less time there will be for pointless small talk and rambling).” A culture of 15-minute scripted meetings is a culture of homogeneity. Where is the trust in the assumption that others will waste your time? This idea might hold for rote cognitive labor where there’s no need for chit chat, but in creative labor this mindset reinforces ingroup-outgroup divides and reduces innovation.
Distributed Fairness
I have spent the last several thousand words presenting evidence that diversity, combined with trust, lifts innovation and productivity. In fact, I’ve spent a good portion of the last 10 years arguing the business case for inclusion56. Scientists like to call this an instrumental argument, which is to say that there is a direct, tangible benefit to you. The instrumental argument for diversity and inclusion is real. It is more than valid. We all genuinely stand to benefit from overcoming the Diversity-Innovation Paradox. And yet, despite all of the force of this argument, it is not enough. Our choices must start with fairness first.
Fairness first means building a diverse team must come before all other considerations—it is non-negotiable57. It is not an argument about the potential business gains or the strategic value of balancing one objective against another. Fairness is about what is right. One might assume that the “rightness” of inclusion comes from its measurable value in innovation and beyond, but it is not enough to make a rational argument when so much of the research I’ve cited above reveals our deep and persistent irrationality in the face of difference. A recent study in the Proceedings of the National Academy of Science finds that instrumental arguments for diversity in universities appeal more strongly to White than Black families, and that universities that rely on instrumental arguments over fairness tend to have greater White–Black graduation disparities. When diversity and inclusion become mere considerations in a transaction, balanced against so many other pressing considerations, our long-term collective intelligence falls.
The constant demand to deliver new research, new products, new customers, new markets, places cognitive and emotional load on us all. Managing these daily stressors competes directly for the same neural resources needed to deploy outgroup trust, even as ingroup trust comes nearly automatically. Treating decisions about diversity as transactional might seem rational on its surface, one of many considerations you must balance for the good of your organization, but those same competing demands directly decrease our ability to see value in difference. This is why the Paradox persists.
If you believe, as I do, the copious research demonstrating that increasing the diversity of your organization will lead to increases in innovation and collective intelligence, you must then set that instrumental argument aside. Abandon the “business case for diversity”. It is still true. It can be the argument that moves you to act, but it cannot be the act itself. Fairness first abstracts away from the transactional nature of the business case and simply states that fairness is a foundational principle to be supported independent of other considerations.
A few years ago, I developed an interest in how Hollywood balances creativity and industry, and so I began listening to The Writers Panel, a podcast about the “practice and business” of writing for television. The host interviews a panel of writers, often including a head writer, known as “showrunners”. There were two fascinating commonalities in hiring practices of showrunners. First, nearly every one of them would say that there is an endless supply of professional writers in LA who can deliver a script, but that what they actually needed was someone who could contribute an original idea the showrunner would never have had by themselves. This has been my hiring principle for years; I thought I was so clever and unusual, but I was just rediscovering what every good showrunner in Hollywood already knew58.
Second, a number of successful showrunners would insist that they couldn’t take the risk of hiring anyone other than friends and long-term colleagues. The theoretical value of diversity meant nothing to them against the daily pressures of delivering a high-quality show every week. One showrunner proudly stated that he would happily pay studio fines rather than diversify his writers room59. Another defensively argued the value of hiring only close friends while simultaneously complaining about how he had been marginalized for his own differences. Despite the universal agreement that new ideas are the currency of their industry and the causal relationship between those ideas and diverse rooms, the Diversity-Innovation Paradox persists, even in Hollywood. In one of the most competitive, high-stakes industries in the world, only those showrunners that practiced fairness first diversified their rooms.
None of this is meant to imply that fairness first comes at no cost. Building a culture that embraces the tension between trust and diversity takes time and effort. If a diverse writers room or innovation cell is non-negotiable, something else must give–usually, time. In the tech industry, the first qualified candidate who walks through the door is very likely to look and think like everyone else already in the room. Building a diverse team requires the time necessary to find a candidate that is not only individually qualified but also raises collective intelligence through complimentary diversity60. That inevitably requires more time and more effort on the part of recruiters, hiring managers, and the team as a whole. When a recruiter has 30 hiring managers scream at them every week for not filling open positions, you can easily understand how the instrumental value of diversity falls away as a consideration. Fairness first insists that a candidate is not qualified if they don’t increase the complimentary diversity of the team.
The additional short-term costs of diversity don’t end with recruiting. Even after a new member has joined a cell, outgroup trust is effortful, and that effort will inevitably reduce the short-term productivity of that cell. Expect to put additional work into maintaining psychological safety as you remix teams and recognize that this likely means short periods of decreased innovation as the room again finds its allostatic balance between trust and diversity. The long-term payoff for the initial effort will outweigh the lazy benefits of ingroup collaboration.
Many organizations don’t allow the time needed to hire for complimentary diversity even as they talk about the importance of inclusion. It’s one thing to understand intellectually that diverse teams and communities outperform in the long-run, but in a culture obsessed with short-term gain, the upfront costs of diversity can feel like losing. Fairness first frees us to see individuals as more than tools to complete this week’s deliverables.
Of course, there are other strategies for increasing diversity. Many organizations offer incentives for ”diversity” hires or include team diversity in managers’ annual reviews. Research has shown that these strategies do have an impact, and yet the Paradox persists. It should tell us something about ourselves that we must pay people to make choices that are already in their best interest. I’ve been known to occasionally promise my kids ice cream and cartoons in exchange for a vaccine shot; doing the equivalent with grown ass adults just to build a qualified team is absurd.
Alternatively, some organizations fall back on the dirtiest word in HR: quota. On the surface, quotas might look like a form of fairness first–you must simply hire a certain number of a certain kind of person without influence from other considerations. And if people are using the q-word around your organization, there is probably a serious problem in need of a drastic solution61. But in practice, reliance on quotas isn't about fairness. Rather than making a commitment of time and effort to finding a candidate that helps build the best possible team, the appeal of quotas is that they are quick and easy. Just fill the diversity hire role. But diversity is not a quality of an individual hire, and focusing purely on individuals and not the composition of the team both ignores the importance of trust in the room and is an invitation to game the system62. Stop paying lip service to diversity. Stop making the easy arguments for diversity that ask little sacrifice and offer little gain.
Just as an individual must be qualified, teams must also be qualified. A team is not qualified if it is not diverse.
