Why Men Got Picked Over Women in a Blind Review of Science Grants

A new study finds men use broad language in grant proposals that leads them to win more science funding. (Images credit: National Institute of Environmental Health Sciences, National Bureau of Economic Research)

A recent study of a science grant application process at the Bill and Melinda Gates Foundation found male applicants received higher scores than women, even in a blind review. At the foundation’s request, a team from the U.S. National Bureau of Economic Research analyzed this imbalance and reported that factors like scientific discipline and position, publication record, and grant history were not factors — the main difference was in the language used in proposal titles and descriptions. According to their working paper, men were found to use more words described as “broad,” while women chose more words labelled “narrow.” The broader word choices were preferred, especially by male reviewers. But, as in most research relating to complex issues of sex, bias and language, the story is more nuanced.

The Use of Common and Broad Words by Gender in Grant Proposals

The researchers examined 6,794 two-page proposals submitted by U.S.-based researchers between 2008 and 2017 to the Gates Foundation’s Grand Challenges Explorations program, which gives out grants of between $100,000 and $1 million in the field of global health. Kuheli Dutt, who works in academic affairs and diversity at Columbia University, pointed out to Nature that the trend of men using broader terms might align with other research that finds men more likely to overstate their performance. As we recently covered in the world of dance, men are also sometimes more likely to self-advocate in professional situations. These trends can be tied to cultural gender norms — beliefs about how men and women should behave.

Within the exploration of the language differences between men and women, researchers used specific interpretations of broad and narrow language. Broad words were those used commonly across many topic areas, while narrow words were used frequently only within specific content areas, such as HIV or malaria. This frequency-of-use system led to some results that might be surprising. For example, “bacteria” was counted as broad, while “community” was marked narrow.

Within this analysis, men employed more broad, or more common, words and received more grants. But these language choices did not lead them to have greater success after the awards were given. When women secured grants, they generally outperformed men in terms of post-funding publication and future funding.

Julian Kolev, assistant professor of strategy and entrepreneurship at Southern Methodist University’s Cox School of Business and lead author, said he and his team “would be hesitant to recommend that women adopt this language… The narrower and more technical language is probably the right way to think about and evaluate science.”

As more tools that automatically analyze text become available, linguistic and cultural research is on the rise. Recent studies have turned up somewhat divergent results — an earlier paper in 2019 found grant abstracts that are longer, contain fewer common words and “are written with more verbal certainty” received more National Science Foundation funding.

Potential Bias in the Review Process

Kolev thinks organizations should spend more time looking into potential reviewer biases. He suggests reviewers could be trained to be more aware of and less influenced by communication style differences. Also, he said, “We consistently show that female reviewers’ scores do not favor proposals from male applicants in the way that male reviewers’ scores do, so increasing the number of female reviewers is one potential way to mitigate the effects we find.”

The Gates Foundation uses a “champion-based” review process for these awards, in which applications are more likely to be chosen if they’re given a single high review. According to Science Magazine, the reviewers are from “a variety of disciplines and perspectives” and have “less-specialized expertise” when compared to an organization such as the National Institutes of Health (NIH). Professor of Economics at the University of Kansas Donna Ginther, who studies disparities in NIH grant funding, said Gates reviewers might, therefore, be “susceptible to grantsmanship like claims, like ‘I’m going to cure cancer’ as opposed to ‘I’m going to understand how this molecule interacts with a cell.”

Ginther hopes this type of linguistic analyses will be used to study other facets of diversity in science (racial disparities are present in NIH funding). And, given that subcultures, socioeconomic experience, age, gender identity (as opposed to biological sex) and many other factors can affect our language choices, there is plenty of territory left to explore. This case certainly raises some interesting questions about foundation grant review processes.

In a written statement, the Gates Foundation said it is “committed to ensuring gender equality” and “carefully reviewing the results of this study—as well as our own internal data—as part of our ongoing commitment to learning and evolving as an organization.”

What’s Next for Women’s Philanthropy? Funding Collective Impact for Gender Equity

Editor’s Note: Betsy McKinney, Founder and CEO of It’s Time Network and author of this post, was recently invited to speak at an event in honor of Women’s History Month at the U.S. State Department. She gave an overview on the need for collective impact infrastructure and initiatives in the women’s sector, and explained the purpose of It’s Time Network and the Network City Program.

Everyone responded vigorously during the presentation when Betsy said that we need a collective impact structure that acts as an AARP for women, and that we can and should fund it ourselves as women over time. People also responded well to the need for shared measurement and the Women’s Well-Being Index. At the end, women from Malaysia, Nepal and Afghanistan asked how they can join the Network City Program. Betsy gave them copies of ITN’s Mayors Guide and they are eager to consider how they can also use the guide and recommendations.

After the unprecedented success of the Women’s Marches, everyone is asking, “What’s next?”

It’s time to build and fund women’s collective power at the city, state and national levels and beyond.

While writing postcards to members of Congress, donating to women’s organizations, participating in online petitions, and running for office are all critically important individual actions that woman can take, we need to consider long-term, collective action as well. Collective action requires that we connect in new ways to build common agendas, work together more effectively and track progress (and regression) in the areas that matter to us as women.

Betsy McKinney, Founder and CEO of It’s Time Network

This work is not a sprint. It’s a marathon that we can “train” to achieve sustained impact in addition to short-term milestones. It’s time to build and fund network infrastructure at the local, state and national levels to support robust cross-sector collaboration and achieve the outcomes that are possible through collective impact work.

The Stanford Social Innovation Review popularized the theory of collective impact and notes that large-scale social change requires broad cross-sector coordination, yet the social sector remains focused on the isolated intervention of individual organizations. Currently, people working in environmental issues are often separate from social and racial justice leaders and many organizations are still too isolated. That isolation is a result of both segmented issues and incentive structures that lead to competition for limited resources. Most organizations compete for funding from the same sources and find it hard to collaborate with other organizations even when they have common interests. Without a permanent structure for supporting collaboration within an issue area or even across issues, such efforts are often only temporary campaigns for one specific goal rather than sustained coordinations.

The Network City program is building out gender equity from the local level, starting with Denver and San Francisco.

At It’s Time Network we are building a national Network City Program to create the capacity for collective action beginning at the local level. With two pilot cities, in San Francisco and Denver and as more cities join the network, the capacity for collective action can begin to scale to the state and national levels. Additionally, each city and or state has international organizations that can join the network, which strengthens our global connections as well.

The work ahead lies beyond simple partisan divides. People from every part of the political spectrum are waking up and exercising their civic muscles. It’s not just about women and women’s rights, either. It is pro-democracy, pro-“justice for all”, pro-equality, pro-inclusion, and pro-love and non-violence. It’s about building bridges. Women have an important role to play in healing divides, modeling cooperation, and leading truth and reconciliation processes. Women are actively building inclusive, compassionate communities that can work together.

While it’s imperative for women to respond to immediate concerns in our world, the next steps must also identify and establish what we want and how to achieve it.

The Mayors Guide to Accelerating Gender Equality is part of a dashboard of tools for our Network City Program that details solutions. The guide is a readily accessible “toolkit” that provides recommendations, resources, and a checklist of actions a city can take in 11 different issue areas to improve the lives of women and girls and to strengthen communities. It is a tool for sharing best practices from city to city, and currently, the guide is being used to build a common women’s agenda for Denver. This spring, It’s Time Network is partnering with the Denver Office on Women and Families and the Mayor of Denver to produce  It’s Time 2017: Denver Gender Equity Summit on May 31st.

Getting clear about exactly what we want is important as we use data to understand the current status of women to inform any actions that we take. What is the current status of women? And how can we meaningfully compare our circumstances from one geography to another? The California Women’s Well-Being Index is an important new tool for comparing the status of women county by county across the state. Developing this tool and creating a Well-Being Index for every state is critical for using data to inform our work together. As we identify areas of greatest need in each state, strategic collective impact initiatives can be designed to engage diverse organizations and stakeholders, and to support collaboration across sectors and among non-profits, business, government, private donors and others. By being data-driven and with tools to measure goals and outcomes, we can achieve long term change and impact. The Network City Program taken to scale, will be a powerful organizing structure for women to use in every community to ground the immense power and passion that has arisen over the past few months.

Building and maintaining a robust national collective impact infrastructure requires transformative funding. This work has been designed, is being piloted and is ready to go to scale. While it’s critical to fund the further development of this program, it is equally important to ensure that the long term funding of this work is a “collective ownership model” and is not forever reliant upon outside funding.

It’s time for women to “own our power” and to own the infrastructure and services that support us. With an innovative funding model, It’s Time Network is pioneering the concept of “women’s collective economic independence.” We cannot rely long term upon the government, corporations, large foundations, or even large private donors. The initial support they give is essential to seed this work, and women can and will always work with these vital funding partners and allies. Yet, it’s critical for us as women to grow the number of women who become participants in the national network so we can build our micro-funding capacity. We can and must rely upon ourselves and build a culture of women’s collective independence from generation to generation.

The Women’s Future Fund at It’s Time Network is part of the ownership model for building women’s collective economic independence. Growing this unprecedented national, collective asset, is tied to the growth of the Network City Program to ensure a distributed decision making model with diverse women leaders at the grassroots (inclusive of women’s city or state foundations) determining the allocation of funds, city by city.

Women have an essential role to play in what’s next not just after the election, but in all aspects of decision making about our world. Anxiety is high as the global challenges seem daunting. As women, we must ground our efforts in stable, loving, creative and collaborative actions that demonstrate our ability to heal and transform our world. It’s time to build our partnerships, grow our collective capacity and promote a vision of a world that we know is possible.

Betsy McKinney is the Founder and CEO of It’s Time Network

Belief-based Social Innovation: Gender-Lens’ Next Frontier | Stanford Social Innovation Review

Coming up soon on Inside Philanthropy: interview with Emily Nielsen Jones, co-founder of the Imago Dei Fund! To get you started on understanding this amazing leader, check out this article she co-wrote with Musimbi Kanyoro about new ways funders are using a gender lens to choose where to put their money. From SSIR:

Philanthropists and for-profit investors alike today are apt to talk of using a gender lens when screening opportunities to fund social change. When my husband and I (Emily) began our foundation—the Imago Dei Fund—in 2009, I gravitated immediately to the idea of empowering women and girls. Little did we know then that it would grow into a powerful movement changing the face of philanthropy.

At the cusp of a new round of global gender goal setting, we find ourselves asking: Where is the gender-lens movement going, which now takes as conventional wisdom that gender balance is a lynchpin of global progress? The answer lies in moving beyond redress, mitigation, and even women’s empowerment programs—though these are still sorely needed—to more directly fund culturally led efforts to re-examine and transform underlying beliefs that systematically disempower females in the first place.

Source: Belief-based Social Innovation: Gender-Lens’ Next Frontier | Stanford Social Innovation Review