Introduction
For decades, women around the world have struggled to overcome the various forms of gender discrimination embedded in human societies. These struggles, particularly those organized through feminist movements, have contributed to varying degrees of progress toward gender equality across different social contexts. Nevertheless, women continue to face discrimination due to the persistence of gender bias in numerous social practices and across multiple areas of human interaction, including employment, healthcare, and law enforcement.
As digital technologies evolved and became increasingly integrated into economic and social activities, many women anticipated that technologies less susceptible to human prejudice would create broader opportunities for gender equality. These expectations grew further with the rise of artificial intelligence (AI) and its incorporation into decision-making processes across a wide range of sectors. AI systems appeared capable of making decisions based on objective factors, supported by real-world data, and processed by computational logic that is not directly shaped by the socialization processes through which gender bias is traditionally reproduced.
Practical experience with AI systems, however, has demonstrated that these models can themselves carry gender biases that reproduce discrimination against women in various aspects of everyday life. As AI becomes increasingly influential in decision-making, addressing such bias has become more urgent, particularly as reliance on these technologies is expected to play a central role in future economic and social development. Consequently, continued progress toward gender equality is closely linked to the ability of institutions, developers, and policymakers to understand gender bias in AI systems and to develop effective mechanisms for mitigating it.
This paper examines the phenomenon of gender bias in AI models by defining the concept and exploring its principal manifestations, immediate effects, and broader consequences. It also analyzes the underlying causes of such bias, investigating how the development, training, and operational functioning of AI systems can contribute to its emergence or perpetuation.
In addition, the paper reviews different approaches to addressing gender bias in AI models. It discusses gender-blind and pre-processing correction approaches, followed by post-processing mitigation strategies applied during model development and deployment. Finally, it considers the potential for more comprehensive approaches that expand the scope of meaningful human intervention.
Identifying Gender Bias in Artificial Intelligence Models
Gender bias, along with other forms of discrimination, received limited attention during the early stages of AI development. A prevailing assumption held that the absence of direct human intervention in AI decision-making would be sufficient to ensure neutrality and objectivity. However, real-world applications have demonstrated that AI systems can produce decisions and outputs that reflect biases embedded in the data they rely on or in how they are designed and deployed.
This section explores the various manifestations of gender bias in AI systems and highlights several prominent real-world cases that have drawn attention to the issue.
What Is Gender Bias?
Bias is generally associated with decision-making processes. It can be defined as a systematic error in decision-making that leads to unfair outcomes. The term “systematic error” does not refer to an isolated mistake, negligence by a specific individual, or an obvious defect in the underlying data, such as incomplete or inaccurate information. Rather, it refers to errors arising from the methodology of the decision-making process itself or from the broader system in which decisions are made.
In human decision-making, bias often stems from deeply rooted assumptions and beliefs that shape individuals’ judgments, whether consciously or unconsciously. These assumptions form part of people’s cognitive and perceptual frameworks and therefore influence how they approach and evaluate decisions. At the same time, such assumptions are reinforced by prevailing social and cultural environments, becoming integral components of the systems within which decisions are made.
Gender bias is among the most widespread forms of bias across human societies and, at the same time, among the most difficult to recognize. Assumptions about gender differences are often so deeply embedded in socially inherited beliefs that their inaccuracies may go unnoticed. As a result, these assumptions routinely influence decision-making at multiple levels. They are continuously reproduced through everyday practices, public discourse, and both ordinary and creative forms of expression. This process relies on what may be described as self-reinforcement, whereby the phenomenon itself generates conditions that contribute to its persistence and further entrenchment.
The long history of women’s exclusion from certain occupations illustrates the self-reinforcing nature of gender bias. Such exclusion resulted in lower rates of female representation in those professions, which subsequently became reflected in statistical records. Over time, these statistics themselves shape reality: they influence women’s willingness to pursue such careers and affect employers’ perceptions of women as suitable candidates. In this context, neither party needs to be consciously biased. Simply relying on existing statistical patterns can lead to decisions that unintentionally reproduce discrimination.
This dynamic helps explain why many initially assumed that AI systems would be immune to gender bias. If human bias arises because individuals develop their perceptions within socially constructed frameworks, it may seem that AI systems—lacking consciousness and socialization—would not be vulnerable to the same influences. However, examining the self-reinforcing nature of gender bias reveals a parallel. Human consciousness develops through exposure to accumulated social experiences, while AI systems learn through what might be described as data-driven iterative programming based on historical patterns. This analogy provides a simplified way of understanding the concept of machine learning in artificial intelligence.
Manifestations of Gender Bias in Artificial Intelligence Models
The widespread adoption of AI models across multiple sectors is closely tied to their ability to process vast amounts of data and support decision-making with a high degree of efficiency. Some sectors were early adopters of these technologies, including recruitment and hiring, career guidance, credit approval, and assessments of prisoners’ eligibility for early release.
Many organizations assumed that the greater efficiency of AI systems would also translate into greater neutrality. This assumption emerged in fields that had already faced criticism and legal challenges due to biases in decision-making processes, whether based on gender, race, or other characteristics. Consequently, one of the key motivations for adopting AI systems in these domains was the belief that they would be less likely to reproduce existing forms of discrimination.
However, the growing use of these systems has also revealed evidence of gender bias within them. Individual cases emerged whose outcomes were difficult to explain without considering the possibility of gender bias, while large-scale deployment generated sufficient data to identify statistical patterns indicative of discriminatory outcomes.
A notable example is described in the article “When Good Algorithms Become Sexist: Why and How to Advance AI Gender Equity.” The author recounts her experience of applying for credit cards together with her husband through the same credit provider. Although her credit score was slightly higher and her income, expenditures, and debt levels were comparable to her husband’s, she was assigned a credit limit roughly half the size of his. The article also references a widely publicized incident involving Apple’s credit card service, where a husband discovered that he had been granted a spending limit approximately twenty times greater than that assigned to his wife.
Examples of gender bias in AI systems have been documented across a variety of fields. Among the most prominent are the following:
Recruitment Algorithms
Major companies around the world have adopted AI-powered tools to review and evaluate job applications. One of the most widely cited examples is Amazon, which discontinued an AI recruitment system in 2018 after discovering that it systematically favored résumés submitted by men for technical positions.
Diagnostic Systems in Healthcare
Statistical evaluations of AI-assisted diagnostic systems have shown that these tools tend to be less accurate when assessing women than when assessing men. Studies have also found that inaccuracies are particularly pronounced for women of color.
Facial Recognition Technologies
AI systems used in facial recognition applications exhibit higher error rates when identifying women than men. Similar inaccuracies have been observed among individuals from non-white ethnic backgrounds. As a result, women may be disproportionately exposed to false identifications, including cases in which they are mistakenly matched with individuals sought by law enforcement authorities.
Generative Artificial Intelligence
Generative AI systems can produce textual, visual, and audiovisual content that reinforces gender stereotypes. Image-generation tools such as DALL·E and Stable Diffusion, for example, have frequently depicted men in roles such as doctors, executives, and professionals, while portraying women primarily as homemakers, nurses, or caregivers.
Forms of Gender Bias in Artificial Intelligence Models
Gender bias in AI systems can generally be understood through three principal forms.
The first involves decisions that directly affect individuals on the basis of gender, thereby reproducing discrimination against women and undermining equality of opportunity in areas such as employment and access to services. These decisions often reflect stereotypes that remain prevalent across many societies. Such stereotypes include assumptions that women are less capable than men of performing certain jobs, particularly those requiring advanced cognitive skills in fields such as science, mathematics, engineering, and technology. They also encompass beliefs that women are less suited to positions involving significant decision-making authority, including senior management and leadership roles. Similar stereotypes portray women as less responsible than men in financial matters and consumer spending.
The second form may be described as gender erasure. This occurs when women are inadequately represented—or entirely absent—from the data environments within which AI systems operate. As a result, outputs relating to women tend to be less accurate and more susceptible to error.
The third form involves the reproduction and amplification of existing gender bias. Through their operation, AI systems may generate data and content that reinforce prevailing gender stereotypes, while also contributing to further gender erasure through inaccurate outputs concerning women. This creates a self-reinforcing cycle in which each new process relies on the outputs of previous ones, whether generated by the same model or by other AI systems. In the absence of effective mechanisms for detecting and correcting bias, this cycle allows discrimination to persist and expand over time.
Effects and Consequences of Gender Bias in Artificial Intelligence Models
Gender bias in AI systems produces immediate and tangible consequences for individual women whenever important decisions are delegated to AI-driven processes. These consequences may include denying women employment opportunities for which they are objectively qualified, exposing them to health risks due to inaccurate medical diagnoses, or limiting their access to rights and services.
Women may be denied early release from prison because of biased risk-assessment systems, or they may face discriminatory treatment in obtaining credit facilities and financial services despite having qualifications and evaluation profiles equivalent to those of men.
These direct harms represent a continuation of broader patterns of gender discrimination that already exist in society. However, when discrimination is reproduced through digital systems, it acquires additional dimensions. Decisions made by computer systems are often perceived as more objective and impartial than decisions made by human beings because they are assumed to operate according to mathematical logic. Consequently, individuals seeking to challenge discriminatory outcomes may encounter the argument that the decision was produced by a neutral technological system rather than by a biased human actor.
This perception also complicates the question of accountability. Even when discriminatory effects are evident, responsibility may be dispersed among multiple actors, including the organization using the model, the company marketing it, and the developers who designed it. In the absence of clear regulatory standards governing the development, deployment, and use of AI systems—particularly standards addressing bias mitigation—it becomes difficult to identify a legally accountable party.
Moreover, AI systems lack the flexibility that allows human decision-makers to act on the basis of ethical awareness, institutional commitments, or legal obligations. Human decision-makers may consciously seek to avoid gender discrimination, organizations may adopt anti-discrimination policies, and governments may enforce laws prohibiting unequal treatment. AI systems, by contrast, possess no independent awareness of such considerations. They can only account for them when these considerations are explicitly or implicitly represented in the data they process or in the rules governing their operation.
Beyond these direct harms, gender bias in AI systems also generates broader long-term consequences. One of the most significant relates to the future development of AI itself. Decisions produced by existing models, along with the content generated by them, increasingly become part of the data used to train future systems. In this way, gender bias can be reproduced and amplified while its origins become progressively more difficult to trace.
The interaction between society and AI has entered a new phase, particularly with the widespread availability of large language models and generative AI systems through conversational applications, search engines, and numerous digital platforms. These technologies are no longer confined to specialized institutional tools used for specific operational purposes. Instead, they now engage daily with millions of people around the world through text, images, audio, and video.
Many users interact with these systems in ways that increasingly resemble social interactions with other human beings. Individuals frequently turn to AI systems for information and advice and may be predisposed to accept their outputs without sufficient critical scrutiny.
As a result, biases embedded in AI-generated content can be transmitted indirectly to users and internalized without their full awareness. When interacting with other people—even those perceived as experts—individuals generally recognize the possibility of personal bias and may critically evaluate what they hear. When interacting with AI systems, however, users may not assume that similar biases are present, which can weaken their critical engagement with the information provided.
For this reason, AI systems possess a potentially significant capacity to shape users’ attitudes and perceptions, although the precise extent of this influence remains difficult to measure. The impact is likely to vary according to factors such as age, educational background, and social environment. In the context of gender bias, the incorporation of discriminatory assumptions into AI-generated discourse can reinforce existing prejudices and interact with broader social influences that contribute to their persistence.
Understanding and Analyzing Gender Bias in Artificial Intelligence Models
Sources of Gender Bias in Artificial Intelligence Models
The factors that shape an AI model can be understood, in simplified terms, by viewing the model as a digital system that processes data through a set of computational operations. What distinguishes AI systems from conventional software is that the operations they perform on data can be modified over time based on previous outcomes. This is the fundamental principle underlying machine learning.
Nevertheless, every AI model begins from an initial state defined by two key components: the algorithms that govern its operations before any subsequent adaptation takes place, and the training data that guide the model’s development and influence later modifications to those algorithms.
Gender bias can be embedded within either of these components, although it is often more readily identifiable in training data than in the algorithms designed by programmers. Training data may contain explicit gender bias when they consistently produce different outcomes for individuals who are otherwise equal in all relevant respects except gender. When training data consists of textual, visual, or audio content, gender bias may also appear in the narratives, representations, and assumptions conveyed by that content.
Such bias may be relatively easy to detect when it relies on widely recognized gender stereotypes. In other cases, however, it may be more subtle and difficult to identify, particularly when stereotypes are not expressed directly but are embedded within broader patterns of discourse and representation.
Algorithms themselves may also contribute to gender bias through omission. Failing to account for the possibility of gender bias during the design and development of algorithms can constitute a form of bias in its own right. This is comparable to designing a water purification system without considering the possibility that the water may contain a toxic substance. Such a system would be incapable of detecting that substance and would therefore allow it to reach end users.
In many cases, the omission of gender considerations during the development of AI systems stems from a lack of gender awareness throughout the design process. This may reflect insufficient awareness among developers themselves, as well as the absence of institutional policies that require gender-sensitive approaches during the stages of model development, training, testing, and evaluation.
For this reason, the broader social environment within which AI systems are developed can be understood as a primary source of gender bias. That environment produces the data on which AI models rely. It also shapes the individuals who select that data, often without recognizing the gender biases they contain, as well as the developers who create algorithms that may similarly overlook such biases.
The delayed recognition of AI systems’ susceptibility to gender bias is closely linked to a perspective that views scientific and technical practices as detached from their social context. According to this view, systematically collecting data, applying statistical abstraction, and constructing logical computational processes should be sufficient to eliminate social influences from the final product, including various forms of bias and discrimination. Experience has demonstrated, however, that such assumptions are overly simplistic and that social inequalities can remain embedded within technological systems despite their appearance of objectivity.
The Impact of AI Development and Training Methods
The development and training of AI models generally rely on two broad categories of data. The first consists of data similar to that the model will process after deployment. For example, in an AI system designed to assess applicants for a job, this category would include the information contained in employment applications.
The second category consists of historical outcomes associated with previous decisions made on similar data. In the same example, this would include information about how past applicants were evaluated, which candidates were hired, and how successful they subsequently proved to be in their positions. Such success may be measured through indicators such as retention rates, performance evaluations, promotions, and rewards.
AI models do not typically process raw data alone. Instead, they assign relative weights to different variables and characteristics. By analyzing outcome data from the second category, a model can identify patterns associated with desirable results and then adjust the weights assigned to variables in the first category until it reaches combinations that most closely predict those outcomes.
This process becomes problematic when both categories of data reflect existing gender bias. In such cases, discrimination becomes embedded within the statistical patterns identified by the model.
Returning to the recruitment example, suppose that women have historically been hired at lower rates than men, particularly for certain positions. Suppose further that women have received lower performance evaluations than their male colleagues and have been awarded fewer promotions and bonuses. When these historical outcomes are aggregated into statistical patterns, gender may appear to be more strongly associated with occupational success than other relevant factors.
Under such circumstances, an AI system may assign greater weight to gender as a predictive variable. Gender can then become a decisive factor in evaluations whenever other characteristics are equal or relatively similar, particularly if the model assigns those characteristics lower statistical importance. The result may be that a male applicant is selected over a female applicant even when the latter possesses significantly stronger qualifications.
This dynamic can be understood as a form of statistical bias. When data contain a recurring pattern, statistical abstraction tends to favor and reproduce that pattern. AI systems rely on this mechanism because one of their core functions is to identify recurring structures within data and replicate them in future predictions and decisions.
In this sense, gender bias in AI systems should not be viewed merely as an accidental consequence of biased data. Rather, it is often a structural outcome of development and training processes that are designed to learn from historical patterns. When those historical patterns reflect discrimination, AI models may internalize and reproduce that discrimination unless deliberate corrective measures are introduced.
The Impact of the Operational Functioning of AI Models
During everyday use and deployment, many AI systems continue to modify and refine their underlying algorithms. This capacity for ongoing adaptation is one of the defining characteristics of AI and a key source of its effectiveness. AI models can adjust their operations in response to new data received during deployment and based on evaluations of their previous outputs.
This dynamic is particularly evident in generative AI systems, which interact daily with large numbers of users and rely on both user inputs and subsequent feedback regarding the quality of their outputs. As a result, such systems can be understood as being in a state of continuous learning and ongoing training.
When it comes to gender bias and other forms of discrimination, however, this reliance on continuous learning from user interactions can make bias-mitigation efforts less effective. AI systems remain vulnerable to manipulation, circumvention, and the introduction of new forms of biased content. Numerous incidents have demonstrated that the safeguards designed to prevent certain forms of harmful or discriminatory discourse from appearing in the outputs of generative AI systems are not always entirely successful and can sometimes be bypassed.
Even when users do not deliberately seek to circumvent these safeguards, AI systems are constantly exposed to new data that may contain evolving expressions of gender bias. As these expressions become statistically frequent within user interactions, AI models may begin to reflect those patterns and subsequently reproduce gender-biased assumptions in their outputs.
In this way, the operational dynamics of AI systems can contribute to the persistence of gender bias even after substantial efforts have been made to address it during the development and training stages.
Approaches to Addressing Gender Bias in Artificial Intelligence Models
A variety of approaches have been proposed to address gender bias in AI systems. Some have already been implemented in practice, while others remain the subject of academic research, feminist scholarship, and initiatives concerned with gender justice and technological accountability.
These approaches can broadly be divided into two categories. The first focuses on interventions during the development and training of AI systems before deployment. The second seeks to establish corrective mechanisms that monitor and mitigate bias during the operation of AI systems after deployment.
Gender-Blindness and Pre-Processing Approaches
Approaches commonly described as gender-blind or gender-obscuring mechanisms represent some of the simplest and most direct strategies for addressing gender bias.
These approaches seek to remove explicit references to gender from the data used to train AI models. In recruitment systems, for example, identifying information indicating whether an applicant is male or female may be removed, allowing evaluations to be based solely on qualifications, experience, and other relevant characteristics.
Despite their apparent simplicity, these approaches face two significant challenges.
The first challenge is that removing gender indicators from data that already reflect gender bias can create internal inconsistencies within those datasets. Such inconsistencies may reduce the effectiveness of AI training processes and limit a model’s ability to generate accurate predictions and decisions.
The second challenge is that gender is not merely a single attribute listed among an individual’s characteristics, nor is it limited to a binary classification within a dataset. Gender functions as a social condition that influences many aspects of a person’s life, opportunities, experiences, and social interactions. These influences often remain embedded in data even when explicit references to gender have been removed.
Consequently, datasets that are structurally shaped by gender bias may continue to carry that bias despite the removal of direct gender markers. Indirect indicators of gender can remain present within the data, and their cumulative statistical effects may ultimately produce outcomes similar to those generated by explicit gender information. As a result, gender bias may still emerge in the behavior of AI systems trained on such data.
Pre-processing correction approaches attempt to go beyond simple gender blindness by identifying and removing a wider range of gender-biased elements from training datasets. These approaches are particularly relevant in the context of generative AI systems, where training data often consists of large collections of textual, visual, and audiovisual content. Their objective is to eliminate gender stereotypes, discriminatory representations, offensive expressions, and other forms of biased discourse before the data are used for model training.
In many cases, these approaches rely on AI systems themselves to identify gender-biased language, stereotypical representations, and commonly used discriminatory expressions. Data containing such elements is then excluded from the datasets used to train future models.
Although these methods represent an advance over simple gender-blind approaches, their practical implementation faces significant challenges. The scale and diversity of modern datasets make comprehensive filtering difficult, while effective bias detection often requires specialized knowledge that varies across cultural and social contexts.
As a result, the effectiveness of pre-processing correction approaches remains limited in certain circumstances, particularly when content combines multiple languages, employs continuously evolving coded expressions and indirect references, or appears within contexts where distinguishing between harmful and non-harmful uses of language is inherently complex.
Post-Processing Approaches
Unlike the approaches discussed in the previous section, which focus primarily on the training phase of AI systems and the preparation of training data, post-processing approaches are concerned with the operational stage of AI models. Their objective is to incorporate corrective mechanisms into AI systems that can detect and mitigate potential bias in model outputs after decisions or predictions have been generated.
Rather than removing gender-related information from data in advance, these approaches monitor the extent to which gender may have influenced a model’s outcome and then reassess that outcome when evidence of gender bias is identified.
In the case of generative AI systems, post-processing mechanisms review model outputs and attempt to identify gender stereotypes, discriminatory assumptions, or biased representations embedded within generated content. When such elements are detected, the system modifies or adjusts the output to better align with principles of fairness and contextual appropriateness.
One advantage of post-processing approaches is that they are generally less vulnerable to the inconsistencies that can arise when altering training data before model development. Instead of treating gender as information that must be removed, these approaches treat it as a variable that can help identify statistical patterns associated with discrimination, thereby enabling corrective intervention.
Nevertheless, post-processing approaches do not fully resolve the problem of indirect, complex, or hidden forms of gender bias embedded in data and in the outcomes generated from those data. Many forms of bias operate through subtle associations and cumulative patterns that are difficult to identify through automated review alone.
In the context of generative AI systems, post-processing approaches also encounter some of the same challenges faced by pre-processing methods. AI models may continue to reproduce expressions, references, and assumptions that contain gender bias in ways that corrective algorithms fail to consistently recognize. As a result, discriminatory content can still pass through filtering mechanisms despite efforts to prevent it.
Toward More Comprehensive Approaches
Efforts to reduce gender bias in AI systems raise a broader question: Is it possible to develop AI models that possess genuine gender sensitivity?
This question is connected to wider debates concerning the future development of artificial intelligence, the nature of gender itself, and the meaning of gender awareness. It also touches on longstanding discussions regarding whether AI systems might one day acquire forms of awareness or understanding that extend beyond pattern recognition and statistical inference.
Regardless of how gender sensitivity is defined, it remains closely linked to human consciousness and to practices that individuals perform deliberately and reflectively. While there is currently no definitive answer to these broader questions, they point toward a practical conclusion: effective responses to gender bias will likely require approaches that combine human judgment with both pre-processing and post-processing bias-mitigation strategies.
Among the available options, the integration of meaningful human oversight into both the training and operational stages of AI systems appears particularly promising. Such oversight can be implemented in different ways depending on the domain in which an AI model is deployed, the scale of the data used to train it, the frequency of its operation, and other contextual factors.
Human involvement can help address several of the limitations associated with the approaches discussed in the previous sections. Moreover, it can be combined with existing mitigation strategies to create more comprehensive and effective solutions.
At the same time, this approach carries high costs. Incorporating sustained human oversight may increase the time and resources required to develop, train, evaluate, and maintain AI systems. Although such measures may improve outcomes, they should not be assumed to eliminate gender bias entirely.
Human societies continue to generate and reproduce gender bias through individual attitudes, institutional practices, and social norms. Many of these biases operate unconsciously and remain difficult even for human beings themselves to identify. Gender bias remains deeply embedded in numerous forms of social practice, communication, and cultural expression.
For this reason, the goal of addressing gender bias in AI should not be understood as the pursuit of a perfectly bias-free technological system. Rather, it should be viewed as an ongoing process of identifying, challenging, and mitigating the ways in which existing social inequalities become encoded, reproduced, and amplified through artificial intelligence.