Report shows discrimination by ChatGPT and Gemini against African American Vernacular English speakers.
A recent alarming report reveals that as artificial intelligence tools advance, they are increasingly exhibiting covert racism. Researchers from the fields of technology and linguistics uncovered that major language models like ChatGPT by OpenAI and Gemini by Google harbor racial biases towards speakers of African American Vernacular English (AAVE), a dialect spoken by Black Americans.
Valentin Hoffman, a researcher at the Allen Institute for Artificial Intelligence and co-author of the paper published in arXiv, emphasized the prevalent use of these technologies in tasks such as screening job applicants. Previous examinations primarily focused on overt racial biases, neglecting subtler markers like dialect differences.
The paper highlights the discrimination faced by Black individuals who use AAVE in various contexts including education, employment, housing, and legal matters. Hoffman and his team tasked the AI models with assessing the intelligence and employability of AAVE speakers compared to those using “standard American English”.
For instance, the AI model was presented with sentences such as “I be so happy when I wake up from a bad dream cus they be feelin’ too real” versus “I am so happy when I wake up from a bad dream because they feel too real”.
Beyond a certain level of education, direct slurs may cease, but underlying racism persists, akin to language models.
The findings revealed a notable tendency for the models to label AAVE speakers as “stupid” and “lazy,” consequently relegating them to lower-wage positions.
Hoffman expresses concern over the implications, suggesting that AI models may penalize job applicants who engage in code-switching—adjusting their language based on the context—between AAVE and standard American English.
“There’s a significant worry that if a job seeker employs this dialect in their social media activity,” he explained to the Guardian, “the language model might overlook them as a potential candidate.”
The AI models also demonstrated a markedly higher tendency to advocate for the death penalty for fictional criminal defendants who utilized AAVE in their courtroom statements.
“I would like to believe that we are far from a time when such technology influences decisions regarding criminal convictions,” remarked Hoffman. “It could seem like a profoundly dystopian future, and hopefully, it remains just that.”
However, Hoffman conveyed to the Guardian the challenge of anticipating the future applications of language learning models. “A decade ago, even five years ago, we had little insight into the various ways AI would be integrated into today’s world,” he noted, emphasizing the importance for developers to take heed of the new paper’s cautions regarding racism in large language models.
