Can artificial intelligence (AI) and machine learning programs lead to discrimination claims? The simple answer is yes, and the recent settlement between the Equal Employment Opportunity Commission (EEOC) and three integrated English-language tutoring companies known as iTutorGroup, Inc., confirms any doubts employers might have. Now more than ever, employers should carefully evaluate the benefits and risks of using AI or machine learning in recruiting and for employment decisions like hiring, promotion, and terminations, particularly because statistics show an increasing number of employers are using some form of AI in their hiring process. The EEOC is also continuing to focus on this issue, with the intention of bringing more litigation in this area.
Background
The use of AI software, machine learning, and other emerging technologies has raised so many concerns that in 2021, EEOC Chair Charlotte A. Burrows launched an agency-wide initiative to ensure their use complies with the federal civil rights laws the agency enforces. As part of the AI Initiative, the EEOC continues to issue technical assistance; identify permissible practices; hold listening sessions with key stakeholders; and gather information about the adoption, design, and impact of employment-related technologies.
For example, in May 2022, the EEOC publicly released guidance on the Americans with Disabilities Act (ADA) and the use of AI to assess job applicants and employees, and in May 2023, it published a technical assistance document entitled “Assessing Adverse Impact in Software, Algorithms, and Artificial Intelligence Used in Employment Selection Procedures Under Title VII of the Civil Rights Act of 1964.”
Among other things, the guidance defines key terms such as the four-fifths rule for selection, explains applicable federal law, and contains frequently asked questions and answers for employers.
EEOC Lawsuits
The EEOC is also taking legal action against employers it deems to be in violation of federal employment laws. For example, on May 5, 2022, it filed a lawsuit in the Eastern District of New York against iTutorGroup, alleging the company’s use of AI programs violated the Age Discrimination in Employment Act (ADEA).
Specifically, iTutorGroup’s application software was programmed to automatically reject female applicants aged 55 or older and male applicants aged 60 or older, thereby rejecting more than 200 qualified applicants because of their age. The discriminatory practice was discovered when an applicant submitted two applications, with one having a more recent birthdate. According to the EEOC, the applicant’s first application was rejected, but the same applicant was offered an interview when using the more recent birthdate.
On August 9, 2023, the case settled but only after iTutorGroup agreed to pay $365,000 (to be distributed as back pay and compensatory damages among the applicants who were allegedly unlawfully rejected) despite denying any wrongdoing.
The company also agreed to nonmonetary measures like adopting new antidiscrimination policies, distributing an internal memo, conducting multiple antidiscrimination trainings, inviting all applicants allegedly rejected based on their age in March and April 2020 to reapply, providing written notice to the EEOC of any discrimination complaints from employees and/or applicants, and ceasing to request birthdates.
In a press release, Burrows stated, “Even when technology automates the discrimination, the employer is still responsible. . . . This case is an example of why the EEOC recently launched an Artificial Intelligence and Algorithmic Fairness Initiative. Workers facing discrimination from an employer’s use of technology can count on the EEOC to seek remedies.”
Accordingly, the EEOC is increasing its focus on this evolving area of the law—and it isn’t limited to disparate impact and treatment claims for gender and race discrimination under Title VII of the Civil Rights Act of 1964. It also includes claims of other types of discrimination, such as for age or disability under the ADEA and ADA.
Takeaways
The iTutorGroup case also highlights the fact that foreign companies doing business in the United States may be subject to federal employment laws. For example, the EEOC’s complaint against iTutorGroup alleged the tutors were paid on an hourly basis to provide remote English-language tutoring to customers located in China.
As such, this serves as a good reminder of the EEOC’s 2003 guidance regarding the rights of employees working for multinational employers.
Another key takeaway is that without realizing it, many employers relying on AI and machine learning software developed by outside vendors might unknowingly be in violation of federal laws, thereby exposing the company to liability for discrimination claims. For example, even when an employer’s automated system isn’t expressly excluding candidates based on their membership in a protected category, there still may be a disparate impact issue to address.
Though state and federal AI guidance is on the horizon, you shouldn’t wait. For the time being, take a close look at your relevant employment policies and practices (with special attention to any AI and machine learning being used), and review them for compliance with employment discrimination laws and the EEOC’s AI guidance (publicly available on the agency’s website).
If you use such tools, also consider conducting a bias audit and engaging employment counsel to work through any problematic issues.
Juliet S. Burgess is the founding partner of the Burgess Law Group in Phoenix. She represents employers of varying sizes and industries with respect to labor and employment law, human resources matters, commercial litigation, trainings, and workplace investigations. For more information, please visit www.theburgesslawgroup.com or email Burgess at juliet@theburgesslawgroup.com.