AI Bias: the impact of AI on the workplace and recruitment

Artificial Intelligence (AI) is set to have a seismic impact on the workplace, as businesses seek to use it to streamline operations, build resilience and boost profits. Our Risk & Resilience research1 found that 25% of the global executives we surveyed plan to invest in AI this year2, and that 68% agree AI will replace jobs in their company, rising to 80% in Singapore3.  

Increasingly, AI is being brought into Human Resources (HR) departments, but AI is known to have innate biases which can skew its decisions, and with its lack of accountability employers can ultimately be held responsible for inequitable outcomes. Resulting in over-reliance on AI potentially leading to accusations of unfairness in the workplace or discrimination and employment practices liabilities (EPL) claims. 

Automating HR

AI is starting to be used by HR departments to streamline a host of tasks, including recruitment, contract drafting and analysing employee data. The technology has revolutionised recruitment, screening candidates and sifting through CVs to identify the most suitable applicants.

The technology is also lifting the burden of administrative tasks from HR departments by drafting job descriptions, codes of conduct and interview questions. At the same time, AI has automated routine functions such as payroll and benefits administration, while AI data analytic tools can help HR departments make staffing projections and review the performance of employees.

In the US, it is important to watch the US Equal Employment Opportunity Commission (EEOC) closely. It recently published its 2024-2028 Strategic Enforcement Plan and at the top of that list was the use of AI in hiring, managing or terminating employees and the potential discriminatory impact. The EEOC estimates that 80% of employers are using some form of AI in their employment decisions and its spent a further $15M in training its employees on how to identify the discriminatory use of AI. This is an area that is ripe for class action so it’s important that while HR teams are using this technology to lift the burden, it is also balanced with positive use of the technology.  

Pitfalls for businesses  

There is still a lot that we don’t know about AI, and the lack of transparency around how the technology makes decisions has led to widespread concern around AI-bias and hallucinations4. For HR departments this presents a particular issue for recruitment, with a recent study by Bloomberg revealing that potential candidates were being filtered out by ChatGPT 3.5 on the basis of their racial background5. The investigation concluded that ‘that using generative AI for recruiting and hiring poses a serious risk for automated discrimination at scale’.

With HR departments holding sensitive, personal data about their employees, there are also growing concerns around privacy. Employers are ultimately responsible for ensuring the safety of their workers, and relying on AI systems to collect and analyse data can make it hard to contain data breaches. 

Getting it wrong

HR departments need to ensure their AI-based systems do not discriminate against or show bias towards certain groups or demographics. While this is difficult to prove, increasingly, there are claims that AI technology in the workplace has unintended consequences and is showing bias against gender, race, older staff, those with disabilities, or those from a disadvantaged background6.

The use of AI could lead to a host of claims around discriminatory bias, unfair recruitment and even redundancy claims. Against the backdrop of these emerging risks, businesses need to ensure the coverage provided under their EPL policy is fit for purpose and that it provides adequate protections from AI-related risks. Going forward employers must be able to demonstrate that any AI-systems used are handling and processing recruitment and employee data in a safe, secure, and fair manner.

Back to Spotlight on Boardroom report