ICO Publishes Regulatory Sandbox Report on Onfido’s Approach to Measuring & Addressing AI Bias

At Onfido, privacy is paramount to everything we do. When creating AI and biometric technologies, we must ensure that they operate fairly for all individuals and ensure that privacy is respected and upheld.

 

This is why we decided to work with the UK’s Information Commissioner's Office (ICO), which upholds information rights, entering into its Regulatory Sandbox with the aim of ensuring that our research to reduce and mitigate algorithmic bias, was carried out in a manner which respected the rights and freedoms of individuals when processing their personal data.

 

The ICO has published its Regulatory Sandbox Final Report: Onfido. The report summarises the ICO’s conclusions on key data protection issues to ensure that our facial recognition technology is fair and inclusive for all data subjects.

 

Crucially for Onfido, the ICO has concluded that while we are a Controller of the personal data we process to develop our technology, biometric data is not a special category of personal data when the processing activity undertaken does not require individuals to be uniquely identified, and therefore the use of biometric data in the context of training Onfido’s AI is not subject to heightened restrictions under GDPR.

 

AI-based systems are historically biased in their ability to distinguish faces of ethnic diversity. We entered the Regulatory Sandbox to explore how we could lawfully measure ethnic or racial bias, in order to train our systems to reduce and mitigate any such bias. As personal data revealing a person’s ethnic or racial origin may only be processed where additional safeguards apply, we sought guidance specifically on this point. Here, the ICO has concluded that where data that reveals a person’s racial or ethnic origin is used for research, the most appropriate condition for processing special category data under GDPR is likely ‘substantial public interest’.   Onfido identified the substantial public interest in our research initiative as being the prevention of discrimination. UK law permits processing of personal data relating to protected characteristics for the purposes of identifying or monitoring the existence or absence of equality as between different groups of people.

 

Throughout our work with the ICO, we were able to demonstrate the steps we take to ensure fairness and transparency to individuals whose data is used for research related processing and received invaluable advice to further enhance our practices.

 

Working closely with the ICO, our research team will continue to drive for the development of standards to ensure the risk to individuals are minimised in the way that AI and machine learning is developed and then subsequently used by organisations.  Participation in the Regulatory Sandbox provided an invaluable opportunity for our research team to understand practically how compliance with data protection and privacy laws can ensure that machine learning does not negatively impact any individual and how it can enhance the explainability of its models.