Algorithmic accountability for the public sector

Project background

Governments are increasingly turning to algorithms to automate decision-making for public services. Algorithms might, for example, be used to predict future criminals, make decisions about welfare entitlements, detect unemployment fraud, decide where to send police, or assist urban planning.¹

Yet growing evidence suggests that these systems can cause harm and frequently lack transparency in their implementation, including opacity around the decisions about whether and why to use them. Many algorithmic systems lack political accountability as they replicate, amplify, and naturalise discrimination against people who’ve borne the brunt of historical oppression and discrimination.

Algorithmic systems can also facilitate new forms of privacy intrusion, producing determinations about people that have lasting echoes throughout people’s lives. These determinations can be hard to contest, and are often illegible to those whose lives they shape. This has spurred opposition across continents from researchers, civil society groups, organised tech workers and communities directly impacted by these systems.

In recognition, policymakers have turned to regulatory and policy tools, hoping to ensure ‘algorithmic accountability’ across countries and contexts. With many challenges and open questions arising from their early stages of implementation, the Ada Lovelace Institute,  AI Now Institute and the Open Government Partnership (OGP) partnered to launch the first global study evaluating this initial wave of algorithmic accountability policy.

Project overview

While there have been some efforts to evaluate algorithmic accountability within particular institutions or contexts (e.g, Shadow Report to the NYC’s Automated Decision System Task Force and OGP’s informal ‘Open Algorithms’ network), there have been few systematic and cross-jurisdictional studies of the implementation of these policies.

This project outlines the challenges and successes of algorithmic accountability policies by focusing on the experiences of the actors and institutions directly responsible for their implementation on the ground.

Through this project, we aimed to:

1. Review the existing policies for algorithmic accountability in the public sector to understand their challenges, successes and how they were implemented. These include Algorithmic Impact Assessments, Algorithmic Audits, Algorithm/AI registers and other measures intended to increase transparency, explainability and public oversight.
2. Provide practical guidance for policymakers and public-sector workers to design and implement effective policies for algorithmic accountability.
3. Identify critical questions and directions for future research on algorithmic accountability that can inform and address the challenges emerging from contexts where policies are already being trialled.

Combining our respective organisations’ work in the field, our final report, Algorithmic accountability for the public sector, reviews existing algorithmic accountability policy frameworks and provides practical guidance to policymakers and public-sector workers at the helm of the latest wave of algorithm accountability policy.

You can also read about this project on the AI Now Medium.

Team:

Project Leads:  At the time of publication Jenny Brennan was a Senior Researcher at the Ada Lovelace Institute, Tonu Basu is the Deputy Director of Thematic Policy Areas at the Open Government Partnership, Amba Kak is the Director of Global Policy & Programs at the AI Now Institute at New York University.

Lead Researcher: Divij Joshi is a lawyer and researcher interested in the social, political and regulatory implications of emerging technologies and their intersections with human values.

About the partners:

For the Ada Lovelace Institute, this research forms part of our wider work on algorithm accountability. It builds on existing work on tools for assessing algorithmic systems, mechanisms for meaningful transparency on use of algorithms in the public sector, and active research with UK local authorities and government bodies using machine learning.

For the AI Now Institute, law and policy mechanisms are a key pathway toward ensuring that algorithmic systems are accountable to the communities and contexts they are meant to serve. This research builds upon a wider body of work including their framework for Algorithmic Impact Assessments (AIA) and the Algorithmic Accountability Toolkit. In the spirit of proactive engagement with the policy process, alongside a broad civil society coalition, they also published the Shadow Report to the New York City Automated Decision Systems (ADS) Task Force to detail accountability mechanisms for various sectors of the city government.

For the Open Government Partnership, a partnership of 78 countries and 76 local jurisdictions, advancing transparency and accountability in digital policy tools is a critical part of a country’s open government agenda. OGP members work with civil society and other key actors in their countries to co-create and implement OGP action plans with concrete policy commitments, which are then independently monitored for ambition and completion through the OGP’s Independent Reporting Mechanism.

While several OGP countries are implementing their digital transformation agenda through their engagement in OGP, a growing number of OGP members are also using their OGP action plans to implement policies that govern public sector use of digital technologies. Among these, accountability of automated decision-making systems and algorithms has seen increasing interest. OGP convenes an informal network of implementing governments, mobilising a cross-country coalition of those working on algorithmic accountability. Given the rapid evolution of the issue, OGP members would benefit from a more comprehensive effort that documents what works (and doesn’t) on the issue, across different country contexts.

Image credit: krasnopolski