Gender and AI
How does the use of data-driven systems exacerbate inequalities in access to primary healthcare for transgender and non-binary people in the UK?
Project background
Transgender and non-binary people experience many barriers to healthcare access, which can have a huge impact on their health and wellbeing. The use of data-driven systems further entrenches existing inequalities in healthcare. A London Assembly Health Committee investigation found, among other things, that NHS IT systems are not able to record trans status consistently and inclusively, which affects the care received by transgender and non-binary people.
Data-driven systems are often designed for a ‘typical’ user and, consequently, as fixed and concrete (such as ‘male’, ‘female’, and ‘other’ or simply ‘transgender’). This normative, binary understanding of gender is embedded in various data-driven systems in the UK, and results in the exclusion of people who do not fit any of these categories/identities. The often opaque design of these systems means that there are still many unknowns around how they work – both for people seeking care as well as those entrusted with delivering it.
In order to assess the consequences of these gendered systems, we need to meaningfully consider and document the lived experiences of the transgender and non-binary people who regularly interact with them. We also need to understand how, when and where assumptions about gender enter the development of these data-driven systems.
This project’s research question is: How does the use of data-driven systems, which rely on and reinforce binary data categorisation, exacerbate inequalities in access to primary healthcare for transgender and non-binary people in the UK?
Project aims
We will examine the challenges faced by transgender and non-binary people face in providing information that is ‘legible’ to data-driven systems, and parallelly, the challenges of collecting and storing data about ‘non-typical’ users. An intersectional feminist lens will be adopted, focusing on the co-production of knowledge with those people most affected by these data-driven systems.
The key objectives for this project:
- Better understand transgender and non-binary people’s experiences of data-driven systems used in primary healthcare in the UK.
- Analyse how the gendered design of these systems makes it challenging to record information about those outside of what the system understands as ‘typical’.
- Co-develop, with transgender and non-binary people, a framework of recommendations for embedding values of gender equality and inclusivity in the design of public sector data-driven systems.
Upcoming workshop
We will be holding a virtual workshop to better understand how gender is recorded in NHS systems. We want to centre the lived experiences of trans and non-binary people accessing primary care in the UK. We would like to hear more about your experience – especially if you have been through the process of, or are considering, changing your gender marker in your health record, or if you have faced challenges in having your gender identity recorded correctly while registering with a GP.
The workshop will be conducted through Microsoft Teams and will last 2 to 2.5 hours. Please reach out to Kavya Kartik kkartik@adalovelaceinstitute.org by 27 November if you are interested in participating. Participants will be provided with a consent form providing information on how individual contributions will be used in the research project.
Aims of the workshop
- To better understand trans and non-binary people’s experiences with registering for or interacting with primary care services, and any resulting challenges in having gender identity recorded accurately
- To better understand the process of getting a new NHS number when changing gender marker in health records, and the potential loss of relevant medical history
- To develop recommendations to ensure that:
- The design and deployment of data-driven systems in primary care are able to respond to existing experiences and structural inequalities that disproportionately impact trans and non-binary people
- Trans and non-binary people are meaningfully included in the development process of these systems
Project publications
‘The computer won’t do that’
Exploring the impact of clinical information systems in primary care on transgender and non-binary adults