Case Study:
Transforming LGBTQ+ Research Design
TLDR;
I reconstructed UCSD’s Graduate Well-Being survey to remove offensive material and better communicate with the underserved LGBTQ+ population it was originally meant to reach. Using journey maps, survey design, and primary source research, I was able to transform UCSD’s survey design and influence future research for the graduate program.
Context
Client: University of California San Diego (UCSD), a leading public university known for their competitive sciences programs.
Product: The University of California’s graduate population survey, which covers topics such as financial confidence, housing, and food security.
Background: In 2020 the University of California Office of the President (UCOP) surveyed the graduate population to compare the health and security of marginalized students to their peers. The survey specifically highlighted the LGBTQ+ graduate population as high risk, and asked multiple sentiment questions of those students.
of students at UCSD
of students at UCSD
Users and Audience
Stakeholders
The University of California Office of the President (UCOP)
Users
Graduate students at University of California schools. Queer graduate students at UCSD made up 10% of surveyed population as of 2016, and transgender/gender non-conforming (GNC) graduate students made up 2% of surveyed population as of 2016.
Audience
Potential graduate students of a UC school who also identified as a marginalized identity.
Scope and Constraints
Time frame: The survey is given yearly, and is open for several weeks for responses.
Limiting factors
As the survey has a limited time window, it was vital to get information to researchers so they could consider updating their terminology before the survey closed.
Roles and Responsibilities
My responsibilities
As a recipient of this survey during my time as a graduate student at UCSD, I contacted the Office of the President to present concerns on the inaccuracies of the gender and sexuality content. I proposed solutions for future interactions of such surveys and worked with the survey team to ensure a more inclusive experience. I served as the only UX Researcher for this project.
Problem
The survey in its released form was exclusionary, discriminatory, and offensive to members of the LGBTQ+ community. Because this survey is exclusionary towards queer and transgender students, the data and statistics from the survey would result in an inaccurate portrayal of the lived realities of those participants. In addition, the questions used were heavily partisan, and had the potential to skew the final analysis.
Business impact: This survey and its results were publicly available on UCSD’s website, and that data was specifically referenced in its diversity material. Any offensive material in said surveys could directly impact recruitment of diverse communities.
User’s primary pain points: Users who were queer or trans could often not complete the survey, as the questions would be highly offensive or lack an appropriate option for them to choose.
Objectives
User objectives: Remove all bias and offensive language from the survey as soon as possible.
Business objectives: Update all public material to scrub offensive material, and review previous data to assess whether bias was present in results. Research and confer with experts to better understand appropriate language to use in future surveys.
Metrics of success: Current and future surveys would remove offensive material, steps would be taken to research more respectful language, and results would be reviewed for potential bias issues.
Research
Methods
I outlined a journey map for a LGBTQ+ user completing this survey, highlighting pain points and exclusionary content. I then worked to rewrite the survey using updated definitions consistent within the queer community and major advocacy organization terminology (as seen in GLADD Media Reference Guide).
User insights
Because this survey was using extremely out of date terminology or references, many participants lacked an option to indicate their true identity.
Process Map - Graduate School Survey Case Study.pdfResults
Performance against objectives
Although data bias was not fully addressed, the Office of the President has taken clear steps to remove offensive material for future surveys and implement respectful language in future surveys.
Impact on design
While the impact was not immediate, the Office of the President did eventually make changes in their future surveys to be more accommodating to queer and transgender students, as shown through their 2021 survey results.
Areas for future improvement
After further experience in diversity and data work, there are some changes and recommendations I would make to create a fully functional survey template. The first would be to create a transgender question, that allows participants to self identify as transgender if they wish to do so. In addition, I would have definitions available for survey users to reference if they need further clarification. Finally, I would include a use guide to provide recommendations for application in a survey. Here is an updated version of my LGBTQ+ friendly survey, with optional info pop ups and use recommendations.