Accessibility Operations

Accessible best practices individually designed to mimic a playing card, that are loosely piled to evoke the feeling that they're in use.
Product Design Architect
Audits, Project Management, Tool & Training Development, UX Ops, Design Implementation

Chrome OS, Android, iOS, Mac OS, Windows

2018-2020, and ongoing

Published by Amplify

One of the firm requirements for any product used by some state school boards is 508 compliance — an education-focused act related to ADA compliance. The exact specifications for accessible products vary by state but many use WCAG 2.0 AA as their standard. During the five years I worked at Amplify, I led accessibility efforts for our entire supplemental educational division, auditing the existing games we had already designed and built, developing tools and action plans for addressing any gaps uncovered, and setting up both my immediate team and other teams within the games portfolio with training, toolkits and playbooks for integrating accessible design into their work streams. This work has been the most rewarding so far in my career: including accessibility from the beginning of the design process not only creates better experiences for disabled people who are often left with unusable experiences, but always creates better and more intuitive experiences for everyone.

Screenshot of Alice Wong, an Asian American woman in a power chair with a mask over her nose speaking to W. Kamau Bell, a Black American man, sitting in a cafe. Caption text reads from the speaker Alice, "It is a part of my cultural identity."
United Shades of America with W. Kamau Bell speaking with Alice Wong, founder and director of the Disability Visibility Project.

A note on language

I use the terms ‘disabled person’, ‘disabled people’, ‘disabled students’, etc. throughout this case study. One may use the more common ‘person with a disability’ and I recognize that there is no single term that everyone prefers for self-identification. Over the numerous qualitative interviews I’ve held, participants have shared their preference for disability-first language. I.e. a deaf person might identify directly with their disability and all that comes with it, preferring not to make it a modifier to their identity, as in referring to them as a person with deafness.

The problem

After the successful beta launch of our Amplify Reading supplemental educational product, we began to selectively work with additional school boards to integrate the product into their curriculum. When the state of Texas decided to roll the product out to their entire student base, it raised a new set of challenges: none larger than the firm Section 508 requirement (WCAG 2.0 AA or better) for any product used in Texas schools. Our beta product had been intentionally lean — educational games were an experimental new area for Amplify which needed to prove product market fit. The Texas school board contract was the validation needed, but also meant many of the challenges we had pushed to “post MVP”, including 508 compliance, now needed to be addressed. I led the accessibility effort for Amplify Reading and expanded our efforts into tools and processes to incorporate accessibility from the start on any new products we, or other teams at Amplify, developed.

First things first: Auditing our existing games

Taking an entirely new product for our company and “making it accessible” felt like an enormous task. Where to start? Our suite of supplemental reading web apps and sites still needed refactoring and some design clean up, so an audit followed by prioritized fixes to gaps in the accessibility (a11y) of the experience was the first step.

Getting started: working with a third party auditor and developing process

Taking an entirely new product for our company and “making it accessible” felt like an enormous task. Where to start? Our suite of supplemental reading web apps and sites still needed refactoring and some design clean up, so an audit followed by prioritized fixes to gaps in the accessibility (a11y) of the experience was the first step.

Government agencies use a framework called VPAT (Voluntary Product Accessibility Template) to assess whether a product is sufficiently accessible. As specified in this framework, we worked with a third-party company to complete audits of all our products. A11y is not something that design can tackle alone: it requires close partnership with product and engineering to ensure the work is properly documented and prioritized. Some issues will be trivial to fix and others will require entire redesigns of specific flows—so making sure communication lines are set up early and are working well is essential.

Translating audit findings into product fixes

I partnered with a product owner and engineering lead within the Amplify Reading team to establish a working group that would translate our audit findings into prioritized fixes, and more importantly, would build tools and processes as we went so that the work would scale.

Together, we took the third-party audit results and internal review documentation for more than 50 applications, websites, dashboards, and eReaders and designed a full spectrum of user personas from kindergarten children through to adults, of varying levels of ability. Some of the specific tools and techniques that were developed for our team included:

  • Ticket templates (including a severity scale) that would take the violation reported by our auditor and translate it into an actionable format that worked within our existing agile development processes;

  • Internal consultation with other teams who were starting to incorporate audit findings into their roadmaps;

  • Workshops that shared vocabulary, process best practices, and common pitfalls across various product teams;

  • Internal documentation for designers and developers who were actually carrying out the work so they could better understand the context around specific fixes and recommended solutions if any were available.

Visually accessible component examples from Figma library. Components shown are keyboard functionality instructions, highlighting, settings, color palettes, focus indicator styling, closed captioning, and play/pause interactions.
The original Amplify Reading product was built in React with libraries in Sketch. I have since recreated all of our accessible components and guidelines into a Figma library (along with its own Accessibility features page), that framework engineers have created a Storybook library for. For more on this process, please see my K–5 Design System case study.
Two screenshots of visual components. The "before" component shows a small speaker icon above a tile. The "after" component shows both a speaker and checkmark button above the tile.
The image on the left is an example of a WCAG auditor approved component of a tile with "image-to-speech" and "select" interactivity. The image on the right is what we updated the component to, after user testing. Making it both a better user experience and WCAG compliant.

Establishing continuous usability testing for accessibility

Audits are a useful and necessary tool for creating accessible products, but they aren’t sufficient to ensure true a11y: first, they are a snapshot of a single point in time and can quickly become stale and second, sometimes the fix recommended by the auditor, while technically compliant with WCAG, may not actually be usable.

To address this and ensure that our a11y effort was achieving more than just checking a box, I developed a continuous democratized usability testing program, for Amplify Reading.

Finding and recruiting participants

Recruiting participants can be challenging, but recruiting participants with specific disabilities is especially so. Various laws exist to protect disabled people from being targeted based on their disability- something that makes recruiting participants difficult. I worked closely with our legal department and HR team to develop legally-sound and inclusive questionnaires for potential participants by  asking them to identify areas of research they would be interested in to help further Amplify’s accessibility program. I also partnered with a third-party organization called AbleGamers, which organizes panels of gamers who have some level of disability and are interested in improving the field of accessible gaming.

Once we had the legal work out of the way, we were able to start recruiting and running evaluative research. We opted to test only with adults, despite the product being designed for kids and adults, for two important reasons:

  • There are additional laws protecting children and disabled students from being targeted. The additional red tape would severely slow an already tedious process and limit the number of participants we would be able to work with.

  • Children don’t have the vocabulary or level of conceptual mind to work through any friction points that assistive technologies, prototypes, and the limits of videoconferencing might introduce into sessions. Young children have low digital literacy and disabled children must also contend with being new at using assistive technologies as well.

This introduced possible bias into our research: disabled adults, while the closest proxy to disabled children we could work with, might not react to our products in the same way a child might. Regardless, as long as we were conscious of this source of bias while interpreting findings, the research was still highly valuable.

1 of 4

In a user testing session, participant with a motor disability showing their assistive technology, which is a mouse with 12 different buttons.

Participant with low vision is using a magnified cursor and zoom function.

Participant with low vision and a motor disability is using an iPad as their assistive technology.

Prev
/
Next
“Retrofitting accessibility would be like adding more milk or eggs to a cake that’s already been baked.”

–Screen Reader participant

Accessibility Ops: Turning insights into action

Audit results provided a backlog of fixes that the design, product and engineering teams could work together to address and continuous usability testing evaluated our solutions, but one piece was missing: a strategy for including a11y in all future designs at Amplify.

Establishing foundations for scalable accessibility

Over the past two years, I have worked to tie accessibility into our product discovery lifecycle, design process, and software delivery process. Initiatives can be grouped into four categories.

  • Training and Workshops. I consolidated and distilled the learnings from our initial audits and usability testing and prepared a series of interactive training sessions that could be run with other teams at Amplify. These focused on fostering empathy and demonstrating the value of designing and building with a11y in mind from the start; best practices for incorporating a11y into a team’s agile process and throughout the product development lifecycle; and a playbook for conducting qualitative research with disabled persons. I also kickstarted a monthly onboarding workshop for new employees to help internalize why we design with a11y in mind and to orient participants to the various tools available.

  • Designer Resource Center. I documented glossaries, guide books, how-to’s and checklists for the broader design organization at Amplify. In particular, I summarized the WCAG 2.1 standards for our specific context so that designers could understand the guidelines without having to pore over the original (and dense) guidelines. I also implemented an open-source change log, so that the material can be transparently versioned as guidance evolves over time — for example, the document is currently being prepared for the anticipation of WCAG 2.2.

  • Internal Design System Interactions, Components and Patterns. I worked directly with our internal design system team (designers, PM’s and UX engineers) to incorporate the latest a11y practices into our broader universal design system for the benefit of all products at Amplify. We also developed templates and patterns across many tools the design teams at Amplify use (Miro, Figma, Sketch, Slides, GitHub, Storybook.js, etc.)

  • Tracking Quality. Working directly with our Research & Analytics team, we were able to intentionally roll out and isolate a11y improvements to directly measure the impact they had on user behaviors. Through this quantitative analysis we were able to pinpoint the changes that had the greatest impact for disabled users and the general population as a whole.

1 of 4

Some examples of internal documentation created for teams. This is an accessibility checklist for designers with sections organized by type of design like Visual Design, User Experience, and Animation.

A glossary of do's and don'ts for teams with specific visual examples within Amplify products. All categories have noted WCAG criteria that correspond with the guidance. This was inspired by the BBC and IBM's accessibility resources.

Broken down list of WCAG criteria explained in 1–2 sentences for teams. Also organized by type of design like Visual Design, User Experience, and Animation.

A filled out tracker, developed for Product Managers, with rows of different WCAG criteria and columns of specific ownership of criteria, notes, locations, and checked off compliance.

Prev
/
Next
Three presentation slides from an accessibility training. Slides read, "Solve for one, extend to many." Accompanying headshot of Paris remotely hosting with large headphones.
Leading training with product teams.
Third party compliance data results shown in a circle graph that reads 71% to 93%.
We had drastically decreased our violation instances and brought our total accessible compliance percentage up. From auditing and interviews I learned that 100% compliance is near impossible, but necessary to strive for.

Outcomes & learnings

The audit results and our qualitative usability sessions were eye-opening. A combination of quick-win changes, as well as some more complicated UX flow and visual design rework allowed us to put out several rounds of more accessible versions of our product and track the effects on student interaction, completion, and scores. Not only did disabled students benefit from the changes implemented but we often saw the entire population benefit, particularly in the kindergarten to third grade group, who are still learning how to use technology and may struggle with specific UI’s or interactions.

It was not easy taking initial findings for one suite of products and expanding it to a program that could be used by many teams at the company. Some of the learnings that came from iterating on process:

  • Always prioritize. There are more fixes than any team can afford to implement in a single product. So, which ones block users the most and will have the most impact when addressed? Having a precise priority scale helps the team weigh conflicting priorities with finite time.

  • Keep knowledge open. While I started as the point person for our a11y efforts, a single point of knowledge and review isn’t scalable. Investing in creating documentation, tools, templates, components, and libraries allowed us to have an outsized impact on what one person could achieve alone - plus as more people became interested in and comfortable with the material, more people began to contribute!

  • WCAG in isolation isn’t enough. It is a helpful standard, and an important foundation for any a11y work, but can never capture all use-cases. Some guidelines, when followed to the letter, don’t work for all users. Creative interpretation of the standard’s intent is required for developing truly usable solutions.

  • Accessibility must be more than an OKR. Without tight integration into existing teams’ agile processes, and without tools and training to facilitate that integration, a11y goals set at the OKR level will perpetually be kicked to “next quarter”.

  • Flexibility is needed. Teams work differently and require different approaches to integrating a11y into their work. Being open to these unique needs will help ensure everyone can create accessible experiences.

User interface component of a toggle button between English and Español.
Spanish accessibility supports in the product: Spanish alt text and text-to-speech. This has contributed to Amplify Reading helping English Learners close the opportunity gap.

Intersectionality with Diversity, Equity, and Inclusion

The Individuals with Disabilities Education Act (IDEA) mandates free appropriate public education (FAPE) in the least restrictive environment (LRE) for students with qualifying disabilities. 14% of students aged 3-21 receive education under this act, a figure which is likely underreported due to the strict criteria for diagnosis and documentation. Of this group, almost half are represented as black and American Indian/Alaska Native and 58% of the student base using Amplify Reading identified as black or Hispanic/Latinx. Any a11y work done by our team would directly impact the most vulnerable students who are often left behind by the public education system.

-> amplify.com