You’ve probably noticed by now that Intuit® considers accessibility a priority. We’ve tackled the subject on many fronts, from why accessibility matters to setting priorities for accessibility issues. We’ve also looked at how to help developers write accessible code, including the creation of an open-source vs-code plugin: accessibility-snippets by Intuit software engineer Kendall Gassner. She believes developers should prioritize accessibility efforts during the development process.
As an Intuit senior software engineer, I recently spoke on this very topic. My goal for the presentation was to prove that making accessibility a priority is worth it, and I can prove it by tracking keyboard events in our applications. My experience working with QuickBooks® Time, previously TSheets, provides plenty of analytical evidence to substantiate this claim, and I want to share some of that information with you today.
Using telemetry principles to capture analytics
During my presentation, “Proving Accessibility is Worth It With Analytics,” I asked three questions:
- How do you know when an accessibility update is actually being used?
- How can you track what changes are most successful?
- How do you prove time spent towards a code change has value?
Thankfully, I already had the answers.
At QuickBooks Time, we’ve been using anonymous keyboard activity to understand how our customers are using the product and to understand our accessibility impact. The first prioritized fix was simple: refactor a link so that it would respond to keyboard “clicks.” This particular link was missing an href attribute, a property that was necessary for keyboard interactions to function, so it required users who prefer to primarily use their keyboard to switch to their mouse and click the link.
We used telemetry principles that were important to our organization to capture keypress analytics in order to identify when this fix was discovered and used for the first time. Telemetry is described as “the collection of measurements or other data at remote points and their automatic transmission to receiving equipment for monitoring.”
In software organizations, telemetry principles are used in gathering application feedback for business layer metrics such as A/B test results, customer usage, number of completed orders, and any number metrics that are important to the business. These analytics range from a link or button being clicked to a signal at the end of a workflow and the data that was submitted as a part of that workflow.
Historically, these analytics are used to evaluate experiments and how customers are interacting with features. For example, we might harness an experimental menu button with a call to our analytics service each time it is clicked, along with a call for the menu option that was selected. With this data, we can easily see if the menu button should be enhanced with further functionality or if we should pivot to a different solution.
The payload for the analytic is a very simple object that contains the following properties: category, action, label, and other. The “category” and “action” properties are used to describe where the event took place (the specific feature) and what the event was, respectively. The “label” property is optionally used to further “tag” an event, and the “other” property is also optionally used to provide any supplemental data.
An example payload could be an object with three fields:
{
“category”: “quick-add button”,
“action”: “clicked quick-add option”,
“label”: “add time”
}
The “category” signifies our experimental “quick-add” menu button, the “action” signifying that an option was selected, and the “label” containing which option had been selected. Once the event is processed by the server, the data that is stored in our events database also includes other anonymous information that could be beneficial to analyzing this data.
Accessibility analytics: Solving for one, extending to many
As a part of our first accessibility-focused effort, for the link that couldn’t respond to keyboard “clicks,” we added a keypress handler to the link. This handler would support the keyboard “click,” which means using the keyboard to focus on the link and then pressing the ENTER key to activate the link. Since analytics are an important part of our organization, we decided that the keypress handler should also include sending an analytic event to log the key that was pressed. We chose to add this simple analytic to begin proving value towards future accessibility efforts.
Our analytic payload followed the expected structure, but we used the “label” property to tag the event as “a11y” and even added the specific key that was pressed in the “other” property. For this analytic, the payload was:
{
“category”: “edit_time”,
“action”: “changing job code using keypress event”,
“label”: “a11y”,
“other”: “keyCode: ENTER KEY (13)” }
}
This same payload structure and keypress handler method has been used in many places across our application, specifically where we have fixed a feature or component to respond to keyboard interactions where it didn’t before.
For example and as explained in Julie Elliott’s Solve for One – Extend to Many, we fixed our Schedule feature so that a customer could create a schedule using only their keyboard. Each component that was fixed was given a unique analytic so that we could track the first time and how often it was used.
Julie writes, “The customer’s journey was an incredible experience to see. From watching their struggle to creating a solution, their delight was extremely rewarding for our team. We were also pleasantly surprised when we saw how many other customers were able to find the solution.”
From this one fix, we have been able to demonstrate solving a problem for a single user that is later discovered and extended to more than 6,000 users.
Evolving analytics and discoveries
Further efforts have since modified how the analytic is created and handled.
We still use a keypress handler to capture the ENTER or SPACE key (depending on the native role of the component), but now we add an “accessibility-event” attribute onto the component that contains the keypress information we want in our analytic and then trigger the click event. The click handler then submits the analytic event for us but uses the “accessibility-event” attribute details as the “other” property value. This way, we get a single analytic that contains the native “click” event and the “keypress” event details, and the data analyst can either use the data in whole or separate the data by mouse clicks and keyboard clicks.
All of this is handled by a single helper method that is available across the application, and the only work that is required by the developer is to ensure that the component has both the click handler and the keypress handler.
Referring back to the “quick-add” menu button example, the analytic that includes the keypress event would resemble the following:
{
“category”: “quick-add button”,
“action”: “clicked quick-add button”,
“label”: “add time”,
“other”: “a11y: clicked with enter key 13”
}
We’ve learned several things from this data.
- The difference in usage between mouse clicks and keyboard clicks. In the past, our remediation efforts were sporadic with no real prioritized focus. This information has allowed us to begin prioritizing and focusing our efforts on areas of the application that are seeing the heaviest keyboard activity. For example, QuickBooks Time has at least three different features where a user can clock in or out. We can now see that one of these three features is used more heavily by keyboard users versus mouse users and ensure that that feature is remediated before the other two features.
- Discoverability. Referring back to our initial accessibility-focused effort, this simple fix was released March 2019. Using this data, we saw that the fix was first discovered that same day it was released and was in fact used 140 times that day. Other efforts, like fixing the logout functionality or our “Contact Us” feature to work for keyboards, are discovered on average 5.4 days after they’ve been released. This discoverability data has proven valuable not only to validating the efforts but also to reinforce and encourage the developers who are making the changes.
What can’t be derived from the data is whether someone is using assistive technology or not. Since assistive technology acts as middleware between the user and the browser, all keyboard interactions using the assistive technology capabilities are intercepted and translated into a traditional click event in the browser, so we do not ever receive the original keypress event. However, if the user continues to use a native keypress without the assistive technology capabilities, we can still capture their keypress.
Here are examples of data that will be further researched and shared:
- 77% of companies using QuickBooks Time have workers who depend on keyboard functionality.
- 7.5% of QuickBooks Time workers depend on keyboard functionality.
- Average number of days to discovery: 5.4 days.
- Total number of unique analytic events that are being tracked: 156 events across 14 categories.
Prioritize accessibility efforts, provide value
Ultimately, the goal is to encourage developers to prioritize accessibility efforts, communicate usage and discovery, and to consider accessibility early and often by tracking keyboard events in your application. The end result is valuable to both the developer, who designs an accessible app, and to the customers, who will have a delightful, accessible experience.