Making accessible accessibility reports

Talk by Calum Ryan

How do we approach accessibility testing in a way that’s sufficiently comprehensive from a technical perspective, but at the same time presentable to clients without technical knowledge and who may themselves have a disability?

In this talk I’ll explain some of the approaches and attention to detail I take to test and present accessibility issues in websites that’s accessible:

  1. Using WordPress to help make accessible accessibility reports for clients.
  2. The conversations I have with clients as part of the accessibility testing process before and after
  3. What the benefits are of using this approach

Watch Calum’s Presentation


Ahmed: Hello everyone. Welcome once again to WordPress Accessibility Day 2020. This is a 24 hour long event where we talk about accessibility. My name is Ahmed Kabir Chaion and I’m working as a Business Development Manager at weDevs. I’m also a regular contributor to the making WordPress documentation, design, polyglots, accessibility, hosting, community, learn WordPress, and Marketing group.

I will be your host for this session. You’re here for the 10 am UTC talk with Calum Ryan. Please feel free to add your question to the YouTube chat kit and we’ll answer those at the end. So how do we approach accessibility while testing in a way that’s sufficiently comprehensive from a technical perspective, at the same time presentable to clients without being technically knowledge and who may themselves have a disability. So that’s a question posed specifically for this talk.

So in this talk the speaker will explain some of the approaches and attention to detail it takes to test and present accessibility issues in websites that’s accessible. The points that are will be covered using WordPress to help make accessible, accessibility reports for clients, the conversations with clients as part of the accessibility testing process before and after, and what the benefits are of using this approach.

Our speaker Calum is an Accessibility Consultant for dxw, helping the public sector make their websites more accessible. His background is in front and web development and user interface design, having worked in the web industry for around 10 years, which brings to this session accessible, um, which brings to this session making accessibility, making accessible accessibility reports.

Apologies for that Calum, take it away.

Calum: Thank you. So I was really in pleased to give this talk about creating accessibility reports, which are accessible. And I, I work for a company called dxw who are based in the UK in London and Leeds has around 70 people roughly. And we design, build, operate through digital public services.

And within dxw I’m, I work in GovPress which specifically works with WordPress. We have lots of clients who use WordPress. They’re all public sector mainly. And throughout the past few months I’ve been working on making the websites more accessible and this is for the latest legislation which is now a legal requirement in the UK for all public sector websites to be accessible. If you take a look at there’s more explanation on what that involves,
the regulations and who that applies to.

So within organization of GovPress, I’m the only accessibility expert and it’s been my responsibility to interpret a lot of that information from the WCAG guidelines, the latest guidelines which all public sector organizations must comply to. So if some of you, I mean it depends, um I’m sure there’s a whole range of people listening to this talk um, but if you’re more technically minded you might have already seen these guidelines. They are an extensive documentation and listing of all the different requirements for having an accessible website.

I’ve probably spent some, quite a few hours bedtime reading for this. It’s uh, it’s not the sort of thing you can just read off in an hour. It’s certainly something you need to get your head around when you’re looking at how accessible websites should be.

In those guidelines I can summarize that websites need to be perceivable, operable, understandable and robust.

So that’s a lot of criteria to present to clients and what I’m looking at, in terms of report, is um for clients to understand the importance of having accessible websites and for that reports being useful to them, that not only before, but after the report, they can see which fixes have been done and which fixes are the most critical, need to be done and which are less important, but still need to be looked at at some point.

In terms of the report going forward as well, it’s a useful tool for the clients that we have to create their own accessibility statement which is a requirement for this guidelines at the moment. For certainly in the UK for the legislation that even if your website is fully accessible you still have an accessibility statement.

So in terms of actually creating an accessibility report there’s a methodology which you can look at from which gives a good in-depth listing of all the different criteria to look through and how to actually structure your accessibility report.

Now this is alone something we wouldn’t present to our clients because it is a very, uh detailed and quite technical presentation on highlighting what has been tested and who has tested it and the types of tests that have been formed. So the challenge that I wanted to get around was looking at what accessibility reports do at the moment and how we can make that more accessible to not only the people who are going to be fixing the technical issues in the website, but the content issues, the overall readability of the content.

So typically we’ll have a lot of different people looking at the report so it’s important that we make that report as accessible as possible to everybody who is not only reading it, but actually putting in information and issues into that report.

Typically there’ll be many issues which are the same over different pages on websites and the similar solutions as well to fix those issues.

And also in terms of our own example where we have many different clients, we need to have multiple people editing those reports or reviewing those reports. So we needed a solution to allow different people to look at those reports at the same time. We also had quite a short deadline to create all of these reports for different clients so having a facility to make those reports in the most efficient and accurate way was very important and having a way to present very technical language in a way which doesn’t completely take over the report and make it difficult for somebody who isn’t technically minded to get to the most important points in the report.

And also a lot of our clients are just used to having a pdf to print off whenever they look at documents. So, although pdfs aren’t inherently that accessible, we still have that option for the report to be printed off as a pdf or just as a paper printout.

So creating accessibility reports, we decided to use WordPress because WordPress is what we use day-to-day in our company. We have a whole team of people who can not only work on the backend, but develop a front-end user interface, can customize themes and WordPress just generally has got a great community of people to make the whole content management system a lot more robust.

Um it’s important to say we don’t actually use Gutenberg in our reports. We still use the Classic interface and that goes back to my, my intention is to make this report not only accessible for the front end, but accessible for people who are creating reports, so the whole way of entering information into reports is using the standard WordPress content management system, without any of the Gutenberg changes.

So starting with how we create the reports, we use, um if anybody is familiar with WordPress, on the backend we use a number of existing plugins, where we use the Classic Editor and the override, so we don’t get the Gutenberg presentation. We use Custom Fields, Advanced Custom Fields which gives us a few more options in terms of adding extra fields to content and making that more easy to add repeated content.

So if you’ve got like a repeated issue or repeated criteria, we can just start typing the words for that criteria and it will populate field without us having to go through and repeat the same thing for every single web book.

We, we use Custom Post Types to store the solutions and the criteria as a centralized area of content so we can refer back to those pieces of content and put those into each report, and that helps make the whole process a lot more efficient and accurate.

And in terms of presenting the reports to clients we use the standard password protection on posts when we publish them, so that only their client with a password can review that report.

In terms of actually viewing the reports, as a user we try and comply to WCAG 2.1 (AA) standard and so everything we do is client compliant to that level. You can navigate the entire report using keyboard, screen reader, and each of the sections are all accessible through a single navigation. And [unintelligible] say that the whole report is a single page. So if you want to print it off you can, just print off as a pdf if you wish.

And in terms of beyond the technical standards, we also look at how readable language is so if we’ve got a technical term such as ARIA, a-r-i-a, we explain what that abbreviation means and in terms of the whole sort of general design system, we try to apply standards which are accessible. So we have has their own design system and we take influence from that so, things such as the issues which I’ll show you in a short while uh in reports are all using standard components, which are accessibility tested.

So let’s look inside the accessibility report.

Initially the report was a prototype which I developed without WordPress. So this report is flexible, we can use what um,
anybody could use the report in whatever CMS or even just a static template if they wanted to. The the current set we have is a WordPress setup that is fairly customizable to however you need it to be.

So we start off with saying who created the report, outlining the person that was optional if they want to have their avatar or any other information about when the report was done, what their role is in the organization and outlining if there was anybody else who quality checked the report and the overall compliance of report. So in some cases we have clients who stipulate they want their website to be up to AAA standard, so we’ll make a differentiation between websites which have been checked to AA or AAA.

So in terms of outlining what we tested, we created a list of say how many pages or pdfs have been tested and they’re now writing how it’s being tested, so we always um specify the automated and manual tests which we’ve done. We have different browsers and [unintelligible] devices we use in the different combinations we use. Typically that will involve a few screen readers. We have a limited team and you can actually do that so although we have, there are several different screen readers, we only test with what we have available at the moment.

We test with VoiceOver, NVDA and the mobile equivalent. So we’ve got VoiceOver on iOS and TalkBack on Android and Narrator on Windows.

And we’ll probably look at having some more when we maybe expand actually a bit more in the accessibility side of things. So this is giving an outline of just how it’s being tested and that is before any of the issues or outlining how we fix those issues.

Now the issues identified in report, we we typically have four user impact levels. So that starts with minor, moderates, serious and critical.

These are user impact levels we take from the axe tool by Deque and we use that as platform automated tests to get an overview of what are the the main issues with websites before moving on to the manual tests. And so within our report we we can use things such as the accordions from the gov uk design system to make the whole um collection of issues more simplified and more collated.

So if there’s an issue it’s the same issue on different pages it’s a lot easier to just go to that single issue and outline it within a certain section. So for each user impact level we have these accordion items which as a description of the image that we have to the right of the slide is the critical issues title, with the issue, the type of test, so in this case it’s an automated test and where the issue is located. So in this case we have the home page navigation and then after that we say the level that it’s been assessed to.

So in this case the level has been WCAG 2.0, AA and the specific criteria that the issue is not complying with, is 1.4.3 contrast minimum. So following those details we outline what is the problem and who it affects.

So in this case we have multiple instances of contrast between foreground and background colours. It doesn’t meet the contrast ratio threshold and this includes [unintelligible] post states this has a serious user impact because there are many types of vision impairments where poor colour contrast could affect their ability to view the website.

Then after that we have screenshots of where that is present. Now this also helps for again people who aren’t technically minded to actually get an idea of where the problem is. So in this case we have four small screenshots which can be opened in a light box kind of presentation, if you want to expand it, and within that we may also have code samples.
In this example we don’t have that, but in some we, we would do.

And finally in the issue is the recommended solution.

We decided to put the solutions in a separate section. So in this case we have a small summary of what the solution is and an anchor link to the link to the solution further down.

So the solutions section is again a collated list of what the solutions are. Typically a solution may be applicable to different issues and in this case we have an image again on the right slide with a few issues of being given solutions.

And in this case we have one about ARIA, removing ARIA, uh one about changing HTML here because bottom element isn’t used,
and another one for changing the colour contrast because background colour. So within each of these solutions we give the outline of the solution, how to fix it, in some cases the solution is aimed at a technical person. In some cases the solution is aimed as a content person. So in this example, we don’t actually specify that, but in a future iteration of this whole platform we’re going to make a lot clearer on who that issue, who that solution is aimed at in terms of responsibility.

And finally we refer again to the criteria that the solution is applicable to and again that is another anchor link to a section further down in the report.

And finally in the report we have the references section, Uh this again links you to the WCAG reference for the criteria. In each case we also have a secondary link to an explanation of how to understand that criteria, a bit more in both cases the explanation is a lot more than we can write and report. And one of the reasons why I decided not to include an entire description of the criteria in the report itself, again it’s just not to make it so overwhelming for different people in the report and to make the report a reasonable level for everybody using it.

So what have we learnt from creating this platform?

Certainly early on we’ve found there’s a lot of information to include in a report and using visual space effectively is certainly something which you need to do when you’ve got so much information to interpret to a vast range of different people you might be viewing the report.

We often have different clients and this is an example, we have the office for national statistics and in such an example as that, we may be creating reports to a more customized level than maybe some others in terms of the people viewing it, in terms of what they need from the reports. There are ways that we might consider making it more customizable than it is at the moment.

In terms of again going, going back to who the issues are applicable to, who’s responsible for fixing those issues, we should maybe look at making that bit more clearer um.

In the discussions we’ve had with clients I’ve found that often many will feel that the report is really just for a technical person to look at and really they’re just there just to um view reports and summarize that the timing, ah actually understand how much time they need for the fixes to be made. Whereas in many cases the fixes are for their own team of content editors and decision makers to spend time making lots of fixes themselves

And then also finally looking at the report in terms of how crucial it is to making websites more accessible, it’s only a complement to discussion and ongoing discussion with clients to understand about how to make the websites more accessible and going forward that the report is only a one-off snapshot in time. That they need to understand that making their website accessible is an ongoing process which they need to revisit frequently and not just as a one-off to meet this legislation.

So what’s next for our report platform? We currently have the platform in our own private repository. It’s not really in a state where we could open source it right now.

We would like to make it open source as potentially a WordPress plugin and also just as a static template that you can work as your own presentation for any content management system.

It would also be great to get users to test out the platform, not only as content editors, but as users viewing the report and we’d like to do some usability testing with people to really understand what are the problems are in interpreting that information and how to make reports more accessible. And then finally also to make the whole process a lot more streamlined we’d like to link those issues into a tracking integration.

So for example, if you’re using a system such as GitHub or Trello potentially having a link with it in that, so potentially maybe it’s something like a API so that we could make that whole information a lot more open and easier to link to certain aspects through the report. And not only developers, but anybody else who wants to see the progress of changes being fixed and any other information basically is a lot more easily accessible.

So really I’m probably finishing quite early for this, but um just as a roundup really I just wanted to say thank you to everybody who’s organized this event .

And I am happy to take questions now or later. The company, do a whole range of work in the public sector. And if you are based in London Leeds or anywhere within the UK, uh we’re happy to talk about how we can work for you in making not only the web more accessible but and helping public sector services and service design. Thank you.

Ahmed: Thank you so much Calum, I believe we have a lot of time for questions.
We can, but we certainly are not seeing that much questions in the chat so I would like to ask you a personal question. What would you like to see in the future? You mentioned about open source as a plugin, would you give us a bit more details about that?

A little bit more in-depth idea about how the future, accessible future to you.

Calum: Sure, em, so my background is in mainly, sort of front end user interface design and I think making the whole um developer process a lot more friendly in terms of the linking of issues to a system which makes things like continuous integration, so developing a way for accessibility testing, an automated way a lot more easy to include in the build process.

And if that can be, uh I think mainly in my report context, but generally the availability of accessibility testing in continuous integration and development make that process look more well documented and promoted that would be great. Making the web more accessible I think is something which a lot of the developers I’ve spoken to aren’t really aware of enough and how to make that process maybe more easier, I think.

Ahmed: Thank you. So my next question to you is based on your experience, where do you think accessibility is successful more? Is it total is it more leaning towards technology or to acceptance of the use of technology?

Calum: Sorry, if you could repeat that again, sorry.

Ahmed: Sure. So where do you think the success lies for accessibility, overall accessibility? Is it on the use of more technology or making it aware for people to be able to use those technologies?

Is it on awareness or on technology? How would you prioritize it, the success of accessibility?

Calum: Yeah, I think certainly both. There’s room for improvement. On the latter point, I think certainly more in terms of awareness and making it more attractive, taking away some of the barriers to how we present that accessibility information. How we make it easier for developer relations to and not only just see it as an afterthought, but something you start doing as as soon as you start writing one line of code.

You know how you’re going to test it and where there might be potential problems for different users.

Ahmed: Right, so um another question. These are all coming from me so I really appreciate this opportunity for me to get first hand experience of asking the question. So I want to know while trying to explain accessibility to people who is not totally aware of such a thing which exists, would you share some of your experience of struggling uh and and trying to find it very difficult to make someone understand the importance of having accessibility.

Calum: Sure I think the perceptions of accessibility certainly on the web are very limited from my experience of speaking to developers and being able to convince them that what they’re doing is is important and that i think it’s something where they’re certainly improving.

And certainly events like this are helping with that process, but just as general awareness is again something which uh I see lacking in my experience of speaking to people and if there’s a way of making that process just a little bit easier in terms of the testing process and auditing websites to, to understand that, is certainly some way to go I think. And there are tools which are making that progress easier and the browser tools which I mentioned, axe and white tiles to some extent as well.

There are tools which I found have opened up some awareness more in terms of accessibility. I’ll still say that there’s some way to go.

Ahmed: All right, thank you so much. We did get a question from the chat. So the question is, how can we follow process for the code deliberation from your accessibility report plugin? And there’s a form of comment, I’m sure that WordPress a11y community could help to translate and contribute.
And also this the person says, thanks for the top down.

Calum: So thank you and so yeah the report tool is uh something which it is intention to make it open source. The code is all pretty much just WordPress based templates and WordPress based plugins. So things like custom fields and the presentation itself is relatively easy just to share and I’d like to do that within the next couple of months hopefully.

We are open about doing it and not just as an internal tool, but something to help other organizations with.

So certainly, um we’ll try and put up on GitHub for the next few months if we can.

Ahmed: Absolutely so thank you so much Calum for taking the time and initiative for speaking today at this event, 24 hour long talking about accessibility being part of it. What other initiatives would you encourage the stakeholders, if they could make a change, they could make a difference, who would you encourage to step forward so that accessibility can become more easily available for everyone to understand accept and apply.

Calum: So do you mean in terms of the organizations?

Ahmed: Yeah what’s the big ones, policy makers, government.

Calum: Yeah I think certainly credit to some of the people in the government who are stepping forward. Maybe not so much the current government, but in terms of the people working on terms as the government UK guidelines and the the overall drive to make at least the government’s website more accessible, that’s um certainly setting a good standard.

I think in terms of private sector as well, there’s some great examples of organizations I’m trying to think of one now, but uh potentially Monzo who are the, one of the organizations in Ukraine online banking and if they were possibly more vocal on what they’re doing for accessibility that’s a good example there, but also some of the more established private sector organizations and the uh, the banking sector certainly because so many banks are essential to making our day-to-day lives um possible through accessible websites. So that’s certainly somewhere which would be great to see. I know Barclays in the UK do a great effort towards not only accessibility, their own website, but talking about it, being more open.

So that’s certainly one I’d reference.

Ahmed: All right Calum, with that we wrap up the Q A session as well. Thank you so so much once again for your support towards the initiative and we hope to see you again in future WordPress Accessibility Day celebrations such as this one.

I would also like to thank everyone for attending the session with Calum Ryan and myself. You can continue the conversation using the #wpad, #wpad2020 and @wpaccessibility.

Also don’t forget to attend the next stop Digital Transformation or Social Digital Transformation with Merary Alvarado at 11 am UTC sharp, here in the same track. The host for the next few talks would be Amanda Bailey. I would like to personally give shout out to the session manager Stefano; our chat monitor, Monique, Hareesh, and also to Joe Dolson for being the lead organizer and putting up such an amazing team with efficiency and success. Until the next talk happens, we want to say goodbye to the viewers from Calum and myself. See you right here after the break, bye bye.

Questions on “Making accessible accessibility reports

  1. How can we follow process for the code liberation from your Accessibility Report Plugin? I am sure that WP A11Y community could help to translate and contribute. Thanks for the talk Calum Ryan