Testing Moodle for Accessibility

This report is presented on one page; it is approximately 9,000 words long. This version was published September 10, 2009.

Summary

Everyone I encountered and tested said clearly that Moodle was the best Learning Management System they'd used. There are many points in its favor.

  • Moodle is pedagogically sound, designed by people with a passion for education.
  • Moodle is a long-standing and mature product, built by and for its users.
  • Moodle is open source. Moodle is almost pure HTML, relying on standard links and form controls.
  • Moodle avoids common barriers to accessibility like Flash, Java, and image maps.
  • Moodle is relatively well-structured semantically.

The vast majority of Moodle is technically accessible: blind and vision-impaired users can accomplish nearly all the tasks set before them. However, much work remains to prevent Moodle from being a frustrating experience for these users. Many details that make sense to sighted users, or are entirely ignorable, are stumbling blocks for users of screen reading or magnification software.

Brian Charlson at the Carroll Center for the Blind is fond of saying that sighted users go from the macro to the micro. That is, we scan pages rapidly, building an increasingly accurate mental model as we narrow in on the details. Blind users go from the micro to the macro. They experience only the details, and have to infer relationships between them in order to build a larger picture.

It is in these details that Moodle needs the most attention.

About this document

Author and sponsors

This document was prepared by Randall Hansen at OpenSourcery for the CANnect consortium, with a grant from The Alfred P. Sloan Foundation. It is the result of direct experience with Moodle, robust testing with blind and vision-impaired users, and engagement with the Moodle community.

This document contains a survey of existing accessibility work in Moodle, descriptions of how blind and vision-impaired users interact with web-based materials, and many specific recommendations to improve Moodle. There is also a basic development plan to implement these improvements.

Audience

This document's primary audience is the Moodle community and Moodle developers. Without them Moodle would not exist, and would not continue to improve. Its secondary audience is anyone interested in accessibility, and for these people I make special effort in two areas. First, I try to avoid jargon and overly-technical language. Second, I try to summarize findings and to allow them to be applied outside Moodle, to other web-based systems.

Helpful for the reader will be general knowledge of learning management systems, of HTML and CSS, and of the basics of web-based applications.

Agenda

My intent is not to criticize any work done on Moodle, rather, to offer constructive suggestions based on real testing. Time and time again I saw blind users succeed brilliantly with Moodle, but in the interests of brevity I have documented only failures and frustrations. This document would be much longer were I to document successes as well.

I believe that Moodle should be accessible by default. That is, Moodle should provide the best possible experience for blind and vision impaired users with no special intervention or configuration on their part. In any apparent conflict between the needs of blind and sighted users, blind users should take priority.

Examples of more accessible Moodle

Moodle is in use at numerous institutions, educational and otherwise, but it's difficult to find examples where accessibility has been enhanced. If those institutions make changes to Moodle but do not share those changes publicly, no one is the wiser and no one benefits. This is unfortunate, but not a violation of Moodle's open source license.

When institutions do share public changes they are often adopted by the Moodle community. For instance, the Open University followed a 2006 accessibility study with development work to implement some of their recommended changes.

I have not been able to find any work enhancing Moodle's accessibility that has not been accepted into the existing Moodle code.

About the tests

Method

All of my recommendations are based entirely on the most important part of the work: testing Moodle with blind and vision-impaired users.

I did several other types of work in preparing this document. I worked with Cheryl Edmonds at CANnect and her team, reading and reviewing their work on Universal Design. I had long conversations with Brian Charlson about the challenges of technology for blind users.

I worked extensively with a copy of Moodle 1.9.5 installed on OpenSourcery's servers. I added and customized courses as a teacher. I registered as a student and took the Moodle features demo course, both visually and using JAWS with my screen turned off.

The test program

I worked with one blind and one vision-impaired user at the Carroll Center for the Blind in Newton, Massachusetts, and one blind user at the Washington School for the Blind in Vancouver, Washington. None of these people had used Moodle before, although all had experience with learning management systems.

I used the same test for all users. My methodology is based on my experience with usability testing. I gave users one clear goal and stayed quiet while they tried to accomplish it. Only if the user admitted failure and gave up did I provide direction.

The two courses I used were:

  • Moodle features demo course, available from on the Moodle demo site.
  • Introduction to PowerPoint XP with JAWS, a course developed by the Carroll Center for the Blind.

This is an outline of the tests:

  1. Register a new account and login.
  2. Enroll in the "Introduction to PowerPoint with JAWS" course.
  3. Take the pre-quiz.
  4. Complete lesson #1.
    • I broke this up into several parts, asking users to complete one section at a time.
    • The rest of this course is identical in structure, so we did not complete every lesson.
  5. Enroll in the "Moodle Features Demo" course.
  6. Starting with lesson 1, complete the course.
    • I broke this up into several parts, asking users to complete one section at a time.
    • Students completed every item in this course except the SCORM lesson.

Users were asked to complete assignments, and then to find their grades or respond to teacher comments.

I learned useful things even when users strayed from these instructions. For instance, I observed difficulties with the enrollment workflow when a user started a course without first enrolling.

Students used JAWS version 9 and 10, Screen Access, Apple's VoiceOver, and ZoomText. We used Internet Explorer 8, Firefox, and Safari on Windows XP and Mac OS X.

How do blind and vision-impaired people interact with web-based applications?

Without exception I was surprised at how proficient all my tested users were and how easily they used Moodle. This shows mostly that I had low expectations, which is an unhealthy bias. Blind users navigate the web clearly and quickly.

There are a number of important differences between the blind and sighted experience of the web. Most important for blind users I tested were a consistent heading hierarchy, useful heading text, and useful link text. These items are so important that JAWS starts each new page telling the user how many heading and and links appear.

All blind users I tested switched between three primary modes of interaction. First was reading: they listened while the screen reader read the page to them. Unless they were reading a large block of text, they interacted with the page almost constantly, e.g., skipping text they thought useless, going back to reread text that was unclear.

Secondly, they interacted with a list of links. All screen readers in the test could show users a list of all links on the page, allowing them either to read the links one at a time, or quickly jump to one using the keyboard.

Thirdly, they interacted with headlines. Screen readers allow users to cycle through all headlines of a given hierarchy; that is, the user can read all level-one headings in order, and then all level-two headings under a particular level-one heading.

Blind users incur a higher cost for failure. It can take a blind user very long to realize that they have performed an incorrect action.

Working with data tables requires a lot of memorization, even if the tables are well-constructed. Blind users must typically memorize all the headings so they don't lose their place in the data rows.

"Skip navigation" or "skip to content" links are often too slow for power users of assistive technology. Many users ignore these features, preferring to navigate their own way.

Blind users set their screen readers to speak very, very fast; much faster than I can comprehend and use. This is a tremendous benefit in many ways, but does mean that sometimes they miss important details.

Blind users have a very active cruft detector. Wikipedia defines cruft as, "any accumulation of obsolete, redundant, irrelevant, or unnecessary information." The blind experience on the web is that of swimming in a sea of tiny details, most of which are not relevant to the task at hand. Maddeningly, each of these details must be experienced in full and judgment passed on them not just once, but on every page. The result is that blind users are very quick to skip past any content which appears not to satisfy their immediate goals. This is one reason that good headings are so important.

Development plan

If a person or group were to implement all of these recommended changes, how should they go about it?

Method

First and most importantly, the development team must test all their changes to make sure accessibility really is being enhanced. It's not difficult for even an experienced and professional sighted developer to make well-intentioned changes which are harmful for blind users. A good test program will prevent this.

Secondly, a team will likely be more successful than a single individual. Someone experienced enough in accessibility or usability testing to carry off that part of the work may not know enough about software development or about PHP to do other parts.

Third, existing Moodle contributors should be strongly considered for the work. Moodle is complex, and having at least one team member with real experience should be very helpful.

Skills required

Required skills for this team, Moodle experience aside, are:

  • Technical project management
  • Usability and/or accessibility testing
  • User interface design
  • HTML and CSS
  • PHP programming
  • Database design

Development priority

Some recommendations are designated "high" or "medium" priority. High priority recommendations are intended to address the most serious defects, problems which can cause task failure for a blind or vision-impaired user. Medium priority recommendations are intended to address problems causing great confusion, frustration, or lost time. Other items are not as important and, other things being equal, resources should not be spent on them until the most important issues are resolved.

While I tested many Moodle features, I do not know the frequency with which these features are used in production. After the high priority issues, items should be address in order of maximum effect on users; that is, in order of most-used features.

Development budget

Many unknown factors affect the cost of development, including the quality of existing Moodle code, the scope of the changes, how well the Moodle community is engaged, the desired pace of development, and how many of these issues are to be addressed. My very rough estimate is that all items identified in this report could be accomplished by a team allocating between six and eighteen months of work. That is, between .5 and 1.5 developer-years.

Testing results and recommendations

By far the most common complaint of all users tested was, "there's just too much stuff." It's very easy for a sighted user to ignore repeated content, secondary content, and images. Users of assistive technology often do not have this luxury. When making changes to Moodle or adding new features, the most important consideration for accessibility should be to ensure that every readable element on the page is important for all users. If elements are judged not to be important they should be removed, not hidden or placed elsewhere in the layout.

Results and recommendations are grouped by features.

Feature: Application-wide details

Autofocusing of form fields can be confusing (high priority)

Summary

On some pages (e.g., login) one form field is automatically "focused" when the page is loaded. That is, the cursor is placed in that field just as if the user had selected it. Screen readers read this form field immediately, without starting at the beginning of the page.

In 2006 the Open University's accessibility study pinpointed this issue:

"The automatic placing of the cursor in the 'username' field is useful for sighted users (including dyslexic, partially sighted, hearing impaired etc.) who will see that it is there, but for screen readers this can be difficult because the screen reader focus does not start at the top of the page and will miss any instructional text such as 'Login here using your username and password.' It is difficult to know how to resolve this conflict of needs and so this requires further consideration."

Why it matters

This is one of very few accessibility problems in Moodle that caused total failure for a screen reader user. Other problems caused or contributed to wasted time or lack of understanding, but did not cause a user to give up entirely.

Vision-impaired users had no trouble with this feature. While I'm sure that the Open University's concern about dyslexic users is valid, the focus of this study is blind and vision-impaired users. For blind users I think the conclusion is clear: automatic focusing of form fields is harmful.

Possible solution

No form fields should be focused automatically. This change will reduce convenience for sighted users but will not prevent them from accomplishing tasks. It is a good candidate for a user preference, but this preference should be off by default.

Some tables do not have headings

Summary

Some tables (e.g., "List of categories," discussed elsewhere) have content rows but no headings.

Why it matters

Headings make interpreting a table easier for everyone, particularly screen reader users.

Possible solution

Every table should have headings.

Some help icons need better labels (medium priority)

Summary

This is what screen reader users hear when reading past the help icon when posting a new discussion topic: "Read carefully, Write carefully, Ask good questions, How to write text and Use emoticons."

This was often interpreted as paternalistic advice, and never understood as a help icon. Other help icons on that page are much better; e.g., "Help with About formatting text (new window)."

Why it matters

Verbosity is one of the prime enemies of screen reader users; anything that sounds like it might not be directly related to the task at hand is usually skipped. Labels starting with "Help" were more likely to attract attention and less likely to be frustrating.

Possible solution

Every help icon's ALT attribute should being with, "Help with" and follow with the shortest possible description of help content. "(New window)" isn't necessary for most screen reader users, but is very helpful for vision-impaired users, where the popup may appear in an area of the screen they cannot see.

Interstitial pages can be confusing (medium priority)

Summary

In a number of places in Moodle (e.g., the "Database" feature) user actions sending data are followed by an interstitial page containing minimal content and a message like, "Your entry has been saved." These pages are set to redirect to another page, using JavaScript, after three seconds.

Why it matters

Three seconds isn't nearly enough time for a screen reader or magnification user to understand a page and read the content on it. Every user was initially confused by this feature, and eventually very frustrated.

Possible solution

Remove all interstitial pages. Confirm user actions with a message on the next page.

Image tags are used to convey information

Summary

Throughout Moodle image tags are used for a number of purposes:

  • To indicate "required" or "advanced" elements. To indicate content type in lists.
  • To show a clickable help icon.
Why it matters

Screen readers deal with these relatively well, typically reading the content of the ALT attribute. Still, they preface this text with the word "graphic," which is clutter.

Possible solution

All information should be conveyed in plain text. Icons should be displayed using CSS instead of image tags.

Help popups are context-free (medium priority)

Summary

Moodle has a fair amount of well-written and useful online help. Help icons appear throughout the application near the topics they address. Clicking a help icon opens a new window.

That new window always has a generic title of "Help," and its first headings make no mention of Moodle or of the specific feature in question.

Why it matters

Blind users typically have little trouble with popups. They do, however, have some trouble with window management. For this they rely on the title of the window and the first lines of content on the page. In both these areas Moodle could be improved.

Possible solution

The help window's title should be useful, telling users that the page is a Moodle help page for a given topic. The page's first heading should say the same thing.

Dialect can be difficult to understand

Summary

Australian and British English is a little different from US English. Moodle uses "surname" where US residents commonly say, "last name." Moodle uses "marks" where US residents almost always say, "grades."

Why it matters

Blind users have far fewer contextual queues for the meaning of words.

Possible solution

Consider localized English language packs, e.g., "US English."

Pluralization is often incorrect

Summary

Often in Moodle a label is associated with a variable number. The number may change from zero to two or more. Because the number isn't known in advance the label is pluralized with a trailing "s" surrounded by parentheses, e.g., "This quiz is limited to 1 attempt(s)."

Why it matters

While a sighted person has no trouble understanding the metaphor, a blind person hears, "This quiz is limited to two attempt open parenthesis ess close parenthesis." Several times I observed users backtracking and listening to this word again in order to understand it.

Possible solution

The best solution would be a robust pluralization algorithm. Many languages and frameworks have this already. Removing the parenthesized "s" entirely would be more accessible.

Language selection control is always present, even if only one language is available.

Why it matters

A sighted user can scan this form control once and then ignore it, since its content and placement are similar on every page. Blind users have to go out of their way to ignore it every time they read past it. This is especially frustrating if the control is not useful.

Possible solution

Do not show the language control if only one language is available.

"Hide" and "Show" blocks links are very verbose

Summary

Depending on how Moodle itself and each course are configured, there will be a number of blocks on the left side of the page. Typical is a block titled "Administration," which for a student may contain two links:

  • Grades
  • Profile

Screen reader users would hear this: Skip administration same page link. Hide administration block button. Administration heading level two. Grades link. Courses link.

Why it matters

Blind users' greatest enemy is cruft; in this case a screen reader user will hear 17 words, while a sighted user sees only three. For the blind user to be successful, all these words must be heard and interpreted.

Possible solution

In this particular case, neither of those links needs to be in this block. "Profile" is accessible elsewhere, and "Grades" belongs somewhere prominent, not in a block that can be removed.

In the general case, the utility of the "skip" and "hide" features should be evaluated. They should be kept only if their utility is high.

WYSIWYG editor is difficult to use (high priority)

Summary

Several pages use a JavaScript WYSIWYG editor for entering text. These are notoriously difficult for blind people.

Why it matters

The Moodle WYSIWYG presents blind users with a large table of unlabeled and inaccessible (e.g., no "alt" attributes) images and form controls. The best case for a blind user is some wasted time figuring out they're in a WYSIWYG editor and then exiting it.

Possible solution

WYSIWYG should be turned off by default.

Skip to content wasn't used by any user

Summary

This is not a defect, but a data point. There has been much debate about whether these links should be "Skip navigation" or "Skip to content," or whether they are valuable at all. I didn't test a large enough group of users to reach a conclusion on such an important topic, but I have two anecdotes.

First, no screen reader user I tested used these links. Second, several blind educators told me that numerous screen reader users depend on links like this.

So my only conclusion here is that while further testing is still needed, appropriate "Skip" links appear to do little harm, and are used by some proportion of screen readers users.

Feature: Navigation

Grades can be difficult to find

Summary

A student loading the "Grades" page sees a table showing every gradable item in the course. These items don't appear to be in any particular order, and are not grouped at all.

Why it matters

Sighted users can quickly scan the page and find entries in the "Grade" column which are not blank. Blind users must either read every item, or skip directly to a course name they know.

Possible solution

Show quizzes and assignments in the same order as the course home page. Show different lists for assignments completed and not completed.

Course home page can be difficult to find (medium priority)

Summary

All users had trouble finding the course home page from within the course. The most common strategy was to go to the site home page and find the course again. No user associated the course number in the breadcrumbs with the course they were taking.

Why it matters

After performing an activity, i.e., reading a lesson, users often return to the course home page. Given the difficulties with the jump menu, this is often the only way to navigate.

Possible solution

Each page within a course should have a link to the course home page called, e.g., "course home page."

Breadcrumbs can be difficult to use

Summary

Breadcrumbs don't contain useful words (e.g., "Site home," "Course home").

Why it matters

Sighted users understand breadcrumbs as a progression, with each item separated by arrow characters. Blind users see a list of links with no explanatory heading and no implied hierarchy.

Possible solution

Add a heading to the breadcrumbs list, hidden from sighted users; e.g., "Breadcrumbs." Provide better labels for standard locations, like the user's home page or the course home page.

User tools placement is inconsistent

Summary

By "user tools" I mean the group of controls showing the current user's name (linking to their profile) and a logout link. On some pages these are at the top right; on most pages they're also at the bottom.

Why it matters

Visually-impaired users with ZoomText are particularly susceptible to changes like this. Once a user gets accustomed to a control or set of controls being in a certain location they will often return there. If the controls are not where they expect, it is often a laborious hunt to find them again.

Possible solution

User tools should be consistently placed on all pages, with no exceptions. Both at the top right and the bottom would be best.

Feature: User login

Registration can be difficult to find (high priority)

Summary

The login page does double-duty. There is no dedicated registration link, so users must select "Login" then select "Create new account." In the reading order of the page, "Create new account" is near the end. On loading the page, the user name form control is focused.

The combination of these factors caused one user to mistake the login form for a registration form.

Why it matters

This was a very frustrating experience for the user. After entering a user name and password they submitted the form and the page reloaded. The password field was focused, and they missed the "Invalid login" message. They thought they had made a mistake with their user name or password, so kept trying different combinations, wondering if what they were entering was too long, too short, or contained forbidden characters.

After some time the user gave up in frustration, and had to be explicitly directed to the "Create new account" button.

Possible solution

Moodle should have a "Register" or "Signup" link preceding the "Login" link.

User name is easy to mistype or forget

Summary

After registration, users receive an email with an activation link. Upon clicking this link they are presented with a login page. The user name field is empty.

Why it matters

One failure experience by a vision-impaired user was linked to mistyping their user name. They never caught this error, and eventually registered again. If the user name had been already entered for them this would not have been a problem.

Possible solution

After account activation, user name field should be correctly populated.

Feature: User profile

Profile page can be difficult to find (high priority)

Summary

The vision-impaired user found the profile page quickly, by looking at the top right corner and then clicking their name. Blind users had much more trouble, often opening a list of links and laboriously reading through each one, trying to find "settings" or "profile." One user failed entirely, and had to be told which link to use.

Why it matters

It's important not only that users can find their profile page but that Moodle inform them it exists. Without a clear label and path to this page some users may not know that the page exists at all. Those who searched a list of links typically ignored their own name.

Possible solution

Each page should have a link to the user profile page called "your profile" or "your settings."

Advanced options can be difficult to find (high priority)

Summary

Moodle's user profile has "regular" options and "advanced" options. The advanced options are not displayed by default, but appear only when the user has activated the "Show advanced" button.

Once activated, the advanced options are blended in with the regular options. E.g., the options list may start like this:

  • Regular option 1
  • Regular option 2
  • Regular option 3

After "Show advanced" is activated, options may appear this way:

  • Regular option 1
  • Advanced option 1
  • Regular option 2
  • Regular option 3
  • Advanced option 2
  • Advanced option 3
Why it matters

Every user had much trouble with this arrangement. One user failed entirely and had to be told how to find the advanced options. Every user had to read the entire list of options again to find the advanced ones, and sometimes they missed one and had to read the list again.

Possible solution

"Advanced" options should be shown by default; the "Show advanced" button should be removed.

In the user profile, options can be difficult to find

Summary

There are thirty-two settings in the user profile, in four groups:

  • General
  • Picture of
  • Interests
  • Optional

Finding a single option (e.g., to receive plain text email instead of HTML) requires either searching or reading every option in sequence.

Why it matters

"Picture of" and "Interests" are good category names; "General" and "Optional" are not. Users have no way of skipping to a group of options they may be interested in, like email.

Possible solution

Options should be categorized better.

Accessibility options can be difficult to find (medium priority)

Summary

Accessibility options are very difficult to find and to enable.

Why it matters

Accessibility options are unique in that they target a particular group of people, unlike other profile options which are either universally applicable, or useful based on interest or context.

Possible solution

In user profile, accessibility options should be grouped at top in their own section.

In user profile, accessibility options are turned off for new users (high priority)

Summary

Moodle has several accessibility options, among them:

  • Email format
  • When editing text (WYSIWYG controls)
  • AJAX and JavaScript
  • Screen reader

All of these are set to the least accessible choice by default.

Why it matters

Finding and enabling accessibility options is difficult for screen reader users; more difficult than finding and disabling them is for sighted users.

Possible solution

In user profile, accessibility options should be turned on by default.

"When editing text" option can be difficult to understand

Summary

The user profile has an option to turn on or off the WYSIWYG editor. The label for this option is "When editing text." Its possibilities are:

  • Use standard web forms
  • Use HTML editor (some browsers only)
  • No screen reader user found this option on the first attempt.
Why it matters

"WYSIWYG" is a very common acronym, and all users tested understood exactly what it meant. "When editing text" could refer to any number of activities or options.

Possible solution

Make the label more declarative, and include the acronym WYSIWYG. Make the options simpler, e.g., "yes" and "no."

"Screen reader" option can be difficult to understand

Summary

The user profile has an option called "Screen reader." There is no other explanatory text, and no help available. No user understood what this option does.

Why it matters

All screen reader users were at first happy to see this option, but quickly frustrated by its lack of explanation and apparent lack of efficacy.

Possible solution

Use a better label, or include a help icon or more explanatory text.

Feature: User home page

"List of categories" block can be difficult to understand

Summary

The "List of categories" block contains a table with two columns: a category name and the number of courses in that category. There are no labels or column headings to help users understand what those columns represent.

Why it matters

Envisioning a table is much easier for sighted users than screen reader users.

Possible solution

The best solution would be to use a simple list, e.g.,

4 courses in "Beginner" 2 courses in "Prerequisites"

This needs no additional labels or explanations, maintains a column layout, and is easier to read.

"Course categories" title for combo list can be difficult to understand

Summary

It's possible to add several lists of courses to the front page, among them are "List of courses," "List of categories," and "Combo list." These are all useful, but both "List of categories" and "Combo list" are titled "Course categories."

Why it matters

Screen reader users often navigate by page header, and it's impossible to tell from the headers which list they need. This is a non-issue for sighted users, who will focus on the content.

Possible solution

Create a better heading for the combo list.

Course list indication of guest access can be difficult to understand

Summary

Courses may allow or deny access to guest (i.e., non-enrolled) users. In the "Available courses" list, this is indicated by the presence or absence of the "guest" icon. If guest access is allowed this icon is appended to the course title. The icon is also linked to to course home page, and its ALT attribute is, "This course allows guests to enter."

Why it matters

The primary problem is that the icon is also a link. This means that in one of the most common screen reader navigation patterns, reading through a list of all links on the page, the text, "This course allows guest access" appears once for every course. These links are useless in this context because the link text doesn't tell the user where the link leads.

The second problem is that an important course property (guest access not allowed) is indicated only by absence of this icon. If all courses are so configured no indication is provided.

Possible solution

The icon should not be a link. It might be very successful as plain text (e.g., "Open to guests" or "Guest access allowed"). If an icon is desirable and understandable to sighted users, the could be replaced by an icon using CSS.

Feature: Courses

"You must enroll" workflow can be confusing

Summary

When attempting a quiz for a course in which they are not enrolled, the user is presented with the message, "You need to enrol in this course before you can attempt this quiz." There is also a "Continue" button.

The "Continue" button takes the user back to the course home page without enrolling them. Every user thought that this action would enroll them in the course.

Why it matters

For a sighted user this is a relatively quick process, even if they have to perform every step twice. They'll soon learn that "Continue" does not enroll them and search elsewhere for a way to enroll.

Screen reader users do the same thing, but it takes the much, much longer, and causes more frustration.

Possible solution

Add an "Enroll" button; replace the "Continue" button with a "Do not enroll" or "Take me back" link.

Starting a lesson can be difficult

Summary

Each lesson begins with minimal instruction. A course page is crowded and content-rich (on a representative page, JAWS tells me, "page has 24 headings and 96 links"), and it's not easy to know where to start.

Why it matters

Sighted users can easily see that the middle column is likely primary: it's much wider than the other two, and for anyone familiar with the web it looks like content, while the other two columns look like navigation or tools.

Blind users have no way of perceiving this. The "Topic outline" header isn't very meaningful, and there's no explicit "Start here" marker

Possible solution

Each course should start with a small amount of narrative. Even a "Start here" heading with copy like, "Read assignment #1 and follow the links under it" would be very helpful.

Feature: Quizzes

Users couldn't tell the length of a quiz

Summary

A sighted user can quickly see that a quiz has three pages, and tell the length of each page by the size of the browser scroll bar or by quickly scrolling the page. From here they can guess or estimate the number of questions.

The best case for a blind user would be to read up the page from the bottom to determine that the last question is number ten; in my testing no user did this.

Why it matters

A 25-question quiz is a serious time investment for a blind user. Without notice at the beginning they have no way to estimate how long they'll need.

Possible solution

Start each quiz with a brief summary of its properties, e.g., "This quiz has 25 questions on 3 pages."

Quiz workflow can be difficult to understand

Summary

I observed this workflow on multiple occasions:

  • User reads a quiz question.
  • User selects answer and submits it.
  • Page loads at the start of the question, and JAWS starts reading it again.

Users had a variety of reactions, based on their mental model of how the quiz worked:

Some thought they were finished with question #1 and were being read question #2. Some thought a software error had occurred and their answer had not been accepted.

No user, on their first attempt, understood what actually happens: that their answer had been accepted and scored, that they were being told their answer status and resulting score, and that they were presented with the question again in order to read that result.

Why it matters

A sighted user sees the color-coded result right away, and after this doesn't need to read the question. A blind user must read the entire question to understand what has changed, because the answer and score are at the bottom of the question.

Possible solution

This is another difficult problem. I know that Olli Savolainen has been working on this since at least 2008, but his work isn't completed yet.

This is a fundamental part of Moodle, and more design iteration and testing is necessary to ensure that changes do no harm. Were it me, I'd be tempted to start testing a quiz interface which presents only one question at a time.

Visual cues in "Marks" can be confusing

Summary

Each multiple-choice quiz question contains text like this: "Marks: --/1"

Why it matters

To a sighted user this likely means, "You've not yet received a score for this question, and one point is possible." A screen reader user hears, "Marks colon dash dash slash one." This is difficult to understand.

Possible solution

If a question has not been attempted, do not show this text. If a question has been attempted, explicitly state the result: "Marks received: 1 of a possible 1."

Quiz gives number of attempts, but doesn't describe consequences or possibilities

Summary

At the beginning of a quiz the user is told twice how many attempts they are allowed, but is never told the consequences of those attempts. It's not until after the first attempt that some users realize they're penalized for an incorrect first guess. No user understood why they were allowed to make attempts which resulted in no marks (e.g., a third attempt).

Why it matters

Quizzes affect users' grades, something they care about very much. Moodle should be explicit about how those grades are calculated so users can make the best possible decisions about how to take the quiz.

Possible solution

On the quiz summary page, instead of text like this:

  • Attempts allowed: 2
  • Grading method: Last attempt

Moodle should be explicit and more verbose about what exactly that means.

Answer result can be difficult to find

Summary

After answering one quiz question, the page is reloaded at the beginning of that question. The entire question and all answers are then read again. A message about whether the answer was correct doesn't appear until the end of the question.

Why it matters

One user answered the question three times before understanding that the answer had been accepted.

Possible solution

Show something like, "You answered question #1 correctly and received 1 point." Or, "You answered question #2 incorrectly, received zero points, and .5 points penalty. Click here to try the question again."

Answer result can be difficult to interpret

Summary

When a user answers a quiz question the page reloads with either "Correct" in a green box or "Incorrect" in a red box.

Why it matters

It's not immediately clear to a blind user that this refers to their answer. Green and red are strong signals to most sighted users, and that helps them form the association. For blind users it's just one more word.

Possible solution

"Your answer was correct."

Button labels can be confusing

Summary

There are four button labels on every quiz page. "Submit" after every question, and then three at the end of the page:

  • Save without submitting
  • Submit page
  • Submit all and finish

It's not clear what "Submit" means in each context; these labels should be improved.

Why it matters

An incorrect button choice can lead to losing points in a quiz, and users know this. Most were nervous about using the last three buttons, unsure of what would happen.

Possible solution

Potential new labels:

  • Save without scoring (or "marking")
  • Grade all answers on this page
  • Grade all answers on this page and finish the quiz

Obviously those labels are very long, and different labels may be more successful. Regardless of the specific text, new users unfamiliar with Moodle must be the primary audience for these labels.

Final buttons on quiz page can be difficult to find

Summary

No user independently found the last three buttons on a quiz page until they reached the bottom of the page.

Why it matters

The "Save without submitting" button seems particularly useful, but users didn't learn about it until far into the quiz.

Possible solution

An introduction to each quiz, describing how the quiz works and what features are available, would help most users.

Feature: Online text

"Edit my submission" label in online text assignment can be confusing

Summary

Some assignments are of type "online text." The user enters text in a single large textarea, then submits it for grading. The first page in this type of assignment is an introduction. To complete the assignment users must click a button labeled, "Edit my submission." Users had trouble understanding that this was the next step. A typical reaction was, "Edit what submission? I didn't complete this before, did I?"

Why it matters

For a sighted user, this button is clearly placed at the center of the page. There are no other actions near it, and it seems obvious that, regardless of the label, it is the next step. Screen reader users have no idea of placement, or of how the button relates to the assignment.

Possible solution

Regardless of specific terminology, the important thing is for the button to indicate that it is the next appropriate step for a user who has not yet attempted the assignment.

If the user has not completed the assignment, label the button, "Start the assignment." If the user has completed the assignment, keep the label the same, or perhaps change it to, "Edit your assignment."

Feature: Surveys

"Default" label can be confusing

Summary

The COLLES and ATTLS surveys are standard and ship with Moodle; they cannot be edited through the Moodle interface. They are well put-together with appropriate labels, and generally screen reader users had no trouble.

The survey offers six choices to users. Five of these choices have column heading and labels; the sixth has only a label, "Default." "Default" is not an acceptable answer for any question. The survey will refuse to submit if any question is answered with "Default."

Why it matters

Sighted users don't see the "Default" label, they see that this column of radio buttons doesn't have a heading and thus appears not to be an available choice. Screen reader users do not see the headings at all times, relying instead on the labels. Some did not understand what "Default" meant, and others thought it was a valid choice.

The price of confusion for screen reader users is high: after completing the survey, leaving a few questions as "Default," they tried to submit and received an error. They had to read back through each question and its answers, searching for ones answered "Default."

Possible solution

First, a note should be made at the beginning of the survey that all questions are required and must be answered. Second, the "Default" choice should be removed, and the middle option ("Sometimes") selected by default instead.

Certainly there are arguments against this approach, but I can't think of a label for the "Default" option which makes sense in this context. More design and testing is needed.

Feature: Forums

"Subscribe to forum" action can be confusing

Summary

Course members have a "Subscribe to this forum" link in every forum for a course. It's not clear what this link does. The link's title attribute is, "Send me email copies of posts to this forum" and that is very clear, but the screen readers tested read /either/ the link's text or its title, not both.

Why it matters

A blind user has a higher cost for failure than a sighted user, and might waste several minutes clicking a "Subscribe" link and figuring out what it does.

Possible solution

The link text should be clearer, e.g., "Receive email updates from this forum."

The course forums page can be difficult to use

Summary

Each course with forums has a page listing all forums. The forums list is a table with four columns:

  • Forum
  • Description
  • Discussions
  • Subscribed

The "Subscribed" column contains either the word "Yes" or a button labeled "No." These labels are context-free (aside from the table header which the blind user must memorize), and difficult to use.

Why it matters

Blind users searching for forums are as likely to land on this page as on the home page for a specific forum. Also, this is the only page which shows forum activity and subscription status for all forums in the course.

Possible solution

For each forum, provide a set of controls. Either: "You are subscribed" text with a link to "Unsubscribe." or, "You are not subscribed" text with a link to "Subscribe."

Feature: "Choices" feature (poll)

Options layout can be confusing (medium priority)

Summary

A poll is displayed as two table rows, one of answers and another form fields to select those answers.

Why it matters

This layout requires blind users to memorize each answer and count its column position before selecting an answer.

Possible solution

There is a "Display Mode" setting for polls which allows the axis to be flipped, showing one column, with each row containing the form field and associated answer. This should be the default, and the "Display horizontally" option should be removed.

Options can be difficult to use

Summary

Each form control has associated text, but none of this text is wrapped in a LABEL tag.

Why it matters

LABEL tags are used to semantically link a form field and its description. They're a quick way to increase usability for all users.

Possible solution

LABEL tags should be used in polls.

Results table with can be difficult to read

Summary

The results display for a poll is a table, with answers represented as columns. There are three rows:

A header row with the answers. A middle row with image tags displaying a bar graph. A final row showing the results as numbers.

Why it matters

For one thing, a blind user must memorize column headers in order to read the table. The middle row is of no user to blind users.

Possible solution

The display would be more successful if the answers and results were directly linked, e.g., in a list:

  • Red: 4 answers, 50% of total
  • Blue: 2 answers, 25% of total
  • Green: 2 answers, 25% of total

A list like this can easily be turned into an attractive bar graph with CSS.

Feature: Database feature

"http://" content in "add link" text box can be confusing

Summary

Some users ignored this prefix text and typed in their own. This either caused validation errors or corrupted links.

Why it matters

It's generally best practice to avoid partially populating input controls. In this case,it isn't a necessary helper for any user, blind or sighted.

Possible solution

Code to detect a missing "http://" and prepend it to links is trivial.

Navigation for database can be difficult to use (high priority)

Summary

Navigation for the database feature is a list of links, by default:

  • View list
  • View single
  • Search
  • Add entry

This is presented to sighted users as a tab bar, and to screen reader users as a list.

"View list" wasn't used when asked to tell me how many links existed, instead read all link numbers.

Why it matters

Neither a list or tab bar is appropriate in this context. , "Tabs should be reserved for letting users change the view while staying in the same place." This list mixes search, selection, and other controls in one place.

This may seem like a purely academic abstraction, but in fact it matters greatly: the essence of a list is that the items are related to one another. For screen reader users, deeply accustomed to skipping what they perceive as cruft, if the first item or two of a list does not immediately interest them the rest of the list is likely to be skipped.

No screen reader user found the "Search" or "Add entry" features organically, only when explicitly prompted, and then only by searching a list of links. No screen reader user found and used the "View list" feature.

Possible solution

"Add entry" and "Search" should be featured near the top of the page, and not in a list. "View list" should be made renamed "Return to list" and made the default view. The link should only visible when /not/ in list view. "View single" should be removed.

Feature: Chat

Chat with JavaScript can be very difficult to use (high priority)

Summary

JavaScript-based chat was effectively impossible to use. The frames made it difficult for users to form a mental model of the page. The new messages, brought in by JavaScript page refreshes, weren't visible to JAWS and thus weren't read.

Why it matters

Screen reader users cannot participate in the conversation.

Possible solution

More clearly differentiate the two paths into chat, e.g.:

"Enter chat with JavaScript and frames (difficult for users of assistive technology)." "Enter chat with no JavaScript and no frames (more accessible)."

Chat without JavaScript can be difficult to use

Summary

The non-JavaScript chat interface presents three headings on a single page: "Participants," "Send message," and "Messages." New messages appear only when the user clicks "Refresh," a button underneath the "Send message" controls. This button was difficult to find.

Since the page is refreshed often, the participants list is read every time. Without reading the entire list, though, it was impossible to know if anyone had entered or left the chat.

New messages appear at the bottom of the screen, but again, without reading it's not possible to tell how many new messages there are.

Why it matters

It's important to make every effort to include blind and vision-impaired users in this real-time communication. With some changes this feature could be very usable.

Possible solution
  1. Include a short use note at the top of the page, telling users that they must manually refresh the page to see new messages.
  2. On every page load, include a short description of what is new, e.g., "There is 1 new user and 7 new messages."
  3. Change the order to "Messages," "Send message," and "Participants."
  4. Break messages into two groups: new messages since last page load, and old messages.

Feature: Jump menu

The jump menu can be difficult to use (high priority)

Summary

This is well-known in the Moodle development community, but it's still a very important issue: the jump menu is like quicksand for a screen reader user. The jump menu contains a list of every page in the course. Once a page is selected, the user is taken there automatically, with no further action.

This is a problem in a common use case, observed in all screen reader users:

  1. User tabs into the control.
  2. User scrolls up and down, reading the options.
  3. User tabs out, intending to go to the next item on the page.
  4. At this point, the last page they read in the list is loaded.
Why it matters

Once a user focuses the control it's very difficult to exit it without switching to another page. Because this takes place without explicit action on the part of the user it is unexpected behavior, and causes frustration and confusion.

Possible solution

Remove or refactor the jump menu. Even a small change like removing the JavaScript which moves to a new page and replacing it with a button would be helpful. Since it's of little use to blind users, it could be made a user preference, as long as it's turned off by default.

Grouping of items in jump menu can be difficult to discern

Summary

The jump menu uses the OPTGROUP tag to group options. For a sighted user, the effect is similar to a nested list, e.g.:

  • (OPTGROUP) Topic 1
  • (OPTION) Lesson 1: Getting Started
  • (OPTION) Video 1: Starting PowerPoint, Editing the Title Slide
  • (OPTGROUP) Topic 2
  • ...

The OPTIONs are selectable (and selecting them may perform certain actions), but the OPTGROUP heading (e.g., "Topic 1") is not selectable; it is only a label. The problem is that JAWS does not read this label.

Why it matters

Long lists of options are more difficult for blind than sighted users, since each item must be memorized, and moving around the list must be done more intentionally. In this case, OPTGROUP tags may be providing Moodle authors with the illusion that they are adding more accessibility to the jump menu, when in fact they are not.

Possible solution

This is a difficult problem. The WCAG explicitly recommends OPTGROUP tags. Their presence does no harm to blind users but neither do they help.

"Previous activity" and "Next activity" can be confusing

Summary

No user understood what these two links did. Some users interpreted them as breadcrumbs, some could not guess even after using them.

Why it matters

This control is near the top of the page, and unless skipped it's read on nearly every page inside a course. Screen reader users quickly learned to avoid the jump menu; these two links were quickly treated as cruft and ignored by all users.

Possible solution

If the feature is valuable to non-blind users, consider putting it in the bottom of the page source and using CSS to place it at the top. Sighted users will receive the same benefit, and screen reader users won't trip over it as often.

Acknowledgements

Many thanks:

  • to Cheryl Edmonds at CANnect, for months of guidance and good cheer;
  • to Brian Charlson and Mark Sadecki at the Carroll Center for the Blind for several days of assistance and education;
  • to Sherry Hahn at Washington School for the Blind, for good advice and for arranging testing;
  • and of course to my intrepid testers, for literally and metaphorically opening my eyes: Cory Kadlik, Rick Morin, and Jim Eccles.

Without the help of these wonderful people this project would not have been possible. Nonetheless, all errors and omissions are mine and mine alone.

© 2009 CANnect. Licensed as Creative Commons BY-NC-SA.