The use of multimedia content has increased dramatically over the last few years, but people with limited or no vision have not been able to fully enjoy the benefits of these advances. This new multimedia browsing accessibility tool provides people with visual impairments a level of control more comparable to a sighted person using a mouse. For example, while enjoying a streaming video, visually impaired people can now select the play button by simply pressing a predefined shortcut key instead of searching in the content for buttons that control the video.

Users can also adjust the volume of an individual source in order to identify and listen to different sound sources without losing track of the screen-reading software because of the sound of a video. If a content creator wants to provide a voice narrative for a video, he can write a text script as a piece of metadata; the tool adds the audio descriptions by using text-to-speech engines. Future plans for extending this technology include enabling flexible audio speed control and contributing this work to an open-source development project. Such contribution will accelerate development and adoption of tools that make Web-based multimedia content accessible to the visually impaired.

How does it work?

Usually, people with visual impairments browse Web pages using either screen-reading software (such as JAWS for Windows) or a voice-enabled browser (such as IBM® Home Page Reader). However, these tools cannot handle multimedia applications properly. Visually impaired users cannot see the multimedia control buttons that appear on a screen. In addition, the audio of a streaming video -- which automatically starts playing after the page is loaded -- interferes with the synthesized assistive voice generated from screen-reading software, a vital assistant for visually impaired users. Furthermore, most multimedia content operates only with a mouse rather than a keyboard, making it virtually impossible for visually impaired people to appreciate multimedia content.

IBM Accessibility Internet Browser for Multimedia is built on top of the Eclipse Rich Client Platform, and it works as a stand-alone application. After a user opens a Web page, the tool automatically analyzes the multimedia content embedded inside the page. Then the tool lets the users control the multimedia content -- letting them play, stop, or pause the videos, change the replay speed, adjust the volume, and so on -- by using simple predefined shortcut keys.

The tool also has a function for providing an alternative text-based interface for the content based on XML metadata. By using metadata, the tool can reorganize or simplify the original content and can provide additional information or navigation methods. Although someone must create the metadata manually, once created, the usability of the site is drastically improved. Usually, multimedia content is designed only for sighted users with mouse operations, so it is impossible or difficult to understand the visually rich content with a voice interface and or to control it by using a keyboard.

This tool also provides functions for adding audio descriptions to movies based on XML metadata by using text-to-speech engines. Audio descriptions are usually created by content creators as an additional sound track for a movie. For example, we are presently using an existing text-to-speech engine to add audio descriptions. Not only developers and creators, but also volunteers are able to add audio descriptions easily and cost-effectively.

The system can also provide additional information or navigation methods, thus allowing users to enjoy the content using their preferred methods.

The tool can work with screen readers (JAWS and Window-Eyes) by making it speak as well as it might work as a self-talking browser.

Hisashi "Himi" Miyashita has been with IBM Research at the Tokyo Research Laboratory since 2003. He works on XML processing, distributed computing, and accessibility. Mr. Miyashita has contributed many free software and open-source projects such as the Meadow project and the IIIMF project at OpenI18N WG.

Daisuke Sato joined the IBM Tokyo Research Laboratory in 2005. His research interests include human computer interaction, Web accessibility, and Web usability. Mr. Sato is a member of the Information Processing Society of Japan (IPSJ).

Shin Saito joined the IBM Tokyo Research Laboratory in 2001. His research interests include Web accessibility, Web usability, and static analysis of mark-up and programming languages. Mr. Saito is a member of the Association for Computing Machinery (ACM) and JSSST.

Takashi Itoh is an advisory software engineer at the IBM Tokyo Research Laboratory. He has many years of software development experience in a variety of IBM products, including operating systems, device drivers, speech synthesis, Eclipse tools for WebSphere, and Home Page Reader.

Kentarou Fukuda, Ph.D., joined the IBM Tokyo Research Laboratory in 2000. His research interests include Web accessibility, Web application development, and multimedia systems. Dr. Fukuda is a member of the Institute of Electronics, Information, and Communication Engineers (IEICE) and IPSJ.

Tatsuya Ishihara""s research interests include Web accessibility, Web usability, and pattern recognition. He is a member of the ACM and IEICE.

Hironobu Takagi, Ph.D., joined the IBM Tokyo Research Laboratory in 2000. Since then, he has conducted research and development on nonvisual computer interfaces. Dr. Takagi is a member of the ACM, IPSJ, and JSSST.

Chieko Asakawa, Ph.D., is an IBM Distinguished Engineer. Since joining the Tokyo Research Laboratory in 1985, she has conducted research and development on nonvisual computer interfaces for the blind; these interfaces include the Home Page Reader (1997). Dr. Asakawa received recognition from the Japanese Ministry of Health and Welfare in 1999 and was inducted into the Women in Technology International Hall of Fame in 2003. She is a member of the IBM Academy of Technology, ACM, IEICE, and IPSJ.