Algorithms and Interfaces for Automated Non-Visual Skimming

Abstract

In our information-driven web-based society, we are all gradually falling victims to information overload [9]. However, while sighted people are finding ways to sift through information faster, computer users who are blind are experiencing an even greater information overload. These users access computers and Internet using screen-reader software, which reads the information on a computer screen sequentially using computer-generated speech through an audio interface and allows users to navigate using keyboard shortcuts or gestures. This interface does not give them an opportunity to know what content to skip and what to listen to. So, they either listen to all of the content or listen to the first part of each sentence/paragraph before they skip to the next one. In this dissertation, I investigate and develop methods for non-visual skimming that will empower visually impaired users to access digitized information significantly faster and, thus, reduce the cognitive load associated with non-visual browsing. Visual skimming involves quickly looking through content while picking out words and phrases that are emphasized visually and/or carry the most meaning. My preliminary user study with 20 visually impaired participants who were blind has shown that users were able to skim and comprehend text much better if the summary included connected word phrases, as opposed to separate unrelated words, e.g., "hands down" instead of "hands". In this dissertation, I investigate the techniques employed by sighted users to skim web content, and design algorithmic methods to enable a computer-assisted skimming experience for screen-reader users. I design and evaluate algorithms for generating variable-length summaries to support skimming text at different levels of granularity and speed. I also design and evaluate a touch-based and shortcut driven screen-reader interfaces to support non-visual skimming. I have used participatory design approaches to devise all user interfaces. I conducted controlled and in-situ real-world experiments to evaluate the approach and interfaces. The Major contributions of this thesis include: 1) novel interfaces for non-visual skimming on regular computers and touch-screen devices; 2) novel computer algorithms that support skimming interfaces.