Microsoft today announced the general availability of Immersive Reader, an Azure Cognitive Services offering that allows developers to embed text reading and comprehension capabilities into apps to make them more inclusive. Immersive Reader is designed to help users — including those with dyslexia, dysgraphia, and attention deficit hyperactivity disorder (ADHD) — improve their reading skills on any device.
About 30 million U.S. teenagers and adults find reading and writing challenging, according to a 2009 U.S. National Center for Education Statistics survey. Troublingly, low literacy levels are correlated with higher rates of crime, problems navigating the health care system, and financial illiteracy. While factors like income, education, and language background play a role, 35% of the aforementioned 30 million individuals have learning, vision, or hearing disorders.
This reality spurred the launch over three years ago of Microsoft’s Immersive Reader, which saw a 560% uptick in usage between February and May. According to Microsoft, schools are increasingly turning to the tool as they transition to distance learning in response to the pandemic.
Hackathon origins
Immersive Reader is part of Learning Tools, Microsoft’s suite aimed at helping improve reading and writing, especially for people with disabilities. As of June 2018, more than 13 million people were using Learning Tools — up from 100,000 in June 2017 — across apps and platforms in over 40 languages. In June 2020, Microsoft reported that more than 23 million people were using Immersive Reader each month.
Learning Tools was borne out of a project spearheaded by Microsoft Windows accessibility program lead Jeff Petty in 2015. Petty, who was looking for ways to improve accessibility in Windows, reached out to Microsoft’s Advanced Reading Technologies team and consulted with the division’s researchers about which features might be most helpful for struggling readers.
“[Petty] looked at how to bring all of the best science and research around meeting and inclusive design principles to empower students of all abilities,” Eric Boyd, corporate vice president of Azure AI at Microsoft, told VentureBeat in a phone interview. “[The team] saw that this had a lot of value, particularly in the education segment.”
Above: Immersive Reader in OneNote.
With a hand from Mike Tholfsen, principal product manager for Microsoft Education, Petty assembled a group of more than a dozen people to work on a dyslexia-focused toolkit for Microsoft’s 2015 companywide hackathon, including engineers from Microsoft’s office in Vancouver who had developed a prototype for a dyslexic mode in OneNote. The team created an extension for OneNote that offered text-formatting tools to make reading, writing, and note-taking easier, edging out over 3,300 other projects to win the hackathon.
At the time, Tholfsen was in contact with teachers and school leaders around the U.S. about a related product called OneNote Class Notebook, which lets teachers collaborate within personalized workspaces. Immersive Reader was designed with only black and white backgrounds, but after teachers told Tholfsen that some students with Irlen syndrome — a disorder that impacts the brain’s ability to process visual information — preferred reading with backgrounds in specific colors, the design team incorporated the colors.
Immersive Reader debuted in January 2016. Soon after, with the support of Microsoft corporate vice president Eran Megiddo, who oversees education products, Tholfsen pushed to incorporate the functionality into additional apps, platforms, and languages.
Reading access
At its core, Immersive Reader is an application of audiovisual techniques intended to help people read more effectively. A Read Aloud feature reads text out loud with highlighting aimed at improving decoding, fluency, and dictation. Another feature optimizes font spacing in a narrow column view to bolster reading fluency for people with visual crowding concerns. (Visual crowding refers to the inability to distinguish between words or letters when they’re clustered together.)
“Immersive Reader brings together a number of services across Azure into one,” Boyd explained. “It’s things like translations and text-to-speech … And we really wanted to bring something that connected with as many of our developers as possible.”
Immersive Reader also shows breaks between syllables to enhance word recognition and decoding; supports writing instruction and grammar understanding by identifying verbs, nouns, and adjectives; and can be used alongside Microsoft Translate to translate documents and websites into over 70 languages. One of its newer modules is a dictionary that surfaces picture representations of words to increase comprehension — when enabled, selecting a word shows a picture that best describes the word.
Microsoft says there’s evidence Immersive Reader’s tools benefit a broad cross section of readers — not just people with learning disabilities. A 2017 RTI International study of students across grade levels in Bellevue, Washington found that reading comprehension increased 10 percentage points on a nationally normed assessment among those who used Immersive Reader.
In 2017, Immersive Reader came to Outlook on the web, Word Online, OneNote Online, and the OneNote for Windows 10 app with expanded language (e.g., for Gaelic, Maori, and Arabic) and region support. In 2019, beta integration with Microsoft Forms, Whiteboard, and Team Assignments arrived, followed by a feature that shows formatting like bold, underline, italics, and emojis. More recently, Immersive Reader gained natural-sounding neural text-to-speech voices in English, Spanish, German, Chinese, and Italian and became available in Office Lens for iOS and Android, coinciding with its launch in Minecraft: Education Edition. In Minecraft, Immersive Reader can read and translate text, including the text found in menus and character dialogue boxes.
Above: Immersive Reader in Office Lens.
May 2020 marked Immersive Reader’s jump to Microsoft Edge, preceded by the announcement that Immersive Reader would become available as an Azure AI service to let partners leverage it in their products. Beginning with partners like Code.org, Safari Montage, Oxford University Press, Wakelet, Follet, Pear Deck, and Firefly, Microsoft has worked with students and schools to test and solicit feedback about the newly expanded Immersive Reader.
The latest version of Immersive Reader — version 1.1 — supports having pages pretranslated and read aloud automatically. It also adds 15 new neural text-to-speech voices and new languages, including Odia, Northern and Central Kurdish, Pashto, and Dari.
Microsoft says it will roll out over 10,000 new images in the picture dictionary as Immersive Library comes to Outlook Desktop later this fall, the Mail App on Windows 10 in Q4 2020, and Forms on both Outlook.com and mobile devices.
Expanding reach
As of today, Immersive Reader is available as an Azure API with an accompanying software development kit to facilitate the app integration process. (Using it requires a free or paid Azure account.) Boyd anticipates the tool will make its way into other first-party products as the pandemic drives further growth.
“We will work very closely with researchers in the space, as well as [looking] for feedback from customers and teachers and particularly places where we have partnerships with people,” Boyd said. “We will solicit input on things that are going to be the most impactful for customers — that’s the thing that really guides all of our thought processes.”
Author: Kyle Wiggers.
Source: Venturebeat
