Huawei recently launched a new mobile app that uses artificial intelligence (AI) to translate children's books into sign language to help deaf children learn to read at a young age.
Through an app called StorySign, the technology is able to read texts in children's books through a phone camera and convert them into sign language, thanks to a character on the screen. The app, in collaboration with the European Union of the Deaf and the British Deaf Association, was born as Huawei believes in using technology for good and pushing the boundaries of what humans can achieve.
There are 32 million deaf children worldwide, and many struggle to learn to read because they can't hear their parents reading them a bedtime story or a teacher repeating sentences – all key milestones to help a child learn at critical stages of learning.
“StorySign is a world first – nothing like it exists yet, yet the concept is incredibly simple, but the technology that makes it work is very complex… it has the potential to change the lives of deaf children,” Huawei UK Global Senior Product Marketing Manager, Peter Gauden, told Mirror UK.
As for how the app works, it’s very simple. Once StorySign is downloaded, the child’s parent can open the app and select a book from the app’s library. All they have to do is point their smartphone camera at the first page of the book and it will highlight each word in the story. A friendly avatar called Star will then read and understand the words and translate them into sign language. This allows parents and children to learn to sign and read together, at their own pace.
However, it's worth mentioning that the app doesn't work with all children's books. Books that StorySign processes will need to be programmed with the app first, but the app library is expanding and will become more extensive in the near future, Huawei said in an article in the Mirror UK.
StorySign is currently only available in the UK and Europe with plans to expand to Asia next year. The app is available for free on Android via the Google Play Store and Huawei AppGallery in 10 languages.
