Edmund West explores the history of the Deaf
community's use of technology to communicate with the hearing world
It may soon be possible for the deaf to
sign at a webcam and immediately see their gestures become words on a screen,
like a deaf version of voice to text.
For those born deaf, life can be very
lonely. Speech does not come naturally to you if you've never heard the sound
of your own voice, and neither does lip reading. What does come naturally to
the deaf is sign language. A group of deaf children who have never been taught
to sign will make up their own. Sign is the basis of deaf culture.
In deaf education, there has always been a
debate between oralists (who believe they should be taught with lip reading) and
manualists (those who believe they should be taught with sign language).
Signing was virtually banned at the 1880 Milan congress. This is despite the
fact that the deaf do as well as everyone else when taught with sign. When they
are orally taught, they fall further and further behind because they have to
spend nearly half their education just learning lip reading Making Waves
Webste:
pslt.org
The Portable Sign Language Translator
(PSLT) app (www.pslt.org) has been created by Technabling, a company of less
than ten people. Technabling uses software to assist in the day-to-day living
of vulnerable people.
The company was founded by Dr Ernesto
Compatangelo, a lecturer in computing science at the University of Aberdeen. He
explained, "The aim of the technology is to empower sign language users by
enabling them to overcome the communication challenges they can experience,
through portable technology. Their signs are immediately translated into text,
which can be read by the person they are conversing with. The intention is to
develop an app that is easily accessible and could be used on different
devices. One of the most innovative and exciting aspects of the technology is
that it allows sign language users to actually develop their own signs for
concepts and terms they need to have in their vocabulary."
This last feature is vital, because some
signers have other problems with hand to eye coordination. For example, they
may not be able to sign in the normal way.
The app has been prototyped for the Making
Waves competition, part of the government's Small Business Research Initiative
(SBRI).
The software is able to convert the
gestures into a video stream, match this to its library of signs and create
sentences in real time. It can work on smartphones, tablets and laptops running
Android, Linux or Windows.
What if one sign has more than two
meanings? On the PSLT forum, Dr Compatangelo acknowledged the problem:
"The BSL signs for 'ball' and 'world' are the very same... we have
equipped the PSLT with a natural language disambiguation mechanism. For
instance, if you sign something like 'I kick the ball/world' the PSLT will
decide that what you are likely to mean is 'I kick the ball' rather than 'I
kick the world'. Of course, the angry person who really means that s/he wants
to 'kick the word', in a figurative way, I suppose, will either have to
rephrase the sentence or to provide a further sentence for clarification.
Again, the very same happens when talking: if I say that I want to burn a log,
does it mean that I want to burn a piece of wood or just a list of all my
access records?"
Dr Ernesto Compatangelo will be speaking in
Glasgow on 8th June for the Joint Information Systems Committee Regional
Support Centre's annual conference 'Here Be Dragons' (on Twitter, #iDragon).
Other companies in the Making Waves
competition are Hassell Inclusion, Gamelab UK and Reflex Arc. Together they
have created uKinect, which uses Microsoft's motion tracker for the same
purpose as the PSLT except it converts Makaton to text.
The software is still in Phase 1. Jonathan
Hassell, head of Hassell Inclusion said, "We'd like to invite colleges,
training organisations and employers that work with people with learning
difficulties, people on the autistic spectrum, and people who have communication
difficulties after suffering a stroke, to get in touch with us if they'd like
to help us shape the future of our innovative product."
In Phase 2, Gamelab will use its share of
the funding to build new apps. These include the following: an app designed for
autistic people, being able to track progress in learning signs, video
demonstrations of unfamiliar signs, educational games of varying difficulty and
an app for creating sign dictionaries quickly and cheaply.
In the future, it plans to convert signs to
sounds and vice versa. On 4th April, it set up an online forum where users can
exchange comments and sign dictionaries for free. Technabling will get $240,000
(of a total $800,000) of SBRI funds for Phase 2, where it plans to include the
whole BSL vocabulary and Makaton. It will also add background noise removal and
an avatar that will convert your typing into signs.
What if one sign has more than two
meanings? On the PSLT forum, Dr Compatangelo acknowledged the problem:
"The BSL signs for 'ball' and 'world' are the very same... we have
equipped the PSLT with a natural language disambiguation mechanism. For
instance, if you sign something like 'I kick the ball/world' the PSLT will
decide that what you are likely to mean is 'I kick the ball' rather than 'I
kick the world'. Of course, the angry person who really means that s/he wants
to 'kick the word', in a figurative way, I suppose, will either have to
rephrase the sentence or to provide a further sentence for clarification.
Again, the very same happens when talking: if I say that I want to burn a log,
does it mean that I want to burn a piece of wood or just a list of all my
access records?"
Dr Ernesto Compatangelo will be speaking in
Glasgow on 8th June for the Joint Information Systems Committee Regional
Support Centre's annual conference 'Here Be Dragons' (on Twitter, #iDragon).
uKinect
is an innovative tool which uses Microsoft’s Kinect technology to recognise the
Makaton sign language used by people with learning difficulties
Other companies in the Making Waves
competition are Hassell Inclusion, Gamelab UK and Reflex Arc. Together they
have created uKinect, which uses Microsoft's motion tracker for the same
purpose as the PSLT except it converts Makaton to text.
The software is still in Phase 1. Jonathan
Hassell, head of Hassell Inclusion said, "We'd like to invite colleges,
training organisations and employers that work with people with learning
difficulties, people on the autistic spectrum, and people who have
communication difficulties after suffering a stroke, to get in touch with us if
they'd like to help us shape the future of our innovative product."
In Phase 2, Gamelab will use its share of
the funding to build new apps. These include the following: an app designed for
autistic people, being able to track progress in learning signs, video
demonstrations of unfamiliar signs, educational games of varying difficulty and
an app for creating sign dictionaries quickly and cheaply.
