Image Based Interactive Device for Effective coMmunication
TIDE Project 1038
The main objective of IBIDEM is to develop a videophone useful for
lip reading by hearing-impaired people as well as provide
capabilities for remote monitoring based on a novel type of
space-variant visual sensor and using standard telephone lines.
Long-distance communication for both social and practical purposes is
becoming an increasingly important factor in every-day life.
Hearing-impairment does, however, prevent many people from using
normal voice telephones for obviuos reasons. A solution to this
problem for the hearing-impaired is the use of videophones. Currently
available videophones working on standard telephone lines (PSTN) do, however,
not meet the dynamic requirements necessary for lip reading. The
spatial resolution is also too small.
In order to facilitate lip reading, signing, and finger spelling,
IBIDEM will develop a videophone based on a novel type of visual
sensor matching the resolution of the human retina in both the spatial
and temporal domains (a retina-like or space-variant sensor). Members
of the IBIDEM consortium already hold a patent in Europe and the US
for a prototype of such an imaging device. The geometry of the visual
sensor, similarly to the human retina, has a high resolution in the
central part and a degrading resolution in the peripheral visual
field, as shown in the figure. This solution results in a
reduction of the number of pixels of
the acquired image (allowing a higher transmission rate on standard
telephone lines) without degrading the perceptual appearance of the
image, as can be seen from figure.
The figure shows in the background the layout of a space-variant sensor
having a resolution of 128 photoreceptor on each of the 64
eccentricity; the total amount of pixel is then 8192.
The circular image represents the output of such a sensor during
"finger spelling" (this is the image that will be shown) while the rectangular
image on the right shows (enlarged) the log-polar representation of the image
(this is the image that is being received/transmitted).
A second objective of IBIDEM is the use of the same equipment for
remote monitoring of health status. The system can be used for
obtaining information about the status of a client in the form of
images and could be extended to include various physiological
parameters like heart rate, blood pressure etc.
The IBIDEM project will construct a videophone using a camera with the
retinal sensor, a motorized system for moving the point of view of the
camera as well as a LCD to display the transmitted images. This
videophone will be a high-quality, low-cost aid for both the
hearing-impaired as well as being useful for remote monitoring.
The videophone will be designed with active participation of members
of the deaf and hard-of-hearing community, and will be demonstrated by
inter-personal communication between two speakers one or both with
hearing disabilities as well as in a situation with remotely
monitoring health conditions using audio and visual information.
Click here to view a sequence of a "finger spelling" experiment.
This sequence has been obtained using a simulated sensor of about 8,000
pixels. The sequence shows the images as they could appear on the
screen of the videophone (sorry for the greenish appearance, in
reality will be B/W).
Dr. Fabrizio Ferrari tel. +39-10-3620102
UNITEK Consortium fax. +39-10-314873
via Pisa 12/1 e-mail: firstname.lastname@example.org
16147 Genova - Italy
Participants Role (Nation) Principal Investigators
UNITEK Consortium Coordinator (I) Jan Nielsen, Andrea Mannucci
IMEC Partner (B) Belgium Lou Hermans, Bart Dierickx
Thomson-CSF Partner (FR) Eric Auffret, Christophe Guerin
Knossos Technologies Partner (GR) Andreas Manganas, Antonis Argyros
DIST - Univ. of Genoa Partner (I) Giulio Sandini, Paolo Questa
Scuola Sup. S. Anna Partner (I) Paolo Dario, Alessandra Sassi
Instituut voor Doven Associate (NL) Han Frowein, Wendy Woelders
Scienzia Machinale Srl Subcon. (I) Luca Bosio
A.F.A. Centro R.E.U.L. Subcon. (I) Silvana Baroni, Claudio Rossi