Adaptive Smart Environments: Detecting Human Behaviour from Multimodal Observation

It is desirable to enhance the social capabilities of
a smart home environment to become more aware of the
context of the human occupants’ activities. By taking human
behavioural and contextual information into account, this will
potentially improve decision making by the various smart
house systems. Full mesh Wireless Sensor Networks (WSN)
can be used for passive localisation and tracking of people
or objects within a smart home. By monitoring changes in
the propagation field of the monitored area from the link
quality measurements collected from all the nodes of the
network, it is feasible to infer target locations. It is planned
to apply techniques from Radio Tomographic Imaging (RTI)
and machine vision methods, adapted to the idiosyncrasies of
RTI, which will facilitate real-time multiple target tracking in
the University of Hertfordshire Robot House (UHRH). Using
the Robot Operating System (ROS) framework, these data
may then be fused with concurrent data acquired from other
sensor systems (e.g.) 3-D video tracking and ambient audio
detection in order to develop a high level contextual data model
for human behaviour in a smart environment. We present
experimental results which could provide support for human
activity recognition in smart environments.