Designing and evaluating multimodal interaction for mobile contexts

Saija Lemmelä, Akos Vetek, Kaj Mäkelä, Dari Trendafilov

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    43 Citations (Scopus)


    In this paper we report on our experience on the design and evaluation of multimodal user interfaces in various contexts. We introduce a novel combination of existing design and evaluation methods in the form of a 5-step iterative process and show the feasibility of this method and some of the lessons learned through the design of a messaging application for two contexts (in car, walking). The iterative design process we employed included the following five basic steps: 1) identification of the limitations affecting the usage of different modalities in various contexts (contextual observations and context analysis) 2) identifying and selecting suitable interaction concepts and creating a general design for the multimodal application (storyboarding, use cases, interaction concepts, task breakdown, application UI and interaction design), 3) creating modality-specific UI designs, 4) rapid prototyping and 5) evaluating the prototype in naturalistic situations to find key issues to be taken into account in the next iteration. We have not only found clear indications that context affects users' preferences in the usage of modalities and interaction strategies but also identified some of these. For instance, while speech interaction was preferred in the car environment users did not consider it useful when they were walking. 2D (finger strokes) and especially 3D (tilt) gestures were preferred by walking users.

    Original languageEnglish
    Title of host publicationICMI'08: Procs of the 10th Int Conf on Multimodal Interfaces
    PublisherACM Press
    Number of pages8
    ISBN (Print)9781605581989
    Publication statusPublished - Dec 2008
    Event10th International Conference on Multimodal Interfaces, ICMI'08 - Chania, Crete, Greece
    Duration: 20 Oct 200822 Oct 2008


    Conference10th International Conference on Multimodal Interfaces, ICMI'08
    CityChania, Crete


    • Design
    • Experimentation
    • Human factors
    • Theory


    Dive into the research topics of 'Designing and evaluating multimodal interaction for mobile contexts'. Together they form a unique fingerprint.

    Cite this