Towards Improved Replicability of Human Studies in Human-Robot Interaction: Recommendations for Formalized Reporting

Shelly Bagchi; Patrick Holthaus; Gloria Beraldo; Emmanuel Senft; Daniel Hernández García; Zhao Han; Suresh Kumaar Jayaraman; Alessandra Rossi; Connor Esterwood; Antonio Andriella; Paul Pridham

doi:10.1145/3568294.3580162

Towards Improved Replicability of Human Studies in Human-Robot Interaction: Recommendations for Formalized Reporting

Shelly Bagchi, Patrick Holthaus, Gloria Beraldo, Emmanuel Senft, Daniel Hernández García, Zhao Han, Suresh Kumaar Jayaraman, Alessandra Rossi, Connor Esterwood, Antonio Andriella, Paul Pridham

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

18 Downloads (Pure)

Abstract

In this paper, we present a proposed format for reporting human studies in Human-Robot Interaction (HRI). We call for details which are often overlooked or left out of research papers due to space constraints, and propose a standardized format to contain those details in paper appendices. Providing a formalized study reporting method will promote an increase in replicability and reproducibility of HRI studies and encourage meta-analysis and review, ultimately increasing the generalizability and validity of HRI research. Our draft is the first step towards these goals, and we welcome feedback from the HRI community on the included topics.

Original language	English
Title of host publication	HRI '23: Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction
Publisher	ACM Press
Pages	629-633
Number of pages	5
ISBN (Electronic)	9781450399708
DOIs	https://doi.org/10.1145/3568294.3580162
Publication status	Published - 13 Mar 2023

Keywords

guidelines
human studies
replicability
reporting
reproducibility

Access to Document

10.1145/3568294.3580162Licence: Unspecified

Accepted_Version
© 2023 ACM. This is the accepted manuscript version of a conference paper which has been published in final form at https://doi.org/10.1145/3568294.3580162
Accepted author manuscript, 487 KBLicence: Unspecified

Cite this

Bagchi, S., Holthaus, P., Beraldo, G., Senft, E., Hernández García, D., Han, Z., Jayaraman, S. K., Rossi, A., Esterwood, C., Andriella, A., & Pridham, P. (2023). Towards Improved Replicability of Human Studies in Human-Robot Interaction: Recommendations for Formalized Reporting. In HRI '23: Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction (pp. 629-633). ACM Press. https://doi.org/10.1145/3568294.3580162

@inproceedings{6da45d47ef644a1ab76021c56ea40ced,

title = "Towards Improved Replicability of Human Studies in Human-Robot Interaction: Recommendations for Formalized Reporting",

abstract = "In this paper, we present a proposed format for reporting human studies in Human-Robot Interaction (HRI). We call for details which are often overlooked or left out of research papers due to space constraints, and propose a standardized format to contain those details in paper appendices. Providing a formalized study reporting method will promote an increase in replicability and reproducibility of HRI studies and encourage meta-analysis and review, ultimately increasing the generalizability and validity of HRI research. Our draft is the first step towards these goals, and we welcome feedback from the HRI community on the included topics.",

keywords = "guidelines, human studies, replicability, reporting, reproducibility",

author = "Shelly Bagchi and Patrick Holthaus and Gloria Beraldo and Emmanuel Senft and {Hern{\'a}ndez Garc{\'i}a}, Daniel and Zhao Han and Jayaraman, {Suresh Kumaar} and Alessandra Rossi and Connor Esterwood and Antonio Andriella and Paul Pridham",

note = "{\textcopyright} 2023 ACM. This is the accepted manuscript version of a conference paper which has been published in final form at https://doi.org/10.1145/3568294.3580162",

year = "2023",

month = mar,

day = "13",

doi = "10.1145/3568294.3580162",

language = "English",

pages = "629--633",

booktitle = "HRI '23: Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction",

publisher = "ACM Press",

address = "United States",

}

Bagchi, S, Holthaus, P, Beraldo, G, Senft, E, Hernández García, D, Han, Z, Jayaraman, SK, Rossi, A, Esterwood, C, Andriella, A & Pridham, P 2023, Towards Improved Replicability of Human Studies in Human-Robot Interaction: Recommendations for Formalized Reporting. in HRI '23: Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction. ACM Press, pp. 629-633. https://doi.org/10.1145/3568294.3580162

Towards Improved Replicability of Human Studies in Human-Robot Interaction: Recommendations for Formalized Reporting. / Bagchi, Shelly; Holthaus, Patrick; Beraldo, Gloria et al.
HRI '23: Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction. ACM Press, 2023. p. 629-633.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Towards Improved Replicability of Human Studies in Human-Robot Interaction: Recommendations for Formalized Reporting

AU - Bagchi, Shelly

AU - Holthaus, Patrick

AU - Beraldo, Gloria

AU - Senft, Emmanuel

AU - Hernández García, Daniel

AU - Han, Zhao

AU - Jayaraman, Suresh Kumaar

AU - Rossi, Alessandra

AU - Esterwood, Connor

AU - Andriella, Antonio

AU - Pridham, Paul

PY - 2023/3/13

Y1 - 2023/3/13

N2 - In this paper, we present a proposed format for reporting human studies in Human-Robot Interaction (HRI). We call for details which are often overlooked or left out of research papers due to space constraints, and propose a standardized format to contain those details in paper appendices. Providing a formalized study reporting method will promote an increase in replicability and reproducibility of HRI studies and encourage meta-analysis and review, ultimately increasing the generalizability and validity of HRI research. Our draft is the first step towards these goals, and we welcome feedback from the HRI community on the included topics.

AB - In this paper, we present a proposed format for reporting human studies in Human-Robot Interaction (HRI). We call for details which are often overlooked or left out of research papers due to space constraints, and propose a standardized format to contain those details in paper appendices. Providing a formalized study reporting method will promote an increase in replicability and reproducibility of HRI studies and encourage meta-analysis and review, ultimately increasing the generalizability and validity of HRI research. Our draft is the first step towards these goals, and we welcome feedback from the HRI community on the included topics.

KW - guidelines

KW - human studies

KW - replicability

KW - reporting

KW - reproducibility

UR - http://www.scopus.com/inward/record.url?scp=85150429038&partnerID=8YFLogxK

U2 - 10.1145/3568294.3580162

DO - 10.1145/3568294.3580162

M3 - Conference contribution

SP - 629

EP - 633

BT - HRI '23: Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction

PB - ACM Press

ER -

Towards Improved Replicability of Human Studies in Human-Robot Interaction: Recommendations for Formalized Reporting

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this