Identifying Ethical Hazards in Safety-Critical Systems: The Role of Creativity

Safety-critical systems can present a varied range of ethical hazards to users, operators and other stakeholders. Some of these hazards, such as a lack of fairness or transparency, have been discussed extensively in existing literature and appear in guidance documents and international standards. Others, such as cultural flattening, anthropomorphism, automation bias and systemic racial discrimination, are typically harder to identify and consequently harder to mitigate against. This paper presents an argument that creativity and collaboration play an essential role in ethical hazard analysis, and introduces a category of HAZOP-based techniques which can be used for a structured and creative discussion of such ethical hazards.