Trust, systems and accidents: Designing complex systems

Darren Dalcher

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)


Trust is easier to destroy than create, hence, trust in overly applied automation may erode following accidents. This paper looks at the crash of a revolutionary supersonic fighter that resulted from over-reliance on protection technology. The protection system has been automated to the extent that it was impossible for the pilot to regain control and convince the system that there was a problem. Complete trust in the safety of the system has thus been translated into a new kind of computer-assisted error, where in the interest of safety the only possible exit strategy is outlawed by the system leading to the potential (or in this case, the actual) destruction of the system it was meant to protect. Overall, trust appears to be an emergent function that takes in safety issues affecting the entire system as well as the relationship between the product, client and developer and the trade-offs that underpin decisions. Operators, working as part of a system, abrogate some of their responsibility by implicitly or explicitly passing on control to computerised systems. While operators function as an integral part of systems, their trust in the ability and safety of the system plays a key role in the ensuing success or failure.

Original languageEnglish
Title of host publicationProcs10TH IEEE Int Conf and Workshop on the Engineering of Computer-Based Systems
Subtitle of host publicationECBS 2003
EditorsS. Kawada
Place of PublicationLOS ALAMITOS
Number of pages8
ISBN (Print)0-7695-1917-2
Publication statusPublished - 2003
Event10th IEEE International Conference and Workshop on the Engineering of Computer-Based Systems - HUNTSVILLE, United States
Duration: 7 Apr 200310 Apr 2003


Conference10th IEEE International Conference and Workshop on the Engineering of Computer-Based Systems
Country/TerritoryUnited States


Dive into the research topics of 'Trust, systems and accidents: Designing complex systems'. Together they form a unique fingerprint.

Cite this