Classic essay listing common reasons complex systems fail and how to think about system fragility and accidents.
How Complex Systems Fail
Complex systems are more than the sum of their parts. They fail in ways that are non-obvious and non-linear.
Complex systems fail at the seams: interfaces and interactions between subsystems are frequent sites of failure.
Complex systems are difficult to test. Real-world usage reveals failures that isolated tests miss.
Complex systems often exhibit emergent behavior; small changes can cascade into large failures.
Complexity grows over time. Adding parts increases unexpected dependencies and failure modes.
Human operators compensate for system shortcomings, creating brittle reliance on tacit knowledge.
Safety measures and redundancy can create new failure modes through complacency or hidden coupling.
Failures are often preceded by warning signs that are misinterpreted or ignored.
Blame focuses on individuals, but failures usually reflect systemic design flaws.
Improving resilience requires simplifying interfaces, better monitoring, fail-safe defaults, and designing for graceful degradation.
(Original essay outlines typical patterns and recommendations for managing complex engineered and socio-technical systems.)
