The failings of the July outage were due to a mismatch between inputs.
Last month’s CrowdStrike outage was caused by an out-of-bounds memory error.
In a root cause analysis published this week, CrowdStrike said the outage was caused by an out-of-bounds memory read error beyond the end of the input data array.
Reported by SC US, security professionals say the outage that led to system crashes on numerous Windows devices was caused by a faulty sensor included in an update in CrowdStrike’s Falcon software.
Rapid Response Content
According to the root cause analysis, Rapid Response Content (behavioral heuristics) is delivered through Channel Files and interpreted by the sensor’s Content Interpreter, using a regular-expression based engine. Each Rapid Response Content channel file gets associated with a specific Template Type built into a sensor release. The Template Type provides the Content Interpreter with activity data and graph context that’s matched against the Rapid Response Content.
CrowdStrike said the new interprocess communication (IPC) Template Type for Channel File 291 defined 21 input parameter fields, but the integration codes that invoked the Content Interpreter with Channel File 291’s Template Instances supplied only 20 input values to match against it.
So a large cause of the outage, said CrowdStrike, was the mismatch between the 21 inputs validated by the Content Validator versus the 20 provided to the Content Interpreter, causing the latent out-of-bounds read issue in the Content Interpreter. CrowdStrike acknowledged that another factor for the outage was the lack of a specific test for "non-wildcard matching criteria" in the 21st field.
“While this scenario with Channel File 291 is now incapable of recurring, it also informs process improvements and mitigation steps that CrowdStrike is deploying to ensure further enhanced resilience,” the root cause analysis states.
Wakeup Call
In the opening keynote at Black Hat USA in Las Vegas, the outage was described as a “wakeup call for European and U.S. cybersecurity leaders.”
“Sadly, it was an interesting lesson for the bad guys. [They learned] It was one mechanism that started the entire process,” said Hans de Vries, COO of the European Union Agency for Cybersecurity, commenting on the CrowdStrike bungled software update.
Jen Easterly, director of the U.S. Cybersecurity and Infrastructure Security Agency, warned there has been a lot of “irresponsible noise on the CrowdStrike incident” saying the gravity of the outage can’t be ignored.
“It just reinforced what [CISA] has been saying about the importance of technology vendors, developing, designing, testing and deploying software that is secure by design,” she said.
Written by
Dan Raywood
Senior Editor
SC Media UK
Dan Raywood is a B2B journalist with more than 20 years of experience, including covering cybersecurity for the past 16 years. He has extensively covered topics from Advanced Persistent Threats and nation-state hackers to major data breaches and regulatory changes.
He has spoken at events including 44CON, Infosecurity Europe, RANT Conference, BSides Scotland, Steelcon and ESET Security Days.
Outside work, Dan enjoys supporting Tottenham Hotspur, managing mischievous cats, and sampling craft beers.
