Sometimes when a complex story takes us by surprise or knocks us back on our heels, it pays to revisit the events in a somewhat linear fashion. Here’s a brief timeline of what we know leading up to last week’s mass-hack, when hundreds of thousands of Microsoft Exchange Server systems got compromised and seeded with a powerful backdoor Trojan horse program.
When did Microsoft find out about attacks on previously unknown vulnerabilities in Exchange?
Pressed for a date when it first became aware of the problem, Microsoft told KrebsOnSecurity it was initially notified “in early January.” So far the earliest known report came on Jan. 5, from a principal security researcher for security testing firm DEVCOR who goes by the handle “Orange Tsai.” DEVCOR is credited with reporting two of the four Exchange flaws that Microsoft patched on Mar. 2.
Reston, Va.-based Volexity first identified attacks on the flaws on Jan. 6, and officially informed Microsoft about it on Feb. 2. Volexity now says it can see attack traffic going back to Jan. 3. Microsoft credits Volexity with reporting the same two Exchange flaws as DEVCOR.
Danish security firm Dubex says it first saw clients hit on Jan. 18, and reported their incident response findings to Microsoft on Jan. 27.
In a blog post on their discovery, Please Leave an Exploit After the Beep, Dubex said the victims it investigated in January had a “web shell” backdoor installed via the “unifying messaging” module, a component of Exchange that allows an organization to store voicemail and faxes along with emails, calendars, and contacts in users’ mailboxes.
“A unified messaging server also allows users access to voicemail features via smartphones, Microsoft Outlook and Outlook Web App,” Dubex wrote. “Most users and IT departments manage their voicemail separately from their email, and voicemail and email exist as separate inboxes hosted on separate servers. Unified Messaging offers an integrated store for all messages and access to content through the computer and the telephone.”
Dubex says Microsoft “escalated” their issue on Feb. 8, but never confirmed the zero-day with Dubex prior to the emergency patch release on Mar. 2.
How long have the vulnerabilities exploited here been around?
On Mar. 2, Microsoft patched four flaws in Exchange Server 2013 through 2019. Exchange Server 2010 is no longer supported, but the software giant made a “defense in depth” exception and gave Server 2010 users a freebie patch, too. That means the vulnerabilities the attackers exploited have been in the Microsoft Exchange Server code base for more than ten years.
The timeline also means Microsoft had almost two months to push out the patch it ultimately shipped Mar. 2, or else help hundreds of thousands of Exchange customers mitigate the threat from this flaw before attackers started exploiting it indiscriminately.
Here’s a rough timeline as we know it so far:
- Jan. 5: DEVCOR alerts Microsoft of its findings.
- Jan. 6: Volexity spots attacks that use unknown vulnerabilities in Exchange.
- Jan. 8: DEVCOR reports Microsoft had reproduced the problems and verified their findings.
- Jan. 11: DEVCOR snags proxylogon.com, a domain now used to explain its vulnerability discovery process.
- Jan. 27: Dubex alerts Microsoft about attacks on a new Exchange flaw.
- Jan. 29: Trend Micro publishes a blog post about “Chopper” web shells being dropped via Exchange flaws.
- Feb. 2: Volexity warns Microsoft about active attacks on previously unknown Exchange vulnerabilities.
- Feb. 8: Microsoft tells Dubex it has confirmed its findings.
- Feb. 18: Microsoft confirms with DEVCOR a target date of Mar. 9 for publishing security updates for the Exchange flaws.
- Feb. 26-27: Targeted exploitation gradually turns into a global mass-scan; attackers start rapidly backdooring vulnerable servers.
- Mar. 2: A week earlier than previously planned, Microsoft releases updates to plug 4 zero-day flaws.
- Mar. 3: Tens of thousands of Exchange servers compromised worldwide, with thousands more servers getting freshly hacked each hour.
- Mar. 5: KrebsOnSecurity breaks the news that at least 30,000 organizations in the U.S. — and hundreds of thousands worldwide — now have backdoors installed.
- Mar. 5: Wired.com confirms the reported number of victims. White House makes statement expressing concern over the size of the attack. Former CISA head Chris Krebs tweets the numbers are bigger than what’s been reported.
- Mar. 6: CISA says it is aware of “widespread domestic and international exploitation of Microsoft Exchange Server flaws.”
- Mar. 7-Present: Security experts continue effort to notify victims, coordinate remediation, and remain vigilant for “Stage 2” of this attack (further exploitation of already-compromised servers).