Rackspace hosted Exchange suffered a devastating outage beginning December 2, 2022 and is still ongoing as of 12:37 AM December 4th. At first described as connection and login concerns, the guidance was ultimately upgraded to reveal that they were dealing with a security incident.
Rackspace Hosted Exchange Issues
The Rackspace system went down in the morning hours of December 2, 2022. At first there was no word from Rackspace about what the problem was, much less an ETA of when it would be resolved.
Consumers on Buy Twitter Verification reported that Rackspace was not responding to support emails.
This has been quite the day with #Rackspace. Every hosted exchange customer has been down for 14 hours approximately. Assistance isn’t reading/responding to tickets. Updates are unhelpful.
I am worried now that they fell victim to something bad like the ProxyNotShell PoC hack. https://t.co/jchKsAO3Z7
— Joe Sinkwitz (@CygnusSEO) December 2, 2022
A Rackspace client independently messaged me over social media on Friday to relate their experience:
“All hosted Exchange customers down over the previous 16 hours.
Not sure the number of companies that is, but it’s substantial.
They’re serving a 554 long delay bounce so people emailing in aren’t familiar with the bounce for several hours.”
The official Rackspace status page used a running update of the outage however the initial posts had no information other than there was a failure and it was being investigated.
The first authorities upgrade was on December 2nd at 2:49 AM:
“We are examining a concern that is impacting our Hosted Exchange environments. More information will be published as they appear.”
Thirteen minutes later Rackspace began calling it a “connectivity concern.”
“We are examining reports of connectivity problems to our Exchange environments.
Users might experience a mistake upon accessing the Outlook Web App (Webmail) and syncing their email client(s).”
By 6:36 AM the Rackspace updates explained the ongoing problem as “connectivity and login issues” then later that afternoon at 1:54 PM Rackspace revealed they were still in the “examination stage” of the interruption, still trying to figure out what went wrong.
And they were still calling it “connection and login problems” in their Cloud Workplace environments at 4:51 PM that afternoon.
Rackspace Recommends Moving to Microsoft 365
4 hours later on Rackspace described the scenario as a “substantial failure”and began offering their consumers complimentary Microsoft Exchange Strategy 1 licenses on Microsoft 365 as a workaround until they comprehended the problem and might bring the system back online.
The official guidance mentioned:
“We experienced a considerable failure in our Hosted Exchange environment. We proactively closed down the environment to avoid any additional concerns while we continue work to bring back service. As we continue to resolve the origin of the issue, we have an alternate solution that will re-activate your ability to send and receive e-mails.
At no charge to you, we will be offering you access to Microsoft Exchange Plan 1 licenses on Microsoft 365 up until further notice.”
Rackspace Hosted Exchange Security Event
It was not up until almost 24 hr later at 1:57 AM on December 3rd that Rackspace formally revealed that their hosted Exchange service was experiencing a security occurrence.
The statement even more exposed that the Rackspace professionals had powered down and disconnected the Exchange environment.
“After more analysis, we have figured out that this is a security occurrence.
The known impact is isolated to a portion of our Hosted Exchange platform. We are taking needed actions to examine and secure our environments.”
Twelve hours later on that afternoon they updated the status page with more information that their security team and outside specialists were still dealing with solving the failure.
Was Rackspace Service Impacted by a Vulnerability?
Rackspace has actually not launched details of the security event.
A security occasion generally involves a vulnerability and there are two severe vulnerabilities currently in the wile that were covered in November 2022.
These are the two most existing vulnerabilities:
Microsoft Exchange Server Server-Side Demand Forgery (SSRF) Vulnerability
A Server Side Demand Forgery (SSRF) attack permits a hacker to check out and alter information on the server.
Microsoft Exchange Server Remote Code Execution Vulnerability
A Remote Code Execution Vulnerability is one in which an assaulter has the ability to run destructive code on a server.
An advisory published in October 2022 described the effect of the vulnerabilities:
“An authenticated remote opponent can perform SSRF attacks to escalate advantages and carry out arbtirary PowerShell code on vulnerable Microsoft Exchange servers.
As the attack is targeted versus Microsoft Exchange Mail box server, the opponent can potentially get to other resources via lateral motion into Exchange and Active Directory environments.”
The Rackspace failure updates have actually not indicated what the specific problem was, just that it was a security event.
The most existing status update since December 4th stated that the service is still down and customers are motivated to move to the Microsoft 365 service.
Rackspace posted the following on December 4, 2022 at 12:37 AM:
“We continue to make development in addressing the incident. The accessibility of your service and security of your data is of high significance.
We have actually committed substantial internal resources and engaged first-rate external know-how in our efforts to reduce unfavorable impacts to clients.”
It’s possible that the above kept in mind vulnerabilities relate to the security event impacting the Rackspace Hosted Exchange service.
There has actually been no statement of whether client details has been jeopardized. This event is still continuous.
Featured image by Best SMM Panel/Orn Rin