Navigating the Global Tech Outage: Essential Steps to Recovery

On Friday, businesses worldwide faced a significant challenge. Late Thursday evening, Microsoft users began experiencing Windows Blue Screen of Death (BSOD) errors and reboot loops due to a third-party update from cybersecurity vendor CrowdStrike. This issue has impacted various industries, including banking, travel, medical, government, and manufacturing.

Understanding the Impact

CrowdStrike issued a statement saying they resolved the root cause, but many systems remain down. Also, Microsoft is experiencing outages related to the same update in its Azure and Office 365 services. This situation underscores the importance of robust contingency plans and disaster recovery.

Mitigation Steps:

If rebooting doesn’t resolve the issue, follow these steps:

  1. Boot into Safe Mode or Windows Recovery Environment
  2. Navigate to C:\Windows\System32\drivers\CrowdStrike
  3. Delete the file matching “C-00000291*.sys”
  4. Boot the host normally

For remote access, use tools like Screen Connect or Splashtop if configured to run in safe mode. Be cautious of third-party solutions from unfamiliar vendors.

Azure Virtual Machine Recovery

For Azure VMs, Microsoft has provided steps for offline repair using at attached unmanaged disk. If the VM’s OS disk is encrypted, additional steps are required to unlock it before proceeding with the repair.

Backup and Recovery

If Safe Mode is inaccessible, consider using your Backup and Disaster Recovery (BCDR) solution to virtualize in the cloud or on a local appliance. Choose a recovery point before 19:00 UTC on July 18th.

Support and Resources

Cloud9 is here to support you. Reach out to our team at 508-978-9225 for assistance. We are committed to helping you navigate this challenging time.