You've probably heard about this by now. The only place I ran into it was Thursday night when I tried to log into the site the company has for logging hours at the end of the day. The site was down, I didn't think anything of it at the time since the site has been down before. The next day there were memos about what to do if employees get a BSOD. I was surprised all of their other services worked when I signed in Friday since they're very Windows centric, though by then it was early afternoon.
A faulty software update from cybersecurity vendor Crowdstrike crippled countless Microsoft Windows computers across the globe today, disrupting everything from airline travel and financial institutions to hospitals and businesses online. Crowdstrike said a fix has been deployed, but experts say the recovery from this outage could take some time, as Crowdstrike’s solution needs to be applied manually on a per-machine basis.
Earlier today, an errant update shipped by Crowdstrike began causing Windows machines running the software to display the dreaded “Blue Screen of Death,” rendering those systems temporarily unusable. Like most security software, Crowdstrike requires deep hooks into the Windows operating system to fend off digital intruders, and in that environment a tiny coding error can quickly lead to catastrophic outcomes.
In a post on Twitter/X, Crowdstrike CEO George Kurtz said an update to correct the coding mistake has been shipped, and that Mac and Linux systems are not affected.
“This is not a security incident or cyberattack,” Kurtz said on Twitter, echoing a written statement by Crowdstrike. “The issue has been identified, isolated and a fix has been deployed.”
Posting to Twitter/X, the director of Crowdstrike’s threat hunting operations said the fix involves booting Windows into Safe Mode or the Windows Recovery Environment (Windows RE), deleting the file “C-00000291*.sys” and then restarting the machine.
No, I hadn't heard about that, but it's not surprising. Security software is often the cure that's worse than the disease. For that type, where it has to load filter drivers, it's a disconnect in between whatever it was filtering. I hate that shit. I remember hunting and poking for norton and mccrappy drivers and registry entries. I'd rather have to format my fucking computer than run that kind of shitware.
I had a security filtering service called "proofpoint" wreck my day the other day. Suddenly, our emails to financial advisors and stuff were bouncing, my IP address (this server) was blacklisted. Muggins here had to deal with it pronto. I don't think it was me specifically, but a misconfiguration because it resolved itself too quickly the next day. I submitted a "false detection" request, to try to get removed and also Mom got the bank's people on it, who said that shouldn't be happening. Mom said that by morning she was able to email people there. It wasn't only the bank, but other financial services too. Proofpoint customers actually have some control over that, they have a form to access, I noted while at the site but even that was too fast for the right hands to get on it.
LOL, actually I did read some headlines yesterday that a big IT outage caused a lot of havoc, but I thought it was some cloud thing, not that it was a bunch of Windows computers blue screening because of security software filters. That's so 2001.
I had a couple of disconnects from the web here yesterday around noon. They only lasted about a minute each time. Could've been related to that I guess. I hadn't heard about the Cloudstrike thing at the time. I read about it later in the afternoon.
I imagine they will lose a big chunk of their customers over this.
I saw a few idiot news media sites calling it a 'Windows outage' ..
I'm not sure how that would affect your web connectivity, you wouldn't be routed through Windows servers. Maybe some web servers running IIS shutting down, but I can't see backbone and trunk routers etc. being shut down.
That's right that people don't understand it. For example my parents were asking me why their systems weren't affected by this. "Because I don't put stupid fucking security software on your computers!" (then I explained that it was this specific security software etc.)
Grogan wrote: Sun Jul 21, 2024 3:07 am
I'm not sure how that would affect your web connectivity, you wouldn't be routed through Windows servers. Maybe some web servers running IIS shutting down, but I can't see backbone and trunk routers etc. being shut down.
That's right that people don't understand it. For example my parents were asking me why their systems weren't affected by this. "Because I don't put stupid fucking security software on your computers!" (then I explained that it was this specific security software etc.)
That's kind of what I thought too, but it was just something strange that had never happened before. Usually it would be a problem with the local provider and would need a router reboot, or wait until they reboot everybody.
Ya, we can't expect normal people to understand a thing like that, especially when the mass news media always sensationalize any tech news to scare people.
Now that I think about it a little more, the people operating the equipment are probably using Windows. Control panels, terminals etc. So if routing tables need manual intervention or something, it's pandefuckingmonium at your ISP or whatever peering communications companies etc.
Antivirus filters can cause blue screens, instability and performance problems. If they get broken, there's a disconnect in the i/o that it was usurping. I used to spend hours hunting and poking for McNorton drivers before those cunts came out with the best programs they ever wrote: The Norton Removal Tool and the McAfee Consumer Product Removal Tool (MCRT)
I remember myself, around 2007 when I resigned to keeping a Windows install for gaming (Vista at that time) even my favourite, ultra efficient Nod32 used to piss me off. I'd be annoyed in Call of Duty (and by the way, that shit brought out the worst in me) and suddenly my motion would be all jerky. I found it was nod32 updating and scanning etc. GONE... after that, my favourite antivirus software was none.
Speaking of removal tools, this popped up on my work laptop today:
FalconServiceUninstallToolFail.jpg (70.32 KiB) Viewed 56050 times
That's Crowdstrike, they pushed this out right at the end of the workday - for normal people that is. For me it was the middle of my workday. And that window could not be closed (pops right back) or minimized. Logging out or rebooting might get rid of it but it would take 10+ mins to get everything back up, so I just pushed it off to the side so I could get my work done. I'll deal with it later.
..
