‘Tis the Season for the Wayward Package Phish

November 4, 2021
26 Comments

The holiday shopping season always means big business for phishers, who tend to find increased success this time of year with a lure about a wayward package that needs redelivery. Here’s a look at a fairly elaborate SMS-based phishing scam that spoofs FedEx in a bid to extract personal and financial information from unwary recipients.

One of dozens of FedEx-themed phishing sites currently being advertised via SMS spam.

Louis Morton, a security professional based in Fort Worth, Texas, forwarded an SMS phishing or “smishing” message sent to his wife’s mobile device that indicated a package couldn’t be delivered.

“It is a nearly perfect attack vector at this time of year,” Morton said. “A link was included, implying that the recipient could reschedule delivery.”

Attempting to visit the domain in the phishing link — o001cfedeex[.]com — from a desktop web browser redirects the visitor to a harmless page with ads for car insurance quotes. But by loading it in a mobile device (or by mimicking one using developer tools), we can see the intended landing page pictured in the screenshot to the right — returns-fedex[.]com.

Blocking non-mobile users from visiting the domain can help minimize scrutiny of the site from non-potential victims, such as security researchers, and thus potentially keep the scam site online longer.

Clicking “Schedule new delivery” brings up a page that requests your name, address, phone number and date of birth. Those who click “Next Step” after providing that information are asked to add a payment card to cover the $2.20 “redelivery fee.” Continue reading →

The ‘Groove’ Ransomware Gang Was a Hoax

November 2, 2021
7 Comments

A number of publications in September warned about the emergence of “Groove,” a new ransomware group that called on competing extortion gangs to unite in attacking U.S. government interests online. It now appears that Groove was all a big hoax designed to toy with security firms and journalists.

“An appeal to business brothers!” reads the Oct. 22 post from Groove calling for attacks on the United States government sector.

Groove was first announced Aug. 22 on RAMP, a new and fairly exclusive Russian-language darknet cybercrime forum.

“GROOVE is first and foremost an aggressive financially motivated criminal organization dealing in industrial espionage for about two years,” wrote RAMP’s administrator “Orange” in a post asking forum members to compete in a contest for designing a website for the new group. “Let’s make it clear that we don’t do anything without a reason, so at the end of the day, it’s us who will benefit most from this contest.”

According to a report published by McAfee, Orange launched RAMP to appeal to ransomware-related threat actors who were were ousted from major cybercrime forums for being too toxic, or to cybercriminals who complained of being short-changed or stiffed altogether by different ransomware affiliate programs.

The report said RAMP was the product of a dispute between members of the Babuk ransomware gang, and that its members likely had connections to another ransomware group called BlackMatter.

“[McAfee] believes, with high confidence, that the Groove gang is a former affiliate or subgroup of the Babuk gang, who are willing to collaborate with other parties, as long as there is financial gain for them,” the report said. “Thus, an affiliation with the BlackMatter gang is likely.”

In the first week of September, Groove posted on its darknet blog nearly 500,000 login credentials for customers of Fortinet VPN products, usernames and passwords that could be used to remotely connect to vulnerable systems. Fortinet said the credentials were collected from systems that hadn’t yet implemented a patch issued in May 2019.

Some security experts said the post of the Fortinet VPN usernames and passwords was aimed at drawing new affiliates to Groove. But it seems more likely the credentials were posted to garner the attention of security researchers and journalists.

Sometime in the last week, Groove’s darknet blog disappeared. In a post on the Russian cybercrime forum XSS, an established cybercrook using the handle “Boriselcin” explained that Groove was little more than a pet project to screw with the media and security industry.

“For those who don’t understand what’s going on: I set up a fake Groove Gang and named myself a gang,” Boriselcin wrote. The rest of the post reads:

“They ate it up, I dumped 500k old Fortinet [access credentials] that no one needed and they ate it up. I say that I am going to target the U.S. government sector and they eat it up. Few journalists realized that this was all a show, a fake, and a scam! And my respect goes out to those who figured it out. I don’t even know what to do now with this blog with a ton of traffic. Maybe sell it? Now I just need to start writing [the article], but I can’t start writing it without checking everything.”

A review of Boriselcin’s recent postings on XSS indicate he has been planning this scheme for several months. On Sept. 13, Boriselcin posted that “several topics are ripening,” and that he intended to publish an article about duping the media and security firms.

“Manipulation of large information security companies and the media through a ransom blog,” he wrote. “It’s so funny to read Twitter and the news these days 🙂 But the result is great so far. Triggering the directors of information security companies. We fuck the supply chain of the information security office.”

Image: @nokae8

Continue reading →

Advertisement

‘Trojan Source’ Bug Threatens the Security of All Code

November 1, 2021
54 Comments

Virtually all compilers — programs that transform human-readable source code into computer-executable machine code — are vulnerable to an insidious attack in which an adversary can introduce targeted vulnerabilities into any software without being detected, new research released today warns. The vulnerability disclosure was coordinated with multiple organizations, some of whom are now releasing updates to address the security weakness.

Researchers with the University of Cambridge discovered a bug that affects most computer code compilers and many software development environments. At issue is a component of the digital text encoding standard Unicode, which allows computers to exchange information regardless of the language used. Unicode currently defines more than 143,000 characters across 154 different language scripts (in addition to many non-script character sets, such as emojis).

Specifically, the weakness involves Unicode’s bi-directional or “Bidi” algorithm, which handles displaying text that includes mixed scripts with different display orders, such as Arabic — which is read right to left — and English (left to right).

But computer systems need to have a deterministic way of resolving conflicting directionality in text. Enter the “Bidi override,” which can be used to make left-to-right text read right-to-left, and vice versa.

“In some scenarios, the default ordering set by the Bidi Algorithm may not be sufficient,” the Cambridge researchers wrote. “For these cases, Bidi override control characters enable switching the display ordering of groups of characters.”

Bidi overrides enable even single-script characters to be displayed in an order different from their logical encoding. As the researchers point out, this fact has previously been exploited to disguise the file extensions of malware disseminated via email.

Here’s the problem: Most programming languages let you put these Bidi overrides in comments and strings. This is bad because most programming languages allow comments within which all text — including control characters — is ignored by compilers and interpreters. Also, it’s bad because most programming languages allow string literals that may contain arbitrary characters, including control characters.

“So you can use them in source code that appears innocuous to a human reviewer [that] can actually do something nasty,” said Ross Anderson, a professor of computer security at Cambridge and co-author of the research. “That’s bad news for projects like Linux and Webkit that accept contributions from random people, subject them to manual review, then incorporate them into critical code. This vulnerability is, as far as I know, the first one to affect almost everything.”

The research paper, which dubbed the vulnerability “Trojan Source,” notes that while both comments and strings will have syntax-specific semantics indicating their start and end, these bounds are not respected by Bidi overrides. From the paper:

“Therefore, by placing Bidi override characters exclusively within comments and strings, we can smuggle them into source code in a manner that most compilers will accept. Our key insight is that we can reorder source code characters in such a way that the resulting display order also represents syntactically valid source code.”

“Bringing all this together, we arrive at a novel supply-chain attack on source code. By injecting Unicode Bidi override characters into comments and strings, an adversary can produce syntactically-valid source code in most modern languages for which the display order of characters presents logic that diverges from the real logic. In effect, we anagram program A into program B.”

Anderson said such an attack could be challenging for a human code reviewer to detect, as the rendered source code looks perfectly acceptable.

“If the change in logic is subtle enough to go undetected in subsequent testing, an adversary could introduce targeted vulnerabilities without being detected,” he said.

Equally concerning is that Bidi override characters persist through the copy-and-paste functions on most modern browsers, editors, and operating systems.

“Any developer who copies code from an untrusted source into a protected code base may inadvertently introduce an invisible vulnerability,” Anderson told KrebsOnSecurity. “Such code copying is a significant source of real-world security exploits.”

Image: XKCD.com/2347/

Matthew Green, an associate professor at the Johns Hopkins Information Security Institute, said the Cambridge research clearly shows that most compilers can be tricked with Unicode into processing code in a different way than a reader would expect it to be processed.

“Before reading this paper, the idea that Unicode could be exploited in some way wouldn’t have surprised me,” Green told KrebsOnSecurity. “What does surprise me is how many compilers will happily parse Unicode without any defenses, and how effective their right-to-left encoding technique is at sneaking code into codebases. That’s a really clever trick I didn’t even know was possible. Yikes.”

Green said the good news is that the researchers conducted a widespread vulnerability scan, but were unable to find evidence that anyone was exploiting this. Yet.

“The bad news is that there were no defenses to it, and now that people know about it they might start exploiting it,” Green said. “Hopefully compiler and code editor developers will patch this quickly! But since some people don’t update their development tools regularly there will be some risk for a while at least.” Continue reading →

Zales.com Leaked Customer Data, Just Like Sister Firms Jared, Kay Jewelers Did in 2018

October 28, 2021
16 Comments

In December 2018, bling vendor Signet Jewelers fixed a weakness in their Kay Jewelers and Jared websites that exposed the order information for all of their online customers. This week, Signet subsidiary Zales.com updated its website to remediate a nearly identical customer data exposure.

Last week, KrebsOnSecurity heard from a reader who was browsing Zales.com and suddenly found they were looking at someone else’s order information on the website, including their name, billing address, shipping address, phone number, email address, items and total amount purchased, delivery date, tracking link, and the last four digits of the customer’s credit card number.

The reader noticed that the link for the order information she’d stumbled on included a lengthy numeric combination that — when altered — would produce yet another customer’s order information.

When the reader failed to get an immediate response from Signet, KrebsOnSecurity contacted the company. In a written response, Signet said, “A concern was brought to our attention by an IT professional. We addressed it swiftly, and upon review we found no misuse or negative impact to any systems or customer data.”

Their statement continues:

“As a business principle we make consumer information protection the highest priority, and proactively initiate independent and industry-leading security testing. As a result, we exceed industry benchmarks on data protection maturity. We always appreciate it when consumers reach out to us with feedback, and have committed to further our efforts on data protection maturity.”

When Signet fixed similar weaknesses with its Jared and Kay websites back in 2018, the reader who found and reported that data exposure said his mind quickly turned to the various ways crooks might exploit access to customer order information.

“My first thought was they could track a package of jewelry to someone’s door and swipe it off their doorstep,” said Brandon Sheehy, a Dallas-based Web developer. “My second thought was that someone could call Jared’s customers and pretend to be Jared, reading the last four digits of the customer’s card and saying there’d been a problem with the order, and if they could get a different card for the customer they could run it right away and get the order out quickly. That would be a pretty convincing scam. Or just targeted phishing attacks.” Continue reading →

FBI Raids Chinese Point-of-Sale Giant PAX Technology

October 26, 2021
165 Comments

U.S. federal investigators today raided the Florida offices of PAX Technology, a Chinese provider of point-of-sale devices used by millions of businesses and retailers globally. KrebsOnSecurity has learned the raid is tied to reports that PAX’s systems may have been involved in cyberattacks on U.S. and E.U. organizations.

FBI agents entering PAX Technology offices in Jacksonville today. Source: WOKV.com.

Headquartered in Shenzhen, China, PAX Technology Inc. has more than 60 million point-of-sale terminals in use throughout 120 countries. Earlier today, Jacksonville, Fla. based WOKV.com reported that agents with the FBI and Department of Homeland Security (DHS) had raided a local PAX Technology warehouse.

In an official statement, investigators told WOKV only that they were executing a court-authorized search at the warehouse as a part of a federal investigation, and that the inquiry included the Department of Customs and Border Protection and the Naval Criminal Investigative Services (NCIS). The FBI has not responded to requests for comment.

Several days ago, KrebsOnSecurity heard from a trusted source that the FBI began investigating PAX after a major U.S. payment processor started asking questions about unusual network packets originating from the company’s payment terminals.

According to that source, the payment processor found that the PAX terminals were being used both as a malware “dropper” — a repository for malicious files — and as “command-and-control” locations for staging attacks and collecting information.

“FBI and MI5 are conducting an intensive investigation into PAX,” the source said. “A major US payment processor began asking questions about network packets originating from PAX terminals and were not given any good answers.”

KrebsOnSecurity reached out to PAX Technology’s CEO on Sunday. The company has not yet responded to requests for comment.

The source said two major financial providers — one in the United States and one in the United Kingdom — had already begun pulling PAX terminals from their payment infrastructure, a claim that was verified by two different sources. Continue reading →

Conti Ransom Gang Starts Selling Access to Victims

October 25, 2021
13 Comments

The Conti ransomware affiliate program appears to have altered its business plan recently. Organizations infected with Conti’s malware who refuse to negotiate a ransom payment are added to Conti’s victim shaming blog, where confidential files stolen from victims may be published or sold. But sometime over the past 48 hours, the cybercriminal syndicate updated its victim shaming blog to indicate that it is now selling access to many of the organizations it has hacked.

A redacted screenshot of the Conti News victim shaming blog.

“We are looking for a buyer to access the network of this organization and sell data from their network,” reads the confusingly worded message inserted into multiple recent victim listings on Conti’s shaming blog.

It’s unclear what prompted the changes, or what Conti hopes to gain from the move. It’s also not obvious why they would advertise having hacked into companies if they plan on selling that access to extract sensitive data going forward. Conti did not respond to requests for comment.

“I wonder if they are about to close down their operation and want to sell data or access from an in-progress breach before they do,” said Fabian Wosar, chief technology officer at computer security firm Emsisoft. “But it’s somewhat stupid to do it that way as you will alert the companies that they have a breach going on.”

The unexplained shift comes as policymakers in the United States and Europe are moving forward on efforts to disrupt some of the top ransomware gangs. Reuters recently reported that the U.S. government was behind an ongoing hacking operation that penetrated the computer systems of REvil, a ransomware affiliate group that experts say is about as aggressive and ruthless as Conti in dealing with victims. What’s more, REvil was among the first ransomware groups to start selling its victims’ data.

REvil’s darknet victim shaming site remains offline. In response, a representative for the Conti gang posted a long screed on Oct. 22 to a Russian language hacking forum denouncing the attack on REvil as the “unilateral, extraterritorial, and bandit-mugging behavior of the United States in world affairs.” Continue reading →

Missouri Governor Vows to Prosecute St. Louis Post-Dispatch for Reporting Security Vulnerability

October 14, 2021
173 Comments

On Wednesday, the St. Louis Post-Dispatch ran a story about how its staff discovered and reported a security vulnerability in a Missouri state education website that exposed the Social Security numbers of 100,000 elementary and secondary teachers. In a press conference this morning, Missouri Gov. Mike Parson (R) said fixing the flaw could cost the state $50 million, and vowed his administration would seek to prosecute and investigate the “hackers” and anyone who aided the publication in its “attempt to embarrass the state and sell headlines for their news outlet.”

Missouri Gov. Mike Parson (R), vowing to prosecute the St. Louis Post-Dispatch for reporting a security vulnerability that exposed teacher SSNs.

The Post-Dispatch says it discovered the vulnerability in a web application that allowed the public to search teacher certifications and credentials, and that more than 100,000 SSNs were available. The Missouri state Department of Elementary and Secondary Education (DESE) reportedly removed the affected pages from its website Tuesday after being notified of the problem by the publication (before the story on the flaw was published).

The newspaper said it found that teachers’ Social Security numbers were contained in the HTML source code of the pages involved. In other words, the information was available to anyone with a web browser who happened to also examine the site’s public code using Developer Tools or simply right-clicking on the page and viewing the source code.

The Post-Dispatch reported that it wasn’t immediately clear how long the Social Security numbers and other sensitive information had been vulnerable on the DESE website, nor was it known if anyone had exploited the flaw.

But in a press conference Thursday morning, Gov. Parson said he would seek to prosecute and investigate the reporter and the region’s largest newspaper for “unlawfully” accessing teacher data.

“This administration is standing up against any and all perpetrators who attempt to steal personal information and harm Missourians,” Parson said. “It is unlawful to access encoded data and systems in order to examine other peoples’ personal information. We are coordinating state resources to respond and utilize all legal methods available. My administration has notified the Cole County prosecutor of this matter, the Missouri State Highway Patrol’s Digital Forensics Unit will also be conducting an investigation of all of those involved. This incident alone may cost Missouri taxpayers as much as $50 million.”

While threatening to prosecute the reporters to the fullest extent of the law, Parson sought to downplay the severity of the security weakness, saying the reporter only unmasked three Social Security numbers, and that “there was no option to decode Social Security numbers for all educators in the system all at once.”

“The state is committed to bringing to justice anyone who hacked our systems or anyone who aided them to do so,” Parson continued. “A hacker is someone who gains unauthorized access to information or content. This individual did not have permission to do what they did. They had no authorization to convert or decode, so this was clearly a hack.” Continue reading →

How Coinbase Phishers Steal One-Time Passwords

October 13, 2021
30 Comments

A recent phishing campaign targeting Coinbase users shows thieves are getting smarter about phishing one-time passwords (OTPs) needed to complete the login process. It also shows that phishers are attempting to sign up for new Coinbase accounts by the millions as part of an effort to identify email addresses that are already associated with active accounts.

A Google-translated version of the now-defunct Coinbase phishing site, coinbase.com.password-reset[.]com

Coinbase is the world’s second-largest cryptocurrency exchange, with roughly 68 million users from over 100 countries. The now-defunct phishing domain at issue — coinbase.com.password-reset[.]com — was targeting Italian Coinbase users (the site’s default language was Italian). And it was fairly successful, according to Alex Holden, founder of Milwaukee-based cybersecurity firm Hold Security.

Holden’s team managed to peer inside some poorly hidden file directories associated with that phishing site, including its administration page. That panel, pictured in the redacted screenshot below, indicated the phishing attacks netted at least 870 sets of credentials before the site was taken offline.

The Coinbase phishing panel.

Holden said each time a new victim submitted credentials at the Coinbase phishing site, the administrative panel would make a loud “ding” — presumably to alert whoever was at the keyboard on the other end of this phishing scam that they had a live one on the hook.

In each case, the phishers manually would push a button that caused the phishing site to ask visitors for more information, such as the one-time password from their mobile app.

“These guys have real-time capabilities of soliciting any input from the victim they need to get into their Coinbase account,” Holden said.

Pressing the “Send Info” button prompted visitors to supply additional personal information, including their name, date of birth, and street address. Armed with the target’s mobile number, they could also click “Send verification SMS” with a text message prompting them to text back a one-time code.

SIFTING COINBASE FOR ACTIVE USERS

Holden said the phishing group appears to have identified Italian Coinbase users by attempting to sign up new accounts under the email addresses of more than 2.5 million Italians. His team also managed to recover the username and password data that victims submitted to the site, and virtually all of the submitted email addresses ended in “.it”.

But the phishers in this case likely weren’t interested in registering any accounts. Rather, the bad guys understood that any attempts to sign up using an email address tied to an existing Coinbase account would fail. After doing that several million times, the phishers would then take the email addresses that failed new account signups and target them with Coinbase-themed phishing emails.

Holden’s data shows this phishing gang conducted hundreds of thousands of halfhearted account signup attempts daily. For example, on Oct. 10 the scammers checked more than 216,000 email addresses against Coinbase’s systems. The following day, they attempted to register 174,000 new Coinbase accounts. Continue reading →

Patch Tuesday, October 2021 Edition

October 12, 2021
16 Comments

Microsoft today issued updates to plug more than 70 security holes in its Windows operating systems and other software, including one vulnerability that is already being exploited. This month’s Patch Tuesday also includes security fixes for the newly released Windows 11 operating system. Separately, Apple has released updates for iOS and iPadOS to address a flaw that is being actively attacked.

Firstly, Apple has released iOS 15.0.2 and iPadOS 15.0.2 to fix a zero-day vulnerability (CVE-2021-30883) that is being leveraged in active attacks targeting iPhone and iPad users. Lawrence Abrams of Bleeping Computer writes that the flaw could be used to steal data or install malware, and that soon after Apple patched the bug security researcher Saar Amar published a technical writeup and proof-of-concept exploit derived from reverse engineering Apple’s patch.

Abrams said the list of impacted Apple devices is quite extensive, affecting older and newer models. If you own an iPad or iPhone — or any other Apple device — please make sure it’s up to date with the latest security patches.

Three of the weaknesses Microsoft addressed today tackle vulnerabilities rated “critical,” meaning that malware or miscreants could exploit them to gain complete, remote control over vulnerable systems — with little or no help from targets.

One of the critical bugs concerns Microsoft Word, and two others are remote code execution flaws in Windows Hyper-V, the virtualization component built into Windows. CVE-2021-38672 affects Windows 11 and Windows Server 2022; CVE-2021-40461 impacts both Windows 11 and Windows 10 systems, as well as Server versions.

But as usual, some of the more concerning security weaknesses addressed this month earned Microsoft’s slightly less dire “important” designation, which applies to a vulnerability “whose exploitation could result in compromise of the confidentiality, integrity, or availability of user data, or of the integrity or availability of processing resources.”

The flaw that’s under active assault — CVE-2021-40449 — is an important “elevation of privilege” vulnerability, meaning it can be leveraged in combination with another vulnerability to let attackers run code of their choice as administrator on a vulnerable system.

CVE-2021-36970 is an important spoofing vulnerability in Microsoft’s Windows Print Spooler. The flaw was discovered by the same researchers credited with the discovery of one of two vulnerabilities that became known as PrintNightmare — the widespread exploitation of a critical Print Spooler flaw that forced Microsoft to issue an emergency security update back in July. Microsoft assesses CVE-2021-36970 as “exploitation more likely.”

“While no details have been shared publicly about the flaw, this is definitely one to watch for, as we saw a constant stream of Print Spooler-related vulnerabilities patched over the summer while ransomware groups began incorporating PrintNightmare into their affiliate playbook,” said Satnam Narang, staff research engineer at Tenable. “We strongly encourage organizations to apply these patches as soon as possible.” Continue reading →

What Happened to Facebook, Instagram, & WhatsApp?

October 4, 2021
132 Comments

Facebook and its sister properties Instagram and WhatsApp are suffering from ongoing, global outages. We don’t yet know why this happened, but the how is clear: Earlier this morning, something inside Facebook caused the company to revoke key digital records that tell computers and other Internet-enabled devices how to find these destinations online.

Kentik’s view of the Facebook, Instagram and WhatsApp outage.

Doug Madory is director of internet analysis at Kentik, a San Francisco-based network monitoring company. Madory said at approximately 11:39 a.m. ET today (15:39 UTC), someone at Facebook caused an update to be made to the company’s Border Gateway Protocol (BGP) records. BGP is a mechanism by which Internet service providers of the world share information about which providers are responsible for routing Internet traffic to which specific groups of Internet addresses.

In simpler terms, sometime this morning Facebook took away the map telling the world’s computers how to find its various online properties. As a result, when one types Facebook.com into a web browser, the browser has no idea where to find Facebook.com, and so returns an error page.

In addition to stranding billions of users, the Facebook outage also has stranded its employees from communicating with one another using their internal Facebook tools. That’s because Facebook’s email and tools are all managed in house and via the same domains that are now stranded.

“Not only are Facebook’s services and apps down for the public, its internal tools and communications platforms, including Workplace, are out as well,” New York Times tech reporter Ryan Mac tweeted. “No one can do any work. Several people I’ve talked to said this is the equivalent of a ‘snow day’ at the company.”

The outages come just hours after CBS’s 60 Minutes aired a much-anticipated interview with Frances Haugen, the Facebook whistleblower who recently leaked a number of internal Facebook investigations showing the company knew its products were causing mass harm, and that it prioritized profits over taking bolder steps to curtail abuse on its platform — including disinformation and hate speech.

We don’t know how or why the outages persist at Facebook and its other properties, but the changes had to have come from inside the company, as Facebook manages those records internally. Whether the changes were made maliciously or by accident is anyone’s guess at this point.

Madory said it could be that someone at Facebook just screwed up.

“In the past year or so, we’ve seen a lot of these big outages where they had some sort of update to their global network configuration that went awry,” Madory said. “We obviously can’t rule out someone hacking them, but they also could have done this to themselves.”

Update, 4:37 p.m. ET: Sheera Frenkel with The New York Times tweeted that Facebook employees told her they were having trouble accessing Facebook buildings because their employee badges no longer worked. That could be one reason this outage has persisted so long: Facebook engineers may be having trouble physically accessing the computer servers needed to upload new BGP records to the global Internet.

Update, 6:16 p.m. ET: A trusted source who spoke with a person on the recovery effort at Facebook was told the outage was caused by a routine BGP update gone wrong. The source explained that the errant update blocked Facebook employees — the majority of whom are working remotely — from reverting the changes. Meanwhile, those with physical access to Facebook’s buildings couldn’t access Facebook’s internal tools because those were all tied to the company’s stranded domains.

Update, 7:46 p.m. ET: Facebook says its domains are slowly coming back online for most users. In a tweet, the company thanked users for their patience, but it still hasn’t offered any explanation for the outage.

Update, 8:05 p.m. ET: This fascinating thread on Hacker News delves into some of the not-so-obvious side effects of today’s outages: Many organizations saw network disruptions and slowness thanks to billions of devices constantly asking for the current coordinates of Facebook.com, Instagram.com and WhatsApp.com. Bill Woodcock, executive director of the Packet Clearing House, said his organization saw a 40 percent increase globally in wayward DNS traffic throughout the outage.

Update, 8:32 p.m. ET: Cloudflare has published a detailed and somewhat technical writeup on the BGP changes that caused today’s outage. Still no word from Facebook on what happened.

Update, 11:32 p.m. ET: Facebook published a blog post saying the outage was the result of a faulty configuration change:

“Our engineering teams have learned that configuration changes on the backbone routers that coordinate network traffic between our data centers caused issues that interrupted this communication,” Facebook’s Santosh Janardhan wrote. “This disruption to network traffic had a cascading effect on the way our data centers communicate, bringing our services to a halt.”

“We want to make clear at this time we believe the root cause of this outage was a faulty configuration change,” Janardhan continued. “We also have no evidence that user data was compromised as a result of this downtime.”

Several different domain registration companies today listed the domain Facebook.com as up for sale. This happened thanks to automated systems that look for registered domains which appear to be expired, abandoned or recently vacated. There was never any reason to believe Facebook.com would actually be sold as a result, but it’s fun to consider how many billions of dollars it could fetch on the open market.

This is a developing story and will likely be updated throughout the day.