"Large cloud providers are very competent and never go down", episode over9000.

Seems like FB's authoritative nameservers are down, and long enough for DNS caches to start expiring:

If anyone is wondering, it's not getting any better. Snapchat, AWS, Twitter, several major mobile providers (T-Mobile, AT&T, Verizon) joined the "seems down" club.


The Verge has the scoop:

> Cloudflare senior vice president Dane Knoecht notes that Facebook’s border gateway protocol routes — BGP helps networks pick the best path to deliver internet traffic — have been “withdrawn from the internet."

CloudFlare blog as a good write-up:

(reminder, CloudFlare is part of the problem, not of the solution)

@rysiek @sullybiker GO GO Gadget Strategic Popcorn Reserves!
🍿 🍿 🍿
(I am *always* well stocked)

@rysiek I know it's wishful thinking, but I hope the screwup of this magnitude will makes some people realize they don't really need facebook.

@avolkov @rysiek or at least that they can't afford to bet it all on FB... as many currently do. Poor fools.

@rysiek That "we got 30 times the DNS queries for Facebook domains because everyone and everything was excessively retrying" graph is 🙄

@galaxis now think what this does to any other DNS recursive resolver operator.


The conclusion in their article is a good proof of your remark:

> Today's events are a gentle reminder that the Internet is a very complex and interdependent system of millions of systems and protocols working together. That trust, standardization, and cooperation between entities are at the center of making it work for almost five billion active users worldwide.

While the problem in this case is clearly centralization, which they refrain from saying.

@rysiek explains why I had issues downloading blender earlier today.

@rysiek i heard this might be Azure related from coworkers FYI, haven't validated that yet

@rysiek Fascinating, #FacebookDown even affects their Tor onion service facebookwkhpilnemxj7asaniu7vnjjbiltxjqhye3mhbshg7kx5tfyd.onion/ – supposedly not requiring external DNS – while DNS issues are cited as the main failure reason right now.

@schmittlauch @rysiek
"It's always DNS"
on a more serious note, most likely lots of internal breakage due to domains not resolving. It's not just the frontend that your client needs to talk to that needs to resolve.

@tbr @rysiek Yeah, all these microservices resolving each other through clearnet DNS as well might be struggling, if they're relying on the same servers.

@schmittlauch @rysiek Facebook has a tor service?! Could anybody explain why?

@imon @schmittlauch because your threat model is not my threat model, and these are both not a threat model of an activist in Iran or Egypt.

We might worry about Facebook's surveillance, they might worry about state surveillance. One chooses one's battles.

> Facebook has a tor service?! Could anybody explain why?
competitive advantage. you have to be logged in, so they get to track you as you navigate their walled garden, while nobody else can
@rysiek that's my fault, I was speculating about the effects on the economy of local businesses of Facebook going NXDOMAIN suddenly, earlier this morning in a group chat

@rysiek Ha so noone noticed for a while? Interesting enough 😄

@rysiek It was so funny, because I was JUST TALKING to someone about how even big companies go down sometimes. But I was referring to Amazon and it's West server XD.

Oh how the mighty have fallen

@rysiek decentralization matters. Kudos to fediverse projects' devs, matrix devs and any other people who tries to implement interoperable systems without hard dependency on %BigProviderName% :2bgasm:

Sign in to participate in the conversation
Mastodon for Tech Folks

This Mastodon instance is for people interested in technology. Discussions aren't limited to technology, because tech folks shouldn't be limited to technology either!