This log4j exploit = remote code execution in basically everything

Arbitrary code execution in iCloud, Twitter, Steam, CloudFlare, Amazon, Tesla, Baidu, Tencent

This may well be devastating 0day RCE exploit that has ever been dropped in all of history.

:drake_dislike: cloud computing ☁️
:drake_like: clown computing 🤡

@rysiek Yup.

I'm just wondering why we haven't head about some attackers iterating through entire IPv4 range, sending poisoned requests at ports and in protocols that have reasonable chance of being serviced by a JVM program.

It's either tougher to exploit than it looks, or I just missed the news on this.

@temporal why would they go to all this trouble when Shodan lists hundreds of thousands of open ElasticSearch instances? 🤷‍♀️

@rysiek I would think walking the entire IPv4 range would be conceptually simpler than trying to get a list from Shodan, but in either case, why aren't all those instances pwnd already?

@rysiek You know the story about two economists and a $20 bill on the floor of a busy train station?

Given the apparent ease of exploitation, I'd expect that by now, all those instances were down, with their servers locked by multiple ransomware scripts...

@rysiek It's a joke about Efficient Market Hypothesis.

Two economists walk down the station and spot a $20 bill.

- "Look, there is a $20 bill on the ground!", exclaims one.

- "No, there isn't. If there was a $20 bill on the ground, somebody would have already picked it up." - the other coolly replies.

@temporal ah, makes sense. Yeah, probably applies here. As in: theory and expectation does not always match actual experience.

I mean:

@rysiek Well, holy hell. How come the entire world isn't already on fire?

@temporal but... it is. Wanna ponder the 🗑️ 🔥 happening right now at all these companies?

Hug Your Admin Day, seriously.

@rysiek Ok, perhaps I'm too impatient about the impending doom.

I'm gonna go, hug my admin, and wait for the fireworks to start.

@temporal @rysiek We're seeing people try this exploit against some of our websites at work (which aren't even backed by java).

@rysiek it also affects your self-hostsd Minecraft server, so don't get too excited.

@wolf480pl how unfortunate that I do not self-host a Minecraft server.

@rysiek my point is, self-hosting doesn't really have much advantage over cloud un case of this vuln.

@wolf480pl it gives me a chance to patch it myself.

It also gives me the ability to quickly pull down a vulnerable service completely and reliably if I choose to. And I know what's running on the server, there are no co-tenants that could be exploited and affect me.

Finally, my self-hosted thing is probably not high on the list of potential attackers (which doesn't mean I should not patch immediately).

@wolf480pl plus, self-hosting is not a way to solve this problem. It solves other problems.

A tool can be good and useful even if it does not solve all possible problems.

In fact, most of the time, the fewer problems a tool solves, the better at it it is!

@rysiek yeah I'm not saying selfhosting is bad, I'm saying "cloud sucks because log4j vulns" is a silly argument.

@wolf480pl cloud sucks because of monoculture.

Because of how many customers are affected by such a bug.

Because these huge, gigantic companies paint themselves as so amazingly competent, and yet we get an insane 0day RCE just days after AWS went down for everyone.

This myth of "technical competence"/"best in industry" needs to die.

It is clown computing.

@rysiek for an individual it doesn't matter how many other people are affected by the same vuln. Though good point about amount of code running, and lower chance of being targetted.

@rysiek @wolf480pl Uhm, this monoculture is unrelated to cloud. log4j was/is one of the most popular logging frameworks among Java Developers.

And given how things are currently mitigated, I would say that at least to some degree cloud is even an advantage here, because you get traffic analysis tools and alike included. So you can deploy honeypots and do traffic analysis immediately. Not that you can't do these on premise, it's just more work.

There is no point in blaming this on the cloud.

@sheogorath @wolf480pl yes there is.

Major cloud providers have meticulously built this narrative that it's somehow "safer" in the cloud because they have all the resources and all the technical expertise in the world, and thus can keep you *safer* than the alternatives.

And yet, here we are. Log4shell dropping *days* after a major AWS outage. Turns out they're as crap as all the rest of tech.

They can't have it both ways, and need to be called out for their bullshit every time this happens.

@wolf480pl @sheogorath no, but again: major cloud providers pretend to be *better* than everyone. And they are demonstrably not. It bears reminding everyone anytime this kind of fit hits the shan.

@rysiek @sheogorath has anyone measured how long it takes a cloud provider to apply security updates vs how long it takes self-hosters?

Don't get me wrong, I think clouds aren't faster at all, and then there is higher attack surface. Still, would be interesting to see such a study.

@wolf480pl @sheogorath absolutely.

But it's not just about deploying the patches. It's also about defense in depth. They are a bigger target, the pay-off for a successful compromise is way higher and affecting way more people.

And yet a thing like that slips through. 🤷‍♀️

@wolf480pl @rysiek well, at least Cloudflare managed to identify potentially compromised systems within hours as well as publishing 3 articles on how log4shell works and how they mitigated it internally as well as for customers on Friday.

Don't know about the rest. But what I can say is that on-prem infrastructure especially for Small-Medium-Businesses are off worse here simply due to the typical lack of 24/7 security staff.

@sheogorath @wolf480pl @rysiek relatedly, inventory and environment heterogeneity can be a big problem in major cloud environments. Half the problem is knowing what’s in your inventory and the other half is coordinating across 23,546 special snow flake setups.

Source: I used to work on a major cloud security team.

@zbrown @sheogorath @wolf480pl oh absolutely, I never claimed it's easy.

It is not.

I did infrastructure for a tiny (compared to cloud providers) org, and every snowflake set-up was something I knew would bite me in the arse eventually. It always did.

On the other hand, major cloud providers *create* technological monocultures. And these are surprisingly vulnerable in unexpected ways: Log4shell, Facebook BGP fsckup, and the last AWS "us-east-1 only but not really" downtime are great examples.

@rysiek @sheogorath @wolf480pl ya there’s something to be said for heterogeneity: “well 60% of the environment is unaffected because they’re running a smattering of versions so old that they aren’t impacted”.

@wolf480pl @rysiek Saying "java sucks and so does the whole mindset that usually goes with it because log4j vulns" is on the other hand not so unreasonable.

@zudlig @rysiek
what java-specific mindset are you referring to and how is it any worse than npm?

@wolf480pl @zudlig this is not a race. Both Java and npm are crap.

With Java, it's the "StringFactoryFactoryInterfaceFactory" mindset. Everything is so abstract that writing simple code is nigh-impossible. This also leads to Java being a horrible resource hog (unless you spend days tweaking the runtime parameters).

There are more, but that's off the top of my head.

@rysiek @zudlig hmm yeah I forgot other people didn't grow past the "let's make EnterpriseFizzBuzzLoopingStrategy" stage after finishing high school.

@wolf480pl @rysiek Yeah the people who are into that sort of thing to an unhealthy degree include those who write at least 90% of "enterprise" java code, from what I've seen. And then it seems to me you too often have these massive, over-complicated, poorly-understood libraries included into projects to do relatively simple things (like writing stuff to a log file) without a second thought by people who barely know what they're doing when it comes to writing code of their own. I don't know if it's any worse than what the npm crowd do, but anyway it is at least not ideal for security.

@zudlig @rysiek
you've lost me at "not ideal".
Sure, it's not good. Not even decent.

And you should want decent, and then you should want good on top of that, but you'll never get ideal. By definition.

@wolf480pl It may be different where you are, and where @zudlig is, but in my neck of the woods to say something is "not ideal" is usually to say "it's a complete cluster-fuck".

Emphasis by understatement.

Now that might not be what was meant here, but I just thought I'd mention that in case it helps people get the communication straight and avoid misunderstandings.


@ColinTheMathmo @zudlig @rysiek
I know this is a thing in a US, but like...
it's stupid.
People should stop doing that.

@wolf480pl That ship has sailed. People use language to communicate, and the words, phrases, sentences, and utterances in general do not carry meaning, and do not say literally (and I mean literally) what people mean.

So you many wish it to be otherwise, but that's simply not how language works. That makes it especially hard for people communicating across cultural boundaries, I know, and I personally try to avoid things like this.

But ... <fx: shrug> ... it happens.

@zudlig @rysiek

@ColinTheMathmo @zudlig @rysiek
I will figure out what's intended, and then proceed to misinterpret it intentionally, thank you.

@wolf480pl @ColinTheMathmo @rysiek Not so much in the USA compared to the UK and Canada in my experience, but it's true, for me any statement that would be overly obvious if interpreted literally is a cue to look for some kind of ironic intent.

In this case I did at least write that things are "at least" not ideal, thus avoiding, even under the most literal of interpretations, any commitment to estimating precisely how far they might be from the ideal we all should strive for.

@zudlig @rysiek @ColinTheMathmo
and I do get the sarcastic "not ideal" in spoken language, but it's usually said in italics...

@zudlig @rysiek @ColinTheMathmo
also, as I said elsethread, I sometimes enjoy purposefully misunderstanding things when I want people to be more precise.

@wolf480pl So you will deliberately say what you don't mean because someone else has said what they don't mean, rather than actively asking for precise clarifications.

I'm not sure that really helps people understand each other, not improves communication in general.

It's not the approach I take, but it's entirely your choice.

@zudlig @rysiek

Show newer

@wolf480pl I disagree. "Not Ideal" in general is used to mean "Very much worse than ideal".

But I'm not sure this conversation will go anywhere useful.

@zudlig @rysiek

@wolf480pl And that's my reward for actively trying to help avoid misunderstandings.

Thank you.

@zudlig @rysiek

Show newer
Show newer
@wolf480pl @rysiek It also potentially affects every Android app. I'm expecting we'll see a flurry of updates on f-droid next week.
@rysiek This was apparently fixed over a week ago:

Does that still qualify it as a zero-day? It looks like it was known and more or less quietly fixed, and someone just decided to drop an exploit for it today.

@tadzik only for the latest log4j. Stuff like fully-updated Confluence bundles a way older version of log4j, for example.

@tadzik ooh, also:

If you already upgraded code to use just released log4j-2.15.0-rc1, it's still vulnerable - you now need to apply log4j-2.15.0-rc2 as there was a bypass. They is no stable release which fixes yet.

Sign in to participate in the conversation
Mastodon for Tech Folks

This Mastodon instance is for people interested in technology. Discussions aren't limited to technology, because tech folks shouldn't be limited to technology either!