Why We Can't Let Enormous Weirdos Regulate AI

Why We Can't Let Enormous Weirdos Regulate AI
it's actually just basic bayesian inference

Imagine if instead of regulating drones via the standard lengthy, boring, and bureaucratic process - which we did - we instead decided, in the name of fostering all-important American drone innovation, to avoid regulating drones meaningfully at all.

Also, imagine that we did not pay well-known aviation safety experts, aeronautical engineers, and other grim-faced and highly experienced people in sensible khaki pants to spend years studying the very realistic risks of drones running into passenger jets and falling on people’s heads.

Instead, imagine that we decided to direct almost all of our money and resources towards paying newly-graduated STEM majors and oddball drone bloggers with creative facial hair vast sums of money to think deeply about the possibility (which they all believe is high) of superintelligent dogs sprouting thumbs and using them to fly drones into humans, thereby causing mass panic and the eventual extinction of all life on earth.

Call them Drone Safety Researchers.

the dog, he has thumbs like a man

At the same time, imagine that the sensible and experienced khaki-pants wearing actual experts on aviation safety and policy who aren't part of this clique are now, in many cases, unemployed and sidelined, wondering what the hell just happened.

Imagine that these Drone Safety Researchers largely seem to live in the same Bay Area group houses, read the same esoteric websites, and attend the same psychedelic-drug-fueled sex parties as the even-better-paid Drone Company Employees, who they are supposedly charged with checking and regulating.

Imagine that when you ask these Drone Safety Researchers about exactly how the dogs will become super-intelligent, or where they get the drones, or how the thumb thing works, they immediately dismiss you as a technically illiterate fool.

“Can’t you see that preventing the Genius Thumb Dog Drone Apocalypse is the most important problem that has ever confronted humanity?” they ferociously reply. “How dare you propose that us, the greatest minds of our generation, waste our valuable time on such comparatively puny concerns as “making sure drones don’t take down passenger jets?” they angrily post on their Substacks (they all have Substacks).

Imagine that the US Drone Companies start to propose that to prevent the Genius Thumb Dog Drone Apocalypse, they should band together and form something that looks like a regulatory cartel, with oversight provided by the Drone Safety Researchers (who may or may not be regular attendees at their hot-tub parties). They assure us that this is the only way we can save humanity.

Imagine that all the while, the great unwashed, the general public, keep hearing about how their Great Aunt Trudy had a drone fall on her head right outside the Costco last week, or how yet another Delta flight to Orlando took a drone right in the left wing engine and had to make an emergency, mostly but not entirely fatal, water landing at Epcot.

And imagine that despite this, the Drone Safety Researchers keep insisting that these safety concerns are ultimately trivial irritations as compared to the real specter haunting humanity: Superintelligent Thumb Dogs with Drones.

Thankfully, this is not how we handled the arrival of consumer drones.

This is, bizarre as it sounds, almost exactly how the U.S. is currently handling the arrival of generative AI.


Isn't it weird that although Americans are increasingly worried about AI related fears like jobs, data centers, AI-inspired violence, and scams, so many of the country's most prominent and well-funded AI “safety” experts and organizations seem to be way more focused on talking about “what if we create a superintelligence that unleashes a swarm of self-perpetuating robots that then kills us all with bioweapons, and also something something time-manipulating basilisks"? [[1]]

And isn't it also weird that the most prominent companies that are building this supposedly potentially world-killing technology have gone full-court press on claiming that, yep, their products will kill us all (unless we give them more organized regulatory power)?

How the hell did this happen?

I don’t know. Well, not specifically.

However, I have some broad theories about the cause.

Such as:

yes, we have to talk about the basilisk thing

Many members of America’s world-beating AI companies are all hanging out at the same parties at the same insular Bay Area group-houses - often, quite literally. They’re reading the same oddball rationalist Harry Potter-themed parables, mainlining the same unsettling doomer theories about AI-powered nano robots and basilisks killing us, imbibing in the same mind-altering substances, and (sometimes) attending the same orgies.

And they have all marinated in each others fart-miasma to such an extent that they have absolutely forgotten how to talk to the rest of us NPCs [[2]] in an even slightly normal way about what, exactly, it is they’re doing under there.

well, that's a very weird thing to say, Eliezer Yudkowsky

What's more, thanks to their vast wealth and influence, combined with Trump's war on nonprofits and academia, these doom-obsessed nerds have been able to take over much of the tech regulatory space. Which means they‘re trying to impose their intensely strange and poorly-evidenced ideas about which AI risks and harms are most important on you, me, and everyone else.

Yes, like the AI Doomer Group House contingent, I'm also very afraid of our tech-driven future. It’s not because I share their fears about super intelligent AI killing us, though.

It’s because I’m afraid that this small clique of all-too-human dorkwads are trying to gain even more mind-boggling wealth and even more political and regulatory control over powerful AI technology, in ways that will allow them to further fracture democracy, privacy, and human flourishing.

That is, unless we end their mutual fart-huffing circle before it’s too late.


As summer leaves flicker into fall, we’re witnessing two main powerful tech factions duking it out over a prospective AI slowdown, what that should look like, and to what extent we’re all mega-doomed.

I will define them as such.

The AI Doom Contingent

The first faction are the aforementioned AI Doom Cult Contingent, [[3]] represented by the alliance of Anthropic, OpenAI, Elon Musk, and a bunch of AI employees. Most of these people are themselves linked, to one degree to another, to the massively tiresome Rationalist/Less Wrong/Slate Star Codex bong-rip philosophy-posting clique.

Our current kerfuffle began on September 8th, when a previously low-profile young AI researcher named Jacob Coxon announced on Twitter that he had resigned from a role at Anthropic after only a few weeks (after previously working at OpenAI), due to his belief that both companies were "racing straight to self-improving superintelligence and gambling with our lives."

Coxon's frightening claims went massively viral, bolstered by the claims of a current Anthropic employee who predicted, on the basis of no particular public evidence, that he rates the chance of AI killing all humans at ">10% within the next decade":

These revelations came quick on the heels of a rash of unsettling recent reports on big AI company's poorly-supervised models hacking other companies (including, in the most recent revelation, OpenAI's model hacking an Australian government website). Which means the timing was right for public existential AI fear to go critical. Almost immediately, many other high-profile people started talking ominously about imminent AI doom, including Barack Obama, Hillary Clinton, and a surrealist joint-effort by Bernie Sanders and actual lich Steve Bannon.

A few days later, on September 12th, Anthropic CEO Dario Amodei published an essay called "We Must Pace the Frontier," which cited recent high-profile AI "escape" incidents as proof-positive that AI companies need to slow the rate at which they increase model capabilities.

Amodei made a number of proposals about what such a "pacing" might look like, first among them placing "embedded evaluators" within AI companies to "verify safety practices and report incidents," and second among them the development of shared safety standards and limits among leading frontier AI companies, which would hopefully help convince other AI powerhouse nations, like those in China, to sign on to risk-reducing agreements.

In a little footnote at the end, Amodei acknowledged that such industry-wide coordination efforts might require "government mediation or waivers of antitrust restrictions."

Critics swiftly seized upon this aside, prompting (so far) both a lawsuit and considerable analysis of the legal issues therein. Critics also pointed out that METR, an organization that Amodei explicitly cited as one potential "embedded evaluator," is itself intensely personally and socially intermeshed with the same companies that it would supposedly be keeping an eye on.

Almost immediately, Sam Altman of OpenAI concurred with Amodei's proposal (although not the part about the need for an antitrust exemption). Even Elon Musk took a moment away from completely failing to control the spread of CSAM on Twitter to agree.

Continuing the theme, on September 23rd, both Dario Amodei and Sam Altman addressed the United Nation’s General Assembly, claiming (Amodei) that “AI could be a risk to humanity as a whole,” and (Altman) that we “could lose control of the future to AI.”

Baby, you've got a stew going!

Or, IDK, a regulatory cartel that happens to benefit a very small number of players, many of whom, again, go to the same sex parties and read the same long-winded blogs and ponderous novel-length rationalist [[4]] Harry Potter parables. Depends on how you look at it, I guess.

At this point, you may be wondering why the AI doom contingent does not just stop building frontier models entirely, if they are so convinced that these technologies have a significant risk of killing all humans.

They largely think this question is really stupid.

Most members of the AI Doom Contingent think that, much as the gravity of their actions haunts their dreams at night, they need to keep building AI - albeit in a more coordinated and controlled fashion - because if they don't, someone who isn't as smart, ethical, and good-looking as they are will build the great and terrible Machine God instead. Probably China. Which would be terrible.

Further, while the AI Doom Contingent is largely willing to acknowledge that their imminent IPOs will turn many of them into some of the wealthiest people who have ever walked the planet, they also seem to think we should view this as a good thing.

Consider the rash of recent speculation within philanthropy and Effective Altruist circles about how the new AI elect will turn around and donate vast sums of their newly-acquired wealth to charity. From this point of view, all that AI IPO money could help stop the hideous global suffering and the projected 14 million deaths that will likely be caused by the destruction of USAID, a historically evil act which was largely personally engineered by, ah, er, Elon Musk.

And isn’t that as effective as altruism gets? [[6]]

You may also be wondering why, if these people are all genuinely convinced that the AI technology they’re building is liable to destroy humanity, that they’re not proposing nationalizing their own companies, thus subjecting them to a more democratic, strict, and transparent regulatory process (well, if we assume that Trump is eventually ousted from office).

I think you can guess the answer to that.

Do They Actually Believe the Freak Stuff?

Some claim the AI doom contingent are just saying these upsetting things about total human extinction at the hands of a cruel collective-intelligence machine god because it’s excellent marketing, and that they expect to profit from having their tools appear scarier and more potentially world-destroying than they actually are.

This is, in some ways, a comforting opinion. I also believe it is wrong.

While a few leading AI figures probably are just repeating doomsday predictions in the service of an epoch-making payday, I think the majority truly are high on their own supply.

john calvin be at it again

They really DO believe that they are the elect and elite stewards of a technology capable of god-like power, in a sense that hews unpleasantly closely to the nasty worldview of John Calvin.

They really DO believe that they are the only humans qualified to both build and attempt to control this mighty super intelligence, and that they therefore must do everything and anything (including, potentially, lying a lot) to maintain that control.

We are, unfortunately, dealing with a passel of true believers. And, as is usually the case with true believers, they are both 1. really weird and 2. really insular.

Others have written at great length about the insanely dense web of personal connections and overlaps between the leaders of big U.S. AI companies, “AI safety” organizations, extremely online rationalists into Harry Potter fanfic, and the deep-pocketed Effective Altruists that bankroll much of the work, so I won’t spend too much time reiterating the basics. As well as how the rationalist scene created the Zizians, a spin-off group that eventually committed a number of high-profile murders. (Scroll to the bottom of this piece for more reading suggestions, or click here).

It suffices to say that when I say “weird,” I mean really, really weird.

The Dead-Eyed Capitalist Opposition

Which brings us to the second big-ticket alliance who are mad at the AI Doom Contingent over these calls to "pace the frontier," who I will dub the Dead-Eyed Capitalists for short.

This crew includes NVIDIA's Jensen Huang, Mark Cuban, David Sacks, Marc Andressen, Alex Karp, Mark Zuckerberg, and much of the Trump administration (among others), all of whom believe for various reasons that a coordinated AI slowdown is a bad idea, and that the AI Doom Contingent are up to something sneaky.

While they are not incorrect to point out that the other side seem to be using AI slowdown fears to justify forming a cartel, it is important to note that they are mostly mad about this because they want to be part of the massively-powerful cartel too. I simply do not buy that they are mad because they actually care about the human impact of the further development of AI. They are certainly not mad because they object to ever-worsening rates of income inequality.

This does not mean this group have opinions on AI that largely mirror those of the general American population. Many of them also at least claim to believe in something like the super-intelligent machine god: they just think it will be our friend.

To quote Marc Andressen in his own October 2023 bong-rip tinted ode to technooptimism: “We believe any deceleration of AI will cost lives. Deaths that were preventable by the AI that was prevented from existing is a form of murder.”

Then there's NVIDIA's Jensen Huang, who presents himself as the Sensible AI CEO, but who also just freely admitted in a New York Times interview that he doesn't think it matters that kids in an AI-saturated world increasingly can't do basic math - because, after all, he doesn't know his own home address.

does jensen huang like, need a Life Alert

Meanwhile, Meta's Mark Zuckerberg believes that AI will help us create "personal superintelligence" that will help you "achieve your goals, create what you want to see in the world, experience any adventure, be a better friend to those you care about, and grow to become the person you aspire to be." He reiterated the point in a July 2025 earnings call, claiming that "“I think in the future, if you don’t have glasses that have AI — or some way to interact with AI — I think you’re … probably [going to] be at a pretty significant cognitive disadvantage compared to other people,” Zuckerberg added.

god help us all

For a final scrap of evidence, consider whatever is going on with Donald Trump's ever more precipitous descent into total AI slop-inflected madness. (Try not to think too much about how Donald Trump has the nuclear codes, which I personally find a lot scarier than the unlikely prospect of being eternally tortured by a superintelligent AI).

What About Us?

This brings me to the rest of us, the great masses of mostly normal people watching this crazy-eyed Clash of the Titans and wondering what all this furious billionaire posting means for our future. Mass unemployment and poverty? Being ground into meat-slurry by rapacious gray-goo machines? A utopia overseen by machines of loving grace? Nothing much interesting at all?

I will bravely state that I think both sides are sweaty weirdos who shouldn’t be granted ultimate future-building power. I believe both sides of this debate have deeply suspicious motives in this ferocious struggle for power, influence, and buckets and buckets of money.

go outside!!!

Personally, I'm a member of the rapidly-growing contingent that wants to break up the fart-huffing cycle, splash some ice water over both neurotic dogs locked in this fur-rending fight, and do what needs to be done to ensure that demographically-diverse, level-headed, relevantly-experienced adults who aren’t all in the same group chat or polycule are placed in charge of sensibly regulating AI.

Because we simply cannot let these abject weirdos, these experimental drug-addled billionaires, these spreadsheet-worshipping dorkasses, decide the future for us.

No, I’m not an AI denier. I think generative AI tools can be incredibly useful: I now use them regularly to help me complete certain programming tasks that would have been both mind-numbingly boring and time-consuming before.

I also think they're dangerous, as evidenced both by a quick glance at the headlines of the last two years, and my entire professional career.

I've dedicated much of my life to trying to mitigate the dangers of novel technologies - these days, I spend most of my time on another horrifying cyberpunk nightmare, the expanding use of small cheap drones to murder civilians. And this experience is exactly why I don't buy the AI doomer arguments that LLMs are the Most Dangerous Tools Humanity Has Ever Invented.

Because my work has long required paying attention to technology harms that are very scary and are happening right now - harms which the biggest names in AI safety seem distinctly disinterested in focusing on.

remember this guy

Like the nightmare march of increasingly AI-powered mass surveillance devices across our communities.

The continuing uncertainty around the community and environmental impacts of the data centers that AI companies are rushing to build around the globe.

How some people appear to have been encouraged to commit murders and mass-shootings after speaking with major AI company chatbots, like the tools produced by ChatGPT.

The ever-growing wave of scams enabled by generative AI, which are stealing ever-growing amounts of money from innocent people, especially the elderly.

The insane proliferation of AI-supported child sexual abuse material on Elon Musk’s X/Twitter - the AI industry's favorite social media website - and elsewhere on our increasingly out-of-control, AI-saturated Internet.

I could go on all day, but you get the idea.

The exact kind of tangible, present, frightening technological risks and harms that average people worry about, and that we’re also struggling to do much about right now - partially because the AI Existential Doom Contingent has, blob-like, absorbed huge much of the attention and funding that used to sustain more-grounded ethical tech policy.

Aren't There Reasonable People Who Can Figure Out How to Regulate AI? What Happened to Them?

Like I said above, I work on protecting civilians from small drone threats. Which means I, unfortunately, spend a lot of time watching children and old people around the world get exploded by ever-more-sophisticated tiny flying robots in high-resolution social media videos. And yet, me and my colleagues are struggling to get people to care much about or to fund addressing this very real cyberpunk horror.

I am not the only person in this surreal, moral-injury causing predicament.

I have spent much of my career working with sensible, brilliant, and often grossly underpaid people who specialize in thinking about non-insane, practical ways to regulate and control technology. Today, far too many of those good-hearted people are some combination of unemployed, underemployed, afraid they'll eminently lose funding, or living in fear of harassment. Why is that?

Let us distribute blame fairly here.

it's great how this specific guy is destroying the world right now

This sad state of affairs is partially the fault of Elon Musk's 2022 purchase and subsequent defilement of Twitter, a move that convinced much of big tech that they too could get away with shedding even the pretense of caring about (or cooperating with, or funding) independent-minded ethical tech research.

It is partially the fault of the Trump administration, with Elon Musk and DOGE’s ardent support, destroying much of the nonprofit, humanitarian aid, and academic funding system many ethical tech experts used to rely upon, while further emboldening Silicon Valley's already quite-goddamned-emboldened-enough overlords.

And it is partially the fault of big AI and their AI-doom obsessed fellow travelers, who kicked off the meteoric rise, since the early 2020s, of the so-called "AI safety" and “catastrophic risk” (among other euphemistic terms) research and policy ecosystem.

This is a constellation of organizations and researchers, funded by prominent so-called "effective altruist" founders (who overlap enormously socially and ideologically with the oddball rationalist community) that have become skilled at presenting themselves to the outside world as sensible tech policy experts - but whose primary interests lie in far weirder things than "what happens if AI takes people's jobs."

lol

Backed by vast sums of effective altruist-aligned billionaire money, these so-called AI safety experts have successfully performed the bait and switch move of supplanting the old non-kooky tech policy guard in both public influence and funding. Think of them as parasitic cuckoo birds in expensive cardigan sweaters.


How Effective Altruists Took Over AI Regulation

I used to think effective altruists were a little annoying, but mostly harmless. I raised my eyebrow at their quantification-obsessed approach and the weird directions their Peter Singer-inspired philosophy sometimes took them, but I wasn't lying awake at night worrying about a group that really was dropping a lot of money on fighting malaria.

I changed my mind in early 2023, when Sam Bankman-Fried crashed out, his companies imploded, and a whole lot of stories came out about how EA's leading luminaries were buying $20 million dollar mansions instead of bed-nets.

Apparently, a lot of EA luminaries had by then convinced themselves, in suspiciously Rationalist fashion, that worrying about impending AI superintelligence doom and the happiness of bazillions of people yet to be born (also, sometimes, the happiness of shrimp) mattered more than anything else. Including addressing global poverty and health.

I optimistically hoped that this scandal might teach the community a bit of humility, foster notions of pursuing a reset. Unfortunately, I was wrong.

As the saga of SBF’s fall from grace and entry into prison went down in 2023 and early 2024, Politico's Brandon Bordelon (among others) published a number of well-reported stories laying out how billionaire EA funders like Open Philanthropy/Coefficient Giving's Dustin Moskovitz and Lightspeed Grant's Jaan Tallinn had funded lobbying efforts and fellowship positions for EA-minded researchers across influential political offices and think tanks in DC, including at eminent organizations like the RAND Corporation.

At the same time, while the Effective Altruism world did launch into much-needed internal fighting and debate over the SBF disaster, the movement itself continued to chug along, subsidized by the aforementioned big fat buckets of big tech money.

If the movement writ large learned many lessons or engaged in much transformative soul-searching from the FTX Affair, there seems to be little evidence of that on the ground. Indeed, just a few days ago, one effective altruism fund, controlling millions of dollars, rather defensively announced that it was hiring disgraced Alameda Research and FTX leader Caroline Ellison.

Meanwhile, the EA movement and its support of so-called "AI safety" causes has only continued to gain political influence and power since 2023. They were so successful in these efforts that when Trump 2.0 came along to lay fire and sword to the entire nonprofit, academic research, and aid ecosystem in early 2025, EA-supported AI safety organizations and researchers quickly became arguably the most prominent - and inarguably the best funded - voices in ethical tech policy, research, and advocacy.

And that's the very truncated story of how the US ethical tech policy world very, very quickly became a whole lot weirder and more out-there than it used to be.

joey pretty perfectly summarizes my personal thoughts on EA

This is bad for a bunch of reasons, but I’ll boil it down.

It’s bad because the Effective Altruism movement and the "AI safety" ecosystem its adherents support both tend to suffer from two core problems:

  1. EA movement members overlap enormously in a myriad of ways rationalists and AI safety researchers and AI company executives, which means they're highly incestuous (and biased) and way too interested in AI doom thought experiments, as compared to actual doom happening to actual people right now.
  2. Due to this shared cultural and social world, these groups all share the pathology of assuming that they are the only smart and rational people in the world, and everyone else trying to do aid or donate to aid is probably stupid and irrational.

This arrogance, as Ugandan farmer and social enterprise founder Anthony Kalulu points out, can extend to blowing off the needs and wants of the poor as largely irrelevant - a lot like their rationalist and AI safety siblings are currently blowing off the needs and wants of average Americans around AI.

ah yes

According to Kalulu, while traditional philanthropy was also notoriously subject to condescending, top-down “we know what’s best for you as clever Westerners” attitudes, the sector at least did fund many small smaller, grassroots organizations. Meanwhile, effective altruist's relentlessly quantification-driven approach hugely favors the capabilities of bigger, Western organizations, thereby shutting out local voices, NGOs, and interest entirely. [[5]]

Indeed, effective altruism, unlike the traditional aid sector, seems barely interested in localizing aid at all. And Kalulu is certainly not the only person who's made this argument, among other very salient critiques of effective altruism's approach. Not that this blind spot is that surprising, because according to a 2024 survey, 68.8% of self-identified EAs are men, 75% are white, and 80% attended a top 200 university.

This arrogance also extends to some leading EAs belief that, since everyone else is dumber than they are, they can use their public work on global health and poverty reduction to lure more smart, kind-hearted, and high-earning people into the fold. [[7]] These new adherents can eventually then be convinced that saving children today matters less than focusing on existential AI risk, which saves children thousands of years into the future (maybe, hypothetically). Some are quite willing to talk about this bait-and-switch approach in the open.

For another example of how weird this all has become, look at the priority list publicly posted by 80,000 Hours, a hugely influential EA organization that was co-founded by Effective Altruism also-cofounder William MacAskill (who was pals with Sam Bankman-Fried, and famously introduced him to Elon Musk over the Twitter buy). Specifically, 80,000 Hours is focused on helping highly paid professionals aligned with the EA worldview think about where and how to donate their money.

Let's glance at at how 80,000 Hours ranks its priorities in the year of our Lord 2026:

AI risk is at the top, and while I'll grant that AI leading to extreme power concentration is a very valid concern, the organization's overview is notably reluctant to call out the inherently intensely political nature of the problem (which is a rather common theme across EA literature). Meanwhile, global health and climate change languish at the bottom. And nuclear weapons rate only an honorable mention.

To put it plainly, this is weird.

This is a value system that is fundamentally out of line with what most regular people believe should be top philanthropic priorities. I know this in part because EAs have funded polling research that says the same thing:

Effective Altruist money absolutely has done good things, which I want to acknowledge. Most recently, on September 18th, Dustin Moskovitz and Cari Tuna's Coefficient Giving announced the $165 million Health Aid Transition fund, intended to help low and middle income countries navigate the horrible aid funding crisis created by Donald Trump and Elon Musk.

And yet, great as such a gift is (and it truly is great), we also have to consider the source. The source being an organization that's part of a broader movement populated by people who seem to view reducing global health and poverty as the salad before the main course: good to have, but you know what they're ultimately here for.

With this context in mind, do we believe the EA philanthropy world - which, again, expects to receive most of the gazillions of donated dollars from the Anthropic and OpenAI IPOs - is going to be just as committed to addressing the global aid crisis as they are to funding yet more AI safety research?

Do we think it's OK that they seem to view analyzing oddball science-fiction inflected doomsday scenarios as of at least equal importance to helping the global poor stay alive? (And that many in the movement think researching AI doom is much more important?)

Should we really believe, with the information currently available to us, that these people and organizations are going to be willing to burn both lots of money and political capital on bringing back USAID - a massive government organization that simply can't be replaced by even very deep pocketed private charity?

I don't.

i could have made this meme look better, but i think it has a certain primordial charm as it is

How Do We Stop the Weirdos from Controlling AI Regulation?

First, we need Democrats to win the midterms. I'm feeling better about that right now.

We need to tell everyone we know (friends, family, your political representatives) that America's biggest tech companies are in the thrall of some really weird shit. Such weird shit that they simply can't be trusted to regulate themselves, or to work with government in ways that actually address the genuine concerns of real Americans over AI technology. Check the Further Reading section below for more links you can share.

The AI Doomer Contingent has got away with a lot of this in large part because 1. it sounds nuts when you try to explain it to people who aren't extremely online and 2. too few people have, therefore, attempted to explain it. Luckily, a dam seems to have broken around talking about wacky AI doomer bullshit. And a lot more people are writing good explainers for the uninitiated (see below).

We particularly need politicians to wake up to the fact that if they cozy up to Silicon Valley AI rationalists and EAs, they may be getting a lot more weirdness than they bargained for. You could start by showing them the New York Times article about the AI group house parties that a 13 year old child claimed he was able to sneak into.

We need to bring non-wacko, non-Silicon-Valley-AI-doom-cult, non-DOGE, tech experts back into government and policy. All those sensible, experienced, ethical government tech and tech policy experts I know are still out there, and they would very much like to be working on crafting AI policy and controls that aren't kooky, biased, or weird. Read this great Liberal Currents piece as one example of what that could look like. We have the talent and the expertise required to take back the regulatory future from the weirdos.

We need to maintain hope. Because there is hope. We are not doomed. We can do this together, and we need neither a machine god or flawed people who have appointed themselves proxies of god to do it.

Further Reading

I maintain a master list of reading on weird AI guys here.

Some selections:

General

What Policymakers and the Public Need to Know About the Cult of AI - Nathalie Maréchal, Tech Policy Press

The Techno-Religion at the Center of the A.I. Debate - The Daily (Podcast), New York Times

The A.I. Party House Where Networking Has a Dark Side - Kirsten Grind, New York Times

Stopping the AI Safety Cult ft. Adam Becker and Cal Newport - Better Offline (Podcast)

How Scared Should We Be of A.I. Right Now? - Cal Newport, New York Times

More Everything Forever - Adam Becker (Book)

‘Think Everybody Dead’: How the Threat of AI Is Fueling a New Political Alliance - Calder McHugh, Brendan Bordelon, Politico

Why AI Companies Can't Be Trusted to Self-Regulate - Heidy Khlaaf, Nature

Safety Co-Option and Compromised National Security: The Self-Fulfilling Prophecy of Weakened AI Risk Thresholds - Heidy Khlaaf, Sarah Myers West, Arxiv

Don’t be fooled by this summer of AI hype - Timnit Gebru, Emily M. Bender, MIT Technology Review

Sam Altman May Control Our Future—Can He Be Trusted? - Ronan Farrow, Andrew Marantz, New Yorker

The TESCREAL bundle: Eugenics and the promise of utopia through artificial general intelligence - Timnit Gebru, Émile P. Torres, First Monday

The AI Kids Take San Francisco - Kerry Howley, New York Magazine

Child's Play - Sam Kriss

Rationalists

The Rise of Silicon Valley’s Techno-Religion - Cade Metz, New York Times

The Zizians and the Rationalist death cults - Max Read

Who are the Zizians? - WBUR

Part One: The Zizians: How Harry Potter Fanfic Inspired a Death Cult - Behind the Bastards (Podcast)

Neoreaction a Basilisk - Elizabeth Sandifer

The Irrationality of Rationalists (w/ Samantha Hancox-Li) - Aaron Ross Powell, ReImagining Liberty (Podcast)

Effective Altruism

Effective Altruism’s Bait-and-Switch: From Global Poverty To AI Doomerism - Nirit Weiss-Blatt, Techdirt

The Real-Life Consequences of Silicon Valley’s AI Obsession - Ellen Huet, Bloomberg

The Good It Promises, the Harm It Does: Critical Essays on Effective Altruism, Edited by Carol J. Adams, Alice Crary, and Lori Gruen, Oxford University Press (Book)

Women and Effective Altruism - Keerthana Gopalakrishnan

Effective altruism is worse than traditional philanthropy in the way it excludes the extreme poor in the global south - Anthony Kalulu

The sad decline of effective altruism - Yarrow Bouchard, Strange Cosmos

Effective Altruism Promises to Do Good Better. These Women Say It Has a Toxic Culture Of Sexual Harassment and Abuse - Charlotte Alter, Time

The Deaths of Effective Altruism - Leif Wenar, Wired

[[1]]: Despite their supposedly superior intelligence, it does not seem to have occurred to many members of the self-appointed AI elect that their industry’s ever more catastrophic PR problem and their long-time disdain for communicating with the public might, perhaps, be related phenomena.

[[2]]: "Non-player-characters" as certain people in tech spaces like to refer to us wretched little normies.

[[3]]: For further reading on the world of Weird Little AI Guys, check out this Semble collection I've been putting together.

[[4]]: If you haven't heard about the Rationalists yet (who overlap immensely with big AI companies and Effective Altruism), well, it's unfortunately time you learned.

[[5]]: A glance at GiveWell's published grants from Jan 2014 to August 2026 indicate that over 96% went to organizations headquartered in high-income countries, though I didn't do an in-depth data analysis.

[[6]]: In this way, big tech once again positions itself as both the creator of and the solution to all life's problems.

[[7]]: Much like Eliezer Yudkowsky explicitly used Harry Potter and the Methods of Rationality as a "good way to recruit International Mathematical Olympiadists" (who are teenagers, a group that, in 2010, one could safely assume had read Harry Potter).