Want to wade into the sandy surf of the abyss? Have a sneer percolating in your system but not enough time/energy to make a whole post about it? Go forth and be mid: Welcome to the Stubsack, your first port of call for learning fresh Awful you’ll near-instantly regret.

Any awful.systems sub may be subsneered in this subthread, techtakes or no.

If your sneer seems higher quality than you thought, feel free to cut’n’paste it into its own post — there’s no quota for posting and the bar really isn’t that high.

The post Xitter web has spawned soo many “esoteric” right wing freaks, but there’s no appropriate sneer-space for them. I’m talking redscare-ish, reality challenged “culture critics” who write about everything but understand nothing. I’m talking about reply-guys who make the same 6 tweets about the same 3 subjects. They’re inescapable at this point, yet I don’t see them mocked (as much as they should be)

Like, there was one dude a while back who insisted that women couldn’t be surgeons because they didn’t believe in the moon or in stars? I think each and every one of these guys is uniquely fucked up and if I can’t escape them, I would love to sneer at them.

(Credit and/or blame to David Gerard for starting this.)

  • lagrangeinterpolator@awful.systems
    link
    fedilink
    English
    arrow-up
    7
    ·
    1 day ago

    Oh boy, another AI doom video popped up on my feed. Time for more morbid curiosity. The topic is about Big Yud and Nate Soares’s new book (“If You Build It, Everyone Dies”) about how AI is gonna kill us all. I have better things to waste 30 minutes on, so I’m not watching the full video, but the thumbnail (“The 7 Minute War”) kinda suggests what the contents are gonna be.

    Thankfully, the description of the video has a Google doc with their sources! I’m sure it’s full of hard evidence from careful experiments that logically demonstrate why their doomsday scenario is something to worry about, not just a random assortment of Anthropic blog posts and completely unrelated events.

    Somehow, there are a bunch of sources for the first 2 minutes of the video.

    “In the New York Times’ best-selling book, which was endorsed by Nobel laureates and the godfathers of AI” Geoffrey Hinton — Personal estimate >50% existential risk.

    Geoffrey “All radiologists will be replaced in 5 years” Hinton, Nobel laureate in physics, famous for his work in … physics.

    “researchers from the Machine Intelligence Research Institute describe in detail one potential example future” Machine Intelligence Research Institute — The Sable scenario from If Anyone Builds It, Everyone Dies by Yudkowsky & Soares. Fictional narrative illustrating risks, not prediction.

    This is not the first we’ve seen from MIRI, and I have a feeling it will not be the last. The monster under my bed is a fictional narrative illustrating risks, not prediction.

    “AI researchers have known this has been potentially a very bad idea since at least 2024” Anthropic/Apollo Research — Multiple 2024 papers document deceptive/self-preserving behaviors in controlled evaluations.

    They are still trying to flog the Anthropic/Apollo Research claims that chatbots will lie to you if you tell them to lie to you.

    “They spin up 200,000 GPUs and let Sable think for 16 hours straight” xAI/NVIDIA — Colossus supercomputer in Memphis scaling toward ~200,000 GPUs for Grok training.

    What does this even demonstrate? Some people can do some stuff with some GPUs? I ate some oatmeal today. Now everyone should be thoroughly convinced of my oatmeal-eating abilities.

    I watched for a few seconds around the timestamp, and it seems to be the beginning of their scifi story, I mean, AGI scenario. Yes, if you want to convince people that your scenario is plausible, I’m sure this is the part that you need serious amounts of evidence for. Remember, almost half the sources have timestamps for the first two minutes of the video.

    “a stunt to see if Sable can crack famous math problems like the Riemann hypothesis” Clay Mathematics Institute — Riemann Hypothesis remains unsolved after 160+ years, considered most famous unsolved problem in pure mathematics.

    Again, what does this demonstrate? I tried solving P vs NP with a cheeseburger. That didn’t work either. The only purpose of mentioning this is for narrative window dressing, because Math Is For Smart People.

    These are the sources for just the first two minutes. After that, they get a bit sparse.

    “Back in 2024, smaller models showed flashes of the same behavior” Multiple Papers — Documented deception/scheming findings in frontier models.

    “Claude 3.7 was caught repeatedly cheating on coding tasks even when told to stop”

    More Anthropic blog posts and system cards? Come on, I can’t sneer the same thing twice in one post!

    “Steal cryptocurrency from weak exchanges just like hackers did to Mt. Gox in 2011” U.S. Department of Justice — Russian nationals charged for 2011 Mt. Gox hack. 647,000-850,000 BTC stolen.

    I don’t know what this has to do with supporting the validity of their AI doomsday scenario, but kudos to them for showing why cryptocurrency is also stupid, I guess.

    “or Bybit in 2025” Reuters/FBI — Largest cryptocurrency theft to date. FBI attributed to North Korean Lazarus Group.

    More? I guess this is hard evidence for showing why cryptocurrency is stupid. I still don’t understand how this demonstrates that AI is scary.

    “Reminder, this scenario is based on years of technical research by the Machine Intelligence Research Institute, laid out in the book If Anyone Builds It Everyone Dies” MIRI — Meta-commentary explaining the scenario is illustrative, not predictive.

    I knew MIRI would be back. It’s illustrative, not predictive! Please don’t blame us if none of this even remotely happens! But it’s based on years of technical research. An entire graduate student’s worth of output in a decade.

    “In 2023, a human gave an LLM access to the internet and created an X account, Terminal of Truths, which gained hundreds of thousands of followers and launched its own crypto meme coin that reached a literal billion dollar market cap” Terminal of Truths — Real-world example of AI agent gaining social media following and wealth.

    The link they give references … another one of their own videos. You really are not beating the circular reference allegations here. Even if the purported story is somehow accurate, this again demonstrates how cryptocurrency is stupid. At least they use an LLM as a prop this time.

    “Gain of function research. Any one of them could be hijacked to unleash catastrophe.” Science/CIDRAP — Fouchier and Kawaoka created ferret-transmissible H5N1. Controversial GOF research began 2011.

    I think Yud is obsessed with this topic in particular. Better than diamondoid bacteria, I guess. Again, the AI just magically comes in and uses this stuff somehow.

    “The number one and number two most cited living scientists across all fields think scenarios like this are not only possible but likely to happen. And the average AI researcher thinks there is a 16% chance of AI causing human extinction.”

    Okay, let me be completely serious for this one. What would someone do if they truly believed that their work would lead to a horrible disaster, such as the extinction of humanity? Would they continue to work in the field, let alone make enough contributions to rise to the top? Alright I’m done.

    • fullsquare@awful.systems
      link
      fedilink
      English
      arrow-up
      5
      ·
      1 day ago

      ahem. h5n1 for ferrets was probably made because ferrets turn out to have immune systems similar enough to humans, in that they do get (common strains of) flu and transmit it by sneezing, that is ferrets are good model organisms for flu vaccine development. so if regular ferrets don’t catch h5n1, then you have to modify either virus or ferret because otherwise it won’t work. it’s not some random virologist deciding to wage biological war against fuzzy noodle critters

    • Seminar2250@awful.systems
      link
      fedilink
      English
      arrow-up
      5
      ·
      1 day ago

      The number one and number two most cited living scientists across all fields think scenarios like this are not only possible but likely to happen. And the average AI researcher thinks there is a 16% chance of AI causing human extinction.

      assigning a number to it makes it scientific

      aside/rant

      i wonder to what extent this bullshit works because of people’s fear of math

      i wish i could convince people that STEM skills are no different than a law degree, in essence — you’ll meet dipshits that are excellent mathematicians and you’ll meet smart people that are mediocre mathematicians. i suspect it’s because people view mathematical notation as impenetrable (when that just depends on the same shit any technical writing depends on, like the writer’s skill at communicating, the reader’s familiarity and strength with the prerequisite material, etc.)

      it’s frustrating, given the number of stupid mathematicians i’ve met