From a UX perspective that site is horrible - the auto scroll thing it’s doing is possibly the most disruptive way to do multiple questions I’ve come across
66% correct! I did better than expected! It’s interesting that most people taking the test do better than 50/50. People aren’t just guessing, they can actually tell the difference, though that might be because there’s a few obvious ones with 6 fingers.
Also interesting are the couple photos that are picked wrong often. One is wrong 84% of the time. It’s a landscape in oil on canvas style most believed is human made. I guess both the style and subject are rigid enough that it’s hard to tell. Ask 10 artists to draw the same landscape with oil on canvas and they’d all end up coming out similar making it easy for AI to replicate.
The abstract ones are most difficult.
Something I figured out that was handy for guessing some human pieces was trying to figure out what the prompt would have been for generating the image. If I couldn’t formulate with words what the image was, then it was more likely a human artist.
On the other hand, looking at light, shadow, and reflections to see if everything made sense really didn’t help.
This was really interesting! There were some cases where I felt like the low res quality of the image made it harder than it should be but maybe that was the intention. Does anyone know of a similar resource that also trains you to spot AI creations (outside of the obvious things like weird hands and feet)?
I got 80% correct but I really took my time. 4 of the AI images that I thought were human (and 6 vice versa).
It was more fun than I thought, but I imagine when it gets even harder to tell it will be an agonising task to perform.
Go team human!
Quite interesting, even though the dataset clearly favors generated art.
What do you mean by that? I think there are 25 of both.
Yes, but genereated pictures are deliberetly chosen to contain less elements that “AI” struggles with and human made ones contain quite a bit of bad anatomy ones just to confuse you. It also contains abstract art, which literally strips proper shapes from drawn objects and calls it a stylistic choice.
What I’m getting at is that it’s not a random selection from both categories, they were hand picked. And in my opinion, the selection methodology favors generated art.
Gotcha. Yeah, it’s definitely a curated selection of artwork. I don’t think it’s really fair to say that the human art is all that unrepresentative – yeah there are some intentional red herrings, like the greek text – but I think the most ambiguously AI-looking human art is the abstract stuff or the stuff that is in genres that AI is commonly used for e.g. anime girls. But the AI art is definitely chosen to be above average; most of them pass the filter of stuff the ACX author didn’t immediately recognize as for sure AI. So it’s not like this is test is a particularly objective measure. I think it’s still fun though.
It’s not clear to me what the purpose of that test is, other than a gotcha. Are we going to see a snarky reveal of the hit and miss rates in some clickbaity newsicle next week?
A few images were quite clearly in the supersaturated colour scheme that “genAI” favours; others were artworks or artist’s styles I recognised, like Basquiat and Miro. The harder ones to identify were the generic DeviantArt artworks, and a couple of false positives made by “genAI” that were quite obviously trained on fine art masters.
But again, not a Turing test, only a test of the participants’ art history skills and visual literacy 🤷
I think the purpose is to give a rough estimate about how hard it is to tell humans from AI art. This was adapted from a test posted on ACX, whose author generally likes to look at graphs of things over time, so I imagine he’ll likely post a couple more over the next few years and compare. I wouldn’t say that’s a gotcha.
Yeah, it’s not exactly a Turing test, no; the Turing test is meant to be interactive after all. That’s sort of harder to do with this kind of art than it is with text. You clearly have a greatly above average understanding of art history; I think the vast majority of people don’t really know the styles of more than a couple famous artists or know much about art histories. So for most people this is meant to be a test on vibes; can they detect “humanity” vs “genAI-ness” in a piece. I think you could more or less call it a Turing test.
Thanks for the context, I didn’t see that information on the website at all. Generally, I think it’s just good form to let people know how and to what ends their results are used — but of course, we know it’s not necessarily how things work
The site mentions it’s from ACX at the top. The results on this version are probably not going to be collected though, and it’s just for fun. The original ACX posting was a google form; when the site says “X% of people get this wrong” it’s most likely in reference to the population polled at that time.
The site mentions it’s from ACX at the top
…no? There’s a link mentioning Scott Alexander which does lead to ACX, but without any background or context. It’s a fairly big assumption that others know what “ACX” is and how SA is connected to it.
I didn’t connect the dots immediately, but turns out I tried reading Unsong back in the day. I quit when I realised Alexander’s ties to rationalist and effective altruist thought 🤷
Oh, sorry, I misremembered the link as saying ACX, but you’re right, it does say Scott Alexander.
Loved Unsong, I’m sorry you have had bad experiences with rationalists/EAs. :/ Yudkowsky is pretty weird and egotistical, still enjoyed his writing though. Here is a good essay defending EA, just basically reminding you that fundamentally EA is about convincing rich people to donate to life-saving charities; I don’t really see why this would be harmful. The biggest criticisms of EA I see are “well there shouldn’t be rich people!” – like, I agree, but, how is that relevant?
Different essay, but I love this quote:
Something else happened that month. On November 11, FTX fell apart and was revealed as a giant scam. Suddenly everyone hated effective altruists. Publications that had been feting us a few months before pivoted to saying they knew we were evil all along. I practiced rehearsing the words “I have never donated to charity, and if I did, I certainly wouldn’t care whether it was effective or not”.
Fascinating, but i hate all the red herrings
