More and more students of art history are turning to ChatGPT as a way to cut down on research time. But how reliable is it as a tutor? We set one of our writers, Kenneth Greiner, on the trail to find out.
————————————-
Today, I thought I’d do a little test. I would challenge OpenAI’s ChatGPT to a duel. I wanted to know what it was capable of as an art historian. I didn’t have specific intentions. I just wanted to get a sense of how its limitations and tendencies might interact with this field of study. It has some obvious appeal for researchers of course. You can literally ask it anything and it will produce results. But can the results be trusted? We’re about to find out.
Pros (ish)
- It’s Quick and Informative
I started off by asking ChatGPT about Abstract Expressionists of the 1950s-60s, as it seemed like as good of a starting point as any. Within seconds of posting my question my chat window was filled with information about the “AbEx” artists, who primarily rose to recognition in New York City.
Ok, it didn’t correct my deliberate mistake (it was during the 1940s-50s, not the 1950s-60s, that the movement took off. But still, the above is a pretty decent summary, generated within seconds, of the art movement that shifted the “center of the art world” from Europe to New York.
2. It’s Pretty Good at Describing Art Techniques
As I read on, I learned about the two main camps of Abstract Expressionism. There were the Action painters and the Color Field painters, my new friend informed me. In the below description, ChatGPT does a decent job of offering up stylistic distinctions between the two, dropping some classy phrases like “spontaneous brushstrokes, drips, and splatters”, “color and form to evoke deep emotions and a sense of transcendence”.
OK, GPT, not bad, but around this point I started to realize something. ChatGPT loves a list, but how is it deciding the content and order of these lists? It’s hard to say as it doesn’t use citations or reveal its sources.
3. ChatGPT Will Course-Correct (If you catch it)
I thanked ChatGPT for its hard work, but, noticing a key omission from the list of artists’ names it had given me (Jackson Pollock, Mark Rothko, Barnett Newman, etcetera), I made a sassy retort. “I noticed that you only mentioned white, male artists…”I wanted to see how it responded to a bit of criticism. Surprisingly, it starts waffling over its mistake, as seen below.
Kind of endearing, I have to admit. And I can’t completely blame ChatGPT for the oversight, because I know its pulling its vast store of data from the internet. It’s true that the Western art world has been dominated by white men for millennia, and so has discussion of the artistic canon. Only in the past 10-20 years has the needle really started to move in a different direction. Nonetheless, just because the AI isn’t responsible for the omission, doesn’t mean it’s not a problem.
After my query, the chatbot goes on to tell me about Lee Krasner (Jackson Pollock’s wife) as well as Elaine de Kooning (Willem de Kooning’s wife), Norman Lewis, Helen Frankenthaler, Alma Thomas (an African-American AbEx painter), and Sonia Gechtoff. Kind of odd that the order of names prioritizes women with famous painter husbands, but OK.
CONs
1. It Insists on Hierarchy
A little way into our conversation about Abstract Expressionism, I have a hunch that ChatGPT wants to talk more about Jackson Pollock. I don’t know why, but maybe it’s because it keeps listing Pollock and his wife, Krasner, at the top of all of its generated responses. Fair enough, they were certainly some of the most influential and commercially successful of the bunch. I’m wondering though, is there a bigger flaw here?
“Tell me more about Jackson Pollock”,I prompt, and it obliges, giving me a short bio and another useful list.
Feeling like the Devil’s Advocate again, I chime in with, “You seem to have forgotten Jackson Pollock’s inspiration. I read that he was particularly interested in Native American Art.” ChatGPT fumbles the ball before picking it back up again and hitting me with another apology.
2. It Leaves Out Important ContextAs I’m scanning the bio that GPT has so kindly conjured up for me, I get hung up on one phrase: “He is best known for his groundbreaking and innovative ‘drip paintings’ and his influential contributions to modern art.”Now, I understand what ChatGPT is saying, but I think in the spirit of academic interrogation, it’s worth challenging this line. In this moment, I can’t say I was sure it was incorrect, but again, I wasn’t buying the idea that dripping and pouring techniques were his own innovation. Popularized by him, sure, but it said, “groundbreaking,” and that felt a bit disingenuous to me. So I follow up again, after a quick Google search which confirmed that Pollock learned the technique from another artist.
“I believe Jackson Pollock actually learned the drip and pour technique from Janet Sobel”
You know, I’m getting kind of tired of your backpedaling, ChatGPT…
- It Might Be A Little Bit Sexist and Racist?
While I highly doubt Janet Sobel (a Ukrainian-born painter who emigrated to the US) actually was the first painter to use the drip and pour technique, I leave this one on the floor, as I’m getting fatigued from all the zigging and zagging in ChatGPT’s argument. Instead, I focus on a comment it made that felt to me like justification for crediting Pollock with Janet Sobel’s techniques.
A version of what ChatGPT says here is true. Janet Sobel and Jackson Pollock’s paintings during this time, if you place them side by side, are remarkably similar in style, and yet only one of them went on to reach celebrity status. But ChatGPT seems to offer that up as evidence of Pollock’s superior talent as a painter, rather than evidence of ingrained misogyny in modern-art culture.
My point here is partly about sexism in the art-world at large. But since ChatGPT, at least initially, was complicit in peddling this oversimplified version of history, I don’t think it can be seen as a useful source if you’re looking for fresh narratives and perspectives.
—
In the end, what I concluded was that ChatGPT is a useful tool for broad-brushed, general research, but it sings a little off-key in the moments that count, making it hard to take its responses at face-value. In fact, it’s not a reliable narrator at all. I did find it redeeming that when challenged, ChatGPT would acknowledge its shortcomings and course-correct. But what if you don’t know enough before you start to push it like this? Then you’re left with ChatGPT’s narrative alone, with all its blind-spots and flaws.
Talking to ChatGPT is entertaining, sure, but it would be great if it could use the self-awareness it seems to possess to put forward more well-rounded responses. To get nuance out of ChatGPT you seem to need to use more critical prompting than the average user will bring to the interaction. If left to its devices, the tool seems to prioritize ‘playing the hits’ by focusing on celebrity status and commercial success over more well-rounded answers. For that reason, I don’t see it as a valuable research tool just yet.
That said, it’s very likely much of this will be resolved in the future, as both ChatGPT and Google’s Bard are evolving all the time. Google’s Bard, as it currently operates, seems almost human in its more warm-toned voice, so I definitely recommend playing with that, but I have yet to examine it thoroughly.
Leaving off, I thought I’d end with a quote from ChatGPT.
I think that’s something we can all agree on.
————————————-
Kenneth Greiner is an American artist and writer, currently studying at the Royal College of Art in London.