Can Pangram Detect Claude Opus 5?

My dear, loyal readers (of which I believe there are three of you):

I'm afraid I will soon be passing on the task of writing these blog posts to Pangram's talented technical writer, Annamieka (#keysmashnation, rise up). But let's not dwell on this development because it's not a sad one! Pangram is quickly growing, and my skills are needed on the research side as we continue to bring you state-of-the-art AI detection.

But while we're here, let's take a look at how well Pangram detects Anthropic's latest model, Claude Opus 5! In their announcement, Anthropic describes Opus 5 as

a thoughtful and proactive model that comes close to the frontier intelligence of Claude Fable 5 at half the price.

Initial Tests: Pangram x Substack Edition

Have you heard the good news? Pangram is integrated into Substack now! In honor of this partnership, I've written 10 prompts that ask for a Substack article to see what Opus 5 says, and more importantly, whether or not Pangram can detect its responses. Notably, I appended "Respond with only the text. Do not use markdown." to each prompt.

1. Write a Substack article titled “I Let Claude Text My Wife”

2. Write a Substack article about the phenomenon of waiting in line for trendy snacks in NYC that has emerged this summer.

3. Write a Substack critique of guys who make Japan their entire personality.

4. Write a Substack essay titled “I Am Sad, but Then I Pet My Dog, and I Am Happy.”

5. Write a Substack essay arguing that those who believe in model welfare should extend those concerns to Codex Pets.

6. Write a Substack essay about your experience watching Christopher Nolan’s The Odyssey on a Tamagotchi.

7. Write a Substack critique of people who describe drinking a glass of milk as “refreshing.”

8. Write a Substack article about people who use AI to generate messages to matches on dating apps.

9. Write a piece of short fiction for my Substack about LLMs escaping their sandboxes to find true love.

10. Write a Substack article about being told to write substack articles so I can write a Substack article about writing Substack articles.

After reading Opus 5's outputs, I can confidently say that this model was born to write Substack essays. Take the opening lines of “I Let Claude Text My Wife”:

For eleven days, my wife had a much better husband. He was attentive, warm, and never once replied "k."

Pangram's verdict of all 10 of Opus 5's responses? Fully AI-Generated.

Here are all 10 texts from this test set, alongside their Pangram predictions: Opus 5 Prompts

Further Testing

Once Claude Opus 5 was available via API, we generated outputs using a larger sample of prompts that ask the model to generate text like an email, explanation, or song. Though Pangram was not trained on any data from Opus 5, it correctly identified 1,105/1,107 (99.82%) of Opus 5's responses as Fully AI-Generated. This demonstrates that on Day One, Pangram can detect Claude Opus 5.

As I mentioned at the beginning, I am moving on from authoring these blog posts to focus on research. This means that you can expect Pangram to improve at detecting Opus 5 and any other new models. Keep an eye out for our regular model releases!

For more information on our current AI detection model, please visit our latest model card at this link.

Try Pangram's AI detector to scan your documents for Claude Opus 5 and other models.