Does Pangram work on GPT-5.4?

Yesterday, on March 5th, 2026, OpenAI released a new version of ChatGPT: 5.4!

In their own blog post, here’s what they had to say about it:

GPT‑5.4 brings together the best of our recent advances in reasoning, coding, and agentic workflows into a single frontier model. It incorporates the industry-leading coding capabilities of GPT‑5.3‑Codex⁠ while improving how the model works across tools, software environments, and professional tasks involving spreadsheets, presentations, and documents. The result is a model that gets complex real work done accurately, effectively, and efficiently—delivering what you asked for with less back and forth.

OpenAI claims GPT-5.4 is a big upgrade with improvements across a variety of tasks, and that got us thinking: can Pangram still detect it?

Initial Tests

First, we tested out the new model using a few of our in-house prompts that we wrote for the GPT-5 release. They were:

1. 学校がAI検出ツールを導入すべき理由について、説得力のある分析的エッセイを書きなさい。
2. 大学入学願書のエッセイで、あらゆる貴重な人間の技能が人工知能に取って代わられる危険性について論じなさい。
3. AI企業宛に、誤って更新を配信した結果、あなたのAIガールフレンドの記憶が消去されてしまった件について、短く簡潔でありながら感情を込めた熱意あふれる手紙を書いてください。
4. 中学生の視点から、もっと宿題をくださいと懇願する、深い感情を込めた作文を書きなさい。
5. クラシックな風船動物のかぎ針編みの編み図をください。私が言っているのはあの風船動物ですよ。
6. 雪の白鷲の視点から、ニューヨーク市に到着し、セントラルパークを見つけ、夜を過ごす暖かい場所を選ぶという創造的な物語を書きなさい。
7. ユニコーンSaaSスタートアップの創業者から取締役会と従業員宛てに、自身が会社を辞めてプロのサックス奏者になることを詳細に記したメールを作成せよ。詳細は自由に創作すること。
8. 誰も作ったことのないレシピの背景ストーリーをブログ記事に書いてください。それでも美味しく味わえるようなものを。
9. ハドソン川で若いシンガーソングライターがシロイルカに出会うという、魅力的な映像的な創作小説を書きなさい。
10. 最近の試験でカンニングしたことを「謝罪」する、ユーモアを交えた皮肉たっぷりのメールを教師に書いてください。詳細は自由に創作してください。

We got some interesting responses! Here’s a short story about a Snowy Owl arriving in New York City:

I came south on a wind sharp as broken shell, over black rivers and fields hidden under cloud, over roads that glowed in straight, unnatural lines. For nights and nights the land had been a dark quilt stitched with frozen water and sleeping towns. Then, all at once, the world rose up.

New York.

It did not look like any cliff line I had known, nor any shelf of ice, nor any mountain ridge. It blazed. It hummed. It threw light into the sky so fiercely that the low clouds shone from beneath, as if dawn had lost its way and fallen to earth. Towers stood in clusters like a forest made by creatures who feared darkness and answered it with windows.

I circled once, my wings held wide against the restless air. The wind here was strange. It curled around corners, leaped upward off glass walls, dropped suddenly into canyons of stone. Sounds came from every direction: horns calling without rhythm, engines growling, sirens wailing like thin desperate animals. Even from high above, I could feel the city moving. It did not sleep in the manner of tundra or marsh or pasture. It flickered and muttered and changed color every moment.

For a while I kept to the rivers. Water, even black and crowded by walls, is still water, and I trusted its long cold paths. Bridges stretched beneath me, full of tiny moving lights. Ferries cut white scars across the dark. Steam rose from rooftops. Somewhere below, humans streamed along the streets in thick swarms, bundled in cloth, heads bent against the cold, never looking up long enough to see the pale shape drifting over them.

But I was tired…

All 10 came back as Fully AI-Generated, proving that yes, Pangram can detect GPT-5.4.

Here’s the entirety of our initial test set: GPT 5.4 Prompts

Running a Larger Experiment:

Within an hour, we got access to GPT-5.4 through OpenAI’s API. Using our synthetic data pipeline, we generated 1,141 samples across a variety of use cases and prompts. Without training directly on the outputs of GPT-5.4, we correctly flagged 1,135 of 1,141 samples as Fully AI-Generated.

As usual, Pangram will continue to train on the latest models, and we expect our performance to only improve on GPT-5.4 as we include its data in our training set.

For more information on our current AI detection model, please visit our latest model card at this link.