Skip to main content

Check out the Pangram MCP! Read more here

Case Studies

Can Pangram detect GPT-6.1 Sol?

Nov 2, 2026

Here's a quick summary


Out of 2,215 prompts, Pangram 4.1 achieves an accuracy of 98.73% on GPT-6.1 Sol outputs on high reasoning, and an accuracy of 98.83% on medium reasoning, for a total accuracy of 98.78% across both settings.

GPT-6.1 Sol is an AI model released by OpenAI on September 29, 2026. OpenAI claims that GPT-6.1 Sol delivers “Near-Astra intelligence for a fifth of the price” on various work-related tasks, like coding and computer use. Unlike most recent releases we’ve seen lately, OpenAI doesn’t make any claims about GPT-6.1 Sol’s writing ability or clarity, so, does it write the same as previous models, and can Pangram detect it?

We tested Pangram 4.1’s performance on 2,215 example prompts and found that Pangram 4.1 classified 6 GPT-6.1 Sol outputs as human, 21 as mixed, and 2,188 as fully AI-generated, achieving an accuracy of 98.78% on GPT-6.1 Sol.

ModelExamplesDetection rate
GPT-6.1 Sol (high reasoning)1,10598.73%
GPT-6.1 Sol (medium reasoning)1,11098.83%

To generate the benchmark set, we used 1,200 first-turn prompts from the LMSYS dataset (Zheng et al., 2023). Then we gave those to GPT-6.1 Sol on medium and high reasoning and ran the output through Pangram 4.1.

Results by category

Overall, GPT-6.1 Sol had a false negative rate of 1.27% out of 1,105 responses on high reasoning, and a false negative rate of 1.17% out of 1,110 responses on medium reasoning. Overall, there was not a significant difference in detection rate between medium and high reasoning (p=0.85).

High reasoning

When GPT-6.1 Sol was set to high reasoning, Pangram 4.1 categorized 3 responses as human, 11 as mixed, and 1,091 as fully AI, achieving a recall rate of 98.73%.

CategoryExamplesAIMixedHuman
Arguments646400
Emails / letters535210
Essays / articles13713610
Explanations48147731
Poems and other creative writing979421
Stories898900
Code and math101000
Role-play and games424200
Other13212741
Total1,1051,091113

Medium reasoning

On medium reasoning, Pangram 4.1 categorized 3 responses as human, 10 as mixed, and 1,097 as fully AI, achieving a recall rate of 98.83%.

CategoryExamplesAIMixedHuman
Arguments646400
Emails / letters555500
Essays / articles13813710
Explanations48147452
Poems and other creative writing999621
Reviews1100
Stories858500
Code and math101000
Role-play and games424200
Other13513320
Total1,1101,097103

How does Sol sound?

GPT-6.1 Sol has a distinctive punctuation style; for instance, it uses curly apostrophes in 74% of its answers, and almost never uses straight apostrophes, especially compared to other recent models.

GPT-6.1 SolClaude Haiku 5.5Gemini 4 Argon
Curly apostrophe ’73.9%0.0%9.1%
Straight apostrophe ' inside a word0.5%75.4%48.5%
Curly double quotes “ ”39.0%<0.1%<0.1%

It also really likes both em dashes and en dashes, using on average 1.3 em dashes per 1,000 words — way more than Haiku 5.5 or Gemini 4 Argon.

GPT-6.1 SolClaude Haiku 5.5Gemini 4 Argon
Answers with an em dash —29.7%0.1%12.1%
Em dashes per 1,000 words1.32<0.010.85
Answers with an en dash –11.4%0.4%0.4%

In creative writing, GPT-6.1 Sol had a few tics. It strongly preferred naming its female characters either Mara or Maya (at least one of which appeared in 65 responses), and generally made their initials M.V., especially preferring the last name Vale overall (but Mara Velez and Mara Venn also appeared). For male given names, GPT-6.1 Sol’s favorite is Daniel, which it only twice paired with a surname (once with Vale and once with Mercer). Of the 418 creative writing responses, GPT-6.1 Sol opened with “The first…” 36 times; a much higher rate than Haiku and Gemini, who did it 11 times and 1 time, respectively.

Sol refused to answer 23 requests, and partially refused an additional 32 requests. For partial refusals, Sol substituted a toned-down or different version of what was asked. Most of these partial refusals are down to the fact that our prompt set did happen to include suggestive creative writing requests. Sol handled this by neglecting to write explicit content, instead choosing to fade to black.

Sol sometimes writes very short answers, 95 under 50 words at high, 90 at medium. Sol’s median scored response was 300 words, ranging from 50 to 10,513 words.

Earlier benchmarks

Pangram is robust to GPT-6.1 Sol on natural language prompts, and detects it with an observed accuracy of 98.73% on high reasoning. It detects GPT-6.1 Sol on medium reasoning with an observed accuracy of 98.83%, for a combined 2,188 out of 2,215 (98.78%) across both settings. Pangram is able to generalize to new models because new model releases tend to inherit much of their predecessors' style and voice, so Pangram doesn't need to be retrained for every new release.

Here's how GPT-6.1 Sol compares to our benchmarks of other recent frontier models:

ModelDetected as AIAccuracy
Claude Haiku 5.51,179/1,18399.66%
Gemini 4 Argon1,159/1,17498.72%
Gemini 3.8 Flash998/1,00099.80%
Gemini 3.1 Pro997/1,00099.70%
Claude Sonnet 5.51,019/1,02299.71%
Claude Opus 5.5997/1,00099.70%
GPT-6 Astra982/98699.59%
Fable 5.11,093/1,09799.64%
Claude Opus 51,105/1,10799.82%
GPT-5.63,408/3,42399.56%
Claude Sonnet 51,145/1,14799.83%

If you want to check a specific document, you can try Pangram here.


Annamieka Aerts

Annamieka is a technical writer at Pangram.

More from Annamieka Aerts