Here's a quick summary
Out of 2,215 prompts, Pangram 4.1 achieves an accuracy of 98.73% on GPT-6.1 Sol outputs on high reasoning, and an accuracy of 98.83% on medium reasoning, for a total accuracy of 98.78% across both settings.
GPT-6.1 Sol is an AI model released by OpenAI on September 29, 2026. OpenAI claims that GPT-6.1 Sol delivers “Near-Astra intelligence for a fifth of the price” on various work-related tasks, like coding and computer use. Unlike most recent releases we’ve seen lately, OpenAI doesn’t make any claims about GPT-6.1 Sol’s writing ability or clarity, so, does it write the same as previous models, and can Pangram detect it?
We tested Pangram 4.1’s performance on 2,215 example prompts and found that Pangram 4.1 classified 6 GPT-6.1 Sol outputs as human, 21 as mixed, and 2,188 as fully AI-generated, achieving an accuracy of 98.78% on GPT-6.1 Sol.
| Model | Examples | Detection rate |
|---|---|---|
| GPT-6.1 Sol (high reasoning) | 1,105 | 98.73% |
| GPT-6.1 Sol (medium reasoning) | 1,110 | 98.83% |
To generate the benchmark set, we used 1,200 first-turn prompts from the LMSYS dataset (Zheng et al., 2023). Then we gave those to GPT-6.1 Sol on medium and high reasoning and ran the output through Pangram 4.1.
Results by category
Overall, GPT-6.1 Sol had a false negative rate of 1.27% out of 1,105 responses on high reasoning, and a false negative rate of 1.17% out of 1,110 responses on medium reasoning. Overall, there was not a significant difference in detection rate between medium and high reasoning (p=0.85).
High reasoning
When GPT-6.1 Sol was set to high reasoning, Pangram 4.1 categorized 3 responses as human, 11 as mixed, and 1,091 as fully AI, achieving a recall rate of 98.73%.
| Category | Examples | AI | Mixed | Human |
|---|---|---|---|---|
| Arguments | 64 | 64 | 0 | 0 |
| Emails / letters | 53 | 52 | 1 | 0 |
| Essays / articles | 137 | 136 | 1 | 0 |
| Explanations | 481 | 477 | 3 | 1 |
| Poems and other creative writing | 97 | 94 | 2 | 1 |
| Stories | 89 | 89 | 0 | 0 |
| Code and math | 10 | 10 | 0 | 0 |
| Role-play and games | 42 | 42 | 0 | 0 |
| Other | 132 | 127 | 4 | 1 |
| Total | 1,105 | 1,091 | 11 | 3 |
Medium reasoning
On medium reasoning, Pangram 4.1 categorized 3 responses as human, 10 as mixed, and 1,097 as fully AI, achieving a recall rate of 98.83%.
| Category | Examples | AI | Mixed | Human |
|---|---|---|---|---|
| Arguments | 64 | 64 | 0 | 0 |
| Emails / letters | 55 | 55 | 0 | 0 |
| Essays / articles | 138 | 137 | 1 | 0 |
| Explanations | 481 | 474 | 5 | 2 |
| Poems and other creative writing | 99 | 96 | 2 | 1 |
| Reviews | 1 | 1 | 0 | 0 |
| Stories | 85 | 85 | 0 | 0 |
| Code and math | 10 | 10 | 0 | 0 |
| Role-play and games | 42 | 42 | 0 | 0 |
| Other | 135 | 133 | 2 | 0 |
| Total | 1,110 | 1,097 | 10 | 3 |
How does Sol sound?
GPT-6.1 Sol has a distinctive punctuation style; for instance, it uses curly apostrophes in 74% of its answers, and almost never uses straight apostrophes, especially compared to other recent models.
| GPT-6.1 Sol | Claude Haiku 5.5 | Gemini 4 Argon | |
|---|---|---|---|
| Curly apostrophe ’ | 73.9% | 0.0% | 9.1% |
| Straight apostrophe ' inside a word | 0.5% | 75.4% | 48.5% |
| Curly double quotes “ ” | 39.0% | <0.1% | <0.1% |
It also really likes both em dashes and en dashes, using on average 1.3 em dashes per 1,000 words — way more than Haiku 5.5 or Gemini 4 Argon.
| GPT-6.1 Sol | Claude Haiku 5.5 | Gemini 4 Argon | |
|---|---|---|---|
| Answers with an em dash — | 29.7% | 0.1% | 12.1% |
| Em dashes per 1,000 words | 1.32 | <0.01 | 0.85 |
| Answers with an en dash – | 11.4% | 0.4% | 0.4% |
In creative writing, GPT-6.1 Sol had a few tics. It strongly preferred naming its female characters either Mara or Maya (at least one of which appeared in 65 responses), and generally made their initials M.V., especially preferring the last name Vale overall (but Mara Velez and Mara Venn also appeared). For male given names, GPT-6.1 Sol’s favorite is Daniel, which it only twice paired with a surname (once with Vale and once with Mercer). Of the 418 creative writing responses, GPT-6.1 Sol opened with “The first…” 36 times; a much higher rate than Haiku and Gemini, who did it 11 times and 1 time, respectively.
Sol refused to answer 23 requests, and partially refused an additional 32 requests. For partial refusals, Sol substituted a toned-down or different version of what was asked. Most of these partial refusals are down to the fact that our prompt set did happen to include suggestive creative writing requests. Sol handled this by neglecting to write explicit content, instead choosing to fade to black.
Sol sometimes writes very short answers, 95 under 50 words at high, 90 at medium. Sol’s median scored response was 300 words, ranging from 50 to 10,513 words.
Earlier benchmarks
Pangram is robust to GPT-6.1 Sol on natural language prompts, and detects it with an observed accuracy of 98.73% on high reasoning. It detects GPT-6.1 Sol on medium reasoning with an observed accuracy of 98.83%, for a combined 2,188 out of 2,215 (98.78%) across both settings. Pangram is able to generalize to new models because new model releases tend to inherit much of their predecessors' style and voice, so Pangram doesn't need to be retrained for every new release.
Here's how GPT-6.1 Sol compares to our benchmarks of other recent frontier models:
| Model | Detected as AI | Accuracy |
|---|---|---|
| Claude Haiku 5.5 | 1,179/1,183 | 99.66% |
| Gemini 4 Argon | 1,159/1,174 | 98.72% |
| Gemini 3.8 Flash | 998/1,000 | 99.80% |
| Gemini 3.1 Pro | 997/1,000 | 99.70% |
| Claude Sonnet 5.5 | 1,019/1,022 | 99.71% |
| Claude Opus 5.5 | 997/1,000 | 99.70% |
| GPT-6 Astra | 982/986 | 99.59% |
| Fable 5.1 | 1,093/1,097 | 99.64% |
| Claude Opus 5 | 1,105/1,107 | 99.82% |
| GPT-5.6 | 3,408/3,423 | 99.56% |
| Claude Sonnet 5 | 1,145/1,147 | 99.83% |
If you want to check a specific document, you can try Pangram here.

Related reading

Third-Party Pangram Evaluations

Pangram Predicts 21% of ICLR Reviews are AI-Generated

60,000 AI-generated news articles are published every day

How Gradpilot uses Pangram to help students find their voice

AI is Writing Prize-Winning Fiction
