What does your AI detection score mean? Understanding your AI Scan

You run a student’s essay or a freelancer’s article through an AI detector. A big, bold number pops up on the screen: 65% AI. What do you do next?

AI detection scores aren't like a traditional grading rubric, where "pass" and "fail" are obvious. The nuance between "completely AI-generated" and "AI-edited" is evolving, and so are our detection systems at Pangram.

This guide translates simple percentages into plain English. We go over how an AI score checker calculates percentages, what confidence intervals mean, and how to determine the right AI detector threshold for taking action.

What Does the Percentage Actually Indicate?

If you scan a document with an AI detection tool, you will get an AI percentage score. “50%,” for example. This percentage doesn’t mean that 50% of the document is fake/AI-generated. It means that, according to the AI percentage checker, 50% of this document contains AI-generated or AI-assisted writing.

Example of an AI Detection Score

Advanced AI detection tools don’t treat a document as a single giant block of text. Rather, these tools break down the text into segments, sentences, and paragraphs. These individual chunks are scored as segments.

If you get a 30% score on a 10-page paper, it likely means that roughly three pages worth of text display the patterns that are typical of an LLM. These patterns include things like a lack of burstiness and predictable syntax, among others. The score does not mean that 30% of the document is AI-generated.

Reading the AI Score Spectrum

If you are using an AI detector and a particular document gets a low score (e.g., 30% AI-generated), this usually indicates a hybrid document. A hybrid document is usually written by a human and then edited with the assistance of an AI tool. On the other hand, a high score, like 85%, strongly suggests that a text is entirely AI-generated.

Highly Likely AI

The low-to-mid-AI detection score often occurs when a writer:

• Uses tools like Grammarly.
• Asks an LLM, like ChatGPT, to “smooth out” the paragraphs they wrote.
• Uses AI to translate their native language to English.

A high AI detection percentage often occurs when the linguistic DNA of a text is overwhelmingly AI-generated. It usually means that the writer of that text typed a prompt into an LLM and copied-pasted that LLM’s direct output with some editing.

Are AI Score Checkers Accurate?

We can't validate the results of other tools, but enterprise-grade AI checkers like Pangram are highly accurate (99.98%). To help you assess their accuracy, most enterprise-grade AI checkers provide a "Confidence Interval" showing how certain the model is about its own detection percentage.

The answer to the question “Are AI checkers accurate?” is rooted in two facts: statistical models are used to determine whether a piece of text was AI-generated, and these models operate with probabilities rather than absolute certainties.

A “High Confidence” flag means that the text matches known LLM training data patterns. And, because the text matches known LLM training data patterns, the AI detection percentage is reasonable. This doesn’t mean the AI detection percentage is absolutely accurate, but that it is probably accurate.

A “Low Confidence” flag means the text has some AI traits, but the model doesn’t have enough data to make a definitive ruling. Many “Low Confidence” flags are caused by snippets from a text that are too brief for an accurate assessment.

If you are using an AI checker that has a black-and-white assessment of written content, a tool like Pangram can help you assess the specific sections that indicate AI-writing.

The "Mixed" Result

The most common reality in modern workflows is “Mixed” content that combines human and AI writing/editing. This is why tools like Pangram 3.0 classify text on a spectrum: Fully Human, Lightly AI-Assisted, Moderately AI-Assisted, and Fully AI-Generated.

Mixed AI Score Results

Classifying AI-generated text on a spectrum is important because a student who gets a 10% score for "Lightly AI-Assisted" text, all because they used a spellchecker, should not be treated the same as a student who turned in a 95% "Fully AI-Generated" essay. The segment highlighting shows you exactly which parts were made with AI.

What AI Score Thresholds Warrant Action?

There is no universal "magic number" that requires corrective action, but, as a best practice, AI detection scores below 20% often indicate standard digital writing assistance. Scores above 60% often warrant a direct conversation about the authenticity of someone’s writing.

You should set an AI detector threshold that reflects your AI policies. For example, if your policy allows “AI for brainstorming ideas, but not drafting the text itself,” then a 40% score requires investigation. Or, if your policy says “AI is not to be used for any part of the writing process,” even a score of 15% likely requires investigation.

The AI detection score you receive is a diagnostic tool. It is not a judge and jury. And, if a high score is triggered, you can use Pangram’s highlighted segments and “AI Phrases” report to sit down with the writer and ask them to explain their drafting process. This can lead to clarification, guidance, and the work that you are both looking for.

AI Detection Is Not a Binary "Pass/Fail"

Pangram is a nuanced analytical tool that provides visibility into the processes that define today’s writing. By understanding exactly what an AI score means, professionals can confidently uphold integrity standards while treating writers with fairness.

Stop guessing what the numbers mean. Get granular, plain-English insights into text authorship with Pangram's segment analysis.

Scan Your Document