Paralegal Guide: Detecting AI in Legal Documents

If you are a paralegal, you probably already know this: AI-assisted drafting is now common in contracts, briefs, and discovery.

Even though AI-assisted drafting is now common, undocumented AI usage can introduce hallucinations, compliance issues, and evidentiary challenges that make things more difficult for your clients.

To maintain the integrity and standing of your law firm, you can use AI detection. Not as an engine for generating accusations against those you work with, but as a verification and risk-mitigation tool. AI-generated documents and sections of documents should be reviewed by knowledgeable paralegals.

Detecting AI in Legal Documents

AI detection for legal documents involves analyzing writing patterns to estimate whether the text was likely generated by large language models.

For example, if you use an AI detection tool to scan a brief, the tool will look at the writing patterns that comprise the information within this brief. Doing so will let the detector determine if the brief was generated with AI.

AI legal document review differs from plagiarism checks because it isn’t looking for plagiarism, but instead, specific patterns that suggest a piece of writing was generated with AI.

Evaluating legal writing for AI usage is especially difficult because a formal tone can be a sign of AI-generated content. Many AI detection tools fail to accurately determine AI’s usage in a legal document because of this.

To detect AI within a legal document in an accurate and non-false-accusation manner, Pangram analyzes the structure and predictability of a piece, as well as specific statistical signals.

Pangram - Legal Document AI Detection Features

Can AI Detection Be Trusted for Legal Work?

The answer to this question is “Yes.” But, AI detection - and, more specifically, AI legal document review - can only be trusted for legal work when it is used as a probabilistic signal to suggest that a piece of writing was/was not generated by AI. It should not be used as a source of definitive proof that is beyond criticism.

Even though some AI detectors are very reliable, false positives do happen. If you use an AI detector to check legal documents, and it suggests that a document was generated with AI, you may want to review it before moving ahead.

Reviewing a piece of writing that was, according to an AI detector, generated with the use of AI can allow you to verify that it actually was written with AI. In the event that the piece does appear to be generated with AI, you can discuss this with the person who produced it.

Courts and compliance teams view probabilistic evidence of AI generation with caution. A piece of legal writing that was produced with AI could contain serious errors, inaccuracies, or erroneous claims.

Common Legal Documents Where AI Use Creates Risk

The following common legal documents come with major risks when AI is used to generate them:

• Contracts & amendments: if your contract or amendment features laws/regulations that don’t exist, and were generated with AI, this could invalidate the entire contract/amendment. An AI legal contract checker serves as a layer of protection in this regard.
• Discovery responses: Your discovery response should include the requested information, but if you use AI to generate it, it may include AI-generated content that could harm your client.
• Internal memos and summaries: memos and summaries generated with AI could contain AI hallucinations, such as laws that don’t exist and events that didn’t happen, which can mislead paralegals and lawyers.
• Client-submitted drafts: your client could submit AI-generated documents - contracts, for example - and, if they do so, those documents could lead to a worse outcome for your client if they contain incorrect information due to AI. This is why an AI legal contract checker - among others - is very important.

AI Detection vs. Manual Review: Use Both!

You need AI detection in order to create a program to highlight sections of documents that should be manually reviewed. The reason for this is as follows: AI detection can catch statistical irregularities that suggest AI usage, and manual review can catch content that is incorrect or harmful.

Combining AI detection with manual review reduces downstream liability by allowing you to ensure that a piece of content wasn’t written with AI and, if it was, to remove the legal flaws that might end up hurting your client.

How Paralegals Should Handle a Document Flagged as AI

Paralegals should handle a document flagged as AI using a step-by-step workflow that supports the paralegals they work with, the lawyers that run the firm, and the clients they serve. This step-by-step workflow is as follows:

• Review your internal "acceptable AI-use" policy if you have one.
• If a document was flagged as AI, after AI legal document review, verify this usage by running it through the AI detector one more time and reading it yourself.
• Check the document history and metadata - for example, if you see that the document was empty before a large volume of text was pasted, that’s a good sign the document was produced with AI.
• Use the document history, metadata, and/or AI detector results as evidence and put this evidence together to escalate your claim that a particular document was AI-generated.
• Document all of the evidence you gather and clarify how you got this evidence, what this evidence suggests, and what it means, so that you have an audit trail.

Choosing the Right AI Detection Tool for Legal Teams

Choosing the right AI detection tool for legal teams involves looking for key features that support your work, the work of the lawyers that make up your firm, and the clients that you serve. These key features are as follows:

• The AI detector can verify if a piece of writing was AI-generated across multiple legal writing styles.
• The AI detector has false positive controls to prevent human-written documents from being flagged as AI.
• The AI detector comes with privacy and non-training guarantees to protect your firm and clients.
• The AI detector offers enterprise-level auditability, allowing your entire enterprise to use it.
• The AI detector should be able to quantify the extent of AI usage within a particular document..

Out of all the AI detection tools on the market, Pangram is one of the very best AI tools for paralegals. We offer all the features listed above, as well as high-grade certification to support your legal team’s privacy and security.

Detection tools allow paralegals to preserve document integrity, protect firms, and support attorneys with defensible evidence.

Help your legal team preserve document integrity, protect the firm, and support their attorneys by using Pangram.

Scan Legal Documents for AI