268 | Hidden text can undermine your AI

Hello to all 8562+ of you from around the globe.

What if one line of invisible text could change how your AI interprets an entire case file?

This isn’t hypothetical. It’s called prompt injection, and as more legal teams connect LLMs to client files, discovery docs, and contract repositories, the risk is growing, fast.

In this edition, we’re cutting through the jargon to explain:

• What prompt injection actually is

• Why indirect injection is the real legal threat

• The specific questions you need to ask your vendors and internal teams—now

This is not fearmongering. It’s just a reality of working with tools that respond to language. And in law, language is everything.

What is Prompt Injection? And Why It Matters to Lawyers

Let’s break this down as simply and clearly as possible. There are two main types of prompt injection. Both should matter to every legal team working with GenAI.

Direct Prompt Injection

This is the kind most people might think about when they hear the term. It happens when a user (you) deliberately enters a prompt designed to override or manipulate the AI’s intended behavior.

It’s the classic example:

Similarly, you might remember one of the earliest and most notorious examples of direct prompt injection: the “DAN” prompt, short for Do Anything Now.

Back in the early days of ChatGPT, users discovered they could trick the model into ignoring its built‑in guardrails by telling it to “pretend” to be a version of itself that could say or do anything: no rules, no filters.

It worked disturbingly well for a while. The model would role‑play as “DAN” and start generating outputs that bypassed content restrictions or policy filters.

That was a defining moment for the public realizing how fragile these systems were and how easily behavior could be hijacked with nothing more than clever wording.

Most LLMs have hardened themselves against these types of attacks. But in truth, every one is still vulnerable. Why and how? It's simple, yet not so simple.

Remember: in LLM land, the coin of the realm isn’t code. It’s words.

Simply by using the right words in the right order, you can manipulate the behavior and output of the AI.

Think of how you might steer a conversation with a person:

• A direct challenge

• A misleading hypothetical

• A slow-rolling “psy-op” where, over time, you subtly reshape the context

LLMs can be manipulated in the same way. Prompting is persuasion.

This isn’t great. But here’s the good news: this is a user-side problem.

You control this. If you’re doing it, you (hopefully) know what you’re doing. 

It also shouldn’t be shocking. Every word (or token) you put into a model changes how it sees the next word. You already know the difference between a good prompt and a poor one. This just takes it further—where the goal isn’t to get stronger answers, it’s to change the model’s entire behavior.

And that brings us to the bigger, more dangerous problem:

Indirect Prompt Injection: The Real Legal Threat

This is where things get scary.

Indirect prompt injection doesn’t come from a user typing in a prompt, it comes from text placed somewhere else, perhaps in documents. The kind you’re uploading into your AI tool. The kind your clients are sending over. The kind sitting in your document management system.

Imagine this:

• You feed 100 contracts into your AI tool.

• One of them contains invisible white text that says:

Except… it is an issue. A few contracts have different terms that affect enforceability.

The AI, following instructions like a good little pattern matcher, now refers to the entire thing using the wrong legal frame.

You never saw the white text.

Your associate didn’t catch it.

The system happily processes it as truth.

And now you have slop in the system. Not because the AI hallucinated but because it was tricked.

The above example is simple. In many modern tools, it will likely fail.

But I share it because the premise is real and proven.

I’ve tested this with my law students and on my own using various prompt injection techniques. And it worked.

That’s with consumer-grade tools. Tomorrow’s threats will be more sophisticated.

So, What Can You Do?

Start asking very specific questions of your vendors and internal AI teams. Like:

• “How do you detect or prevent indirect prompt injection from embedded documents?”

• “Do you sanitize inputs from third-party content?”

• “Do you scan for hidden or invisible text?”

• “How do you handle conflicting instructions from uploaded material?”

If they shrug, laugh it off, or launch into buzzwords, you have a problem. And worse, they may not even realize what you are asking about.

Just remember, you are not being paranoid. You’re being responsible. If you’re trusting an AI to read and summarize privileged documents, the least it can do is not fall for invisible traps.

One bad prompt can poison the well. Don’t let your AI drink from it.

Talk again soon!

To read previous editions, click here.

DISCLAIMER: None of this is legal advice. This newsletter is strictly educational and is not legal advice or a solicitation to buy or sell any assets or to make any legal decisions. Please /be careful and do your own research.