Getting useful work out of AI takes practice. These three mental models explain why the first attempt can disappoint, and how clearer instructions, better goals, and more context can improve the next one.
Have you ever used AI for something and thought, that was fine, but I could have done it faster myself?
This is where a lot of us get stuck, and I hear a version of this from lawyers all the time.
This fall, I joined the faculty at the University of Texas School of Law to run their AI training program for students. In that course, I spend less time on which tool to use than on how to think about working with AI. The tools will continue to change, and these mental models will help you effectively use whatever comes next.
Suppose you put in 100% of the effort and get 98% back. That’s pretty close to good. But it’s still useless. You’ve spent more time than you saved. Even a two-percent loss isn’t something you’re going to come back to, and it’s rational not to.
Joshua Wohle, co-founder of Mindstone, explains this well in his interview on the Beyond the Prompt podcast. He calls it the utility threshold: the point where working with AI gives you enough value to justify the effort.
The good news is that there are two ways across. The models can improve, and this is happening all the time. This means that use cases that didn’t work six months ago might work today.
But second, you can get better at using AI. Clearer instructions, more defined goals, and better context can move your existing use cases beyond the utility threshold so that with 100% effort, you are getting 105% or 200% or 300% back. You don’t have to wait for the next model to make something useful out of the one you already have.
All of us have a mental model for how software works. Most software is deterministic, which means that it works the same way every time. If a program doesn’t work the way it always has, it’s broken. You hit Control-S and it saves your document. You hit Control-P and it prints your document. If those commands stop doing what they’re supposed to do, you reasonably conclude the software is broken. And you are reasonably frustrated.
AI isn’t deterministic; it is probabilistic. That difference takes some getting used to. Ask the same question twice and you will get different answers or different framing. When it’s wrong, AI isn’t going to tell you. If it doesn’t know, AI may guess without telling you it’s guessing.
But there’s another habit we need to change: what we do when the answer disappoints us or when we discover that it was wrong?
With AI, we need to develop the resilience to try a different way. Before deciding AI can’t do something, ask yourself: is there something about my prompt or context that caused it to go off the rails? Did I explain what I wanted? Did I give AI what it needed to do a good job?
In my experience, AI can usually do what I’m trying to get it to do. I have had to learn a new mindset to be curious about what went wrong and resilient enough to stick with it and try a different way. But I try to start with the assumption that AI can do what I need it to do; I just don’t know how to get it out of the model yet.
Ethan Mollick offers a useful analogy in Working with AI: Two paths to prompting:
You can (inaccurately but usefully) imagine the AIs knowledge as huge cloud. In one corner of that cloud the AI answers only in Shakespearean sonnets, in another it answers as a mortgage broker, in a third it draws mostly on mathematical formulas from high school textbooks. By default, the AI gives you answers from the center of the cloud, the most likely answers to the question for the average person. You can, by providing context, push the AI to a more interesting corner of its knowledge, resulting in you getting more unique answers that might better fit your questions.
Think of that cloud as holding many possible kinds of response. Without much direction, you tend to get something generic and average. Your prompt and context help steer the model toward a more interesting part of what it learned during training, where the response may fit your task better.
“Draft a client update” leaves a lot for AI to guess. Who is the client? What do they already know? What happened? What decision do they need to make? If you don’t provide an answer, what the AI will give you is the most likely response — an average of all of the possible responses to your question. A response this generic is unlikely to be what you are actually looking for as an attorney.
But answering those questions and providing an example of an update you’d be happy to send pushes the model to a more interesting part of its training data and is more likely to give you the response that you’re looking for. Or to say it another way, a disappointing first answer doesn’t tell you everything about what the model can do.
Pick something you tried with AI that wasn’t worth the effort. Tell the model what a good result would look like. Give it the background, constraints, and examples it was missing. Then see whether the result is more useful.
That’s how these three ideas fit together. Your first attempt may fall below the utility threshold. Trying again takes a different kind of patience than ordinary software asks of you. Learning how to prompt and how to provide the right context is a powerful way to improve the result.
I write about this every week in The Bell Rock Brief. If you’d like the next one in your inbox, sign up in the footer below.