Why Most Lawyers Are Terrible At Assessing Risk
One of my biggest pet peeves with lawyers is when I hear the expression, “there’s a risk here.”
In life, there’s always a risk.
There’s a risk to waking up in the morning and getting out of bed. There’s a risk to walking your dog on the street. There’s a risk to NOT getting out of bed in the morning. There’s a risk to eating peanut butter, drinking white wine, bungee jumping, and base-jumping.
But those risks are not all created equally.
It’s not helpful to merely point out that “there’s a risk,” without providing more information.
To be helpful, to actually serve your clients in a way that helps them make an informed decision, you have to quantify the risks in a way that is useful. You have to quantify the risk in a way that an intelligent person can use that information to make an informed decision.
Very few lawyers do that.
Most lawyers simply use the word “risk” as a means to justify endless billable hours that line their pockets at the expense of their clients.
But you could pay your lawyers millions of dollars every month and never eliminate the risks to your business. The biggest and most successful businesses in the world do pay their lawyers millions of dollars every month and they still get sued, every month (I know this because I used to represent many of those very businesses).
What you need, as a business person, is not a lawyer who will tell you that “there’s a risk,” but who will provide you with a useful assessment of the risk that you can then use to move forward with your business.
For example, if I were to say, “If you apply for a trademark under this brand, there is a risk that it will be rejected by the USPTO on the basis of likelihood of confusion with another brand,” it’s hard to know whether it makes sense to proceed with the trademark application.
But if I say “I estimate there’s a 20% chance that your trademark will be rejected,” then you have a much better sense of whether it makes sense to proceed or not. Similarly, if I say, “I estimate that there’s an 80% chance that your trademark will be rejected,” then you have a very different risk-assessment ahead of you.
Most lawyers are simply allergic to that kind of basic math. But in my opinion, by failing to provide some attempt at precision in your risk assessment, most lawyers do a horrible disservice to their clients that prevent them from making informed decisions.
There’s a book called Superforecasting, written by Philip Tetlock and Dan Gardner, that talks about the differences between those who do well at predictions and those who do not. And the characteristics that distinguish those who do well compared to those who do not is not raw IQ or educational achievement, but rather the simple hygiene of honesty and self-assessment.
Superforecasters, or anyone who does well with risk assessment, make specific, granular predictions. They update their predictions when they learn new facts. And then they look back at their predictions to assess how well calibrated they were.
This concept of “calibration” is essential to the business of predictions. When someone making a prediction provides an estimate (there’s an 80% chance that your trademark will be rejected), that estimate is, in a sense, coming out of thin air. It’s informed by experience and judgment, but in isolation, it’s hard to know whether that prediction was accurate or not. But if you predict something with 80% likelihood, and you have five such predictions, on average four such events should happen and one should not. If you look back at your predictions and your original estimates come out as described in the prior example, then you know that your estimates were well calibrated and that you made good predictions.
This is what superforecasters do. This is what good lawyers should do (but very, very few actually do). This is what people who care about the truth should do.
So whenever you hear a lawyer (or anyone else) tell you, there’s a “reasonable probability that there could be a problem here” or “there could be an issue here,” do yourself and the lawyer a favor by demanding more precision. Does “reasonable probability” mean a 5% chance or a 75% chance? Without some degree of specificity, a mere acknowledgement of the existence of a risk is the opposite of useful information.
To make good decisions, and to provide a useful service where you help others make better decisions, you need to do some math.
 As a total tangent, the best example I can think of when it comes to bad calibration is Spock from the original Star Trek. Whenever Kirk would fly the Enterprise into an asteroid belt or attack 17 Kling-on ships at the same time, Spock would say something like, “The odds of surviving such an encounter is approximately 1 in 3,092,765.” But yet, the Enterprise engaged in these hijinks in nearly every episode and survived every time. Spock’s estimates were therefore horribly calibrated. Spock, for all his apparent math-y-ness and rationality, was unequivocally awful at statistics.
 Of course, some people are very bad about confusing low probability with no possibility. If you hear that a negative result has only a 15% chance of happening, then that may sound like a sure thing. But it basically means that you’re a bad roll of the dice away from a judge ruling against you. And if you have many clients, like we do, for your predictions to be well calibrated, those 15% bad beats must happen about one out of every six times, or else your predictions are imprecisely pessimistic in a systematic way. That doesn’t make it any easier when you’re the one who takes the bad beat, but that’s the way the math works.