How to tell whether a crew really uses a tool

The dashboard that looked fine

A general manager opens the new system's dashboard on a Friday. Every job from the week is in there. Photos are attached. Notes are filled in. The rollout looks like a success.

Then he walks past the office manager's desk and sees a stack of paper tickets, a phone full of texted photos and a second monitor with the new system open. She has been entering the crew's work herself, every morning, from whatever they sent her. The system is full. The crew has never touched it.

This is the most common way a rollout fails without anyone noticing. The data arrives, so leadership believes the tool is used. What is really happening is that one person absorbed the cost of the tool so the crew did not have to. When she goes on vacation, the system goes quiet.

If you want to know whether a crew really uses a tool, you have to measure something harder to fake than a full database.

What does not count, and where the count starts

Start by throwing out the easy numbers.

Logins measure access. A technician can log in, tap around and go back to the clipboard. Training attendance measures whether people sat in a room. It says nothing about the next Tuesday. App installs measure whether someone put an icon on a phone. And "it looks good" from a supervisor measures politeness.

None of these are useless. They tell you whether the basic setup worked. But none of them tell you whether the work moved into the tool. We do not accept any of them as evidence of use, in our own contracts or anyone else's.

Real measurement starts with a question the tool cannot answer: how many jobs should have gone through it?

That number is the denominator, and it has to come from somewhere other than the tool. Your job list, your dispatch board, your work orders or your invoices already record how much work happened. From those, you decide which units are eligible. Maybe it is every completed service call. Maybe it is every job over a certain size, or every inspection of one type. Write the rule down so anyone could apply it.

The rule should settle the awkward cases before they come up. Does a job cancelled on site count? A warranty callback? A job split across two days? A job done by a subcontractor who will never touch your tool? There is no single right answer. What matters is that the rule is written before you count, so nobody adjusts it after seeing the result.

Then count how many of those eligible units were completed in the tool. That is the numerator. The share is your adoption rate.

This sounds obvious, and it is rarely done. Most adoption reports count only what entered the tool, which means they cannot see the jobs that never did. A tool that captured every job it saw can still be missing a large share of the work. The only way to see what is missing is to count from the outside.

Who, when and how complete

Once you have the count, check three things about each entry.

Who made it. The tool should record who created each entry. If the person doing the work made it, that is use. If the office made it the next morning, that is the workaround from the opening scene. Both fill the database. Only one means the crew adopted anything.

When they made it. An entry made at the job, or before the end of the shift, is part of the work. An entry made three days later from memory is a report about the work, and it will be missing details. For most field workflows we write timing into the definition: the job counts only if it was completed in the tool before the end of the shift.

How complete it was. An entry with the required photos, measurements and sign-off is useful. An entry with only a job number is a checkbox. Decide which fields must be filled for the entry to count, and check them. Then look at what the office does after the entry lands. If someone still corrects or retypes it, you are in the re-entry trap with an extra step.

Watch for the shadow system

Numbers get you most of the way. The rest comes from walking around.

Look for paper that should not exist anymore. Ticket books in the trucks. A whiteboard in the shop that still carries the real schedule. A group text where the crew posts job photos. A spreadsheet on someone's desktop that "just tracks a few things." Each is a shadow system, and each tells you what the tool is failing to do.

Shadow systems are not a discipline problem. They are information. The crew keeps them because they do something the tool does not: they work offline, they take fewer taps, they handle the odd job. When you find one, ask the person using it what it gives them. That answer is usually the next thing to fix.

Talk to the crew leads too, away from the office. Ask them what they would take out of the tool if they could. Ask what they do when it does not work. Ask whether anyone has told them what happens to the information after they enter it. A crew that never sees its entries used for anything soon stops caring whether they are complete, and no dashboard will show you that.

Measure late, not early

The first two weeks after launch are the worst time to judge adoption. Everyone is watching. The vendor is on site. Supervisors are reminding people. Use looks high because attention is high.

The honest measurement comes after the attention fades. That is why our acceptance test uses the final ten business days of a 30-day live-use window. At least 80 percent of eligible units must complete the agreed workflow in the tool in that period, counted from the client's own operating records, with quality checks passing and no critical defect open. If a tool is still used when nobody is watching, it is used.

You can apply the same idea to any tool your company already runs. Pick a normal week well after launch. Count eligible jobs from your own records. Count how many were completed in the tool, by the person doing the work, on time and complete. The gap between those two numbers is the real state of adoption.

Then look at the misses, not just the total. Sort the jobs that did not go through the tool by person, by crew, by job type and by day of the week. The pattern usually jumps out. One crew never uses it. Nobody uses it on jobs that run late. It works on service calls and gets skipped on installs. Each of those points at a specific cause: a crew lead who was never brought in, a tool that takes too long at the end of a long day, a screen that does not fit a certain kind of job. A total tells you whether there is a problem. The misses tell you what it is.

What to do next

If the count comes back low, do not start with more training or a mandate. Start with why the tool is harder than the old way. The guide on crews not using a new system walks through how to find out. If you want to see how we write this measurement into a contract before a build starts, read How we work.