How to measure whether an AI pilot changed the process

    Pilot dashboards usually show activity, such as how many times an agent ran. Whether the process itself became faster, cheaper or shorter can only be seen against a baseline: how the process ran before the pilot.

    What should the baseline include?

    The steps, how long each one takes, who owns them, what triggers them and where work tends to wait. Recorded before the pilot, it gives you something to compare with a few months later.

    What can the comparison show?

    Did customer feedback reach engineering faster, or did the agent produce 1,200 summaries that people still re-read and rewrite? Did vendor onboarding go from three weeks to one, or did the delay move from procurement to legal review?

    A pilot can go well and still miss the numbers that matter. It may speed up a step that was not the bottleneck, move work to another team, or push the slow part somewhere nobody measures.

    What else does the baseline show?

    Processes that have no agents yet. They appear as long manual stretches between automated steps, often because no one owned them clearly enough to start an experiment, and they may be the next place to look.

    ← All articles