When human-in-the-loop becomes human-in-the-way

Adding a human review step to a new automation is sensible caution, especially early on, while trust in the system's accuracy is still being established. The problem shows up later, when the review step that made sense at launch never gets revisited, and the automation ends up no faster than the manual process it replaced, because every output still waits on a person to approve it.

This happens quietly because nobody explicitly decided to keep the bottleneck, it's just that removing a review step feels riskier than leaving it, even once months of data show the system's accuracy no longer justifies universal review. Caution has no natural expiry date unless someone sets one.

The fix is deciding upfront what would justify removing or reducing the review step, and actually checking against that threshold on a schedule: if the system's output has been accurate above some agreed rate for some agreed period, review moves from every output to a sampled percentage, then eventually to spot-checks, rather than staying at full manual review by default.

Human-in-the-loop is the right starting point for almost any consequential automation. It's not supposed to be the permanent architecture unless the accuracy genuinely never improves, and most teams never come back to check whether it has.