Asking an operator well

Below the threshold, Sixth Sense does not guess. It asks. How it asks decides whether the operator is a safety net or a bottleneck.

What the question contains

A good question fits on one screen and needs one tap.

• The image, with the item outlined, so the operator sees what the camera saw.
• The reason: which fact was uncertain. “Label partly covered” is useful. “Low confidence” is not.
• The allowed actions, as buttons. The same list the rule engine produced. Nothing the operator could not have chosen from the line.
• The robot’s own preference, marked, with its confidence.

The operator taps one action. That is the whole interaction. There is no free text, no menu tree and nothing to type with gloves on.

Where the question goes

The question goes to whoever is responsible for the cell at that moment, on a tablet they already carry. It does not require walking to a panel. If nobody answers within a set time, the fallback is a rule: hold the item, or stop that lane, but never act on the uncertain choice.

Keeping the question rare

A question that arrives every thirty seconds stops being read. Operators start tapping the first button, and the safety net becomes a formality. Three things keep the question rare and meaningful.

A calibrated threshold. If the confidence score is honest, a threshold of 90 asks about one item in ten at most, and usually far fewer, because most items sit well above it.

Rules for known cases. If the same question comes back every day for the same reason, the answer should become a rule. The console flags repeated questions for exactly this purpose.

A floor that blocks instead of asking. Very low confidence usually means something unusual is on the belt. Those cases are blocked and escalated, not queued as questions.

Measuring it

The log records how long each question waited and what the operator chose. Two numbers are worth watching. The time to answer tells you whether the tablet is with the right person. The agreement rate, how often the operator picks the robot’s preferred action, tells you whether the threshold is set where it should be. A high agreement rate at the current threshold suggests it can be lowered for that task. A low one says the model needs recalibration on those items, or that a rule needs a look.

The answer is data

Every operator answer is a labelled example: this item, these facts, this decision by a person who knows the process. Those examples are what calibration is checked against, and what any later adjustment of the model is measured on. The question is a cost during the shift and an asset after it.