Skip to main content

Lift

Lift is the last stage and the only one that reports on a change already in production.

Who can use it: anyone who can open Studio.

Only after promotion​

The page has nothing to show until the round is promoted to all traffic. Before that it says so and quotes the round's current status, rather than showing zeros.

What it reports​

The Lift stage with a headline saying how many points better the agent is, four metric cards for faithfulness, groundedness, helpfulness and p50 latency each with before, after and change, and a faithfulness over rounds chart whose markers distinguish promoted, rolled back and not promoted.The Lift stage with a headline saying how many points better the agent is, four metric cards for faithfulness, groundedness, helpfulness and p50 latency each with before, after and change, and a faithfulness over rounds chart whose markers distinguish promoted, rolled back and not promoted.
What the promoted round gained, and the trend across rounds. The chart plots the rounds that were rolled back too, so the line is not a success story by construction.

A headline saying how many points better the agent is, then four measures against the previous round: faithfulness, groundedness, helpfulness and p50 latency, each with its before, its after and the change.

These are the evaluation figures the comparison stage used, carried forward. The agent is live on the candidate, so they describe what shipped.

One sentence at the end says the next loop has already started and that yesterday's failures are being clustered. Nothing checks whether that is true. Whether a new round exists is visible on the workspace, and whether clustering ran depends on someone invoking the job.

The trend​

Faithfulness over rounds plots every round that has a measured lift, whatever became of it, with the markers keyed Promoted, Rolled back and Not promoted.

Including the rounds that failed is deliberate. An earlier version plotted only promoted rounds, which made every trend an unbroken success by construction.

The series is assembled in your browser from the agent's round list; there is no history endpoint behind it. Two things follow. Fewer than two measured rounds and the chart says there is not enough to draw a trend. A current round with no measured lift yet is named as absent from the chart rather than left to be noticed.

Reading it honestly​

Lift is measured on the evaluation suite, not on production traffic. It says the candidate answered the suite better than the previous version did. Whether users are better served is a question for the agent's own analytics and the next round's failure clusters.

Describes the platform as of .

Was this page helpful?