Ticket-triage exercise
Prepare labeled support threads, run a supported classification workflow, inspect ambiguous results and measure errors before delivering an external queue.
Measure a classifier against a small labeled set of support threads you provide. Include thread IDs, text, expected topics and ambiguous cases. Keep the expected labels separate so you can compare predictions with a known answer.
- Upload structured rows or connect a source containing the threads.
- Define topic boundaries in a reusable prompt or skill. Choose an installed executable classification transform, or a categorical forecaster backed by a published evidence query. A requested capability must exist before it can run.
- Author the pipeline or forecaster and inspect its input/output schema. Validate configuration, save, and execute a bounded set.
- Compare labels against your expected labels and inspect evidence for errors. Confidence should not be treated as a guarantee of correctness.
- If delivering a queue externally, configure a writable endpoint and enforced policy/permissions, then verify the receiving system after execution.
- Record appropriate artifact/assertion review or prediction evaluations. Update instructions or data deliberately and measure a separate rerun.
Success means measured classification behavior and verified delivery. Human correction does not automatically retrain the classifier.
See Forecasts and Deliver results.
Renewal-risk exercise
Use your own account history to build a probability forecaster, preserve evidence cutoffs and evaluate renewal outcomes without leaking future information.
Quick: pipeline from Postgres to S3
Configure Postgres and S3 endpoints, review a bounded extract pipeline, save and run it, verify the output object and schedule recurring execution.