Skip to main content
Train a decision model to return probabilities for a defined set of outcomes, then compare it with a baseline on held-out data. Your coding agent handles preparation, training and benchmarking from one request. This walkthrough uses the ecommerce fraud dataset: 10,000 synthetic transactions with a supplied fraud label. Fraud detection is the worked example; the workflow also applies to other classification and routing tasks when you supply suitable data and target definitions. For generated text responses or tool calls, follow How to fine-tune a language model.

​
Before you begin

Connect your coding agent and select a project for the tutorial. You do not need an application repository. This workflow includes training and benchmark inference, which can incur provider costs. Your agent explains the proposed model and expected cost before launch. The agent works out the preparation and carries the dataset, model and benchmark context through the workflow. You do not need to issue a new prompt for each stage.

​
What happens next

  1. Prepare the data. The agent inspects the source, prepares the transaction inputs and fraud labels, and separates training, development and held-out examples. It retains the preparation in Overmind so the result can be inspected and reproduced.
  2. Train a model. It selects a compatible decision model, checks the training setup and expected cost, then follows the accepted run through validation.
  3. Benchmark the result. It compares the trained model with a suitable baseline on the same held-out examples and explains the measured results.
Data checks are part of the workflow. Preparation is not a separate stopping point; the agent continues toward the requested benchmark and brings back decisions that need your input. Follow progress in the same conversation or inspect the saved datasets and runs in the Console.

​
Check the result

At the end, the agent should show:
  • The trained model and its completed training run, including the recorded validation results.
  • The benchmark participants, held-out data and number of examples scored by each model.
  • How often the models detect fraud, miss fraud or flag legitimate transactions, with representative mistakes.
  • The comparison with the baseline, including any missing results or limitations.
The original source has 9,580 non-fraud and 420 fraud transactions, so overall accuracy alone can hide poor fraud detection. The supplied is_fraud label is the answer to predict, not part of the model input. The agent handles these checks during preparation; dataset details explain the source and its limitations. These are synthetic examples. Improvement on this benchmark does not establish performance on real transactions, and a trained model may not outperform its baseline.

​
Continue in the same conversation

Use Compare decision models for another comparison, Monitor training to inspect a run, or Serve a model to try the trained result in an application.