Run your first evaluation

This lesson checks whether recorded answers use the expected fraud labels. It introduces evaluation with a deliberate failure, so you can see what a failed check looks like.

​

Before you begin

Use Prepare data to prepare the fraud example dataset in a separate tutorial project. A trained model is not required. This exercise checks answer format; it does not benchmark a trained fraud model. The platform may use its configured model to explain a low score. Your agent prepares a separate exercise dataset, defines the check and follows the evaluation through completion.

​

Check the result

Valid recorded answers use the labels fraud and not_fraud. An intentionally invalid answer, such as unknown, should fail that vocabulary check. The agent should show the evaluated answers, their scores and any errors or unscored rows. This verifies the answer format. A valid label can still be the wrong answer for a transaction. Measuring accuracy requires model predictions compared with separate reference labels on held-out examples. Keep the deliberately modified exercise separate from training and final assessment. Use a tutorial project because enabled project-wide checks can also participate in live scoring.

​

Continue in the same conversation

Once you have a trained decision model, compare it with Jev on held-out transactions.