Skip to content
Permalink
Branch: master
Find file Copy path
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
43 lines (29 sloc) 2.44 KB

Evaluate

Now that you've successfully trained the model, you may want to test its performance before using it in the production environment. The Model Evaluation tool allows you to perform a cross validation on a specified model version. Once the evaluation is complete, you’ll be able to view various metrics that will inform the model’s performance.

This evaluate operation is asynchronous and is currently available on the Portal only.

How It Works

Model Evaluation performs a K-split cross validation on data you used to train your custom model. cross validation In the cross validation process, it will:

  1. Set aside a random 1/K subset of the training data and designate as a test set,
  2. Train a new model with the remaining training data,
  3. Pass the test set data through this new model to make predictions,
  4. Compare the predictions against the test set’s actual labels, and
  5. Repeat steps 1) through 4) across K splits to average out the evaluation results

For a refresher on custom training, please refer to our training documentation.

Requirements

To run the evaluation on your custom model, it will need the meet the following criteria:

  • A custom trained model model version with:
    1. At least 2 concepts
    2. At least 10 training inputs per concept (At least 50 inputs per concept is recommended)

Running Evaluation

You can run the evaluation on a specific model version of your custom model in the Portal. Go to your Application, click on your model of interest, and select the Versions tab. Simply click on the Evaluate button for the specific model version.

model-eval-versions model eval

The evaluation may take up to 30 minutes. Once it is complete, the Evaluate button will become View button. Click on the View button to see the evaluation results.

model eval results

Note that the evaluation may result in an error if the model version doesn’t satisfy the requirements above.

For more information on how to interpret the evaluation results and to improve your model, check out the Evaluation corner under the “Advanced” section below.

You can’t perform that action at this time.