Clarifai Guide
Clarifai Home
v5.10
v5.10
  • Introduction
  • Getting Started
    • Quick Start
    • Applications
      • Create an Application
      • Base Workflows
      • Application Settings
    • Authentication
    • Glossary
  • API Guide
    • API overview
      • API technical references
      • Status codes
    • Data management
      • Supported formats
      • Inputs
      • Patching
      • Pagination
    • Annotate
      • Positive and negative annotations
    • Model
      • Clarifai Models
      • Custom model walkthrough
      • Create, get and update
      • Workflows
    • Predict
      • Images
      • Video
      • Prediction parameters
      • Multilingual classification
    • Search
      • Index images for search
      • Rank
      • Filter
      • Combine or negate
  • Portal Guide
    • Portal overview
    • Data management
    • Annotate
      • Positive and negative annotations
    • Model
      • Clarifai Models
      • Custom model walkthrough
      • Workflows
      • Evaluate
        • Interpreting evaluations
    • Predict
    • Search
      • Rank
      • Filter
      • Combine or negate
  • Product Updates
    • Upcoming API Changes
    • Change log
  • SDK Guide (depreciated)
    • Mobile
    • Android
Powered by GitBook
On this page
  • How It Works
  • Requirements
  • Running Evaluation

Was this helpful?

Edit on GitHub
  1. Portal Guide
  2. Model

Evaluate

PreviousWorkflowsNextInterpreting evaluations

Last updated 5 years ago

Was this helpful?

Now that you've successfully trained the model, you may want to test its performance before using it in the production environment. The Model Evaluation tool allows you to perform a cross validation on a specified model version. Once the evaluation is complete, you’ll be able to view various metrics that will inform the model’s performance.

This evaluate operation is asynchronous and is currently available on the only.

How It Works

Model Evaluation performs a K-split cross validation on data you used to train your custom model.

In the cross validation process, it will: 1. Set aside a random 1/K subset of the training data and designate as a test set, 2. Train a new model with the remaining training data, 3. Pass the test set data through this new model to make predictions, 4. Compare the predictions against the test set’s actual labels, and 5. Repeat steps 1) through 4) across K splits to average out the evaluation results.

Requirements

To run the evaluation on your custom model, it will need the meet the following criteria:

  • A custom trained model model version with:

    1. At least 2 concepts

    2. At least 10 training inputs per concept (At least 50 inputs per concept is recommended)

Running Evaluation

The evaluation may take up to 30 minutes. Once it is complete, the Evaluate button will become View button. Click on the View button to see the evaluation results.

Note that the evaluation may result in an error if the model version doesn’t satisfy the requirements above.

You can run the evaluation on a specific model version of your custom model in the . Go to your Application, click on your model of interest, and select the Versions tab. Simply click on the Evaluate button for the specific model version.

For more information on how to interpret the evaluation results and to improve your model, check out the corner under the “Advanced” section below.

Portal
Evaluation
Portal
cross validation