# Auto Annotation

This tutorial demonstrates how auto-annotation workflows can be configured within Clarifai Portal. With auto-annotation, you can use model predictions to label your inputs. Auto-annotation can help you to prepare training data, or assign other useful labels and metadata to your inputs.

When a concept is predicted by a model, it is predicted with a confidence score between 0 and 1. When your model predictions are confident (close to 1), you can have your data automatically labeled with that concept. When your predictions are less-than-confident, you can have your input sent to a human being for review.

This enables you to speed-up and scale-up your annotation process while ensuring quality standards.

![](https://465453543-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LisemUaXxC3S7Kk9xFz%2Fsync%2F03fbaaa144a9c2b262dfa20806dfc909033aff52.jpg?generation=1593527574463319\&alt=media)

## Create Concepts, Label and Train a Custom Model

To begin, create your application and upload your inputs.

![](https://465453543-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LisemUaXxC3S7Kk9xFz%2Fsync%2F6b97f6ed8cd1f979b217deabae1ef6507ac0d805.jpg?generation=1593527573413046\&alt=media)

![](https://465453543-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LisemUaXxC3S7Kk9xFz%2Fsync%2Fe41fbc70634d57922f14afbe5d671653c37de5d5.jpg?generation=1593527574649877\&alt=media)

Upload files from your computer, or add image and video URLs.

![](https://465453543-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LisemUaXxC3S7Kk9xFz%2Fsync%2F010ba418153a3f6885f907413ee529edfd70a5ae.jpg?generation=1593527573605187\&alt=media)

Create the concepts that you would like to use for your model. Click create new concept in the lefthand sidebar. In this tutorial we'll create concepts that describe people based on activities they do in the park: `walker`, `runner`, `bike_rider`, `rollerblader` and `stroller`.

![](https://465453543-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LisemUaXxC3S7Kk9xFz%2Fsync%2F4235c01561c5bdd646ecaabd1a687541cbc28ed2.jpg?generation=1593527573480169\&alt=media)

Label your inputs with the appropriate concepts by clicking the checkmark box.

![](https://465453543-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LisemUaXxC3S7Kk9xFz%2Fsync%2Ff45fb43de293cfb196a2011d452d036e1e899bd5.jpg?generation=1593527573815700\&alt=media)

## Navigate to the Model Gallery

Now it is time to create some custom models.

![](https://465453543-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LisemUaXxC3S7Kk9xFz%2Fsync%2F8b1e4ec90575c730cfa3f042fc328dee76d16e3d.jpg?generation=1592216039385375\&alt=media)

## Create a Context-Based Classifier

A Context-Based Classifier lets you create a custom classification model that will predict the concepts you have created. Choose a `DISPLAY NAME` and click in the `OUTPUT_INFO.DATA.CONCEPTS` box to select the concepts that you would like included in your model. Finally, click "Create Model" and then click "Train Model" in the next page.

![](https://465453543-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LisemUaXxC3S7Kk9xFz%2Fsync%2F958264607492870e565938d193921e32cc243837.jpg?generation=1593527573849100\&alt=media)

![](https://465453543-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LisemUaXxC3S7Kk9xFz%2Fsync%2Fe279eec1ca84ad10e71701f1f2fac15c59e73482.jpg?generation=1593527572987100\&alt=media)

## Create `GREATER THAN` and `LESS THAN` Concept Thresholder models

Concept Thresholders help you route your data based on the confidence of your predictions. Choose a `DISPLAY NAME` and add the concepts you would like to route under the `OUTPUT_INFO.DATA` heading. You can choose separate concept thresholds for each of your custom concepts. Create one model with high `CONCEPT THRESHOLDS` and select `GREATER_THAN` under `OUTPUT_INFO.PARAMS.CONCEPT_THRESHOLD_TYPE`. Create a second Concept Thresholder model with low `CONCEPT THRESHOLDS` and select `LESS_THAN` under `OUTPUT_INFO.PARAMS.CONCEPT_THRESHOLD_TYPE`. The `LESS_THAN` model will help you screen out concepts that are predicted with very low confidence.

![](https://465453543-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LisemUaXxC3S7Kk9xFz%2Fsync%2F6b0699ac868da0343c51841aaf23c3718269ed84.jpg?generation=1593527574034060\&alt=media)

## Create a `WRITE SUCCESS` and `WRITE PENDING` Annotation Writer models

The Annotation Writer writes each annotation with a specific concept, status and user. Create one Annotation Writer that will write the annotation with the `ANNOTATION_SUCCESS` status and choose the user (possibly yourself) under `OUTPUT_INFO.PARAMS.ANNOTATION_USER_ID`. Create a second Annotation Writer that will write with the `ANNOTATION_PENDING` status.

![](https://465453543-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LisemUaXxC3S7Kk9xFz%2Fsync%2Fa69d2c0d8afdd2341935845a5defef9f32b27cec.jpg?generation=1593527573885027\&alt=media)

## Create the workflow

We will now join all the models together into a single workflow. In the Clarifai platform, the outputs from one model can be used as inputs to another model. Different models accept and produce different types of inputs and outputs.

The Context-Based Classifier will return list of concepts for each of your inputs. These inputs will then be sent to the Concept Thresholder models. `GREATER THAN` model will filter out the concept if it lower than the threshold and send the final concept list to `WRITE SUCCESS` model. Concepts will then be written to the input with `SUCCESS` status. You can train or search on these concepts immediately.

The `LESS THAN` model will filter out the concepts with low prediction scores. The remaining concept list will be sent to `WRITE PENDING`. Concepts will then be written to the input with `PENDING` status.

![](https://465453543-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LisemUaXxC3S7Kk9xFz%2Fsync%2F2ad14882557607ee7b9827a3ef2f8f5888b8c9e4.jpg?generation=1593527573033893\&alt=media)

![](https://465453543-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LisemUaXxC3S7Kk9xFz%2Fsync%2F8d2e145297bdd2ba9a1221b69c00a5f60624f9c8.jpg?generation=1593527574732379\&alt=media)

## Make the new workflow your app's default

Make this the default workflow in the app, so it will run every time we add an input and execute the auto annotation process. If the workflow is not the default workflow of your app you can still use PostWorkflowResults on new inputs to check that you configured the workflow graph and your models properly but the data will not be written to the DB. This is recommended before making it your default workflow and adding inputs to you app.

![](https://465453543-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LisemUaXxC3S7Kk9xFz%2Fsync%2F816caab84cf31a2df875fd37a98fd136dff32738.jpg?generation=1593527574357810\&alt=media)

## Add new images

Adding new images will trigger the default workflow in your app.

![](https://465453543-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LisemUaXxC3S7Kk9xFz%2Fsync%2Feb4cfa1b0634b7a385ce3b4462e5767eedb2390f.jpg?generation=1593527574151092\&alt=media)

## View Annotations in Explorer

Now you can view your annotations in Explorer. Just drag your mouse over the "tag" icon to see annotations associated with the image.

![](https://465453543-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LisemUaXxC3S7Kk9xFz%2Fsync%2F02238dad27f1d2044ec9ecec831c310f5acec79b.jpg?generation=1593527574615001\&alt=media)