# Anomaly Detector

The Anomaly Detector tool recognizes defects, debris, or other faults. Use the Anomaly Detector tool to identify unknown or unanticipated defects on your inspected parts.

The Anomaly Detector tool can perform tasks such as:

- Detecting anomalies and aesthetic defects by learning the normal appearance of an object

- Recognizing scratches on complex surfaces

- Identifying incomplete or improper assemblies

The Anomaly Detector tool requires initial training on a sample image set, with manual classification of views as OK or NG (Not Good). The tool learns the appearance of good parts and bases NG predictions on deviations from that OK state. This means that you can train the Anomaly Detector with only OK views, although for the best results Cognex recommends that your sample set includes NG views as well. NG views are useful when testing and validating the performance of the tool and they influence how the tool calculates the anomaly score threshold. Labeled views of defective parts also help you to determine how accurately the tool finds defects.

To set up the Anomaly Detector tool:

1. ROI The Region of Interest (ROI) is the area of the image where the vision tool operates.: Set the ROI to create views from your images.

2. Label: Select views and label good ones as OK and views with anomalies as NG.

3. Train: Set the Feature Size parameter. Assign roles manually or automatically to views to determine how the views are processed.

4. Process: Click Process All.

   When processing finishes, the results panel shows detailed statistics about the results of the tool.

5. Review the Metrics and Statistics and refine labeling Labeling is the process of marking features or defects in images, or categorizing the images into classes. Labeling is crucial for training Deep Learning Lite and Deep Learning Standard tools because the labels serve as training material to guide the tools how to function correctly. and training as needed. Set anomaly score limits in Processing Options to adjust the threshold of what the tool classifies as OK and NG.

   Setting different upper and lower anomaly score threshold limit numbers can result in a third, Unclassified class.

## Using the Anomaly Detector with Other Tools in the Toolchain

You can use the Anomaly Detector tool flexibly as an upstream An upstream tool comes first in the toolchain and its output defines a downstream tool. An upstream tool can have several downstream tools. or downstream A downstream tool requires a previously configured upstream tool and its configuration as input. The tool is placed after its upstream tool in the toolchain. part of a toolchain Contains a group of tools and the sequence of execution for each tool.. For example:

- The Anomaly Detector tool can pass NG images to a Segmenter tool for further inspection.

- The Anomaly Detector tool can pass NG images to a Classifier to classify the types of anomalies detected.

- The Anomaly Detector tool can take images from a Locator tool to identify defects on specific parts.

## Workflow Steps

The following sections describe the configuration of the tool when you first go through the steps.

How to set the ROI

1. Change the ROIThe Region of Interest (ROI) is the area of the image where the vision tool operates. by moving, resizing, and turning the blue rectangle on the image. Place the blue rectangle on the part of the image you want to check.

   - To move the ROI, click and drag anywhere inside the blue rectangle.

   - To resize the ROI, click and drag any of the sides or corners of the blue rectangle.

   - To rotate the ROI, click and drag the rotate handle on one side of the blue rectangle.

   - Alternatively, you can adjust the position along the X and Y axes, the width, the height, and the rotation of the ROI with numerical values in the ROI Options section.

2. To change the point from where the application calculates the location of the ROI, select Origin or Center.

3. If the tool is in a tool chain after another tool that produces blobs, you can apply a Mask From Blob to narrow down the inspection area. For more information, see Masking.

4. Click Apply ROI when you are done to set the ROI for all the images you uploaded.

How to label with the Anomaly Detector tool

The Anomaly Detector has two preset classes: OK and NG. Label views without defects or debris as OK and views with any anomalies as NG.

To label a view as a class:

1. Select the view in the view panel.

2. Click the corresponding label in the LabelingLabeling is the process of marking features or defects in images, or categorizing the images into classes. Labeling is crucial for training Deep Learning Lite and Deep Learning Standard tools because the labels serve as training material to guide the tools how to function correctly. Options.

You can label several images at the same time by selecting them.

Labeled views have a colored graphical mark in the upper left corner that shows the assigned class.

To remove an assigned label from a view, select the view then click Remove Label.

Note: If you already have a trained Anomaly Detector tool, use the bootstrap labelingBootstrap labeling is a fast labeling method, which involves labeling a small sample of images, training the tool, and reviewing the results. Accept views with correct markings to convert them into labels, and replace incorrect markings with correct labels. method to speed up the labeling process. For more information, see Bootstrap Labeling.

You can train the Anomaly Detector tool even if you only label OK images as the tool learns the appearance of the good parts and it recognizes bad parts based on how much they deviate from that. However, for the best results, also include images in the training image set that show defects and you can label as NG.

How to train the Anomaly Detector tool

After the Label step, proceed to the Train step to train the model.

1. Only one Training Mode is available for the Anomaly Detector:

   Deep Learning Lite - Few Sample (Default): Optimized for standardized objects. Requires only a few images to train.

2. Adjust the training parameters as needed:

   - Resize Parameters

     Feature Size specifies the typical feature diameter in pixels. Adjust this setting to achieve a more accurate result by ignoring smaller image features than the specified Feature Size. The Feature Size approximates the size of a typical feature that is needed for a good classification. For example, If the Feature Size is set to 16 pixels, any image feature smaller than 16 pixels is ignored for anomaly detection.

     Feature Size strongly influences processing time. A Feature Size of 100 is 100 times faster than a size of 10, while a Feature Size less than 8 usually does not yield good results. When setting the Feature Size, consider the equation in regards to processing time (Ptime):

     Ptime ~ F2

3. To auto-assign roles to images, click Auto-Assign Roles. A pop-up window provides you with options for how to assign roles:

   See the Label section for more information.

## Verifying and Refining

The following sections describe the information you see after completing the workflow steps and how to verify results and refine the tool.

How to view the results after the Process step

After clicking Process All in the Process step, the tool provides the following in the results panel:

- Detailed statistics about the performance of the tool in the panel on the right side of the window.

- Graphical markings in the upper left corner of the views:

  Number | Name | Description
  1 | Role | The role that you either manually assigned or auto-assigned to the view. For information on role assignments, see Train.
  2 | Assigned Label | The label you set for the view in the label step.
  3 | Prediction Label and Anomaly Score | The label prediction after the process step is complete. The prediction anomaly score ranges from 0 to 100. A lower value means a less prominent anomaly and a higher score means that the anomaly is more prominent. A 0 score means that the tool considers the view to have no anomalies at all.
  4 | Heat Map | The visual indication of where on the view the tool found anomalies. A darker red overlay corresponds to a higher anomaly score for the area. You can check or uncheck Predictions in the full view of the currently selected image, which also enables or disables the Heat Map overlay. With Heat Map Without Heat Map

- You can verify whether the predictions are accurate and, therefore, how well-trained the tool is.

How to interpret Metrics and Statistics

Review the statistics and metrics that OneVision returns in the right-side panel:

- If the performance of the tool is optimal based on the returned statistics, you are ready with the training.

- If the performance is not satisfying, you can adjust your labelingLabeling is the process of marking features or defects in images, or categorizing the images into classes. Labeling is crucial for training Deep Learning Lite and Deep Learning Standard tools because the labels serve as training material to guide the tools how to function correctly., refine the parameter settings, or provide more labeled data and retrain the tool. For more instructions, see Example of verifying and refining with an initially small train set.

Statistics

Information about processing time and the confidence score of the predicted classes.

Statistics help identify anomalies, for example, if any images fall outside the expected count range, selecting the given bar shows the incorrectly processed images and you can manually correct them.

Metrics

Section | Description
Confusion Matrix | The confusion matrix is a visual representation of the labels matched against the predictions of the tool. The horizontal dimension shows the predictions of the tool. The vertical dimension shows the actual labels you assigned. If the predictions match the labels, the results are good. A darker red color means a higher ratio and a lighter color means a lower ratio. A diagonal line of dark cells in the confusion matrix shows that performance is good. The color of a cell represents the ratio of the predicted views (the value of a cell) to the total number of labeled views (the total value of the row). Click the fields in the Confusion Matrix to filter for the given category of views.
Results Statistics | The table lists the classes and information about each one in the corresponding columns.
Column | Description
Predicted | Shows the total number of views the tool classified as a certain class.
Labeled | Shows the number of the views labeled as a certain class.
Trained | Shows the number of views labeled as a certain class and included in the training set.
Recall | Recall is the percentage of the labeled classes that the tool correctly identifies. OneVision calculates Recall by the fraction of classified views labeled as class i that are correctly classified as class i: For example, a Recall of 90% for the class i means that the neural network modelEach AI tool is a neural network model. A neural network model mimics the way biological neurons work in the human brain. The neural network model consists of interconnected layers of artifical neurons, called nodes, and they have multiple layers. Neural network models excel at tasks like image classification and pattern recognition. correctly classifies a view of class i 9 out of 10 times.
Precision | Precision is the percentage of detected classes that match the labeled class. OneVision calculates Precision as the percentage of correctly classified views predicted as class i: For example, a Precision of 90% for the class i means that the neural network model confuses an view of class i with a different class 1 out of 10 times.
F-Score | The F-Score represents the overall accuracy of the tool. The F-Score is the harmonic mean of the Recall and Precision scores, which provides a single metric that balances both Precision and Recall. The value of the F-score ranges from 0 to 1, with 1 representing the highest accuracy.

Selected View

Detailed information about the selected view only, including image data, the assigned label, and a breakdown of the predicted classes.

Note: Unclassified is only visible as a class on Metrics and Statistics if you have set different anomaly score threshold upper and lower limit numbers in the Process step. For more information on the threshold, see How to set and optimize the threshold.

For more information on interpreting results, see Metrics and Statistics.

How to set and optimize the threshold

The Anomaly Detector tool automatically sets an anomaly score threshold when processing views. To check where the initial processing set the anomaly threshold, filter your view panel by score, either ascending or descending order.

You can adjust this threshold by setting upper or lower limits in the Processing Options. Setting different numbers for the two limits can result in a third, Unclassified result class. Both numbers must be a value between 0 and 100.

- A view is NG if the anomaly score is higher than the upper threshold limit.

- A view is OK if the anomaly score is lower than the lower threshold limit.

- A view is Unclassified if the anomaly score is in-between the two limits.

Setting only one of the limits results in the anomaly score threshold being that value. In the example below, views with an anomaly score lower than 56 are OK and views with an anomaly score of 56 or higher are NG. In this case of only setting one limit, there are no Unclassified view.

In the example below, the lower limit is set to 24 and the upper limit to 58, resulting in the views scoring in-between those two values being Unclassified.

You can use the Unclassified predictions, for example, in the following use cases:

- Manually review Unclassified labels and decide whether they are OK or NG

- Run Unclassified labels through inspection again

- Set Unclassified as the input class for a downstreamA downstream tool requires a previously configured upstream tool and its configuration as input. The tool is placed after its upstream tool in the toolchain. tool for further refinement

Example of verifying and refining the tool

The following example shows the verifying and refining process of the Anomaly Detector tool. In this specific example, the initial setup only used a few labeled views. Out of 80 images in the image databaseThe set of images used to train and test your vision application.:

- A total of six views labeled as OK

- Four views assigned Train roles, two views assigned Test roles

## Viewing the Results

Following the completion of the Process step, you can filter the view panel by Result Class. In this case, filtering for NG displays all the views the tool predicted to have anomalies. Sort by ascending Score to see the lowest scored NG views the tool predicted first:

Use these views to verify how accurate the predictions of the Anomaly Detector tool are.

## Repeating Tool Setup

Go to the Label step and manually label views that are wrongly classified as OK. Label views that the tool correctly predicted as NG as NG to influence the anomaly score threshold and improve the performance of the tool. Experiment with how many views you need to label to get the desired results. In this example refinement process, five more views were labeled as OK and two as NG.

In the example, the tool predicted the following view to be NG with a high anomaly score, even though it is an OK image. Label the view as OK to further train the tool and help it learn what anomalies to look for. The Heat Map helps determine why the tool made the prediction and, in this case, it considered an area of the view where anomalies are not expected. To improve the performance of the tool further, go back to the ROIThe Region of Interest (ROI) is the area of the image where the vision tool operates. step and narrow down the view by adjusting the Region of InterestThe Region of Interest (ROI) is the area of the image where the vision tool operates..

Note: Once you apply a new ROI, the Process step is out of date and you can no longer see the predictions.

You can also assign Train roles to these newly labeled images to make sure the Anomaly Detector tool uses them for training.

To test the performance of the Anomaly Detector tool after the additional manual labelingLabeling is the process of marking features or defects in images, or categorizing the images into classes. Labeling is crucial for training Deep Learning Lite and Deep Learning Standard tools because the labels serve as training material to guide the tools how to function correctly. and ROI reset, train, and process views again and examine the results.

## Verifying the Results Again

The table shows new filtering after completing the steps again. Between the anomaly scores 62 and 72 the predicted views should actually be NG. This shows that the Anomaly Detector tool needs more refinement, and that the anomaly score threshold also needs adjusting.

Result Class: OK, sort by Score descending order | Result Class: NG, sort by Score ascending order
The selected views have wrong predictions. | The lowest scored views do have anomalies and all the NG predictions are correct.

Based on this information, you can set the anomaly score lower threshold limit to 60 in the Process step. Click Process All again and verify that the Anomaly Detector tool is now well-trained.

Result Class: OK, sort by Score descending order | Result Class: NG, sort by Score ascending order
The highest scored view is OK. | The lowest scored view is NG.

Sorting by score also helps spot any false positives or negatives remaining. Check for false positives and retrain as needed, as there might be views with anomalies scoring lower than the highest scoring OK view.