Confusion Matrix of Red Analyze

The confusion matrix is a visual representation of the ground truth Ground truth refers to the actual nature of the problem that is the target of a machine learning model, reflected by the relevant data sets associated with the use case in question. Supervised machine learning models are trained on labeled data that are considered for the model to identify patterns that predict those labels on new data. versus the predictions of the tool.
Number Description
1

The horizontal dimension shows the predictions of the tool.
2 The vertical dimension of the matrix shows actual labels you assigned, which represent the ground truth for the tool.
3

The F1 score of the confusion matrix is the combination of the Precision and Recall metrics. The F1 score is a comprehensive metric for the segmentation Segmentation is the process of selecting a view from an image. performance.

  • Recall shows how well the neural network matches the area of the labeled defect region. For more information, see Recall.

  • Precision shows how well the neural network avoids confusion with other areas in an image. For more information, see Precision.

For more information about how the application calculates the Precision, Recall, and F1 score in the confusion matrix, see Precision, Recall, and F1 Score Calculation Example.

Note: The Precision, Recall, and F1 score of the confusion matrix are calculated differently compared to those in Region Area Metrics section, which are calculated on the pixel level.​

Confusion Matrix and Multiple Defect Classes

If the Red Analyze tool has multiple defect classes, the confusion matrix only shows the results for one class of defects at a time. You can switch between defect classes using the dropdown menu in the upper right corner of the Database Overview.

The confusion matrix only counts the selected defect class as "Bad", and counts all other defect classes as "Good". For example, for the defect class "Blob", "Good" means everything that is "not Blob". This means that both "Good" and defects of other classes count as "Good".

Positive and Negative Results in the Confusion Matrix

You can view A view of an image is a region of pixels in an image. Tool processing is limited to the pixels within the view. You can manually specify a view, or you can use the results of an upstream tool to generate a view. the confusion matrix in terms of true positive, true negative, false positive, and false negative results. The Precision and Recall metrics, and ultimately the F1 score, are calculated based on these results.

For the "Good" class:

  • One "Actual Good - Predicted Good" pair is one true positive (TP)

  • One "Actual Bad - Predicted Good" pair is one false positive (FP)

  • One "Actual Bad - Predicted Bad" pair is one true negative (TN)

  • One "Actual Good - Predicted Bad" pair is one false negative (FN)

For the "Bad" class (or selected defect class):

  • One "Actual Bad - Predicted Bad" pair is one true positive (TP)

  • One "Actual Good - Predicted Bad" pair is one false positive (FP)

  • One "Actual Good - Predicted Good" pair is one true negative (TN)

  • One "Actual Bad - Predicted Good" pair is one false negative (FN)

For more information about positive and negative results, see Positive, Negative, and False Results.

Precision, Recall, and F1 Score Calculation Example

The Precision and Recall metrics, and ultimately the F1 score, are calculated based on the count of true positive (TP), true negative (TN), false positive (FP), and false negative (FN) results. For more information, see Positive and Negative Results in the Confusion Matrix

To calculate the metrics on the confusion matrix example in the image above:

  1. Calculate the Precision, Recall, and F1 score of the confusion matrix for the "Good" class:

    • Precision = TP / (TP + FP) = 136 / (136 + 0) = 1

    • Recall = TP / (TP + FN) = 136 / (136 + 1) = 0.993

    • F-score = 2 * Recall * Precision / (Recall+Precision) = 2 * 0.993 * 1 / 1.993 = 0.996

  2. Calculate the Precision, Recall, and F1 score of the confusion matrix for the "Blob" class:

    • Precision = TP / (TP + FP) = 14 / (14 +1 ) = 0.933

    • Recall = TP / (TP + FN) = 14 / (14 + 0) = 1

    • F-score = 2 * Recall * Precision / (Recall + Precision) = 2 * 1 * 0.933 / 1.713= 0.965

  3. Calculate the F1 score of the confusion matrix:

    • 0.5 * (F-score of "Good" class) + 0.5 * (F-score of "Blob" class) = 0.5 * 0.996 + 0.5 * 0.965 = 0.98