# Train Neural Network

After all views are correctly labeled and split into the training images and the test images, neural network training is performed in the following general way:

1. Configure tool parameters and start the training. Train the tool by pressing the Brain icon. Each image in the image set that is being used for training (defined in the Training Set dialog) is sampled across its full extent, using the specified Feature Size when the architecture is Focused.
2. The resulting samples are provided as input to the VisionPro Deep Learning deep neural network.
3. For each sample, the neural network produces a specific response (depending on the tool type), and this response is compared with the image labeling associated with the sample's location in the training image.
4. The internal weights within the network are repeatedly adjusted as the samples are processed and reprocessed. The network training system continually adjusts the network weights with a goal of reducing the error (difference or discrepancy) between the network's response and the labeling provided by the user.
5. This overall process is repeated many times, until every sample from every training image has been included at least the number of times specified by he Epoch Count parameter.

1. The sampling region.

2. The user-drawn, labeled defect region.

3. The neural network.

4. The response by the network.

5. The iterative process of adjusting the weights to reduce the discrepancy (in other words, error) between the labeled defect (in yellow) and the network response (in blue).

The specific characteristics of the neural network training vary somewhat depending on the type of tool being trained. The goal for network training of the Red Analyze tool in Supervised Mode is to reduce the spatial discrepancy between the defect labeling and the detected defects. For Red Analyze High Detail, network is also trained to locate and identify defect regions within an image, like Red Analyze Focused Supervised. The labeling that you perform for the Red Analyze tool in High Detail labels all of the defect pixels in the labeled image.

- Sampling Region and Sampling Parameter

  While Red Analyze Focused Supervised samples pixels with a sampling region defined by the users, Red Analyze High Detail samples from the entire image, so they don't have a sampling region, not requiring the sampling parameters in training.

- Training with Validation

  Red Analyze High Detail uses validation data set to validate each trained neural network and with these validation results selects the most performing and stable neural network with the given training set.

## Configure Tool Parameters

After a tool has been added, the tool's parameters can be accessed to fine-tune the tool's performance prior to training, and how the tool will process images during runtime operation. The VisionPro Deep Learning tool parameters adjust the how the neural network model is trained, and also how the tool processes statistical results.

When Red Analyze tool's Architecture parameter is set to High Detail, the tool is configured to consider the entire image equally. This option is useful when you want to get more accurate or detailed results in pixel levels, at the expense of increased training and processing times. There are 4 categories of tool parameters for Red Analyze Tool in High Detail mode. You can see more detailed information for each parameter. ​

Tip: In many cases, the default parameter settings perform well against most image sets. For the initial training, attempt to train without any adjustments to the parameters.

### Architecture Parameter

The Architecture parameters selects the type of neural network model that will be used. This option is useful when you want to get more accurate results, at the expense of increased training and processing times. The High Detail and High Detail Quick architecture setting configures the tool to consider the entire image equally, while the Focused architecture setting is selective, focusing on the parts of the image with useful information. Due to this focus, the network can miss information, especially when the image has important details everywhere.

### Mode Parameter

The newly captured images in the front lines of your daily operation can have variations from the training images that constructed your existing tool. With Mode parameter, you can adapt your previously trained tool to the new images with variations coming from your production line. For more details, please refer to Adaptation Mode: Adapting to Line Variations.

### Network Model Parameter

The Network Model parameter allows you to select the size of the trained tool network, which will change the time required for training and processing. In High Detail mode, there are 4 different network models; Small, Normal, Large, Extra Large.

- Normal network model(default) is commonly used for general project.

- Larger network model is useful for more complex images, but this does not mean that its performance will always be better. Larger network model has a risk of overfitting. Larger network model will increase training and processing time.

If you suffer from a poor performance (recall, precision, and F1 score) in the Normal network model, it is generally recommended to switch over to Small network model to enhance the performance.

### Training Parameters

The Training tool parameters control the training process. If any changes are made to the Training tool parameters after the tool has been trained, it will invalidate the training and necessitate that the tool be retrained.

Parameter | Description
--- | ---
Epoch Count | The number of times to train using all training images. Higher the value more the repetition. The value is from 1 through 100000. Tip: High Detail mode uses a different concept of Epoch Count from Focused mode. You’d better use a higher epoch count in High Detail mode than in Focused mode with the same database.
Training Set | The dataset that is used to create a deep learning model. This means that during deep learning, only the features of the images included in the train set are extracted to create a deep learning model. You can select training set in Select Training Set dialog.
Validation Set Ratio | The ratio of the number of views which will be used as a validation set among training set. You can enter the validation set ratio value from 1% to 50%. If you increase the validation set ratio while keeping the number of the training set, the amount of data used for training will decrease. So, setting a high validation set ratio could affect the performance negatively when you are using a small training set. On the other hand, a too low validation set ratio will not be helpful for selecting a good model for an unseen data set.
Minimum Epochs | Due to the validation set, High Detail mode can select the model created from the low epoch as a final model to prevent overfitting. If you set a minimum epoch, High Detail mode selects a model created after this epoch as a final model. If you get a good processing result to the train set, but not a good one to the test set, you may think overfitting. However, if the processing result is not good to the train and test set, it would be underfitting. In this case, it is recommended to use higher minimum epochs. The available value is 0 through the current epoch.
Patience Epochs | Each time a fixed number of iterations is executed, High Detail mode measures the loss of the model. If no drop of loss is observed for N (the value of Patience Epochs) epochs, High Detail mode stops training and selects the model with the lowest loss until the epoch. You can utilize patience epoch for early stopping in case that there is no need to proceed a training process unless loss drops over a certain period. The available value is 0 through 100000. Tip: If it is complicated to set a specific epoch count, you can increase the epoch count and create a model based on patience. You can tract the loss for every epoch with Tool - Inspect Loss. The Loss is calculated with the validation set but not training set.
Patch Size | The size of the square that divides each view into several chunks. Training (feature detection) and processing of each view are executed based on each chunk. Generally, a smaller Patch Size works better for catching small subtle blobs, and a larger Patch Size for large obvious blobs in terms of convergence speed in training.

Note: Patch Size parameter is only available when Expert Mode is enabled. This is enabled via the Help menu.

### Training Parameter Details: Epoch Count

Epoch Count parameter lets you control how much network refinement is performed. As described in the Neural Network Training topic, the training process repeatedly processes input samples through the network, compares the network result with the user-supplied labeling, then adjusts the network weights with a goal of reducing this error. Because of the large number of network nodes (and thus weights), this process can be repeated almost indefinitely, with each iteration resulting in a progressively smaller improvement in the error. Increasing the Epoch Count parameter setting increases the number of iterations of training that are performed. This will reduce the network error on the training images at the cost of requiring more time for training.

It is important to keep in mind, however, that the goal for training the network is to perform accurately on all images, not just those used for training. As the epoch count is increased, the network will tend to experience overfitting (Terminology), where the error on untrained images increases at the same time that the error on trained images decreases. For this reason, you should carefully monitor the network performance on all images as you adjust the epoch count. You have to choose an optimal Epoch for your dataset because its optimal value is different by dataset, particularly the statistical diversity of your dataset.

### Training Parameter Details: Minimum Epochs

Due to the validation set, High Detail mode can select the model created from low epoch as a final model to prevent overfitting. If you set a minimum epoch, High Detail mode selects a model created after this epoch as a final model.
If you get a good processing result to the train set, but not a good one to the test set, you may think overfitting. However, if the processing result is not good to the train and test set, it would be underfitting. In this case, it is recommended to use higher minimum epochs.

### Training Parameter Details: Patience Epochs

Each time a fixed number of epochs (1/8 epochs) is executed, High Detail mode measures loss of the model. If no drop of loss is observed for N (the value of Patience Epochs) epochs, High Detail mode stops training and select the model with lowest loss until the epoch. You can utilize patience epoch for early stopping in case that there is no need to proceed a training process unless loss drops over a certain period.

Tip: If it is complicated to set a specific epoch count, you can increase the epoch count and create a model based on patience.

### Training Parameter Details: Patience Epochs & Minimum Epochs

Minimum Epochs was applied prior to Patience Epochs during training in VisionPro Deep Learning 2.1.1 or lower versions. This means that Patience Epochs started to be counted after Minimum Epochs was elapsed. Contrast to this, in VisionPro Deep Learning 3.0 or higher, Patience Epochs is applied independently from Minimum Epochs in training. As a result, Minimum Epochs and Patience Epochs in VisionPro Deep Learning 3.0 or higher versions work differently compared to the lower versions of VisionPro Deep Learning. Refer to the examples below for how Patience Epochs and Minimum Epochs are applied during training in VisionPro Deep Learning 3.0.

All the examples below are using the following parameter setup:

- Patience Epochs: 5

- Minimum Epochs: 10

Example 1. The tool waited 5 epochs hoping for the best loss so far (0.1960) was updated again regardless of Minimum Epochs. Since the best loss (0.1960) was not updated after the waiting, the tool stops training and picks its best loss among the epochs after Minimum Epochs. The corresponding best loss is 0.2135 at the epoch 14.

Example 2. The tool waited 5 epochs hoping for the best loss so far (0.1911) was updated again regardless of Minimum Epochs. Though the best loss (0.1911) was not updated after the waiting, the tool continued its training toward the epoch 10 to also meet Minimum Epochs condition. The tool picks its best loss when it reached the epoch 10, and the corresponding best loss is 0.1911 at the epoch 3.

Example 3. The tool waited 5 epochs hoping for the best loss so far (0.1965) was updated again regardless of Minimum Epochs. Since the best loss (0.1965) was not updated after the waiting, the tool stops training and picks its best loss among the epochs after Minimum Epochs. The corresponding best loss is 0.2106 at the epoch 11.

### Training Parameter Details: Validation Set Ratio

It is the ratio of number of views which will be used as a validation set among training set. If you increase the validation set ratio while keeping the number of training set, the amount of data used for training will decrease. So, setting a high validation set ratio could affect the performance negatively when you are using small training set.
On the other hands, too low validation set ratio will not be helpful for selecting good model for unseen data set.

### Training Parameter Details: Patch Size

Patch Size is the size of the square that divides each view into several chunks. Training (feature detection) and processing each view is executed based on each chunk. Generally, a smaller Patch Size works better for catching small subtle blobs, and a larger Patch Size for large obvious blobs in terms of convergence speed in training.

### Perturbation Parameters

The VisionPro Deep Learning neural network can only be trained to learn the features in images that it actually sees. In an ideal world, your training image set would include a representative set of images that expressed all of the normal image and part variations. In most cases, however, training needs to be performed with an unrepresentative image set. In particular, image sets are often collected over a short period of time, so normal part and lighting variations over time, as well as changes or adjustments to the optical or extrinsic characteristics of the camera are not reflected.

The VisionPro Deep Learning training system allows you to augment the image set by specifying the types of appearance variation that you expect during operation, through the use of the Perturbation parameters, such as the following:

- Luminance
- Contrast
- Rotation

The Perturbation parameters allow the VisionPro Deep Learning tools to artificially generate images to be trained on, improving results for applications with high amounts of variance. These parameters are common across all of the tools. The Perturbation parameters can also be combined. This allows for the generation of more complex images by using the parameters separately, as well as in conjunction.

Tip: If your Training Image Set does not include all the variations your part may exhibit during runtime, you can use the Perturbation parameters. For example, if your part will rotate by +/-45 degrees, you can set the Rotation parameter, and the software will rotate the images during training by that amount. However, to get the best results, Cognex recommends using actual sample images of the part variations. When a part spins, there may be different shadows, and those will not be captured by the tool when the rotation is based on artificially spinning the image.
Note: Perturbation is not a substitute for collecting and training actual images. In particular, the image perturbations can do no more than approximate the actual changes in the appearance of real parts or scenes.

High Detail mode provides 13 perturbation options. Use only the perturbation options which can actually be acquired at production lines.

Perturbation | Description
--- | ---
Horizontal Flip | It performs flipping in horizontal direction. You may use in most cases where object location and angles are not strictly fixed.
Vertical Flip | It performs flipping in vertical direction. You may use in most cases where object location and angles are not strictly fixed.
Rotation 90° | It performs only +90° rotation. If you check three options ‘Horizontal Flip’, ‘Vertical Flip’, ‘Rotation 90°', it performs 0°, 90°, 180°, and 270° rotations probabilistically.
Rotation | It performs rotation between 0° to 45°. So, applying four options 'Rotation 90°', 'Rotation', 'Horizontal Flip' and 'Vertical Flip' is same with randomly rotating between 0° to 360°.
Contrast | It adjusts contrast by multiplying a random value for all channels. The random value follows the uniform distribution within a range of 0 to 2. You can use this option when the images are obtained with inconstant contrast because of the irregular lighting environment
Luminance | It adjusts luminance by adding a random value for all channels. The random value follows the uniform distribution within a range of - 255 to 255. You can use this option when the images are obtained with inconstant luminance because of the irregular lighting environment.
Colorwise | It adjusts color by multiplying and/or adding different random values per channels. It should be used with Contrast and/or Luminance option. The colorwise option is applied by changing the way of Contrast and Luminance option applied form applying same value for all channels to different random values for each channel. You can use this option when the images are obtained with inconstant color tone because of the irregular lighting environment.
Gradation | It randomly adjust the gradation. You can use this option when the images are obtained with inconstant gradation because of the irregular lighting environment.
Zoom-In | It randomly zooms in the views from the center. The maximum of zooming is 5/6 of original view size. The random variable follows the uniform distribution. You can use this option when the defect to be detected has an irregular product size.
Sharpen | It randomly sharpens the views by image filtering within a range of 0 to 2. You can use this option when the focus issues the image too blurry.
Blur | It randomly applies Gaussian Blur to the views. The random variable follows to the Gaussian Sigma Distribution within a range of 0 to 2. You can use this option when the focus issues the image too sharp.
Distortion | It applies a distortion to the views by picking the points in the views and moving them. The number of points is same or less than 6. You can use this option when the images are distorted due to the deterioration of the optical equipment.
Noise | It applies the noise by multiplying a random value per pixel for all channels. The random value follows the uniform distribution within a range of 0 to 2. You can use this option when the images are distorted or contaminated by dust due to the deterioration of the optical equipment.

Note: All Train images can be applied to perturbation. Every train image has 50% chance to be augmented to every checked options independently.
By the law of large numbers, all perturbation options can be applied equally and trained by increasing the number of epochs.

## Recover Last Parameters: Restore Parameters

Restore Parameter button is designed for the easy turning back of tool parameter values to the values that you chose in the last training task. It remembers all values in Tool Parameters used in the last training session. So, if you changed any of its values and now want to revert this change, you can click it to roll back to the tool parameter values which are used in the last training. Note that it is disabled when the tool has never been trained or there were no changes from the initial set of tool parameter values.

The following steps explain how Restore Parameter works:

1. Restore Parameter button is always disabled when the current tool was not trained.

2. Once the current tool is trained, the checkpoint of parameter rollback is set to the values in Tool Parameters of the last training session. At this point, if you change any value in Tool Parameters, the button is enabled.

3. Click Restore Parameter button and it reverts the changed value to the value of the checkpoint.

4. If you train the current tool again, with some changes in Tool Parameters, then the checkpoint of parameter rollback is updated to the changed parameter values. Again, the button is disabled unless you make another change for the values in Tool Parameters.

5. If you make another change and click Restore Parameter button again, it reverts the changed value into the value of the updated checkpoint.

Note that if you re-process a trained tool after changing the values in Processing Parameters, the checkpoint of parameter rollback is not updated, and thus Restore Parameter remains enabled. The checkpoint is updated only after the training of a tool is completed.

Disabled | Enabled

Note: There are parameters that cannot be restored to the last training session due to the inevitable reasons.

1. Irreversible parameters that changing these parameters will reset the tool

   1. Network Model, Exclusive, Feature Size, Masking Mode, Color, Centered, Scaled, Scaled Mode (Uniform/Non-uniform), Legacy Mode, Oriented, Detail

2. Irreversible parameters that these parameters are not invertible in nature

   1. Low Precision, Simple Regions

3. Other irreversible parameters

   1. Training Set, Heatmap in Green Classify High Detail and Green Classify High Detail Quick(This parameter does not affect the prediction performance)

   2. Overlay parameter in Masking Mode in Blue Read

Note: For only the High Detail (Green Classify and Red Analyze) and High Detail Quick modes (Green Classify), if you abort the current training and choose to save the tool, clicking Restore Parameter will restore the parameter values of this saved tool. If you do not choose to save the tool, clicking Restore Parameter will restore the parameter values of the completed training before the currently initialized training.

## Control Neural Network Training

The training of Red Analyze High Detail can be controlled by configuring the tool parameters and the training set.

### Training Set

The largest single determinant affecting the network training phase is the composition of the Training Set. The best method for controlling the network training phase is to construct a proper training set for your tool. In this way, you can separate images/views into categories that allow you to determine if your tool is generalizing your images/views properly.

### Validation Set and Validation Loss

The use of training set is in common for all tools, but High Detail tools has another data set called "validation set" or "validation data" which is part of the training set, whose amount of data is chosen by users. For High Detail modes, the validation loss (=the loss calculated from the validation data) is calculated for each model during the training phase, and the model who gives the best validation loss in terms of performance and availability is finally selected as the result of training.

The purpose of validation data is among many neural network models generated from the training data choosing the best model as the final output of training. The training strategy that adopts validation data to achieve this goal is here called "training with validation." Unlike Focused mode tools, High Detail mode tools (Green Classify High Detail Mode and Red Analyze High Detail Mode) provide the training with validation, and you can control the network training with monitoring validation loss. During training at the end of every 1/8 epoch, the neural network calculates the loss value from the validation set you previously configured.

The validation loss stands for the performance of your trained network in terms of accuracy of classification (Green Classify High Detail Mode) or segmentation (Red Analyze High Detail Mode), which means that smaller loss generally means a better network. So it is better to have this value close to 0. The validation loss of Red Analyze High Detail Mode is calculated per pixel as the segmentation, which is the binary classification among "Good" or "Defect", is executed on each pixel. Though, to gain the full-sight regarding how your network truly performs well, you have to test the trained network against some separate data (Test Data) to prevent overfitting.

Validation Loss (from 0 through 1)

- 1 - IOU

IOU (unit: %)

- IOU is the intersection over union that measures to what extent predicted areas are equal to their ground truth. The formula to calculate IOU is:

  (Ground Truth Area ∩ Predicted Area) / (Ground Truth Area ∪ Predicted Area)

Tip: You can monitor the change of the validation loss in training for each High Detail mode with Loss Inspector .
Note: For how to speed up training or processing, see Optimize Speed
Note: For the general tips and tricks for training and processing, see Application Design