# ViDiELSegment3D

You can use the ViDiELSegment3D tool for finding flaws or locating features on a 2D projection of a 3D image. You can create classes for the different features, so the ViDiELSegment3D tool can search for several features at the same time. After training, the tool can predict the appearance of the features in the new images.

Note: When using high-resolution images in EL Segment tools, prediction algorithms can produce exceedingly small defect labels and can clutter the training due to the level of detail. You may need to zoom in to see these labels. You can either partially erase the labels on the images, or completely clear the entire label prediction to manage the detailed predictions.

To add the ViDiELSegment3D tool:

1. Drag and drop the ViDiELSegment3D tool into the Spreadsheet.

   The property sheet pops up:

   Note: If you are editing a ViDiELSegment3D tool that has already been trained in an existing job A job is a user-configured program in Spreadsheet consisting of functions and designed for specific applications. You can create and edit jobs using the Spreadsheet interface, and load them on a vision system for deployment in a live environment., the tool automatically shows predictions based on the training the tool received.

2. Set up the Region of Interest using one of the following methods:

   - Double-click the Box3D parameter and adjust the ROI box in the acquisition Acquisition is the process or result of the vision system acquiring a new image. display.

   - Expand the Box3D parameter and define the box parameters manually.

   - Double-click the External Box parameter and select a Box3D function Functions are tools that are available in Spreadsheet for processing and analyzing acquisitions or other results. You can add functions to your Spreadsheet job to create tool chains and produce results for specific applications. cell in Spreadsheet.

3. In the Projection tab of the property sheet, select the Projection Mode, Feature Width, and Feature Height that you want the tool to use for creating the image projections.

   Note: If you change the Projection Mode after you already trained some images, the tool adjusts all the images in the database to that projection as well.

4. Start labeling images.

   Note: Once you have started labeling, you cannot change the ROI size and projection settings.

   You can do the labeling in the following ways:

   - Use the Pen tool by holding down the left mouse button to color the areas you want to mark.

   - Use the Polygon tool by clicking around the edge of an area or feature to draw lines. Double-clicking closes up the lines and creates a colored polygon for you.

   If you do not like the label you created, you can use the Undo and Redo buttons, delete part of the markings with the Eraser tool, or all of them with the Clear button. The Clear button deletes every label in the image, regardless of the class.

   You can rename the class or give the class a new color by clicking on the Edit Class button.

5. Click Accept Labels to train the image.

6. After labeling, you can click the Auto-Tune button in the Projection tab to enable projection auto tuning. If projection auto tuning is enabled, the function analyzes the projection images of the training samples using different projection settings and selects the best projection settings based on the analysis results.

   Note: The system runs a few different projections and changes the parameters automatically, but the selected projection mode is not visible in the property sheet until you close and reopen the property sheet.

7. Show the next image to the tool. At this point, the tool makes a prediction, meaning that the tool tries to find the patterns that you marked in the previous images.

8. Finetune your labeling if needed.

   For example, there can be false predictions in your image. You can delete these faults by using the Eraser tool before accepting the changes.

9. If needed, create more classes by clicking the Add Class button. You can have up to three classes defined. Name your new class and pick a different color.

   Note: Cognex recommends that you label at least five images for each class. The more images you train, the higher the quality of your output is.

   You can view your trained images in the sample viewer. To reach the sample viewer window, click the Edit button on the right.

   In the sample viewer, you can correct and finetune the trained images by clicking on the Edit button. You can also delete them by clicking on the Delete button, or you can edit the classes.

   Click on the Edit Classes button to open the class editor where you can rename and delete classes, or change the class color:

10. Click OK, if you finished labeling. The output of the function appears on the Spreadsheet.

Inputs

Setting | Description
PointCloud A point cloud is a 3D structure that consists of the data points collected by a 3D vision system during acquisition. | Reference to a cell containing a point cloud A point cloud is a 3D structure that consists of the data points collected by a 3D vision system during acquisition.. The default value is $A$0.
Fixture3D | Specifies the point cloud coordinate system in which the 3D fixture A fixture is a coordinate location on the image that keeps the tool in the place determined by the fixture. is defined. It is offset from the PointCloud origin in the following format with the following arguments: {X, Y, Z, Rotation, Tilt, TiltDirection}
X | Offset on the X-axis from the PointCloud origin.
Y | Offset on the Y-axis from the PointCloud origin.
Z | Offset on the Z-axis from the PointCloud origin.
Rotation | Specifies the in-plane rotation in degrees around the PointCloud Z-axis. The value range is -180 to 180. The tool normalizes external references with values outside the allowed range to the allowed range.
Tilt | Specifies the tilt angle in degrees from the PointCloud Z-axis after rotation. The value range is 0-180. External references with values outside the allowed range result in an error.
Tilt Direction | Specifies the direction of tilt in degrees from the rotated PointCloud X-axis of the tool, around the PointCloud Z-axis of the tool. The value range is -180 to 180. The tool normalizes external references with values outside the allowed range to the allowed range.
Note: The Fixture3D parameter is only available if you set External Fixture to 0.
External Feature | Specifies a reference to a Fixture3D structure defined in another cell. The function uses the referenced cell as the fixture of the function. Set to 0 (OFF) to use the internal fixture of the function instead.
Box3D | Specifies the dimensions, offset and angle for the 3D rectangle of the function. It is offset from the PointCloud origin in the following format with the following arguments: {SizeX, SizeY, X, Y, Z, Rotation, Tilt, TiltDirection}
SizeX | The size of the box along the X-axis of the PointCloud. The value range is 0.001 to inf., and the default value is 100.
SizeY | The size of the box along the Y-axis of the PointCloud. The value range is 0.001 to inf., and the default value is 100.
SizeZ | The size of the box along the Z-axis of the PointCloud. The value range is 0.001 to inf., and the default value is 100.
X | Offset on the X-axis from the PointCloud origin. The value range is -inf. to inf., and the default value is 0.
Y | Offset on the Y-axis from the PointCloud origin. The value range is -inf. to inf., and the default value is 0.
Z | Offset on the Z-axis from the PointCloud origin. The value range is -inf. to inf., and the default value is 0.
Rotation | Specifies the in-plane rotation of the box (in degrees) about the PointCloud Z-axis of the tool. The value range is -360 to 360. External references with values outside the allowed range will be normalized to the allowed range.
Tilt | Specifies the tilt angle of the box (in degrees) from the PointCloud Z-axis of the tool. The value range is 0 - 180. External references with values outside the allowed range will result in an error.
Tilt Direction | Specifies the tilt direction of the box (in degrees) from the rotated PointCloud X-axis of the tool, about the PointCloud Z-axis of the tool. The value range is -360 to 360. External references with values outside the allowed range will be normalized to the allowed range.
Note: This parameter is available only if External Box is set to 0.
External Box | Specifies a reference to a Box3D structure defined in another cell to be used with the function. Set to 0 (OFF) to use the internal Box3D structure of the function.
Projection Image A projection image is the 2D projection of a 3D point cloud acquisition. Resolution | Specifies the resolution of the image to be generated from the point cloud within the region of interest. Low, Medium, or High.
Run Parameters | Advanced parameter properties.
Confidence Threshold | Specifies the minimum confidence score (0 to 100) to determine pixel-to-class assignment. The larger the threshold, the more strict the class assignment is.
Minimum Volume | Minimum volume that blobs must have to be included, in real-world units.
Maximum Volume | Maximum volume that blobs must have to be included, in real-world units.
Minimum Area | Minimum area that blobs must have to be included, in real-world units.
Maximum Area | Maximum area that blobs must have to be included, in real-world units.
Minimum Height | Minimum height that blobs must have to be included, in real-world units.
Maximum Height | Maximum height that blobs must have to be included, in real-world units.
Orientation | Bounding box alignment options of result 3D blobs. Aligned to Box Aligned to Z Aligned to Background Surface
Result Sorting | Advanced sorting properties.
Sort Key | Select the sorting key for the sorting rule from the drop-down menu.
Sort Order | Choose Ascending or Descending as the sorting order for the sorting rule.
Sort Tolerance | The threshold to determine is two or more results are tied, in real-world units or degrees. If they are tied, they can be sorted by a subsequent sort. This only takes effect when multiple sorting rules are applied.
Sort Reference | Select the sorting reference for the sorting rule from the drop-down menu.
Show | Select the type of graphic to overlay on the image.

Projection Parameters

Setting | Description
--- | ---
Projection Mode | Specifies the mode used to create the sample 2D image from the point cloud.
Height Map | The Height Map projection mode creates three-dimensional height maps. This projection mode collects 3D data points within the ROI, and transforms them into a 2D height data representation, where the assigned height value for each pixel signifies the average distance between the points within the corresponding XY cell and the bottom of the ROI box. The projection mode fixes minor irregularities in the height data and directly maps it to an 8-bit image. In this mapped image, pixel values range from 0 (bottom of the box) to 255 (top of the box).
Automatic Height Map | The Automatic Height Map projection mode creates three-dimensional height maps, automatically selecting the height range. This projection mode collects 3D data points within the ROI, and transforms them into a 2D height data representation, similar to the Height projection mode. This mode automatically sets the height range for optimal results.
Shape Change | The Shape Change projection mode identifies 3D depth along the XYZ axes, and visualizes 3D structural variations along the X, Y, and Z axes. This projection mode collects 3D data points within the ROI, and projects them into a 2D data representation. Changes are color coded relative to the X, Y, and Z axes in order to highlight changes in the orientation of surface normals. Estimated kernel sizes for surface normals are set based on the parameter feature width. This projection mode amplifies edges, but is less sensitive to height changes.
Vertical Shape Change | The Vertical Shape Change projection mode identifies 3D depth along the Z-axis. The Vertical Shape Change projection mode is designed to identify and visualize 3D structural variations along the Z-axis. This projection mode collects 3D data points within the ROI, and projects them into a 2D data representation, highlighting changes in the orientation of surface normals relative to the Z axis. Estimated kernel sizes for surface normals are set based on the parameter feature width. This projection mode amplifies edges, but is less sensitive to height changes.
Flex Inspect | The Flex Inspect projection mode creates projection images that identify features and anomalies on a smooth freeform surface, where the coverage of features is expected to be a small portion of the entire surface. If the features of interest do not have a sufficiently large and clear background surface area, projection images can have bad contrast between features and surrounding areas. This projection mode estimates the shape of the local background as a freeform surface, and shows height changes relative to the local background shape. This mode computes the average height in the neighborhood of a pixel according to the user-defined feature width, and overlays the difference between pixel height and the computed average height onto the projection according to the user-defined feature height.
Curve Inspect | The Curve Inspect projection modecreates projection images that identify features and anomalies on a smooth curved surface, where the coverage of features is expected to be a small portion of the entire surface. If the features of interest do not have a sufficiently large and clear background surface area, projection images can have bad contrast between features and surrounding areas. This projection mode estimates the shape of the local background as a curved surface, and shows height changes relative to the local background shape. This mode computes a curved surface model based on pixel neighborhoods according to the user-defined feature width, and overlays the difference between pixel height and the surface model onto the projection according to the user-defined feature height.
Greyscale | The Greyscale projection mode identifies overall 2D surface flaws. This projection mode is a direct mapping technique from the height data addressed to an 8-bit image. In this mapping, the value of each pixel in the resulting image corresponds directly to the intensity of the original 3D points.
Feature Width | Specifies the expected width of a feature of interest in millimeters.
Feature Height | Specifies the expected height of a feature of interest in millimeters.
Auto-Tune | Runs the configured function using multiple different projection parameters, and selects the best scoring settings. This button is disabled until you have labeled a sufficient amount of images.

Outputs

Returns | A ViDiELSegment3D data structure, or #ERR if any of the input parameters are invalid.
Results | When initially inserted into a cell, the following functions are automatically inserted into the spreadsheet for each identified feature to create a result table:
Output | Description
--- | ---
Num Found | The number of features identified.
Index | The index of each identified feature.
Label | The class label for each identified feature.
Volume | The volume of each identified feature.
Area | The area of each identified feature.
Height | The height of each identified feature.
Center of Mass | A Point3D structure locating the center of mass for each identified feature.
Bounding Box | A Box3D structure creating a bounding box around each identified feature.