# API Changelog

This topic lists the API changes in previous releases of VisionPro Deep Learning.

## VisionPro Deep Learning 3.0

Among the new features introduced in VisionPro Deep Learning 3.0, some of them are only provided through API as they enhance runtime functionality.

### Classification Batch Processing for Runtime API

Processing multiple views at once through batch processing is now available for Green Classify High Detail. Using the following Runtime APIs can speed up the Green Classify High Detail processing via API.

The steps to use classification batch processing API are summarized as follows.

1. Fix the batch size and prepare the images to process. The maximum batch size is dependent on the current available GPU memory.

2. Create a sample.

3. Add the prepared images to the sample.

4. Run batch processing.

5. Check the processing results.

For more details of utilizing batch processing API, see the below example codes.

Note: If you select a batch size larger than the maximum batch size which is calculated based on the available memory of the current GPU in Step 1, the batch processing will fail to start.

Copy
C++ Example of Batch Processing API for Runtime (Green High Detail)

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    ...
    // The maximum batch size depends on the available GPU memory size.
    // let's suppose the available maximum batch size is 12.
    constexpr int batch_size = 12;

    // you need to set "runtime_parameters/batch_size" as the batch size you chose above.
    // "workspace": the name of your workspace
    // "default": the name of the stream in your workspace
    // "Classify": the name of the Green High Detail tool in your stream.
    string batch_size_str = std::to_string(batch_size);
    vidi_runtime_tool_set_parameter("workspace", "default", "Classify", "runtime_parameters/batch_size", batch_size_str.c_str());
    ...
    // you also need to prepare the same number of the images as the batch size
    vector<VIDI_IMAGE> images(batch_size);
    for (int i = 0; i < batch_size; i++)
    {
        ...
        status = vidi_load_image(image_path.c_str(), &images[i]);
        ...
    }
    ...
    // create a sample for batch processing. Note that a sample is the basic unit of processing task.
    // "my_sample": the name of the sample you create.
    status = vidi_runtime_create_batched_sample("workspace", "default", "my_sample");
    ...
    // add the prepared images to the sample
    for (int i = 0; i < batch_size; i++)
    {
        ...
        status = vidi_runtime_batched_sample_add_image("workspace", "default", "my_sample", &images[i]);
        ...
    }
    ...
    // execute batch processing
    status = vidi_runtime_batched_sample_process("workspace", "default", "Classify", "my_sample", "");
    ...
    // get result of batch processing
    status = vidi_runtime_get_batched_sample("workspace", "default", "my_sample", &result_buffer);
    ...

Copy
C# Example of Batch Processing API for Runtime (Green High Detail)

    ...
    // The maximum batch size depends on the available GPU memory size.
    // let's suppose the available maximum batch size is 12.
    int batch_size = 12;
    ...
    // you also need to prepare the same number of the images as the batch size
    // the paths list should contain the paths of images
    List<IImage> imgs = new List<IImage>();
    for (int i = 0; i < batch_size; ++i)
        imgs.Add(new LibraryImage(paths[i]));

    // create a sample for batch processing. Note that a sample is the basic unit of processing task.
    // "Classify": the name of the Green High Detail tool in your stream.
    using (IBatchedSample sample = stream.CreateBatchedSample())
    {
        // add images to sample
        for (int i = 0; i < batch_size; ++i)
            sample.AddImage(imgs[i]);

        ITool greenTool = stream.Tools["Classify"];

          // set batch size parameter
          var param = greenTool.ParametersBase as ViDi2.Runtime.IGreenHighDetailParameters;
        param.BatchSize = batch_size;

        // run processing
        sample.Process(greenTool);

        // get processing result of batch
        for (int i = 0; i < batch_size; ++i)
        {
            var tags = sample.Tags(greenTool, i);
            foreach (var tag in tags)
                Console.WriteLine($"tag: name={tag.Key}, score={tag.Value}");
        }
    }

Note: Batch processing should not be used in a tool chain. If you want to use batch processing in a chain of tools, create a runtime workspace for each tool and chain them.

### NVIDIA TensorRT Support for Runtime API

NVIDIA TensorRT is an NVIDIA SDK that boosts the inference speed of a deep learning application. It optimizes the inference speed of a deep neural network by NVIDIA GPU model. From VisionPro Deep Learning 3.0, Runtime API supports TensorRT to boost the processing speed at Runtime. To exploit TensorRT for Runtime API, go through the following steps. The TensorRT for Runtime API is supported only for Red Analyze High Detail and Green Classify High Detail runtime environment.

1. In the training environment, create a workspace, stream, and a High Detail tool. Train the tool to make it ready to be deployed in the runtime environment. The training environment can be either VisionPro Deep Learning GUI or API. You can create a workspace, stream, and a High Detail tool and train the tool in GUI. For the more details of the training and runtime environment, see Environments.

2. Export a runtime workspace which includes the trained High Detail tool. You also can create a runtime workspace in GUI (Workspace > Export Runtime Workspace). For the more details of the runtime workspace, see Runtime Workspace.

3. Deploy the runtime workspace and its High Detail tool on the device on your front line. For the more details of the runtime deployment, see Runtime Workspace and Runtime Deployment.

4. Run TensorRT Optimization API to optimize the High Detail tools in your runtime workspace with TensorRT, and save your runtime workspace. The optimization normally takes under 10 minutes to be completed. After the first run of optimization, the tool's neural network is optimized to the specific NVIDIA GPU model you use, and from this point, you can process as many images you want with faster speed.

   Copy

   C++ Example of TensorRT Optimization API for Runtime (Red High Detail and Green High Detail)

       1
       2
       3
       4
       5
       6
       7
       8
       9
       10
       11
       12
       13
       14
       15
       16
       17
       18
       19
       ...
       // open the given workspace.
       // "workspace": the name of your runtime workspace
       //"..\\..\\resources\\runtime\\Green High-detail Tool.vrws": the path to the runtime workspace file
       status = vidi_runtime_open_workspace_from_file("workspace", "..\\..\\resources\\runtime\\Green High-detail Tool.vrws");
       ...

       // optimize the High-detail tool
       // "default": the name of the stream in your workspace
       // "Classify": the name of the Green High Detail tool in your stream.
       clog << "Start optimization. It will take a few minutes." << endl;
       status = vidi_runtime_tool_convert_trt("workspace", "default", "Classify", 0);
       ...

       // save runtime workspace with the optimized tool
       string save_path = "..\\..\\resources\\runtime\\Green High-detail Tool optimized.vrws";
       status = vidi_runtime_save_workspace("workspace", save_path.c_str());
       clog << "The workspace with the optimized tool is saved at " << save_path.c_str() << endl;
       ...

   Copy
   C# Example of TensorRT Optimization API for Runtime (Red High Detail and Green High Detail)

       1
       2
       3
       4
       5
       6
       7
       8
       9
       10
       11
       12
       13
       14
       15
       16
       17
       18
       19
       20
       21
       22
       ...
       // Open a runtime workspace from file
       // the path to this file relative to the example root folder
       // and assumes the resource archive was extracted there.
       // "workspace": the name of your runtime workspace
       ViDi2.Runtime.IWorkspace workspace = control.Workspaces.Add("workspace", "..\\..\\..\\..\\resources\\runtime\\Green High-detail Tool.vrws");

       // Store a reference to the stream 'default'
       IStream stream = workspace.Streams["default"];

       // "Classify": the name of the Green High Detail tool in your stream.
       var greenTool = stream.Tools["Classify"] as ViDi2.Runtime.IGreenTool;

       // Optimizes the High-Detail tool with TensorRT. It takes some time to optimize the tool.
       greenTool.OptimizeTensorRT(0);

       // Save runtime workspace with the optimized tool.
       // You can use this workspace to process with the optimized tool.
       string savePath = "..\\..\\..\\..\\resource\\runtime\\Green High-detail Tool Optimized.vrws";
       workspace.Save(savePath);
       Console.Write($"The workspace with the optimized tool is saved at {savePath}.");
       ...

   If you want to use the Classification Batch Processing for Runtime API together with TensorRT-optimized processing, you need to set batch size before calling the optimization.

   Copy
   C++ Example of Setting Batch Size Before Calling TensorRT Optimization (Green High Detail)

       1
       2
       3
       4
       5
       6
       // Optimize with batchSize=16
       // The 5th(width) and 6th(height) parameter of vidi_runtime_tool_convert_trt() are for target width and height for optimization
       // If width=0 or height=0 is provided, the trained size is used. So we just recommend use width=0, height=0 in normal case.
       int gpu_index = 0;
       int batch_size = 16;
       vidi_runtime_tool_convert_trt("workspace", "default", "Analyze", gpu_index, 0, 0, batch_size);

   Copy
   C# Example of Setting Batch Size Before Calling TensorRT Optimization (Green High Detail)

       1
       2
       3
       4
       5
       6
       // Optimize with batchSize=16
       // The second(width) and third(height) parameter of OptimizeTensorRT() are for target width and height for optimization
       // If width=0 or height=0 is provided, the trained size is used. So we just recommend use width=0, height=0 in normal case.
       int gpuIndex = 0;
       int batchSize = 16;
       greenTool.OptimizeTensorRT(gpuIndex, 0, 0, batchSize);

   Directory to C++ Full Example Codes:

   C:/ProgramData/Cognex/VisionPro Deep Learning/3.0/Examples/c++/Example.Runtime.OptimizeHDTool

   C:\ProgramData\Cognex\VisionPro Deep Learning\3.0\Examples\c++\Example.Runtime.HDGreen.Batched

   Directory to C# Full Example Codes:

   C:/ProgramData/Cognex/VisionPro Deep Learning/3.0/Examples/c#/Example.Runtime.OptimizeHDTool.Console

   C:\ProgramData\Cognex\VisionPro Deep Learning\3.0\Examples\c#\Example.Runtime.HDGreen.Batched.Console

5. Load your runtime workspace in your application and deploy it on the device to process images with the TensorRT-optimized tool. To do this, you need to set a parameter before processing.

   Copy
   C++ Example of Setting Parameter to Use TensorRT Optimized-model for Processing (Red High Detail and Green High Detail)

       1
       2
       3
       4
       5
       // If you want to process with optimized tool, you have to set ProcessTensorRT to true.
       // Setting this option to true will trigger TensorRT processing.
       status = vidi_runtime_tool_set_parameter("workspace", "default", "Analyze", "runtime_parameters/process_with_trt", "true");
       ...
       status = vidi_runtime_sample_process("workspace", "default", "Analyze", "my_sample", "");

   Copy
   C# Example of Setting Parameter to Use TensorRT Optimized-model for Processing (Red High Detail and Green High Detail)

       1
       2
       3
       4
       5
       6
       // If you want to process with optimized tool, you have to set ProcessTensorRT to true.
       // Setting this option to true will trigger TensorRT processing.
       var hdParam = hdTool.ParametersBase as ViDi2.Runtime.IToolParametersHighDetail;
       hdParam.ProcessTensorRT = true;
       ...
       sample.Process(hdTool);

   See the example codes below to process images with a runtime workspace.

   Directory to C# Example Codes:

   - C:/ProgramData/Cognex/VisionPro Deep Learning/3.0/Examples/c#/Example.Runtime.HDGreen.Console

   - C:/ProgramData/Cognex/VisionPro Deep Learning/3.0/Examples/c#/Example.Runtime.HDRed.Console

   Directory to C++ Example Codes:

   - C:/ProgramData/Cognex/VisionPro Deep Learning/3.0/Examples/c++/Example.Runtime.HDGreen

   - C:/ProgramData/Cognex/VisionPro Deep Learning/3.0/Examples/c++/Example.Runtime.HDRed

   Note: TensorRT Optimization API (vidi_runtime_tool_convert_trt, .OptimizeTensorRT) should be called again after your NVIDIA GPU model is changed because its optimization works by NVIDIA GPU model. If you changed your NVIDIA GPU model in your device, repeat the Step 4 and 5.

## VisionPro Deep Learning 2.1.1

In VisionPro Deep Learning 2.1.1, a new processing parameter max_scan_iterations is added only in API for Blue Locate and Blue Read. This parameter limits the maximum number of iterations for image scanning during processing and providing a fixed number for this parameter speeds up the processing.

Note: The features or the character features can be less detected than expected when the value of max_scan_iterations is set too low. The appropriate value for this parameter is dependent on the images.

You can get the value of this parameter by:

- vidi_runtime_get_parameter

- vidi_runtime_tool_get_parameter

You can change the value of this parameter by:

- vidi_runtime_set_parameter

- vidi_runtime_tool_set_parameter

The parameter path for max_scan_iterations is:

- processing/blue/max_scan_iterations

### Examples

This topic introduces the code examples of max_scan_iterations parameter for C and .NET API

Symbol | Definition
--- | ---
WORKSPACE | The name of your workspace.
STREAM | The name of the stream in your workspace.
TOOL | The name of the tool in the stream.
40 | The value of the parameter.

Copy

C API

    1
    VIDI_UINT status = vidi_runtime_tool_set_parameter("WORKSPACE", "STREAM", "TOOL", "processing/blue/max_scan_iterations", "40");

Copy

.NET API

    1
    libraryAccess.SetToolParameter("WORKSPACE", "STREAM", "TOOL", "processing/blue/max_scan_iterations", "40"); // libraryAccess is ILibraryAccess

## VisionPro Deep Learning 2.1

As High Detail Quick modes are added in VisionPro Deep Learning 3.0, there are some changes to API as well.

### Green Classify High Detail Quick Training - C API

Copy

Green High Detail Quick Training with C API

    1
    2
    3
    4
    5
    6
    // Green high-detail-quick mode
    ...
    param_ss << "workspaces/" << workspace_name << "/streams/" << stream_name << "/tools/" << tool_name << "/tool_type";
    status = vidi_training_set_parameter(param_ss.str().c_str(), "green_quick");
    ...

To train Green Classify High Detail Quick, the parameter path should be set to "green_quick" for vidi_training_set_parameter.

The example of parameter path:

- workspaces/WORKSPACE_NAME/streams/STREAM_NAME/tools/TOOL_NAME/tool_type

Symbol | Definition
--- | ---
WORKSPACE_NAME | The name of your workspace.
STREAM_NAME | The name of the stream in the workspace.
TOOL_NAME | The name of the tool in the stream.

To configure the Tool Parameters for Green Classify High Detail Quick, the parameter path also should be set. For example, if you want to change Epoch Count in Training Parameters, set the parameter path as:

- workspaces/WORKSPACE_NAME/streams/STREAM_NAME/tools/TOOL_NAME/training_parameters/count_epochs

Symbol | Definition
--- | ---
WORKSPACE_NAME | The name of your workspace.
STREAM_NAME | The name of the stream in the workspace.
TOOL_NAME | The name of the tool in the stream.

See C:\ProgramData\Cognex\VisionPro Deep Learning\2.1\Examples\c++\Example.Training.HDGreen\example_training_hdgreen.cpp for the more detailed examples of training C API codes.

### Green Classify High Detail Quick Training - .NET API

Copy

Green High Detail Quick Training with .NET API

    1
    2
    3
    4
    5
    ...
    // Green high-detail-quick mode
    hdGreenTool.Database.Tool.Type = ToolType.GreenQuickHighDetail;
    ...

To train Green Classify High Detail Quick, the ViDi2.Training.ITool.Type should be set to ToolType.GreenQuickHighDetail.

To configure the Tool Parameters for Green Classify High Detail Quick, the ViDi2.Training.IGreenHighDetailParameters should be set. For example, you should set the value of CountEpochs for IGreenHighDetailParameters.

See C:\ProgramData\Cognex\VisionPro Deep Learning\2.1\Examples\c#\Example.Training.HDGreen.Console\Program.cs for the more detailed examples of training .NET API codes.

### Green Classify High Detail Quick Processing - C API

The way to process Green Classify High Detail Quick with API is the same as those of Green Classify Focused and Green Classify High Detail. See C:\ProgramData\Cognex\VisionPro Deep Learning\2.1\Examples\c++\Example.Runtime.HDGreen\example_runtime_hdgreen.cpp for the details.

### Green Classify High Detail Quick Processing - .NET API

The way to process Green Classify High Detail Quick with API is the same as those of Green Classify Focused and Green Classify High Detail. See C:\ProgramData\Cognex\VisionPro Deep Learning\2.1\Examples\c#\Example.Runtime.HDGreen.Console\Program.cs for the details.

### Red Analyze High Detail Quick Training - C API

Copy

Red High Detail Quick Training with C API

    1
    2
    3
    4
    5
    6
    ...
    // Red high-detail quick mode
    param_ss << "workspaces/" << workspace_name << "/streams/" << stream_name << "/tools/" << tool_name << "/tool_type";
    status = vidi_training_set_parameter(param_ss.str().c_str(), "red_quick");
    ...

To train Red Analyze High Detail Quick, the parameter path should be set to "red_quick" for vidi_training_set_parameter.

The example of parameter path:

- workspaces/WORKSPACE_NAME/streams/STREAM_NAME/tools/TOOL_NAME/tool_type

Symbol | Definition
--- | ---
WORKSPACE_NAME | The name of your workspace.
STREAM_NAME | The name of the stream in the workspace.
TOOL_NAME | The name of the tool in the stream.

To configure the Tool Parameters for Red Analyze High Detail Quick, the parameter path also should be set. For example, if you want to change Epoch Count in Training Parameters, set the parameter path as:

- workspaces/WORKSPACE_NAME/streams/STREAM_NAME/tools/TOOL_NAME/training_parameters/count_epochs

Symbol | Definition
--- | ---
WORKSPACE_NAME | The name of your workspace.
STREAM_NAME | The name of the stream in the workspace.
TOOL_NAME | The name of the tool in the stream.

See C:\ProgramData\Cognex\VisionPro Deep Learning\2.1\Examples\c++\Example.Training.HDRed\example_training_hdred.cpp for the more detailed examples of training C API codes.

### Red Analyze High Detail Quick Training - .NET API

Copy

Red High Detail Quick Training with .NET API

    1
    2
    3
    4
    ...
    // Red high-detail quick mode
    hdRedTool.Database.Tool.Type = ToolType.RedQuickHighDetail;
    ...

To train Red Analyze High Detail Quick, the ViDi2.Training.ITool.Type should be set to ToolType.RedQuickHighDetail.

To configure the Tool Parameters for Red Analyze High Detail Quick, the ViDi2.Training.IRedHighDetailParameters should be set. For example, you should set the value of CountEpochs for IRedHighDetailParameters.

See C:\ProgramData\Cognex\VisionPro Deep Learning\2.1\Examples\c#\Example.Training.HDRed.Console\Program.cs for the more detailed examples of training .NET API codes.

### Red Analyze High Detail Quick Processing - C API

The way to process Red Analyze High Detail Quick with API is the same as those of Red Analyze Focused Supervised and Red Analyze High Detail. See C:\ProgramData\Cognex\VisionPro Deep Learning\2.1\Examples\c++\Example.Runtime.Red\example_runtime_red.cpp for the details.

### Red Analyze High Detail Quick Processing - .NET API

The way to process Red Analyze High Detail Quick with API is the same as those of Red Analyze Focused Supervised and Red Analyze High Detail. See C:\ProgramData\Cognex\VisionPro Deep Learning\2.1\Examples\c#\Example.Runtime.HDRed.Console\Program.cs for the details.