GPU Memory Optimization

The GPU Memory Optimization feature (which is activated by default, using the default value of 2GB) provides significant speed improvement with Windows Display Driver Model (WDDM) drivers, as well as with the Tesla Compute Cluster (TCC) drivers. However, the reserved memory must be carefully selected, depending on the application. The highest gain in performance is for applications that process small images.

 

Pros Cons

Reserves memory on the graphics card

Difficult to estimate the correct amount of memory to reserve

No more time lost for memory allocation

 

  • If you turn on this option, the system pre-allocates GPU memory for tool optimization. It is recommended to turn on this option when using Focused mode tools (Red Analyze Focused - Supervised/Unsupervised, Blue Locate, Blue Read, Green Classify Focused) to speed up training and processing.

  • If you turn off this option, the system stops to pre-allocate GPU memory. It is recommended to turn off this option in training High Detail modes and High Detail Quick modes (Green Classify High Detail, Red Analyze High Detail, Red Analyze High Detail Quick, Green Classify High Detail Quick) that it slows down their training speed.

Note: It is not recommended to enable Optimized GPU Memory (Help - Compute Devices) for training multiple tools of High Detail/High Detail Quick and Focused mode within a stream. This mix up of 2 or more modes in training greatly suffer from enabling Optimized GPU Memory in terms of speed, regardless of the number of GPUs you use.

 

You can deactivate the feature or change the allocation settings via the API or through command line arguments.

For example, in the .NET API, you would set control.OptimizedGPUMemory(2.5*1024*1024*1024ul);; in the C API, you would set vidi_optimized_gpu_memory(2.5*1024*1024*1024);

 

Graphic Card Memory

 

VisionPro Deep Learning Memory Usage:

  • Display: memory to store images prior to computation
  • VisionPro Deep Learning Reserved: memory reserved for computation
  • Free: memory left free, because the amount of memory used is not known

An example for training:

  • Reserved memory at 2.5GB (the minimal value for training)
  • Free memory at about 1GB
  • Which results in a graphics card with at least 3.5GB of memory

An example for runtime:

  • Reserved memory at 512MB
  • Free memory at about 256MB
  • Which results in a graphics card with no minimal memory required