# GPU Modes with a ViDi Server

The client/server functionality allows up to eight client PCs to make use of a single server for both training and processing. This allows the clients to share the use of one or more GPUs installed on the server.

The ViDi Suite server provides exactly the same capabilities and configuration options as the ViDi Suite GUI, running locally, in regard to making use of multiple GPUs.

- To use multiple GPUs, they must all be of the same type and have the same amount of memory.
- The maximum number of GPUs that can be used by a server is four.
- Individual clients can request a specific GPU device when processing a stream.
- A single ViDi server can be configured to provide a training service (to support clients running the ViDi Suite GUI), a run-time service (to support clients processing images through streams using the run-time interface), or both training and runtime services. Regardless of the origin of the request (training or run-time service), the requests are placed in a single FIFO queue and processed in order, using the first available GPU device.

The GPU Mode on the server is set when the ViDi Suite Service is started, and command line arguments can be used to specify the GPU Mode.

SingleDevicePerTool GPU Mode

By default, the service is configured to use the SingleDevicePerTool GPU Mode. In this mode, each time a tool is processed, a GPU is locked for the duration of tool processing. The GPU is released after the tool is processed.

When a multiple-tool stream is processed, each tool is processed sequentially. If there are no available GPUs when the next tool in the stream is ready to be run, the processing of the stream stops until a GPU is available. If there are already other processing requests in the queue, the tool will be added to the end of the queue.

When a stream is configured with two tools using the same upstream tool as an input, the tools are run sequentially, not in parallel, and stream processing will block other tools until both tools have been processed.

MultipleDevicesPerTool GPU Mode

If the server is configured for MultipleDevicesPerTool GPU Mode, all of the GPUs are treated as a single GPU, all of the time. This means that only one tool can be processed at a time for the entire server.

In the specific case of a Red Analyze Tool, the use of MultipleDevicesPerTool mode may speed up the tool, especially a tool with a high image-to-feature size ratio. However, this potential speed up comes at the expense of latency across all clients.