How many Universe environments could a NVIDIA DGX-1 handle at the same time without slowind down too much? Could it handle something like 100 or 1000 environments, if yes how much slow will it be? It would be nice to make a distributed deep learning development environment with the DGX-1
CUDA is Like Owning a Supercomputer:
The main limitation would be the amount of GPU memory available. The DGX-1 with 8 GPUs has 128GB. So let’s say your neural network has N parameters, each taking up 4 bytes (float32), you can calculate how many parallel models you can fit on your hardware (theoretically). The actual number will be slightly lower than this since memory is also allocated to other tasks.
Nvidia DGX-2 is 2 petaflop AI supercomputer for $399,000: