So, ~2x faster than 2080 Ti, which is 30% faster than 1080Ti.
2x VRAM.
The only real question for me is, will it support Multi-Instance-GPU?
Let me explain why this is important. Now usually when you train a network, you increase your batch-size to fit the VRAM and monitor your IO and GPU load to ensure saturation.
But if a GPU has 2x VRAM and is 2-3x faster than 1080Ti, then maybe you can have multiple instances of your model on you GPU (that matters only for models that do not scale with large batch-sizes easily).
The only problem is that:
- You cannot use DDP in PyTorch (usually it is faster than DP for 4+ devices), because:
DDP processes can be placed on the same machine or across machines, but GPU devices cannot be shared across processes.
- So you will have to invent something / change your code / or maybe even use their bleeding edge RPC functions;
If this function is available on 3090 ... then you could turn your GPU into 2-3 virtual GPUs and use it accordingly? That would be truly epic, especially for production use-cases (yeah I know about their SLA)! Also would be great for teamworking.
#hardware