How much memory does Mistral Large 4 need?
The weights alone need BF16 (2,100 GB), FP8 (1,050 GB) and 4-bit (525 GB). Memory for the context (the KV cache) and the inference software comes on top, so a setup that only just fits will struggle.
Sources: Mistral's model card