How Much Stress Can Your Server Handle When Self-Hosting LLMs? | Towards Data Science

Do you need more GPUs or a modern GPU? How do you make infrastructure decisions?