Reichelt, David Georg and Abitz, Daniel and Gross, Jonathan and Kühne, Stefan (2025) LLMs on Affordable GPUs : A Benchmarking Study. Softwaretechnik-Trends, 45 (4). pp. 23-25.
Full text not available from this repository.Abstract
A variety of Large Language Models (LLMs) exist, which have different characteristics regarding the quality of their answers and their resource and time consumption. For most of the LLMs, the required hardware is costly. Nevertheless, to achieve data sovereignty and privacy, hosting LLMs locally is necessary. Due to the high costs, it is crucial to choose LLMs and configurations that are suitable for the given environment. In this benchmarking study, we examine how to provide LLMs for researchers in the context of a local university computing centre. To do so, we extended an implementation of the LLM-asa-judge principle, and executed it on our infrastructure. Our extended implementation can be used to decide what to host on different hardware, and to decide which new models may fit better.