Introduction to Embedding Small Language Models in Java for Mobile & Edge Devices

Embedding SLM models within Java Applications aims to demonstrate a Java-based approach to load and interact with SLMs, without the need for external services or APIs. We aim to evaluate multiple methods to enable the inference of SLMs from within Java. A Java Native approach to interfacing with SLMs reduces external dependencies, ensure portability, strengthens security and is a seamless fit for enterprise environments which are dominated by Java-based systems. By outlining practical trade-offs with examples, we aim to demonstrate that Java can serve as a self-contained platform for integrating language models at scale.