Inference worker
The inference worker runs on a separate Bazzite GPU workstation rather than the stable Rocky Linux and K3s server. This keeps NVIDIA drivers, large model files, and desktop GPU pressure out of the application host. It also creates an intentional availability boundary: classification can pause while the workstation is used for gaming or maintenance without disturbing the rest of the homelab.
The worker exposes a private, authenticated OpenAI-compatible llama.cpp API. Rootless Podman runs a pinned CUDA-enabled image from a per-user systemd Quadlet, with a checksum-pinned Qwen3.8-27B IQ4-class GGUF mounted read-only.
The API binds to a stable private address, requires an API key, and has neither a router forward nor a public Caddy route. The key authenticates requests but does not encrypt plain HTTP, so that transport is acceptable only on the trusted LAN or WireGuard path.
On-demand lifecycle
The selected profile consumes most of the workstation's 16 GB of GPU memory when loaded, so inference is started for classification work rather than kept resident on a shared desktop GPU. modelctl is the operator interface for the registered model service: it starts, stops, restarts, reports status, and reads the service journal without introducing a second container lifecycle.
The Quadlet deliberately has no boot activation target. It gives user systemd a service that can be started explicitly, but it does not attach that service to the user's default target. Enabling systemd user lingering keeps the user manager alive after the final desktop or SSH session ends; it does not create a dependency that loads the model at boot.
For a timed run, modelctl asks systemd-run --user to create a transient auto-stop timer. The expiry therefore belongs to the lingering user systemd manager rather than to a shell sleep process or the SSH connection that initiated it. A later timed start replaces the existing expiry instead of stacking timers.
Service activation is only the first stage of startup. modelctl polls the private health endpoint while a cold model loads and reports the model ready only after that endpoint responds successfully. The auto-stop timer is committed after readiness, so its duration is not consumed by model loading. If a new service fails to become ready or the wait is interrupted, modelctl stops the service created by that invocation and leaves no timer from it. A service or timer that existed before the command is preserved.
Container and GPU boundary
NVIDIA CDI supplies the GPU device to rootless Podman. During initial integration, the service loaded its model and answered health checks while startup logs showed CUDA unavailable and GPU layers ignored. A healthy HTTP endpoint therefore proved that the server was reachable, but not that inference was running on the GPU.
Adding SecurityLabelDisable=true, matching the NVIDIA CDI requirement for this Podman path, resolved GPU handoff. That setting disables SELinux separation for the container, so the deployment compensates by pinning the image, mounting only the model directory and doing so read-only, avoiding unrelated host mounts, and keeping the authenticated API on the private network with no public ingress path.
Validation evidence
The selected profile uses an NVIDIA RTX 5070 Ti with 16 GB of memory, an 8,192-token context, one inference slot, and text-only operation. Local health and authenticated chat completion passed. On 2026-09-26, the GPU reported 14,067 MiB used and 1,798 MiB free with Qwen loaded at 8K, confirming GPU-backed execution rather than CPU fallback. The private health endpoint was also reachable from both the Rocky Linux host and a Linkwarden pod.
A short timed run loaded the model, retained its transient systemd timer after the initiating SSH connection exited, and stopped automatically at expiry. This exercised the boundary between the remote operator session and the lingering user manager instead of assuming that a scheduled shell process would survive logout.
Those measurements establish cross-host reachability and GPU execution; they do not establish latency, throughput, repeated-request stability, or high availability. Validation therefore treats driver state, CDI discovery, disposable-container GPU visibility, model checksum, generated Quadlet, service state, GPU memory use, health, and authenticated completion as distinct layers. An active service is insufficient when its logs show CPU fallback.