Inference worker
The inference worker runs on a separate Bazzite GPU workstation rather than the stable Rocky Linux and K3s server. This keeps NVIDIA drivers, large model files, and desktop GPU pressure out of the application host. It also creates an intentional availability boundary: classification can pause while the workstation is used for gaming or maintenance without disturbing the rest of the homelab.
The worker exposes a private, authenticated OpenAI-compatible llama.cpp API. Rootless Podman runs a pinned CUDA-enabled image from a per-user systemd Quadlet. The profile described here mounts a checksum-pinned Qwen3.8-27B GGUF with Unsloth Dynamic IQ4_XS quantization read-only.
Linkwarden AI Tagger uses this API for model responses and owns classification policy, validation and writes to Linkwarden. The worker supplies inference capacity; the maintenance workflow connects its readiness and GPU checks with changes to the model, container or driver.
The API binds to a stable private address, requires an API key, and has neither a router forward nor a public ingress route. The key authenticates requests but does not encrypt plain HTTP, so that transport is acceptable only on the trusted LAN or WireGuard path.
On-demand lifecycle
The selected profile consumes most of the workstation's 16 GB of GPU memory when loaded, so inference is started for classification work rather than kept resident on a shared desktop GPU. modelctl is the operator interface for the registered model service: it starts, stops, restarts, reports status, and reads the service journal without introducing a second container lifecycle.
The Quadlet deliberately has no boot activation target. It gives user systemd a service that can be started explicitly, but it does not attach that service to the user's default target. Enabling systemd user lingering keeps the user manager alive after the final desktop or SSH session ends; it does not create a dependency that loads the model at boot.
For a timed run, modelctl asks systemd-run --user to create a transient auto-stop timer. The expiry therefore belongs to the lingering user systemd manager rather than to a shell sleep process or the SSH connection that initiated it. A later timed start replaces the existing expiry instead of stacking timers.
Service activation is only the first stage of startup. modelctl polls the private health endpoint while a cold model loads and reports the model ready only after that endpoint responds successfully. The auto-stop timer is committed after readiness, so its duration is not consumed by model loading. If a new service fails to become ready or the wait is interrupted, modelctl stops the service created by that invocation and leaves no timer from it. A service or timer that existed before the command is preserved.
Container and GPU boundary
NVIDIA CDI supplies the GPU device to rootless Podman. During initial integration, the service loaded its model and answered health checks while startup logs showed CUDA unavailable and GPU layers ignored. A healthy HTTP endpoint therefore proved that the server was reachable, but not that inference was running on the GPU.
On this worker, adding SecurityLabelDisable=true let the container use the GPU. That setting disables SELinux label separation for this container. Its image is pinned, its only host directory mount holds the model and is read-only, and its authenticated API stays on the private network.
A host-local lock coordinates workloads that share this inference capacity. A systemd service must acquire the lock before Qwen starts and holds it while Qwen needs it. The read-only GLiClass experiment used the same lock. If one held it, the other could not start using the same capacity. Desktop applications run outside this arrangement.
Validation evidence
The selected profile uses an NVIDIA RTX 5070 Ti with 16 GB of memory, an 8,192-token context and one text-only inference slot. On 2026-09-26, reported GPU memory use rose from 279 MiB with Qwen stopped to 14,067 MiB with it loaded, with 1,798 MiB reported free. The model weights, working memory for the request and the desktop all need room. The tagger handles a long document across several requests so it can stay within that measured profile.
Local health and authenticated chat completion passed. The Rocky Linux host and a Linkwarden Pod also reached the private health endpoint.
A short timed run loaded the model, retained its transient systemd timer after the initiating SSH connection exited, and stopped automatically at expiry. This exercised the boundary between the remote operator session and the lingering user manager instead of assuming that a scheduled shell process would survive logout.
Those measurements establish cross-host reachability and GPU execution; they do not establish latency, throughput, repeated-request stability, or high availability. Validation therefore treats driver state, CDI discovery, disposable-container GPU visibility, model checksum, generated Quadlet, service state, GPU memory use, health, and authenticated completion as distinct layers. An active service is insufficient when its logs show CPU fallback.