Maintenance
Maintenance keeps a change understandable: identify the current state, preserve a recovery path, make the intended change, and inspect the result. Service availability and security evidence both matter. An application may respond successfully while a configuration difference still needs review.
Lifecycle
The operating system package manager covers only part of the environment. K3s, Traefik TLS state, Kubernetes workload images, classifier dependencies, model files, and the inference container each have their own lifecycle.
Each lifecycle begins with discovery, captures backup or recovery evidence where state can change, validates inputs before mutation, checks stability afterward, and retains a rollback path. Mutable image tags are avoided where recovery identity matters. An application image rollback is never assumed to reverse database or search-index changes, and recovery artifacts remain until the upgraded service has passed its observation period.
Bootstrap and maintenance are separate. Bootstrap creates an absent component and refuses to replace an existing installation silently. Maintenance starts from discovered live state and owns upgrades, migrations, rollback, and recovery evidence.
Review the result of a change
After maintenance, use the component's functional checks and inspect the security report or a fresh doctor diagnosis. Start with the observation time and findings requiring review so an older healthy report is not mistaken for evidence about the new state.
For configuration or trusted local code covered by AIDE, compare integrity differences with the authorized work. For example, replacing a reviewed maintenance tool can change its hash, inode and timestamps, along with its parent directory's metadata. Those differences need an explanation even when the tool runs successfully. Linux Audit can help investigate selected operations through available process and login attribution.
Once the changes are understood, review a candidate AIDE baseline, explicitly approve it, and require a clean check against that new state. An unrelated difference remains an investigation item. The report and doctor provide observations for this decision; approval belongs to the AIDE lifecycle.
Follow the component lifecycle
| Component | Maintenance question |
|---|---|
| K3s | Which cluster, storage and TLS state must survive this change? |
| Linkwarden | Are application, PostgreSQL and Meilisearch versions compatible, and is the required migration recoverable? |
| Calagopus | Do the Panel and Wings remain compatible across their separate update paths? |
| Inference worker | Is the model ready, and is inference actually using the GPU after a driver, container or model change? |
| USBGuard | Will the required USB devices and network recovery path remain usable after a policy change? |
Troubleshooting
Diagnosis proceeds from the outside inward and stops at the first failed boundary:
- DNS and router forwarding.
- Host firewall, native nftables enforcement, and the ServiceLB TCP 443 path.
- Traefik listener, certificate, router, and upstream selection.
- K3s node, namespace, Service, Deployment, pod, and events.
- PVC binding and filesystem or SELinux state.
- Application logs and dependency health.
- Public HTTPS response and an authenticated functional check where applicable.
This order prevents network, ingress, orchestration, storage, and application failures from being diagnosed as one undifferentiated outage. The same principle applies within components: CrowdSec detection is separated from firewall enforcement, and inference health is separated from actual GPU execution.