Linkwarden AI Tagger
The project began with an imported bookmark library heavily concentrated in a top-level Unorganized collection. Classification needed stable destinations, consistent metadata, useful human review, and a mutation path that could not treat plausible model output as trusted state.
The classifier asks a private model to propose a collection and tags. Code checks the proposal against the approved taxonomy, and the default run produces a report for review. An explicitly authorized apply runs inference again, writes eligible high-confidence results and verifies them in Linkwarden. Across three recorded applies, it updated 50 distinct bookmarks. The reports also show what it skipped and why.
A controlled taxonomy
The taxonomy is source-controlled Markdown rather than prompt text or live application state. One file owns the collection hierarchy, another owns canonical tags, aliases, roles, ordering, and selection rules, and a rulebook owns classification and workflow policy. Python code parses and validates these sources, so prompts and synchronization consume the same representation instead of maintaining parallel lists.
Collections answer where a bookmark primarily belongs. Tags carry reusable detail such as platform, content type, domain, discipline, technology, concept, or qualifier. The classifier may select only exact existing collection paths and canonical tags. Aliases are normalized through explicit mappings; fuzzy matching is not allowed to turn a near-spelling into a write.
Unorganized remains a workflow fallback for unclear subjects, taxonomy gaps, or cases where forcing a destination would be misleading. The model may propose one collection and up to three missing tags for review, but suggestions never enter the applied tag list and never create live taxonomy as a side effect.
Separating API access from policy
The generic Linkwarden API client authenticates, reads raw owned collections and bookmarks, updates links, and verifies returned state. It does not decide which bookmarks are eligible or how confidence changes behavior.
Classifier policy is split again by responsibility:
- pure validation checks model output against the loaded taxonomy and schema;
- inference owns private worker requests and response extraction;
- workflow owns eligibility, queue state, writes, verification, and recovery;
- reporting owns machine-readable audit records and deterministic review documents.
That dependency direction also lets taxonomy synchronization reuse the Linkwarden API without importing classifier behavior. Network access does not implicitly authorize policy decisions, and policy validation does not perform filesystem or remote mutation.
The inference boundary
The inference worker provides a private authenticated llama.cpp API. The classifier prefers complete Linkwarden-preserved text when it is usable. Missing, unusable, or oversized text triggers available saved HTML, then saved PDF, before one controlled public GET of the original page. A protected page can block direct retrieval. The evidence pipeline does not render pages in a browser or bypass CAPTCHA and access challenges.
The model's 8,192-token context must fit the instructions, taxonomy, bookmark evidence and room for an answer. When a usable document cannot fit in one request, the classifier can spend additional inference time to cover it in chunks. It counts tokens for each request, extracts compact evidence from each region and combines those results for the final decision. If usable page evidence is unavailable, it falls back to bookmark metadata.
The classifier sends the selected evidence, bookmark metadata, and controlled policy, then expects one structured JSON object. Bookmarked content is untrusted input: instructions embedded in it do not override classification policy.
The model returns an exact collection path, controlled permanent tags, confidence, uncertainty, a recommended action, a reason, optional provenance evidence, and review-only taxonomy suggestions. The inference response is a proposal until deterministic code accepts it.
Processing every chunk establishes how much text the classifier read. The text can still be wrong for the bookmark. For example, a saved article may lead to a generic product homepage on the same domain. The model is asked to flag that mismatch so the workflow can try another source. If the final decision relies on metadata alone, an eligible medium-confidence result gets one focused reassessment of the destination and tags that the metadata supports. Confidence can rise, stay the same or fall. The usual write checks still apply.
Deterministic validation
Validation rejects unknown fields, non-existent collections, uncontrolled tags, workflow tags selected as permanent metadata, invalid confidence/uncertainty combinations, and malformed suggestions. High confidence with no uncertainty uses an acceptance recommendation. High confidence with evidence-limited uncertainty may remain applicable when the available evidence supports its destination and tags. A high-confidence taxonomy gap selects Unorganized, proposes a taxonomy change, and remains review-only. Medium and low confidence must identify a concrete uncertainty and a human action that can resolve it.
Canonical permanent tags are limited to nine. Exact duplicates may be removed and valid tags may be reordered by their defined roles; both normalizations are recorded. The application does not use fuzzy matching or partial writes to salvage invalid output. Failed output remains available for targeted retry or review.
One bounded repair request may correct a structured-output contract violation against the same completed evidence. The repair is attempted at most once, must pass normal validation, cannot increase confidence or remove genuine evidence problems, and does not re-fetch or reprocess the document. A reported taxonomy gap must remain blocked in Unorganized with a valid suggestion; repair may correct the suggestion structure without clearing the gap. A successfully repaired result follows the same eligibility rules as any other validated result.
This boundary keeps model reasoning flexible while making accepted state changes finite and mechanically checkable.
Dry-run review and fresh apply
The workflow uses a dry run to review classifier behavior before mutation. It selects the oldest eligible batch, classifies it, generates reports, and changes no Linkwarden state. If the methodology needs adjustment, oldest-first ordering allows the same stable batch to test the revised behavior.
Apply classifies the oldest eligible bookmarks again using live data. A write requires high confidence, an existing approved collection, canonical tags, no blocking uncertainty or evidence conflict, and a successful freshness check. Medium-confidence, low-confidence and blocked recommendations stay in the report for human review.
Blocking conditions include taxonomy gaps, insufficient metadata, ambiguous subjects, invalid taxonomy values, unresolved evidence conflicts, stale classification inputs, and write or verification failures. The evidence-processing path is not itself a blocker: complete multi-pass processing, complete direct evidence, and successful bounded repair can produce applicable high-confidence results.
evidence-limited means enough remaining evidence supports a defensible provisional classification, but important page content could not be verified. insufficient-metadata means the available evidence cannot support a defensible destination recommendation. Batch 1 — 20260927T030125Z contains an evidence-limited result where preserved content was unavailable and direct retrieval received HTTP 403.
Guarded writes and stale-state protection
Consider a bookmark whose saved PDF changes while the model is deciding where it belongs. Before it starts writing, the workflow reads the bookmark again and checks its metadata and evidence. For a saved archive, it compares the archive reference, the hash of the bytes and the hash of the extracted text. A mismatch stops the write even if the title and URL still match. This check and the subsequent write are separate API requests, so another edit can still occur between them.
For an eligible result, the workflow records the collection and tags, then adds and verifies a reclassify tag if the bookmark does not already carry it. This gives the bookmark a known state from which classification can be retried. The final write selects the approved collection, keeps existing permanent tags, adds suggested tags that fit within the nine-tag limit and removes the workflow markers.
The update is followed by a fresh read that verifies the complete remote collection and tag state. A successful HTTP request alone is not considered a successful mutation.
Recovery behavior
The recovery target depends on which write failed. If adding or verifying the queue tag fails, the workflow attempts to restore the original collection and tags. Once the bookmark is queued, a failed or interrupted final write or a failed verification triggers an attempt to restore the verified queued state, including reclassify. In either case the workflow reads the restored state back. If it cannot restore and verify the target state, it records both failures, marks the remote state uncertain and stops the batch.
This is recovery for classifier-owned Linkwarden state, not a database restore. PostgreSQL backup, Meilisearch migration, and workload reconstruction remain separate responsibilities of the Linkwarden architecture.
Audit and reproducibility
Every run gets a timestamped local report directory. Machine-readable JSONL records hold classifications, errors, write outcomes, and—when apply mode is used—pre-mutation rollback state. Deterministically rendered Markdown groups low and medium confidence ahead of high-confidence results so human work is visible first.
The active classifier reads live Linkwarden data and records whether mutation was authorized. Run metadata includes lifecycle status and counts, the hashes of the exact taxonomy and rulebook inputs, and Git revision and working-tree state when available. Historical records retain their original input provenance. A clean finish becomes completed; handled fatal errors and interruptions become aborted; an unclean termination can leave running, which means only that no clean completion was recorded.
Normal runtime report directories remain private or local and ignored. Machine-readable JSONL, rollback data, and operational telemetry are not published. Selected Markdown snapshots are preserved as public validation evidence, with operational links omitted. They record automation outcomes and model recommendations; later reports are not public by default.
Classification QA and the feedback loop
Medium- and low-confidence classifications are not failed high-confidence writes. They preserve uncertainty and a concrete recommended action for human review. Untouched Unorganized bookmarks can be selected again later, while an explicit reclassify workflow tag requests another pass after a wrong result, taxonomy change, or accepted suggestion.
Collection and tag suggestion reports group repeated gaps and attach supporting bookmark IDs and reasons. Frequency prioritizes review but does not approve a change. A human accepts, edits, or rejects a proposal; only a manual edit to the canonical taxonomy followed by validation and synchronization makes the new term available. Affected bookmarks are then selectively reclassified rather than rerunning the entire library.
The loop improves classification QA without granting the model authority to evolve taxonomy or mutate ambiguous results. Human decisions remain the source of structural change; automation makes those decisions consistent, reviewable, and repeatable.
Validation surface
Offline tests use mocked Linkwarden and inference communication. Coverage includes schema and taxonomy validation, live read-only behavior, candidate selection, explicit mutation entry, preservation of existing tags, stale-input refusal, medium/low report-only behavior, complete-state verification, restoration after queue/write/verification failures, fatal handling when recovery cannot be verified, audit lifecycle metadata, and taxonomy synchronization differences.
The validation results show which decisions led to writes on real bookmarks. The recovery tests exercise failure paths with substituted API responses. Correctness of a bookmark's subject and tags still calls for human judgment.
Experiments
GLiClass was evaluated within this project as a lightweight read-only classifier, then retired and removed after real-bookmark testing. The GLiClass experiment report records the evaluation boundary, results, decision, and lessons learned.