Wallabag AI Tagger

A small, self-contained worker that reads articles from the Wallabag API, classifies them through an OpenAI-compatible LiteLLM router, and adds up to a configurable number of tags.

The supplied defaults use:

• LiteLLM model: qwen-tasks
• Maximum content tags: 3
• Queue tag: ai:pending
• Safe startup: DRY_RUN=true
• Scheduler: five-minute interval

The project uses only Python's standard library. Its Docker image does not need to download Python packages while building.

How it works

1. You create a Wallabag tagging rule that adds ai:pending to new articles.
2. The worker queries entries carrying that tag.
3. Wallabag's already-extracted article HTML is converted to bounded plain text.
4. The article and the current taxonomy are sent to qwen-tasks through LiteLLM's /v1/chat/completions endpoint.
5. The JSON response is validated and sanitized locally.
6. The content tags are added without replacing manual tags.
7. ai:pending is removed only after a successful write.

Failures remain pending for retry. After MAX_ATTEMPTS, the worker adds ai:error and removes ai:pending. Ambiguous results receive ai:review. Status tags do not count toward MAX_TAGS and can be disabled by setting their configuration values to empty strings.

Quick start

1. Create a Wallabag API client

In Wallabag, open Settings → API clients management → Create a new client. Keep the client ID and secret. Wallabag's OAuth password flow also requires the username and password belonging to the entries being tagged.

2. Configure the project

cp .env.example .env
chmod 600 .env

Edit at least:

WALLABAG_URL=http://wallabag
WALLABAG_USERNAME=your-user
WALLABAG_PASSWORD=your-password
WALLABAG_CLIENT_ID=your-client-id
WALLABAG_CLIENT_SECRET=your-client-secret

LITELLM_BASE_URL=http://litellm:4000/v1
LITELLM_API_KEY=your-litellm-key
LITELLM_MODEL=qwen-tasks
MAX_TAGS=3

The Compose file joins the existing external Docker network named server. That matches a common reverse-proxy setup. Change SERVICES_NETWORK if Wallabag and LiteLLM share another network. If they do not share one, either attach both services to a common private network or use reachable HTTPS URLs.

3. Create the Wallabag rule

The project intentionally does not create or modify tagging rules. A suitable rule for ordinary HTTP(S) articles is:

Rule: url matches "http"
Tags: ai:pending

The configured value of PENDING_TAG must match the tag used in the rule.

4. Check both APIs

docker compose build
docker compose run --rm wallabag-ai-tagger check

The check authenticates with Wallabag, reads its tag list, calls LiteLLM's /models endpoint, and reports whether qwen-tasks is visible.

5. Test without writing

Leave DRY_RUN=true, add a test article to Wallabag, and run:

docker compose run --rm wallabag-ai-tagger once

The proposed tags are printed directly by that command. Once the persistent service is running, follow later executions with docker compose logs -f wallabag-ai-tagger.

Dry-run decisions are remembered in the persistent tagger-data Docker volume, preventing the same unchanged entry from being sent repeatedly. Changing PROMPT_VERSION or the article contents makes it eligible again.

6. Enable writes

Set:

DRY_RUN=false

Then start the scheduled service:

docker compose up -d --build
docker compose logs -f wallabag-ai-tagger

Scheduling

Scheduling is entirely configurable and has three modes.

Fixed interval

RUN_MODE=interval
RUN_ON_START=true
INTERVAL_SECONDS=300

The worker runs immediately and then five minutes after each completed run.

Cron expression

RUN_MODE=cron
RUN_ON_START=false
CRON_EXPRESSION="0 */2 * * *"
SCHEDULE_TIMEZONE=Europe/Rome

This runs every two hours in the configured timezone. Standard five-field cron expressions are supported, including lists, ranges, steps, English month names, and English weekday names. When both day-of-month and weekday are restricted, standard cron OR semantics are used.

Examples:

# Every 15 minutes
CRON_EXPRESSION="*/15 * * * *"

# Every day at 03:30
CRON_EXPRESSION="30 3 * * *"

# Monday through Friday at 21:00
CRON_EXPRESSION="0 21 * * 1-5"

One-shot / external scheduler

RUN_MODE=once
RUN_ON_START=true
RESTART_POLICY=no

Or leave the normal service stopped and invoke this from cron or a systemd timer:

docker compose run --rm wallabag-ai-tagger once

MAX_BATCHES_PER_RUN controls how much work one scheduled execution performs:

• 1: one BATCH_SIZE batch per run.
• Any positive number: at most that many batches.
• 0: drain all currently reachable pending entries.

Retries for a failed entry occur on later scheduled executions, not in a tight loop during the same run.

Tag taxonomy

config/tags.txt accepts one canonical tag per line. Blank lines and lines starting with # are ignored. The worker can also retrieve existing tags from Wallabag.

Three policies are available:

TAG_POLICY	Behaviour
existing_only	Reject every model tag absent from the known taxonomy.
prefer_existing	Prefer known tags but allow a concise new tag when necessary. This is the default.
free	Let the model create tags without treating the taxonomy as a restriction.

The application always validates the model output, removes duplicates, rejects commas/newlines/status tags, preserves canonical existing spelling, excludes tags already attached to the entry, and enforces MAX_TAGS locally.

To keep multilingual articles organized under one taxonomy, set TAG_LANGUAGE to the language in which new tags should be written.

For additional classification rules, copy config/prompt.example.txt to config/prompt.txt and set:

PROMPT_FILE=/config/prompt.txt
PROMPT_VERSION=2

Increment PROMPT_VERSION when a materially changed prompt should be treated as a new classifier version during dry runs and backfills.

Backfilling existing articles

Backfill reads existing entries directly; it does not require adding ai:pending to your archive.

Start with a dry run over a small sample:

docker compose run --rm wallabag-ai-tagger backfill --limit 25

Available options:

--limit N                 Process at most N entries; 0 means all
--archive all|unread|archived
--skip-tagged             Skip entries already carrying a content tag
--force                   Ignore successful state from an earlier backfill

After inspecting the output, set DRY_RUN=false and repeat. Successful backfills are tracked using article content hash, model, and prompt version.

Important configuration

Variable	Default	Purpose
LITELLM_MODEL	qwen-tasks	LiteLLM model alias.
MAX_TAGS	3	Maximum new content tags per article.
MIN_TAGS	1	Fewer accepted tags causes review status.
DRY_RUN	true	Classify and log without changing Wallabag.
RUN_MODE	interval	interval, cron, or once.
INTERVAL_SECONDS	300	Delay between interval executions.
CRON_EXPRESSION	*/15 * * * *	Schedule used in cron mode.
SCHEDULE_TIMEZONE	Europe/Rome in example	IANA timezone for cron mode.
BATCH_SIZE	5	Entries fetched per API batch.
MAX_BATCHES_PER_RUN	1	Batches per scheduled execution; 0 drains.
MAX_CONTENT_CHARS	12000	Maximum article text sent to LiteLLM.
CONTENT_TAIL_CHARS	2000	Space reserved for the article ending.
PENDING_TAG	ai:pending	Queue tag created by your Wallabag rule.
REVIEW_TAG	ai:review	Status applied to uncertain output. Empty disables it.
ERROR_TAG	ai:error	Status applied after maximum failures. Empty disables it.
MAX_ATTEMPTS	3	Scheduled attempts before terminal error status.
LLM_RESPONSE_FORMAT	json_schema	json_schema, json_object, or none.
LITELLM_EXTRA_BODY_JSON	thinking disabled	Extra top-level LiteLLM request fields.
STATE_DB	/data/state.sqlite3	Persistent dry-run/backfill/retry state.

See .env.example for every setting.

Structured output and qwen-tasks

The default request asks LiteLLM for JSON-schema-constrained output and sends:

{"chat_template_kwargs":{"enable_thinking":false}}

as extra request data. If qwen-tasks already fixes this at router level, set LITELLM_EXTRA_BODY_JSON={}. If an older LiteLLM or llama.cpp build rejects JSON Schema, the worker automatically falls back to JSON-object mode and then prompt-constrained JSON when LLM_RESPONSE_FORMAT_FALLBACK=true.

All responses are parsed and validated by the worker even when the inference server claims to enforce the schema.

Local development

Python 3.11 or later is required. No third-party packages are needed.

make compile
make test

Run without Docker:

cp .env.example .env
set -a
. ./.env
set +a
PYTHONPATH=src python3 -m wallabag_ai_tagger check

If your interactive shell is Fish, use a dotenv loader of your choice or run through Docker Compose; the application itself does not depend on a shell.

Security notes

• Keep .env mode 0600; it contains Wallabag credentials and API secrets.
• Prefer private Docker networking instead of routing service-to-service calls through public reverse-proxy hostnames.
• Article content is explicitly treated as untrusted prompt data.
• The worker never replaces or removes manual tags.
• TLS verification is enabled by default. Disable it only for a trusted local endpoint with a self-signed certificate.