🌐 llm-wiki

1. 🎯 Core Principle

The LLM Wiki is a persistent, compounding personal knowledge base compiled once and kept current by an LLM-in-the-loop. Unlike traditional Retrieval-Augmented Generation (RAG) which rediscovers knowledge per query, the LLM Wiki synthesizes information, tracks contradictions, establishes cross-references, and integrates new context directly into structured, interlinked markdown pages.

The human curates sources and directs the domain constraints; the agent summarizes, cross-references, directories, and audits consistency.

2. 🗂️ Layout

A standard LLM Wiki resides at a designated directory (defaulting to ~/wiki) with the following hierarchy:

wiki/
├── SCHEMA.md           # Domain conventions, taxonomy, tag constraints, page thresholds
├── index.md            # Alphabetical content catalog grouped by type with one-line summaries
├── log.md              # Chronological record of all updates, rotated at 500 entries
├── raw/                # Layer 1: Immutable source material
│   ├── articles/       # Web articles, text scrapes
│   ├── papers/         # PDFs, ArXiv documents
│   └── transcripts/    # Meeting transcripts, interviews
├── entities/           # Layer 2: Subject profiles (people, organizations, models)
├── concepts/           # Layer 2: Subject topics (techniques, mathematical formulations)
├── comparisons/        # Layer 2: Side-by-side matrices and evaluations
└── queries/            # Layer 2: Filed syntheses of complex user questions

📥 Layer 1: Raw Sources

Raw source documents are immutable. The agent reads from but never writes to this directory. Each source receives frontmatter mapping its origins:

---
source_url: https://0rk.de/article
ingested: YYYY-MM-DD
sha256: <body-hash>
---

The SHA256 digest is calculated over the raw markdown/text body (excluding the frontmatter) to detect content drift or duplicate ingestion.

📄 Layer 2: Wiki Pages

Wiki pages contain synthesized knowledge written in markdown with standard YAML frontmatter:

---
title: Page Title
created: YYYY-MM-DD
updated: YYYY-MM-DD
type: entity | concept | comparison | query
tags: [tag1, tag2]
sources: [raw/articles/source-file.md]
confidence: high | medium | low
contested: false
contradictions: []
---

📐 Layer 3: Schema

Contains the domain definition, tag taxonomy, and page-creation thresholds. It governs what tags are allowed and what topics warrant page creation.

---

3. ⚙️ Core Operations

📥 Ingest Pipeline

When ingesting new material:

1. Source Capture: Save the raw document into raw/ under a lowercase, hyphenated slug. Prepend the origin URL and compute its body SHA-256. If a document with the same URL already exists, verify the hash. Skip if identical; flag drift and overwrite if changed.
2. Entity & Concept Extraction: Identify key subjects that meet the Page Thresholds (appearing in 2+ sources or central to one).
3. Index Scan: Search index.md and check the filesystem to locate existing pages for the extracted subjects.
4. Write/Update Pages:
   • For new subjects, create files in their respective directories. Ensure at least two outbound [[wikilinks]] link them to existing pages.
   • For existing subjects, integrate the new facts. Bump the updated date.
   • If claims conflict, follow the Conflict Policy: do not silently overwrite. Detail both perspectives with dates and sources, mark contested: true, add the conflicting page to the contradictions list, and log the conflict.
5. Navigation Update: Append new pages to index.md and document the action inside log.md.

🔎 Query Flow

When resolving user questions:

1. Orientation: Read SCHEMA.md, index.md, and the last 30 lines of log.md.
2. Search: Execute regex search for keywords across the wiki directory.
3. Synthesis: Synthesize a comprehensive answer, citing relevant pages using wikilinks (e.g., [[transformer-architecture]]).
4. Filing: For highly complex or side-by-side syntheses, create a page under queries/ or comparisons/ to save the work.

---

4. 🧭 Navigation

• index.md: Tracks every page sorted alphabetically within sections corresponding to their type (Entities, Concepts, Comparisons, Queries).
• Scale Rules:
  • Split a section when it exceeds 50 entries.
  • Generate a _meta/topic-map.md grouping pages by theme when total entries exceed 200.

---

5. 🔍 Lint & Audit

The agent must programmatically audit the vault using a script or command to report on:

1. Orphans: Pages with zero inbound wikilinks.
2. Broken Links: Wikilinks pointing to non-existent target files.
3. Index Gaps: Wiki pages present in the filesystem but missing from index.md.
4. Schema Compliance: Frontmatter validation and tags not declared in SCHEMA.md’s taxonomy.
5. Staleness: Pages updated >90 days ago that reference active subjects.
6. Contradictions: Pages with contested: true or unresolved conflicts.
7. Size Limits: Pages exceeding 200 lines (candidates for splitting).

---

6. 🔗 Related

• Skills