tldw
v0.1.31 beta · Building toward The Young Lady's Illustrated Primer

Build your own Primer. Ingest, transcribe, search, and query your own material.

tldw is an open-source, self-hosted research system. Feed it video, audio, PDFs, web pages, and docs. It gives you transcripts, hybrid RAG, notes, study workflows, writing tools, MCP automation, and OpenAI-compatible APIs.

Get Started View on GitHub

Quick Start

Clone the repo, pick one public setup profile, and follow prepare, start, then verify. The default Docker single-user + WebUI path uses same-origin browser API requests through the WebUI proxy. API-only Docker and local dev remain supported alternative setup profiles.

Preflight check 
git clone https://github.com/rmusser01/tldw_server.git
cd tldw_server
make quickstart-prereqs
Shortest alias: Docker single-user + WebUI 
git clone https://github.com/rmusser01/tldw_server.git
cd tldw_server
make quickstart
Docker single-user + WebUI 
git clone https://github.com/rmusser01/tldw_server.git
cd tldw_server
make setup-docker-single
make start-docker-single
make verify-docker-single
Docker multi-user + Postgres 
git clone https://github.com/rmusser01/tldw_server.git
cd tldw_server
export ADMIN_USERNAME=tldw-admin
export ADMIN_PASSWORD="$(python3 -c 'import secrets; print(secrets.token_urlsafe(24))')"
make setup-docker-multi
make start-docker-multi
make verify-docker-multi
Local single-user 
git clone https://github.com/rmusser01/tldw_server.git
cd tldw_server
make install-local
make setup-local-single
make start-local-single
make verify-local-single

PyPI installation is not generally available yet. Use this repository checkout and the quickstart targets above.

Default Docker startup brings up the API at localhost:8000/docs and the WebUI at localhost:8080. Browsers stay on same-origin browser API requests through the WebUI proxy. LAN/custom-host browser access is the advanced configuration path. The local single-user path starts the API at localhost:8000/api/v1/config/quickstart.

Need a team or public deployment instead of the default single-user path? Start with the Docker Multi-User + Postgres guide.

What is tldw?

tldw (Too Long; Didn't Watch) exists because most software for working with source material is either shallow, hosted, or both. This project is for people who want the whole stack on their own machine or server.

The long-term goal is The Young Lady's Illustrated Primer. That is the point of the project. The current codebase is the practical part: ingestion, transcription, retrieval, APIs, notes, prompts, and self-hosting.

If you run `tldw` yourself, you keep the source material, transcripts, prompts, notes, embeddings, APIs, and databases. That matters if your work is private, strange, long-lived, or just not something you want locked inside someone else's product.

  • OpenAI-compatible Chat / Audio / Embeddings / Evals — one API surface instead of custom glue for every model backend.
  • FastAPI backend + WebUI — current stack, current quickstarts, and a path you can actually run right now.
  • Hybrid RAG — full-text search, vectors, reranking, and evaluation in the same system.
  • Local or hosted LLMs — OpenAI, Anthropic, Google, Cohere, Groq, Mistral, OpenRouter, Qwen, Ollama, llama.cpp, vLLM, and more.
  • Multi-provider STT/TTS — transcription, streaming audio APIs, voice catalogs, and local speech stacks.
  • Notes / prompts / chatbooks — keep notes, prompts, chatbooks, and source material in one stack.
  • MCP + tool execution + metrics — not a toy wrapper, an actual extensible stack.
  • Wide format support — video, audio, PDF, EPUB, DOCX, HTML, Markdown, XML, MediaWiki dumps, and more.

What's New in v0.1.31

The latest beta line focuses on onboarding, study workflows, stronger workspace UX, and runtime hardening while the `dev` branch continues to land larger writing, clipper, and sandbox work.

Sharper onboarding and setup

First-run assistant setup, archetype templates, expanded FTUE/FTUX flows, Docker/WebUI quickstarts, and clearer setup verification make the project easier to try from a fresh checkout.

Study and evaluation workflows

Study Packs, flashcard templates, study suggestions, and the recipe-based evaluations framework add guided ways to turn source material into practice and measurable retrieval improvements.

Writing, research, and workspace depth

Writing Suite foundations, Deep Research runs, shared workspace flows, Companion Home, MCP Hub governance, and reference-manager import/sync expand tldw beyond a media-ingest API.

MCP and governed automation

The MCP Virtual CLI and follow-on hardening provide workspace-bounded command execution, policy-aware discovery, approval gates, metrics, WebSockets, and tighter isolation contracts.

Audio and container follow-through

Curated audio setup paths, local TTS/STT provider work, voice catalogs, and GHCR container snapshot publishing make speech and deployment workflows more practical.

Security and reliability hardening

Recent releases tightened AuthNZ, outbound scraping policy, shared SQLite ownership, audit durability, Docker packaging contracts, upload feedback, streaming aborts, and service shutdown behavior.

Features

A self-hosted stack for taking raw source material and turning it into something you can search, query, and keep.

Ingest messy source material

Pull in video, audio, PDFs, web pages, EPUB, DOCX, Markdown, XML, and other inputs without maintaining a separate toolchain for every format.

Transcribe long-form media

Convert recordings into text you can search, quote, summarize, and feed back into the rest of the stack instead of relistening every time.

Retrieve across transcripts, notes, and docs

Use hybrid retrieval across the material you already collected so the useful part of it is still findable weeks or months later.

Run local or hosted models

Use local inference, hosted APIs, or both through one interface instead of wiring every provider into your tooling separately.

Keep notes, prompts, chats, and artifacts together

Notes, prompts, chats, artifacts, and source material live in the same system so you do not have to reconstruct prior work from five different tools.

Self-host and inspect the stack

Run it on infrastructure you control, inspect the code and data paths, and shape the system around your own workflow instead of a hosted default.

FAQ

Is this usable now, or is it still mostly aspirational?

You can run it now if you are comfortable with a beta project. The bigger Primer goal is the reason the project exists, but the current codebase already gives you a working self-hosted stack for ingestion, transcription, retrieval, and API-driven experimentation.

How is my data handled?

The default posture is simple: keep your data on infrastructure you control. No telemetry, no analytics, and no phone-home behavior. You decide how content is ingested, stored, backed up, exported, and deleted.

Which models are supported?

Hosted providers such as OpenAI, Anthropic, Google, Cohere, DeepSeek, Groq, Mistral, OpenRouter, HuggingFace, and Qwen are supported, and so are local options such as llama.cpp, Kobold.cpp, Ollama, vLLM, TabbyAPI, and other OpenAI-compatible endpoints.

How can I contribute?

Start with the good first issues on GitHub, or join the Discord to talk with the community. If you care about self-hosted research tooling and the Primer goal, there is plenty of real work to do.

Community

Join the conversation, get help, share ideas, and help shape the project.

Discord Zulip GitHub