Skip to main contentAccessibility help
Quick Settings
Sign in
AI Mode
All
Images
Videos
Short videos
Forums
News
More
Tools

Search Results

AI Overview
https://media.licdn.com/dms/image/v2/C5603AQGUrXhzYpFNPA/profile-displayphoto-shrink_200_200/profile-displayphoto-shrink_200_200/0/1653416381156?e=2147483647&v=beta&t=bVSRdLlGFbGX_6o77hAKyI0E_jPbXBFfJrz1YOrRP9Y
Most people's experience with AI and documents is Retrieval-Augmented Generation (RAG): you upload files, the system splits them into text chunks, retrieves matching pieces at query time, and has the language model generate an answer. This phrasing echoes Andrej Karpathy's popular critique contrasting standard RAG with persistent, evolving knowledge bases. 
How Standard RAG Works
  • Upload files: Drop raw PDFs, notes, or codebases into a database.
  • Chunk and embed: Split text into small segments and turn them into vector numbers.
  • Query-time retrieval: Search for matching chunks when you ask a question and feed them to the AI context window. 
Limitations of RAG
  • Stateless and transient: The system forgets connections between queries; it re-searches from scratch every time.
  • Context window limits: It struggles to synthesize an entire library of documents at once, often missing global insights or cross-file contradictions.
  • Token heavy: Repetitive retrieval of raw text chunks wastes processing power over time. 
Would you like to explore alternatives to RAG like persistent markdown wikis and knowledge graphs, or do you need help optimizing a RAG pipeline? 
  • Andrej Karpathy's LLM Wiki: Building your own document-to ...
    6 Apr 2026 — Most people's experience with LLMs and documents looks like RAG: you upload files, the LLM retrieves relevant chunks at query time...
    X·TheYotg
  • LLM Wiki - GitHub Gist
    27 Jul 2026 — Most people's experience with LLMs and documents looks like RAG: you upload a collection of files, the LLM retrieves relevant chun...
    GitHub
  • 71.5x Fewer Tokens: How Karpathy's LLM Wiki + Obsidian ...
    11 Apr 2026 — Most people's experience with LLMs and documents looks like RAG: upload files, retrieve chunks at query time, generate an answer. ...
    Substack·Ketan's Substack
  • Karpathy's LLM Wiki: A Knowledge Base That Compounds
    27 May 2026 — The LLM Wiki is a pattern published by Andrej Karpathy in April 2026. It's a pattern that uses an LLM to incrementally build and m...
    AI Builder Club
  • Andrej Karpathy Just Made RAG Obsolete — And All You Need Is Three Folders
    6 Apr 2026 — RAG — Retrieval-Augmented Generation — is the standard playbook when you want an LLM to work with your documents. Upload files, ch...
    Medium
  • Data & Knowledge Layer: Building a RAG Pipeline
    9 Oct 2025 — 🧩 How RAG Works (Deep Breakdown) Chunking You start by breaking large documents, datasets, or codebases into smaller Embeddings E...
    LinkedIn
  • RAG Pipeline + LLM: Yes, You Can Have Your AI Cake and Eat it Too
    RAG pipelines read docs, SQL databases, websites, and can process unstructured data such as extracting information from a pdf docu...
    Focused.io
  • LLM vs RAG vs Agent: AI Types Explained | Mayank A. posted on the topic
    28 Dec 2025 — And this is exactly where RAG comes in. How RAG works First your documents like PDFs notes or data are converted into vectors and ...
    LinkedIn
  • [2509.07666] MoLoRAG: Bootstrapping Document Understanding via Multi-modal Logic-aware Retrieval
    6 Sept 2025 — Retrieval-augmented generation (RAG) methods mitigate this by selecting relevant pages, but they rely solely on semantic relevance...
    arXiv
  • RAG is Dead. Karpathy’s LLM Wiki is the future | Project Explained
    2 May 2026 — RAG is stateless. Every query triggers the same pipeline: retrieve, stuff into context, generate, forget. It works well for simple...
    Artificial Intelligence in Plain English
  • RAG vs Long Context Models [Discussion] : r/MachineLearning
    22 Feb 2024 — RAG requires lots of inferences behind the scenes, lots of database fetches, and just extra complexity everywhere. RAG is a tempor...
    Reddit
  • How to Add a Memory Layer to Your RAG System [Guide]
    24 Oct 2025 — The issue isn't with the documents or the vector search, it's that RAG has no memory of who's asking. Every query starts from scra...
    Unstructured
  • Andrej Karpathy's LLM Wiki: Building your own document-to ...
    6 Apr 2026 — Most people's experience with LLMs and documents looks like RAG: you upload files, the LLM retrieves relevant chunks at query time...
    X·TheYotg
  • LLM Wiki - GitHub Gist
    27 Jul 2026 — Most people's experience with LLMs and documents looks like RAG: you upload a collection of files, the LLM retrieves relevant chun...
    GitHub
  • 71.5x Fewer Tokens: How Karpathy's LLM Wiki + Obsidian ...
    11 Apr 2026 — Most people's experience with LLMs and documents looks like RAG: upload files, retrieve chunks at query time, generate an answer. ...
    Substack·Ketan's Substack
Show all
AI can make mistakes, so double-check responses
Transcribing...

Web results

Andrej Karpathy's LLM Wiki: Building your own document- ...


X · TheYotg
180+ likes · 3 months ago
X · TheYotg
180+ likes · 3 months ago
Most people's experience with LLMs and documents looks like RAG: you upload files, the LLM retrieves relevant chunks at query time, and ...Read more

LLM Wiki


GitHub
https://gist.github.com › karpathy
GitHub
https://gist.github.com › karpathy
2 days ago — Most people's experience with LLMs and documents looks like RAG: you upload a collection of files, the LLM retrieves relevant chunks at query ...Read more

Document to knowledge graph | Paulo Cysne


LinkedIn · Paulo Cysne
6 reactions · 3 months ago
LinkedIn · Paulo Cysne
6 reactions · 3 months ago
... documents looks like RAG: you upload files, the LLM retrieves relevant chunks at query time, and generates an answer. This works, but the ...Read more

Karpathy's LLM Wiki: A Knowledge Base That Compounds


AI Builder Club
https://www.aibuilderclub.com › Blog
AI Builder Club
https://www.aibuilderclub.com › Blog
27 May 2026 — Most people's experience with LLMs and documents looks like RAG: you upload files, the LLM retrieves relevant chunks at query time, and ...Read more

I Used Karpathy's LLM Wiki to Build a Research Brain That ...


Towards AI
https://pub.towardsai.net › ...
Towards AI
https://pub.towardsai.net › ...
19 Apr 2026 — Most people's experience with LLMs and documents looks like RAG: you upload a collection of files, the LLM retrieves relevant chunks at query ...Read more

An Architectural Reading of Karpathy's LLM Wiki


GitHub Pages documentation
https://cozypet.github.io › llm-wiki-schema
GitHub Pages documentation
https://cozypet.github.io › llm-wiki-schema
The core claim: most people's experience with LLMs and documents looks like RAG, and RAG has a fundamental flaw. With RAG, you upload documents, the LLM ...Read more

71.5x Fewer Tokens: How Karpathy's LLM Wiki + Obsidian ...


Substack · Ketan's Substack
7 likes · 3 months ago
Substack · Ketan's Substack
7 likes · 3 months ago
Most people's experience with LLMs and documents looks like RAG: upload files, retrieve chunks at query time, generate an answer. It works, but ...Read more

LLM Wiki


Blake Merryman
https://blakemerryman.com › linked › 2026/04 › llm-wiki
Blake Merryman
https://blakemerryman.com › linked › 2026/04 › llm-wiki
10 Apr 2026 — Most people's experience with LLMs and documents looks like RAG: you upload a collection of files, the LLM retrieves relevant chunks at query ...Read more

Andrej Karpathy's LLM wiki automates knowledge base ...


Facebook · Tech Titans
3 comments · 1 month ago
Facebook · Tech Titans
3 comments · 1 month ago
Most people's experience with LLMs and documents looks like RAG you upload a collection of files, the LLM retrieves relevant chunks at query ...Read more
People also search for
Documents looks like rag llm
Documents looks like rag explained
Claude personal wiki
Karpathy note-taking

Page navigation

12345678910Next

Footer links

Hungary
1138, Budapest
 - From your IP address
 - 
Update location
HelpSend feedbackPrivacyTerms
Google apps