> indyweb breakthrough / epiphany - language IS sparse encoding - agential RAGs are the wrong strategy - the Indyweb is the right direction with modularity and reusability as fundamental information design axioms
- every sentence, every paragraph, every chapter, every essay, every book every paper is intrinsically sparse encoding.
- language fundamentally correlates to the columnar organisation of the neocortical neuronal groups of approximately 10,000 neurons
- and there are approximately 200 million such cortical mini columns of grouped neurons that make up the neocortex
- each spoken human language has a few hundred thousand words in its dictionary group
- specialised domains expand this but
- English has an estimated 171,000 current words,
- dictionaries contain over a million entries, while
- a native speaker's active vocabulary might be closer to 20,000 words.
- sparse coding may be the way brains evolved to deal with the combinatorial explosion of symbolic spoken language
- IPFS employs content addresses that can be mapped to dictionary groups
- this finally breaks through the extremely primitive backwards digital fascimile of the past half century by enabling true modularity of networks of ideas
- making them reusable and modular instead of inflexible fixed text corpus.
- Looking at how AI tools use a technique called agential RAG (Retrieval Augmented Generation),
- it is the fundamentally the wrong way to go about generating modular ideas to store in a graph by chunking a big block of text corpus into smaller ideas.
- This is because the assumption of storing big blocks of text corpus is the wrong assumption to begin with
- this considers modularity and reusability as an afterthought rather than as a fundamental design principle
- incremental change can keep us moving on the wrong direction
- that's when a paradigm shift is absolutely required
- implementing sparse encoding for language itself is pretty revolutionary!
- but it is the only way to mitigate a combinatorial explosion
- a theoretical model for sparse encoding of language in current generation of computing devices can be very useful to operationalize a new digital LANGUAGE system that opens up the latent possibilities of language
- the Indyweb is the system most aligned to developing the next generation of human language
- language itself is a strange loop
- it is simultaneously reality AND a description of reality
- In this sense, it is a cultural artefact of human self-awareness
- Gregg Henrique's uToK that breaks reality into different system levels embedded like nested Russian dolls may be a clue as to reality's inherent self-organisational, autopoeitic, system levels.
Different signs are appropriate for different levels.
- plants cannot directly communicate with bacteria
- cats cannot symbolically communicate with plants
- humans cannot symbolically communicate with cats
- at least not in detail and comprehensively
- in this model, one could interpret "enlightenment" our awakening as an act of getting in touch with the lowest level of our system in relationship to all of reality
- Sparse coding was introduced by Jeff Hawkins of Numenta and the Redwood center for Theoretical Neuroscience
https://redwood.berkeley.edu- the act of writing should be activating word/idea nodes in realtime via some level of sparse encoding to record the idea flow.
- recognized pathways between sequences of nodes can be measured with a distance metric that can trigger reusability.
- novel composition is timebinding, we generate a mixture of novel text strings along with existing ones. It's always mixed mode.
- citation
- new idea
- citation
- new idea
- etc...