13th-Century Visualizations Simplify AI.
Space likes to make it a habit of being the final frontier. Star Trek and Space Force explored it. Earth-bound empires thought deeply about it. Developments in dynastic Central Asia, for example, spun out of a 5th-century BC analysis of two spatial forces: zhongguo and tainxia. That's how the ancient Chinese summed up the friction between a central “capital of all states” and a wider “entire human world.” The Great Wall was the 7th-century Chinese answer for the spatial threat of invaders to prosperous lands to the north. Space properly managed, yields security.
Spatial thinking is again becoming consequential in managing the lucrative emerging terrain of new tools attempting to automate complex human habits. Back in April, machine learning pioneer, and Chief AI Scientist at Meta, Yann LeCun (who we admire for being marvelously dubious about machine automation) published an excellent tree diagram that lays out a spatial narrative of various machine learning tools.
LeCun elegantly maps the various limbs of logic that gave us language models like ChatGPT, Bard, and Bloomz . It clearly captures what’s so easy to forget about AI: Automating the reading and writing of language started around 2017, when engineers at the then-obscure Google Brain project, had an aha moment. What mattered, when it came to automating language, was a reasonably reliable way to predict what the next logical chunk of a sentence might be; followed by a shade more-accurate means of confirming that the previous guess is actually a story and not gibberish.
Somehow this grammatical trick got nicknamed as a transformer. Apparently, the developer’s found themselves transformed by the results. Who knows. Regardless, LeCun maps out the spatial logic between the various software tools based in and around transformers.
LeCun’s AI tree simplifies, but it also clarifies that the roots of language automation as either Encoder-Only, Decoder-Only, or the hybrid Encoder-Decoder approach. For LeCun, software that generates original words and ideas by itself, does so in exactly two ways: The software either munches up, or encodes, text. Or as it spits out, or decodes, that text. Though some software tries extra hard and both munches up and spits out words and ideas.
Understanding how various encoding or decoding works is not especially hard. A young machine learning engineer named Will Keohrsen, has posted a simple-to-follow process on mapping on the spatial relationships between book genres.
It was worth noting that the reality of AI does not seem to match its magic Mr. Keohrsen has exactly 2 active followers on Github.
An Ancient Tree of Knowledge. But what makes LeCun’s AI tree far more intriguing is that tree visualizations such as his, are among the most ancient and efficient means of describing complex spatial relationships. The undisputed Albert Einstein -- or Alan Turing to be more computational about things -- of tree diagrams is a 13th-century Catalan philosopher and logician named Ramon Llull.
Llull is unique among Renaissance thinkers for being well-respected across the full range of Hebrew, Arabic and Latin cultures. His synthesis of contrasting ideas has captured the imagination of influential minds ever since. The 17th-century polymath Gottfried Leibniz, who invented the basic idea of mathematical logic, was influenced by Llull. The 19th-century economist, William Stanley Jevons also used Llull’s logic to create an eerily computer-like so-called Logical Piano that “solved complicated problems faster than humans.” Sound familiar?
Llull’s 13th-century Tree of Knowledge, just like LeCun’s Tree of Ai , organized the then rapidly expanding realms of scientific analysis into easy-to-understand limbs. While the modern mind might quibble with classifying the world as elemental, vegetal, human, or imperial, how Llull then processed those classifications into consistent elemental modules that created accurate predictions, puts LeCun’s tree-like analysis of machine learning into the exact right context:
Nearly a thousand years ago, Llull mapped the challenging spatial narrative of an emerging world of new ideas. His spatial analysis is what the bright minds that followed used to create a better world.
Space mattered then. It matters now.
<