GenAI : 'Back to the Future'?
Nobody would argue if you equated AI with 'the new, new thing'. Yet AI has a retro streak to it!
Context
Since Micheal Lewis wrote The New, New Thing - it has been indelibly associated since (perhaps even encouraged) with Tech and the Valley. GenAI is in theory, The New, New Thing on Steroids - Moore’s laws doubling every 18 months replaced by a new GenAI models every 18 days (roughly the time it recently took Elon to stand up a supercomputer in Memphis, normal cycle time 5 years).
But counter to intuition, AI might actually - 1. bring back mothballed ideas in back-to-the-future fashion and 2. bring an afterglow to fading incumbency - the so called Lindy Effect (what has been around, is likely to stay around).
In other words - AI itself is a new, new thing .. but it shows a taste for the old. Exhibit 1 that started this thought process, was the demise of an ‘easier to use’ React framework enhancement called Svelte (see below).
It brings me no pleasure to say this, but Svelte is dead because LLM base models are better at writing React. — Jess Martin
Storyline
As I thought more about the above statement, Svelte is not an isolated example, so much as GenAI’s first salvo. The broader story:
Systems edifice of the last 50 years anchored in ‘user friendly’
Programming languages with intuitive constructs (e.g. Objects)
UI with intuitive constructs (e.g. Windows, Menus)
Operating systems with intuitive constructs (files, hierachicies)
We are delegating those jobs to GenAI
GenAI doesn’t give a damn about user friendly
In fact GenAI prefers machine friendly
Machine friendly focus in systems was a thing of the 1960s!
What’s to come1
Systems 2.0:A Rebirth of Computer Systems
Once human insights are abstracted into agents, what will human ‘tasks’ will become and therefore what will computer interfaces will evolve to support? CHI 2024 has been a disappointment, so I’ll go it alone and derive my point of view, from a bit of arxiv hunting and personal intuition:
Support for hybrid (declarative-imperative) programming. programming will exist but gradually but eventually become declarative. the transition will be untidy with possible programmatic collage creation (pull pieces of code together and ask LLMs to ‘glue’ them into coherence).
Programs as complex question graphs. As I’ve covered earlier in
programming is going to be more like a socratic dialog with an LLM than churning out code trees.
Rise of ‘metacognitive’ support. I’m not a fan of anything with ‘meta’ in it (other than the company) - tends to be a word that takes a difficult idea to obscure. But this notion of metacognitive support to help you keep track of what you’ve asked and answered (as a control to your thoughts and behavior) will be a huge deal. Without it all dialogs will end in either intellectual vertigo or anger or both! Most of us (especially the crowd that colleges churn out these days) are abysmal at asking questions, and need all the help they can get.
Debugging as theorem proving. Once people get a grasp of LLM programming, the size of systems we will be able to create will magnify. They will work great until they don’t. And when they don’t, it will be beyond human capability to debug. Not only the size of the code, but the amount of shared context that would have transpired (mostly in machine efficient binary), would make debugging feel like the Vesuvius Challenge. As in VLSI, you will see theorem provers get integrated into software testing and debugging - think of them as assertions on steroids. Theorem proving languages like this were a big thing in the 80s-90s, now to make a come back (much as the return of Hinton from the 1980s to the 2010s was a very different outcome).
Coda
There’s a lot more to be said about the future, all of which is uncertain but this will do for the moment. I for one look forward to the time when writing software to compute something may become easier than googling it.
For those whose lives haven’t been scarred by an encounter with APL - https://x.com/venuv62/status/1616634397549117442