Making the Complex Simple

Simply Understood

Large language models (LLM) handle data very differently than a traditional database. What we’ve learned since the introduction of GPT-3 in 2020 is that because of the algorithms behind these models, we can query the data in unstructured and imprecise ways and receive responses that astonish and surprise us. And sometimes alarm us.

A16Z reports that, over the past year, thousands of new consumer products have been introduced, which leverage the ‘magic of AI’. Or in the immortal words of Grady Booch, digital Ouija boards. These language models are complex, compute-intensive, difficult to build and not easily understood. Which is the root of the problem. Should you bet your business on something that is hard to understand?

We’ve written in a previous post that the complexity of foundation models should no longer surprise. OpenAI’s mission to create a general intelligence platform for the good of humanity strikes us as a moon shot, and with it, of course, the complexity mounts. It’s on these platforms that companies are building apps.

Business executives, however, place high value on simplicity, not to make things simple, but to make things simply understood. Dr. Jeremy Avigad, a mathematics professor at Carnegie Mellon University, talked about this as a Theory of Simplicity, something which we believe needs to be embraced in validating the utility of language models. We described in our last post how we took the ‘necessary actions’ to direct the language model in a business workflow for booking reservations. The ability to embed functions in the model process helped in delivering quality, accurate interactions but were not perfect. But were they good enough?

It is interesting to consider that language models today provide no unique differentiated value to users. Every user has access to the same capabilities. Competitors gain little advantage from a model unless they can leverage the models in unique ways to drive productivity or innovation. Without a doubt, companies will eventually build and deploy custom models, trained on their proprietary datasets, and positioned for specific use cases, as compute power and talent becomes more accessible.

So, in this new era of AI apps, how do we make things simply understood, so that businesses can better measure the risks, and gain confidence in the outcomes? In the upcoming series of posts, we will be describing our work in AI testing, and some important guidelines we’ve embraced to make the complex simple, but with reliable proofs. We look forward to sharing more with you then.