Harnessing AI for Production

Navigating the Law of the Horse

With apologies to Judge Frank H. Easterbrook of the United States Court of Appeals for the Seventh Circuit, we think the “Law of the Horse” may have some application for AI in production systems. Judge Easterbrook, of course, made the coherent argument that niche or specialized areas of law, like cyberlaw (analogous to a hypothetical "law of the horse"), is less effective than general legal rules. In other words, we can overregulate a new technology, with cumbersome or intrusive rules, rather than relying on a broader set of regulations and processes to manage outcomes. But this governance approach does not translate well to GenAI.

Executives are wrestling with the deployment of GenAI technologies for their enterprise systems because the technology is unpredictable, and detailed quality assurance is required before it is broadly relied on for core applications. We’ve all experienced the ‘hallucination’ of GenAI, which is radically unacceptable in any production application where precision, accuracy and predictable outcomes are expected in every customer touch point across the company. Appropriately, until governance and controls can be implemented effectively, without impeding the efficient economic delivery of services and products, GenAI will be mostly relegated to internal functions such as marketing content or code generation.

We have been exploring the issues intensively in our work with companies through prototypes, not only identifying the risks, but crafting some very practical solutions in the delivery of AI-driven applications. We have built frameworks for deploying AI in production which permits managers to gain detailed quality control of GenAI while demonstrating compliance with broader policies of the company. In other words, it’s the Law of the Horse on steroids.

But let’s step back for a minute and review the composition of GenAI technology. Yuval Levin explained in a recent WSJ article that GenAi was

... in essence an analytical technology that learns complex patterns from training data and then draws on those patterns to make predictions about new data. With modest computing power, that looks like a guess at the next word in a text message. But with access to gargantuan amounts of data and vast computing capacity, it can look like a clean-prose response to a complex prompt approximating what a human would write.

We like that definition because it captures the essence of the Large Language Models (LLMs), which are the foundation of GenAI applications. But with new features released by OpenAI, and other tech companies, we’re seeing exciting advances which deliver astonishing value for customers. Our work shows that we can steer this capability in unique, affordable and useful ways through the injection of custom data and functions. Of course, we see the greatest productivity gains will be accrued from GenAI by leveraging the models for logic, which we've written about in prior posts.

Over the next few posts, we’ll describe what we’ve learned in harnessing GenAI for production through custom functions. Some of our posts will be technical, with sample code and results. Other posts will center on the policy implications and process transformations that may be necessary to ‘scale up’ with the technology. Rest assured, that while the Law of the Horse may not fit well for national regulations and policies, it works very well for corporate controls and production operations.