Can AI Think?

Understanding if AI is Understanding

We rarely get sidetracked by the metaphysical, but this lastest debate among Industry luminaries was hard to ignore. The question posed among scientists was “Can LLMs understand”? In the normal course of business, we are too focused on building prototypes and delivering code to get distracted by these questions. We usually leave the philosophical to the philosophers and the metaphysical to the mystics.

But on this question, we believe the simple answer is 'Of course not.'

One of the giants of systems architecture put it a little more elegantly than us:

All large language models - by the very nature of their architecture - are unreliable narrators, and in fact are incapable of reasoning or understanding. - Grady Booch

And in case some of us mere mortals didn't understand that, Grady Booch explains further:

Hallucinations are an inescapable consequence of the very architecture of every large language model.

You can learn more about Grady's point of view here. And a counterpoint view from Geoffrey Hinton and Andrew Ng can be found here, which started the great debate.

We have published in prior posts about our views on AI safety and risk. We concluded that properly deployed foundation models have great upside for a company investing in this technology. But the debate on ‘understanding’ strikes us as important because if AI can think, or at least reason about problems, language models would represent a new epoch in the history of computing and software engineering.

While language models are an exciting tool for organizations to use, it is as important to understand their limitations and strengths in order to match the technology with the appropriate use cases.

We like the way Bindu Reddy, the CEO of abacus.ai, frames the LLM picture:

LLMs such as GPT-4, Bard and LLama appear magical and have displayed emergent human-like intelligence.

Fundamentally, they work on a pretty simple principle - next-word prediction.

An LLM uses something called conditional probability to predict the next word in a sequence. Simply put, that's the likelihood of an event happening, given that another event has already happened. For our model, it's the probability of a particular word following a given sequence of words.

How does the model get these probabilities? From the training data

So, that makes sense to us. Models appear magical. Display emergent intelligence. But they are neither. They don't think.

But the Foundation models do astound, and they are becoming more astonishing with time. Which is why it is becoming increasingly important for companies to stand up prototypes and explore how these models can propel their business.

We had an interesting (if not astonishing) recent encounter with GPT-4 in our work. We prompted the model with a fairly complex problem, not unlike what one may encounter in a web development project where a state machine is needed to control workflow.

/*
We have an array of objects. Each object is uniquely identified by an id. 

The relationship between objects is described on each object through an array of 
object ids. 

The relationship array identifies which objects must be processed before it can be 
processed.

The state of the object is set to 'disabled', until all prerequisite objects have been 
processed, with a state of 'executed'.  If an object has no prerequisites, its' 
state should be 'enabled', as identified by an empty array. 

Write an efficient algorithm as measured by O Notation to handle this case. 

A sample set of the array is included below:
*/

const obj = [
  {
    id: 30,
    pre: [10, 20],
    eState: false,
    dState: true,
    isExecuting: false,
  },
  {
    id: 40,
    pre: [10, 20, 30],
    eState: false,
    dState: true,
    isExecuting: false,
  },
  {
    id: 40,
    pre: [10, 20, 30],
    eState: false,
    dState: true,
    isExecuting: false,
  },
]

GPT-4 'thought' about it for a seconds and responded with this code and embedded comments:

function updateStates(arr) {
  // Create a map of id to eState for easy lookup
  const eStateMap = arr.reduce((acc, cur) => {
    acc[cur.id] = cur.eState
    return acc
  }, {})

  // Create a new array of objects with updated dState
  const newArr = arr.map((obj) => {
    // If the prerequisite array is empty, then dState is false
    if (obj.pre.length === 0) {
      return { ...obj, dState: false }
    }

    // Check if all prerequisite eStates are true
    const allPreTrue = obj.pre.every((id) => eStateMap[id])

    // Set dState based on the prerequisite eStates
    return { ...obj, dState: !allPreTrue }
  })

  return newArr
}

const arr = [
  // Your array here
]

const newArr = updateStates(arr)

We were impressed. The code worked. But the magical part was the comment that GPT-4 appended at the end of the response:

The O Notation for this algorithm is O(n).

I also noticed that you have a duplicate object in your array with an id of 40.

We hadn't spotted the duplicate, but GPT-4 'noticed it'? It seemed to make the comment on our test data as an 'after thought'. You can understand why we think that's pretty magical - and astonishing!