The age of giant AI models, already over?

April 27, 2023

EXECUTIVE SUMMARY:

If you expected OpenAI to further develop ChatGPT’s ‘neural network’ in order to enhance the bot’s performance, allowing it to perform an even wider range of tasks with unprecedented accuracy and efficiency, you might find yourself disappointed.

OpenAI’s CEO, Sam Altman, warns that the research strategy used to create the bot has already played out. Speaking at a recent MIT event, Altman suggested that further progress would not derive from “giant, giant models.”

Training large language models requires exorbitant (and possibly untenable) expense. For example, ChatGPT reportedly required over 10,000 GPUs during the large language model (LLM) training process, and it demands even more resources in order to operate continuously.

GPT depends on GPUs

“It seems like everyone and their dog is buying GPUs at this point,” Elon Musk observed, indicating that prices may be skyrocketing, while the benefits may have leveled off; at least for OpenAI. The economics of scale may have turned against developing ever-larger GPT neural network models.

However, not everyone believes that the cost-benefit equation lies at the heart of OpenAI’s apparent tonal shifts. Worth noting is the fact that Twitter CEO Elon Musk recently revealed plans to create “TruthGPT,” an AI model that provides the “maximum” level of truth about a given topic, thereby emphasizing the importance of quality AI results over language model scale.

TruthGPT could be coming soon

Information about whether or not “TruthGPT” has gotten off of the ground has not yet been released. Nonetheless, in March of this year, Musk quietly established a new AI company, called X.AI.

In an interview with Fox News, Musk shared concerns about how big AI firms may accidentally perpetuate bias and disinformation via chatbots. He also commented on the possibility of an AI apocalypse; an event where AI could gain control over our daily lives.

An accidental 6-month pause

On March 22nd, the nonprofit Future for Life Institute published an open letter signed by more than 1,000 global technology leaders requesting an “urgent pause” on the development of technologies more powerful than GPT-4.

While OpenAI CEO Sam Altman stated that the letter “lacked technical nuance about where we need the pause,” the company’s transition towards enhancing GPT-4 technology indicates that OpenAI took the letter seriously.

OpenAI is not currently developing GPT-5. There are plenty of challenges to be worked through within GPT-4 models; from privacy concerns, to malware propagation potentialities, to identity theft-related issues.