How Large Language Models (LLMs) Will Change Our World:

Learning From Our Past

Imagine it's the early 1990s, and the Internet is still a relatively new concept. Back then, many people didn't fully grasp its potential. Some thought of it merely as a way to send emails or browse a few static websites.

I vividly remember getting connected to the Internet in 1992 and being amazed; I had a sense of what it was but not fully grasping what its potential might be.

We can see in old news reports from that time just how limited and far off some people's understanding of what the Internet was and what it might become. They couldn't envision how it would eventually transform nearly every aspect of our lives, from shopping and communication to entertainment and education.

Large language models (LLMs) are at a similar stage today. Just as the Internet was initially seen as a simple tool, LLMs might currently be viewed merely as advanced chatbots or tools for generating text. However, their potential is far greater. And I believe their impact on us will likely be greater than any technological revolution of our past.

How Might LLMs Impact Us?

As with any advanced technology, LLMs are complicated and difficult for the average person to understand. LLMs stand on the shoulders of many other machine learning technologies that have been discovered/invented over numerous decades, with major breakthroughs in the past several years. And to many, AI has become a synonym for LLMs. That is understandable since most people have heard of impressive LLMs like ChatGPT which are a form of AI.

Think of LLMs as the underlying technology that, like the Internet, can revolutionize many fields. It isn't necessarily the underlying technology itself that will bring so much value. The value emerges when the underlying technology is implemented correctly into reliable solutions/applications that solve complex problems. For instance:

• Communication: Just as the Internet enabled new forms of communication like social media, LLMs can enhance how we interact with machines, making customer service, virtual assistants, and automated support more natural and efficient.
• Information Access: The Internet made information globally accessible. Similarly, LLMs can process and summarize vast amounts of information, helping professionals in fields like law, medicine, and finance make better decisions faster.
• Education and Training: The Internet brought online courses and educational resources to the masses. LLMs can provide personalized tutoring, generate educational content, and support interactive learning experiences.
• Entertainment: The Internet revolutionized entertainment with streaming services and online gaming. LLMs can create new forms of interactive storytelling, generate content for games, and assist in creative writing.
• Business and Productivity: Just as the Internet transformed business operations with e-commerce and cloud computing, LLMs can streamline workflows, automate mundane tasks, and provide insights from data analysis.

A central theme to the use cases of LLMs is automation and productivity improvement. We will likely see a massive increase in automation and productivity in the coming years. My hope is that such a boost in productivity and GDP could even be large enough to help the USA grow out of the national debt-to-GDP conundrum we are facing today.

And the most valuable use cases have not likely been defined or understood yet. And with the help of creative user experience professionals and innovative thinkers, the best way to tap the value of LLMs will likely emerge over time.

So Much More Than Just Chatbots

Almost everyone is familiar with ChatGPT. Developed by OpenAI, this platform serves as a versatile general-purpose chatbot that can handle a multitude of conversational scenarios—ranging from simple trivia queries to complex explanations of specialized topics, and even crafting creative outputs like stories or poems. Despite its broad capabilities, ChatGPT, like any generalist tool, is not a deep expert in specific domains and may occasionally produce inaccurate information. The true potential of the GPT model, and other similar large language models (LLMs), is unlocked when they are integrated into more sophisticated systems.

LLMs are far more than mere chatbots. Imagine them as akin to the human brain, equipped not just to converse but to understand and generate text that is remarkably human-like. This allows them to tackle complex problem-solving, execute detailed analyses, craft precise translations, and even aid scientific endeavors by hypothesizing or summarizing extensive volumes of research. This broad spectrum of capabilities renders LLMs indispensable across a wide array of applications, demonstrating their transformative impact beyond straightforward conversational tasks.

Reasoning Engines Need Tools To Do Their Best Work

Assigning complex tasks to the world's best engineers, doctors, lawyers, scientists, and other professionals without allowing them to use tools would likely lead to poor results. Restricting them to rely solely on their minds, with no access to basic aids like pencils and paper for notes, calculators, and other resources, would drastically limit their capabilities. Similarly, expecting an LLM, which functions as a reasoning engine akin to our own brains, to perform such tasks without access to supportive tools is equally unrealistic.

One innovative extension of LLMs that provides "tool" access to the LLM is Retrieval-Augmented Generation (RAG). Much like humans use tools to enhance their calculative abilities or organize vast amounts of information, RAG acts as a cognitive enhancer for LLMs. It combines the generative power of these models with a retrieval system that pulls in relevant information from a vast corpus of data. This method allows the LLM to process and integrate this data in real time, much like a human referring to a library of books to inform their reasoning and responses, thereby greatly enhancing the accuracy and depth of its outputs. This makes citation of the source of content possible, enabling checks and balances when the consequences of getting it wrong can be devastating.

Another powerful extension is Program-Aided LLM (PAL). Where RAG enhances the LLM’s memory and reference capabilities, PAL is akin to providing a computational toolset that extends its reasoning capabilities. By interfacing with external APIs or executing specific programs, an LLM can perform tasks beyond its inherent training—such as complex calculations, dynamic data retrieval, or real-time content analysis. This is similar to how a person might use a calculator for complex math, a planner to organize thoughts, or software to track and review task progress, thus enabling a deeper and more reflective thought process.

These advancements illustrate how LLMs, when augmented by technologies like RAG and PAL, transform from mere conversational agents into powerful, multifaceted cognitive engines that can push the boundaries of what automated systems can achieve.

Enhancing LLM Impact through Reinforcement Learning from Human Feedback (RLHF)

As we consider the potential impacts of large language models (LLMs) on society, it becomes necessary to ensure their development is guided by ethical principles while meeting product/service requirements. Reinforcement Learning from Human Feedback (RLHF) is one method that can help with ensuring LLMs more precisely meet our requirements.

RLHF utilizes human feedback to align LLM outputs with whatever adjustments the trainers of the model desire. This includes guiding the model to abide by principles of helpfulness, harmlessness, and honesty (HHH). By interacting with human trainers, these models learn not just to perform tasks, but to do so in a way that is beneficial, ethical, and truthful. This process ensures that as LLMs take on more roles in our lives, from automating customer service to assisting with medical diagnostics, they do so in a manner that is considerate of human expectations and safety. Clearly getting this done correctly will become increasingly important if we want to move in the direction of increased automation and autonomy given to LLMs.

An integral tool in optimizing RLHF is Proximal Policy Optimization (PPO), an algorithm that facilitates efficient policy iteration. PPO helps streamline the reinforcement learning process by balancing exploration of new behaviors with the refinement of already learned behaviors, making it less likely for the model to engage in harmful or misleading output. This balance is critical in deploying LLMs in diverse and unpredictable real-world scenarios, where the stakes of misinformation or ethical missteps can be high.

The integration of RLHF, guided by HHH principles and enhanced by PPO, not only improves the trustworthiness and reliability of LLMs but also broadens their applicability across various sectors. This approach ensures that LLMs are not just powerful tools for innovation but also responsible entities that enhance our interaction with technology in line with societal norms and values.

Specialized LLMs as Powerful Reasoning Engines

The true potential of LLMs shines when they are used as reasoning engines rather than just brute force generators of content. LLM's perform poorly as databases but quite well as reasoning engines. Through fine-tuning and advanced techniques like RAG and PAL, LLMs can be trained for specific tasks and then given access to various databases and services, achieving performance that can surpass human capabilities in certain domains while ensuring high quality and accurate information.

These specialized LLM applications leverage the power of RAG and PAL to access external knowledge bases and perform sophisticated reasoning tasks. By combining the generative capabilities of LLMs with targeted retrieval of information, along with RLHF to enhance the quality of the models, these models can provide highly accurate and contextually relevant outputs, significantly reducing the risk of hallucinations. This approach ensures that LLMs are not only generating content but also providing well-reasoned, accurate, and reliable information.

For instance:

• Medical Diagnosis: By training an LLM on domain specific medical literature and patient data, or potentially fine-tuning on such content, an LLM can lean the language used in medical practice. Then the LLM can be used in a RAG pipeline to access vectorized databases of medical literature and treatment plans, resulting in an LLM solution that can assist healthcare professionals in diagnosing diseases, suggesting treatment plans, and staying updated with the latest medical research.
• Legal Research: Specialized LLMs trained on legal documents used in applications that have access to vectorized databases of case law can help lawyers quickly find relevant case law, draft legal documents, and analyze complex legal issues with high accuracy.
• Financial Analysis: In finance, LLMs can be originally trained on or potentially fine-tuned to read financial reports and news content to gain an ability to understand the specialized language used in the field. A vectorized database of financial reports and news content is created and updated as new content becomes available. Then the fine-tuned model can be used in an application that gets the latest financial report and news content from a vectorized database to assist quantitative and fundamental analysts in their tasks of analyzing the current state, potential trends and formulating investment strategies.
• Scientific Research: By integrating LLMs with scientific databases, researchers can generate hypotheses, summarize research papers, and identify novel insights, accelerating the pace of scientific discovery.

This list highlights some current applications where RAG and PAL achieve performance on par with or superior to human experts. While these technologies occasionally make mistakes, similar to human errors, they undoubtedly provide a competitive edge to professionals who utilize them. Ongoing research and the consistent improvement in the capabilities and reliability of LLMs suggest a trajectory towards broader adoption across various industries and new use cases. Initially, we expect these tools to augment professional tasks, enhancing the effectiveness and efficiency with which these tasks are executed. Over time, as the quality and reliability of these technologies become indisputably superior to human performance, we anticipate a shift from augmentation to more comprehensive automation.

Time Will Tell

While the Internet connected people and information in ways that were hard to imagine initially, LLMs are poised to enhance how we interact with information and technology across various domains. Just as the true potential of the Internet unfolded over time, the transformative power of LLMs is still emerging, promising to reshape industries and daily life in ways we are only beginning to understand.