Pushing the limits of large language models: The future of LLM research

Large language models (LLMs) have captured the public's attention with their vast potential applications. LLMs have already demonstrated their proficiency in email composition and software code generation, among other areas. As LLMs gain traction, there is increasing apprehension regarding their limitations, which impede their use in various applications. These limitations include fabricating false facts, floundering in tasks requiring common sense, and consuming vast amounts of energy.

Numerous research areas warrant exploration to address these challenges and enable LLMs to be leveraged in a wider range of domains.

What are large language models (LLMs)?

Large language models are artificial intelligence (AI) models designed to process and understand human language. They are typically built using deep learning techniques and trained on vast amounts of text data from the internet. The goal of these models is to learn how language works and how to generate human-like responses to prompts or questions.

Stopping “AI hallucination”

One of the primary challenges posed by large language models (LLMs) like ChatGPT and GPT-3 is their tendency to "hallucinate," which refers to their ability to generate text that is plausible but not grounded in reality, leading to the creation of false information. Users have raised concerns about ChatGPT, highlighting instances where the model has generated text that sounds convincing but is factually incorrect.

Researchers have explored "knowledge retrieval" techniques to mitigate this issue. Knowledge retrieval involves augmenting the LLM with additional context from external sources like Wikipedia or domain-specific knowledge bases. By providing this additional context, the LLM can generate text that is more grounded in reality.

One example of this approach is the "retrieval-augmented language model pre-training" (REALM) introduced by Google in 2020. The REALM model uses a "neural retriever" module to retrieve relevant documents from a knowledge corpus, which are then used as context for the LLM to generate the final output.

Other advancements in knowledge retrieval include "in-context retrieval augmented language modeling," a technique developed by AI21 Labs that allows for easy implementation of knowledge retrieval in different black-box and open-source LLMs.

You.com and the version of ChatGPT used in Bing also utilize knowledge retrieval techniques. After receiving a prompt, the LLM creates a search query, retrieves relevant documents, and generates output using those sources. Links to the sources are also provided, enabling verification of the information generated by the model.

While knowledge retrieval is not a foolproof solution and still has limitations, it represents a promising step forward in addressing the challenges posed by LLMs.

Making AI understand us better

Language models such as ChatGPT and GPT-3 are highly impressive in generating text, but they do not truly understand language or the world as humans do. This means that they can make mistakes that seem nonsensical to us.

Techniques called "prompt engineering" involve crafting prompts to guide LLMs to produce more reliable output. One such technique is "few-shot learning," where a few examples are given along with the prompt to guide the model in producing the output. Creating datasets of few-shot examples can improve LLM performance without retraining or fine-tuning them.

Another technique, called "chain-of-thought prompting," enables LLMs to produce an answer and its steps to reach it. This is particularly useful for applications that require logical reasoning or step-by-step computation. There are different CoT methods, including a few-shot technique that prepends the prompt with a few examples of step-by-step solutions, zero-shot CoT, which uses a trigger phrase to force the LLM to produce the steps it took, and faithful chain-of-thought reasoning that uses multiple steps and tools to ensure that the LLM's output accurately reflects the steps it used to reach the result.

While reasoning and logic are fundamental challenges of deep learning that require new approaches to AI, better prompting techniques can help reduce LLMs' logical errors and help troubleshoot their mistakes.

Fine-tuning AI

Fine-tuning LLMs with domain-specific datasets can greatly enhance their performance and robustness within those domains. This is especially beneficial in cases where a general-purpose LLM, like GPT-3, would perform poorly.

Recent advancements in fine-tuning techniques have further improved the accuracy of models. One such technique is "reinforcement learning from human feedback" (RLHF), which was used to train ChatGPT. In RLHF, human annotators provide feedback on the answers generated by a pre-trained LLM. This feedback is then used to train a reward system that fine-tunes the LLM to better align with user intents. ChatGPT has been highly successful in following user instructions due to the effectiveness of RLHF.

The next step in this field is for companies like OpenAI and Microsoft, which provide LLM platforms, to develop tools that enable businesses to create their own RLHF pipelines and customize models for their specific applications.

Overall, fine-tuning LLMs with domain-specific data and using reinforcement learning techniques like RLHF can greatly enhance the accuracy and reliability of these models.

Reducing the costs of LLMs

One of the major challenges with LLMs is their high costs, making them unaffordable for certain companies and applications. There are ongoing efforts to reduce the costs of LLMs to address this issue. Some of these efforts involve creating more efficient hardware, including AI processors specifically designed for LLMs.

Another promising direction is the development of new LLMs that can match the performance of larger models but with fewer parameters. Facebook has developed a family of small, high-performance LLMs called LLaMA, which are accessible for research labs and organizations lacking the infrastructure to run large models.

According to Facebook, the 13-billion parameter version of LLaMa outperforms the 175-billion parameter version of GPT-3 on major benchmarks, and the 65-billion variant matches the performance of the largest models, including the 540-billion parameter PaLM.

While LLMs face many challenges, including ethical and safety concerns, these developments are promising and could help make LLMs more reliable and accessible to the developer and research communities.