LLMs Aren’t a One-Size-Fits-All: Learn how our best practices can help guide your LLM models

As large language models (LLMs) like ChatGPT continue to make headlines, decision-makers across industries are exploring how these advanced AI tools could benefit their organizations. With the goals of automating processes, summarizing data, and even generating content, LLMs are quickly becoming part of everyday strategic business conversations. In the rush to adopt AI technologies, many companies should be asking: Are LLMs always the best solution, or could simpler, more tailored approaches offer better results?

For leaders tasked with navigating AI integration, it’s crucial to understand where LLMs are truly needed and where they are not. While these models can be powerful in specific contexts, they aren’t a universal solution as some are trying to get us all to believe. Here are some of the common misconceptions about LLMs we see and the framework we use to help choose the right AI tools for our business needs—whether that means leveraging the full power of an LLM or selecting a more targeted, customized model. Hopefully the work that we are doing will help serve as guideposts for other organizations that are trying to create the same kind of framework internally.

LLMs- The Swiss Army Knife

“ChatGPT”, “AI”, “LLMs” - It’s rare that a day goes by that you don’t hear one of those buzzwords thrown around as a vague term for general artificial intelligence (AI). With the public release of ChatGPT by OpenAI in late 2022, AI was thrust into the spotlight and LLMs were touted as a swiss army knife for all our AI needs. Since then, numerous big tech companies have been scrambling to release their own proprietary models to claim a piece of the market. Additionally, companies have been finding a myriad of ways to incorporate LLMs into their products, courses have been developed to teach LLM usage, and job openings for LLM engineers have been flooding the market. 

This gold rush for LLM market share has left many consumers with whiplash trying to keep up with the what, when, and how of LLMs for their business. In particular, data science (DS) problems are not straightforward in how they can benefit from LLMs, which serve as just one more option in a large bucket of DS/AI techniques, methodologies, and tools. Here are a few general guidelines to keep in mind to help navigate these waters and find an effective solution for specific business DS applications.

Who, What, When, Where of Using LLMs

1. Don’t use a jackhammer where a chisel will do a better job

A large continuum represents the complexity of AI techniques and tools. LLMs are at the end as one of the current most complex with simple predictive models taking residence on the least complex side. While a variety of applications could benefit from their use, LLMs can introduce unnecessary complexity for many business problems. It can be very beneficial to evaluate the spectrum of DS solutions and try to identify the least complex tool that will do the job. This will result in a more explainable, effective solution that minimizes overhead and maintenance.

2. Understand that LLMs are not a one-stop shop for all DS/AI needs

The media hype around LLMs has created significant confusion around what they can and cannot do well. In addition to the idiosyncrasies that have gained trending notoriety - such as weird hand rendering and providing wrong answers to simple questions - there are a number of applications that LLMs aren’t good at. For this reason, it’s important to do some research into the best applications of LLMs (such as this one) before implementing a DS solution. In general, LLMs are good at tasks such as summarizing text, aggregating information, generating content, and facilitating coding. This is because these tasks benefit from the aggregation of massive amounts of similar types of data, which is a significant backing of LLMs. Knowing this background can help position an LLM for business applications that will realize the most benefit.

3. Advanced algorithms and large amounts of general data may not be a suitable substitute for domain-specific business data

LLMs are built using enormous amounts of data about every topic under the sun. However, even with all that information, they most likely have not been exposed to data specific to a given business, industry or domain. This means that for problems that surround business-specific data, a more customized (often simpler model) can often demonstrate better performance by being more targeted and not having to combat additional noise introduced by a broad, generalized model. A business should evaluate its data - quantity and quality - as to its utility in training a customized model before spending extra time tailoring an LLM for its business problem.

At Fama, we consider these guidelines as we also navigate the rapidly changing AI landscape. Our AI toolbox contains a number of different techniques and algorithms to suit our different data science needs. Our data science and analytics team looks at each problem individually to ensure the developed solutions are appropriate to satisfy our business needs. For some, an LLM has given us the best results; for others, we’ve achieved highly effective results with a simple customized model. The landscape of data science and artificial intelligence is vast, which provides a wealth of opportunities to create all types of solutions to solve business problems, as long as users keep in mind the pros and cons of all the different tools and apply them intelligently.