3 Areas Where RAG Implementation Can Be Improved

If you are in the business of handling large volumes of data, chances are your people can greatly benefit from implementing generative artificial intelligence (AI) into your practice.

One such framework under generative AI is retrieval-augmented generation — or RAG — which is a technique that allows customers to talk to their data using a chatbot. Specifically, it is an architectural approach that creates embeddings of your curated data and documents, retrieves relevant information based on the questions and prompts by end users, and inferences the responses and retrieved data with a large language model.

Think of your data as a library and your RAG chatbot as a librarian.

With RAG now an industry standard among technology companies, there are pitfalls to watch out for should you consider implementing it.

Here are three areas to prioritize for a smoother RAG implementation.

1. Context

When it comes to handling valuable information, your biggest obstacle might be — wait for it — people. Because people are reliably unreliable, you must put a lot of care into providing as much context as possible when training your AI agents. A good AI implementation should be equipped to understand context from incomplete, fragmented or seemingly obvious end-user prompts. The key here is attention to detail.

An example of a preventable RAG failure might look something like this:

Question: “What can you tell me about hot dogs?”

Answer: “Dogs get hot when they lay outside.”

While a person might naturally understand hot dog is a food item in this sentence, without better context, the language model might interpret the question more literally and provide an answer about your pet getting hot.

People want the right answers on their initial engagement with AI. While training humans to write better prompts can be burdensome, your RAG implementation should account for variations in context from a variety of inquiries.

2. Quantity and Quality of Information

When an end user turns to your chatbot to find a specific piece of information, they will expect an answer that is accurate and relevant. For this to happen, your organization’s leveraged data must also be accurate and relevant.

Just as not having enough data will affect the quality of AI-generated answers, having too much data can lead to inconsistent chatbot responses. If it requires multiple inquiries to the platform to obtain accurate information, especially for an area of your business that’s essential, that’s a red flag for you to re-examine the amount and relevancy of your data.

The benefit of RAG is that it will only respond to inquiries from the data that is uploaded upfront. End users can ask a wide variety of questions about the information or documents. This decreases the likelihood for AI hallucination, the generation of outputs that significantly deviate from the factual grounding of that curated data.

3. Subject Domains/Scaling

RAG is a tool that works well searching within one or two subject domains, but beyond that, it becomes confusing for chatbots. When there are too many subject domains for a RAG chatbot, it doesn’t know exactly where to go with a question. Add to the mix English-language complications — like bass (the fish) versus bass (the musical instrument) — and you can repeatedly encounter a situation where an answer to an inquiry is unhelpful.

This is where MRKL (Modular Reasoning, Knowledge and Language) comes into play.

Pronounced “miracle,” MRKL is a solution that expands on RAG with an agentic approach of putting an AI specialist in front of each data source as necessary. With MRKL, you would essentially leverage AI to select the best agent that has the correct data for the question being asked. Think of MRKL as a smart GPS that selects the best route (AI agent) to your destination (the answer) by choosing the most accurate and relevant source, ensuring the most precise and vetted answer.

If your organization deals with a lot of data on multiple topic areas, this is where MRKL excels.

Whether you need help introducing a chatbot to streamline open enrollment for your human resources department, or advanced analytics on employee questions, CDW can orchestrate AI enhancements for your organization every step of the way. Our rooted partnerships with trusted industry leaders and our knowledgeable solutions architects are our superpowers. Let us work with you to drive the best business results possible.