Fixing LLM Hallucination Conversations

Enhancing LLM Conversations via Semantic Routing

Hi everyone, what's up? how things are up to? i hope its good.

Today I will demonstrate one of the new and the best techniques on how to avoid LLM hallucination which is one of the major problems that we fear of.

To avoid LLM hallucinations we need to steer the input to the right LLM agents. So let’s say example we want to query a database and if we do not have any user input steering mechanism to its right LLM agents then it will look for answers everywhere from query chain to rag to bot conversation and other agents. Hence computationally expensive and ends up with confusion with errors.

Previously we used to depend heavily on Prompt engineering but still, we know we don't have enough confidence in hallucination. Thus another function like langchain Tools came into the picture. Lang-chain tools tries to create ‘observations’ and ‘thoughts’ internally for all the LLM agents( like query chain, rag, conversation bot, other agents etc. ) for n-iterations and give the best summary but again internally it creates a lot of confusion because when you multi-class multi-agents it tries to create a MIXED summary of query_chain output like( select * from tablename where customer_ID = 2) and RAG (“documents”) and other agents. Thus in the end ERROR.

For that, i have observed lang-chain tools are good for reasoning only if it have same type of agents, (like rag chain 1, rag chain2,,,,3,etc) as it combine multi-agents output and creates summarize version of output.

Semantic Routing helps user input to steer to the right multi-class multi-agent and thus helps to avoid hallucinations.

Now let’s get some hands-on to understand how we can apply

First, install the sematic-routing

pip install semantic-router

Then import the packages

from semantic_router import Route
from semantic_router.encoders import CohereEncoder
from semantic_router.layer import RouteLayer

We will create the route objects

# we could use this as a guide for our chatbot to avoid political conversations
politics = Route(
    name="politics",
    utterances=[
        "isn't politics the best thing ever",
        "why don't you tell me about your political opinions",
        "don't you just love the president" "don't you just hate the president",
        "they're going to destroy this country!",
        "they will save the country!",
    ],
)

# this could be used as an indicator to our chatbot to switch to a more
# conversational prompt
chitchat = Route(
    name="chitchat",
    utterances=[
        "how's the weather today?",
        "how are things going?",
        "lovely weather today",
        "the weather is horrendous",
        "let's go to the chippy",
    ],
)

# we place both of our decisions together into single list
routes = [politics, chitchat]

We will also define the text embedding encoder, i would prefer to use openai() to avoid dependency errors

import os
from semantic_router.encoders import CohereEncoder, OpenAIEncoder

# for Cohere
os.environ["COHERE_API_KEY"] = "<YOUR_API_KEY>"
encoder = CohereEncoder()

# or for OpenAI
os.environ["OPENAI_API_KEY"] = "<YOUR_API_KEY>"
encoder = OpenAIEncoder()

The route layer handles our semantic decision-making.

from semantic_router.layer import RouteLayer

rl = RouteLayer(encoder=encoder, routes=routes)

thats it!

output1 = rl("don't you love politics?").name
output2 = rl("how's the weather today?").name

Output1 : politics Output2: chitchat

Again these are based on the utterances the more you add the more will able to detect the user input.

Now all we need is to wrap it in if-else conditions.

if output1 == 'politics':
  print("route1")
  l1 = LLM(....,prompt= output1)
else output2 = 'chitchat'
  print("route2")
  l2 = LLM(....., prompt = output2)

Here is it the pictorial representation.

Semantic-routing LLM

Thanks again, for your time, if you enjoyed this short article there are tons of topics in advanced analytics, data science, and machine learning available in my medium repo. https://medium.com/@bobrupakroy

Some of my alternative internet presences are Facebook, Instagram, Udemy, Blogger, Issuu, Slideshare, Scribd, and more.

Also available on Quora @ https://www.quora.com/profile/Rupak-Bob-Roy

Let me know if you need anything. Talk Soon.

We r Mountain Sherpa