Fixing LLM Hallucination Conversations
Enhancing LLM Conversations via Semantic Routing
Hi everyone, what's up? how things are up to? i hope its good.
Today I will demonstrate one of the new and the best techniques on how to avoid LLM hallucination which is one of the major problems that we fear of.
To avoid LLM hallucinations we need to steer the input to the right LLM agents. So let’s say example we want to query a database and if we do not have any user input steering mechanism to its right LLM agents then it will look for answers everywhere from query chain to rag to bot conversation and other agents. Hence computationally expensive and ends up with confusion with errors.
Previously we used to depend heavily on Prompt engineering but still, we know we don't have enough confidence in hallucination. Thus another function like langchain Tools came into the picture. Lang-chain tools tries to create ‘observations’ and ‘thoughts’ internally for all the LLM agents( like query chain, rag, conversation bot, other agents etc. ) for n-iterations and give the best summary but again internally it creates a lot of confusion because when you multi-class multi-agents it tries to create a MIXED summary of query_chain output like( select * from tablename where customer_ID = 2) and RAG (“documents”) and other agents. Thus in the end ERROR.
For that, i have observed lang-chain tools are good for reasoning only if it have same type of agents, (like rag chain 1, rag chain2,,,,3,etc) as it combine multi-agents output and creates summarize version of output.
Semantic Routing helps user input to steer to the right multi-class multi-agent and thus helps to avoid hallucinations.
Now let’s get some hands-on to understand how we can apply
First, install the sematic-routing
pip install semantic-router
Then import the packages
from semantic_router import Route
from semantic_router.encoders import CohereEncoder
from semantic_router.layer import RouteLayer
We will create the route objects
# we could use this as a guide for our chatbot to avoid political conversations
politics = Route(
name="politics",
utterances=[
"isn't politics the best thing ever",
"why don't you tell me about your political opinions",
"don't you just love the president" "don't you just hate the president",
"they're going to destroy this country!",
"they will save the country!",
],
)
# this could be used as an indicator to our chatbot to switch to a more
# conversational prompt
chitchat = Route(
name="chitchat",
utterances=[
"how's the weather today?",
"how are things going?",
"lovely weather today",
"the weather is horrendous",
"let's go to the chippy",
],
)
# we place both of our decisions together into single list
routes = [politics, chitchat]
We will also define the text embedding encoder, i would prefer to use openai() to avoid dependency errors
import os
from semantic_router.encoders import CohereEncoder, OpenAIEncoder
# for Cohere
os.environ["COHERE_API_KEY"] = "<YOUR_API_KEY>"
encoder = CohereEncoder()
# or for OpenAI
os.environ["OPENAI_API_KEY"] = "<YOUR_API_KEY>"
encoder = OpenAIEncoder()
The route layer handles our semantic decision-making.
from semantic_router.layer import RouteLayer
rl = RouteLayer(encoder=encoder, routes=routes)
thats it!
output1 = rl("don't you love politics?").name
output2 = rl("how's the weather today?").name
Output1 : politics Output2: chitchat
Again these are based on the utterances the more you add the more will able to detect the user input.
Now all we need is to wrap it in if-else conditions.
if output1 == 'politics':
print("route1")
l1 = LLM(....,prompt= output1)
else output2 = 'chitchat'
print("route2")
l2 = LLM(....., prompt = output2)
Here is it the pictorial representation.
Thanks again, for your time, if you enjoyed this short article there are tons of topics in advanced analytics, data science, and machine learning available in my medium repo. https://medium.com/@bobrupakroy
Some of my alternative internet presences are Facebook, Instagram, Udemy, Blogger, Issuu, Slideshare, Scribd, and more.
Also available on Quora @ https://www.quora.com/profile/Rupak-Bob-Roy
Let me know if you need anything. Talk Soon.