Large language models (LLMs) have changed the game of natural language processing in recent times, enabling developers to build complex chat interfaces that can talk like humans. The potential for these interfaces spans customer service, virtual assistants in general, training, and education through entertainment platforms. However, constructing useful LLM chat interfaces is not without its complications and challenges.
I have been working on integrating AI functionalities and researching how chat interfaces can be built to navigate and utilize diverse data sources using LLM and Agents. For this proof of concept, I have used Azure OpenAI and other AI functionalities within Azure. It demonstrates various use cases, design patterns, and implementation options. The goal is to provide a foundation for architects and AI enthusiasts to explore the potential of Azure AI and make informed decisions about solution approaches. These use cases leverage diverse data sources such as SQL DB, Cosmos DB, CSV files, Multiple data sources, etc. The primary goal of this project is not only to showcase different use cases but also to explore various implementation options.
If you don’t have an Azure account set up, you can set one up here with some free credits.
Below is an example that shows how a natural language interface can be built on any CSV file using LLM and Agents. By leveraging the sample code, Users can upload a preprocessed CSV file, ask questions about the data, and get answers from the AI model.
You can find the complete file for chat_with_CSV here.
I have used environment variables. You can have them in a config file or define them in the same file.
|
file_formats = { “csv”: pd.read_csv, “xls”: pd.read_excel, “xlsx”: pd.read_excel, “xlsm”: pd.read_excel, “xlsb”: pd.read_excel,
}
aoai_endpoint = os.environ[“AZURE_OPENAI_ENDPOINT” ] aoai_api_key = os.environI “AZURE_OPENAI_API_KEY” ] deployment_name = os.environ[“AZURE_OPENAI_DEPLOYMENT_NAME” ] aoai_api_version = os.environ[ “AZURE_OPENAI_API_VERSION” ] #”2023-05-15″ aoai_api_type = os.environ[“AZURE_OPENAI_API_TYPE”] |
|
uploaded_file = st. file_uploader( “Upload a Data file”, type=list (file_formats. keys()), help=“Supports CSV and Excel files!”, on_change=clear_submit,
}
if uploaded_file: df = load_data(uploaded_file) |
For this, we need to import AzureChatOpenAI from langchain_openai and use the below params,
|
11m = AzureChatOpenAI (azure_endpoint=aoai_endpoint, openai_api_key = aoai_api_key, temperature=0, azure_deployment=deployment_name, openai_api_version=aoai_api_version, streaming=True) |
You can add more parameters if required; the details are in this link.
For this, we need to import create_pandas_dataframe_agent from langchain_experimental.agents and import AgentType from langchain.agent.
|
pandas_df_agent = create_pandas_dataframe_agent( 11m, df, verbose=True, agent_type=AgentType.OPENAI_FUNCTIONS, handle_parsing_errors=True, number_of_head_rows=df.shape[0] #send all the rows to LLM ) |
The pandas agent is a LangChain agent created using the create_pandas_dataframe_agent function, which takes the below inputs and params,
For this step, we need to use import StreamlitCallbackHandler from langchain.callbacks.
When the run method is called on the panda agent with the input message from the prompt and callback arguments, it goes through a sequence of steps to generate an answer.
|
with st.chat_message(“assistant”): st_cb = StreamlitCallbackHandler (st.container(), expand_new_thoughts=False) response = pandas_df_agent.run(st. session_state.messages, callbacks=[st_cb]) st.session_state.messages.append({“role”: “assistant”, “content”: response}) st.write (response) if __name__ == “__main__”: main() |
Initially, the agent identifies the task and selects the appropriate action to retrieve the required information from the data frame. After that, it observes the output, combines the observations, and generates the final answer.
The sample code below shows how a natural language interface can be built on structured data like SQL DB and NoSQL, like Cosmos DB, with Azure OpenAI’s capabilities. This can be utilized as an SQL programmer Assistant. The objective is to generate SQL code (SQL Server) to retrieve an answer to a natural language query.
You can find the entire file for chat_with_DB here.
I have used environment variables; you can have it as a config file or define it in the same file.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
#Load environment variables load_dotenv (“credentials.env”)
aoai_endpoint = os.environ[“AZURE_OPENAI_ENDPOINT”] aoai_api_key = os.environ[“AZURE_OPENAI_API_KEY”] deployment_name = os.environ[“AZURE_OPENAI_DEPLOYMENT_NAME”] aoai_api_version = os.environ[“AZURE_OPENAI_API_VERSION”] aoai_api_version_For_COSMOS = “2023-08-01-preview” #### for cosmos other API versio aoai_embedding_deployment = os.environ[“AZURE_EMBEDDING_MODEL” ]
sql_server_name = os.environ[“SQL_SERVER_NAME”] sal_server_db = os.environ[“SQL_SERVER_DATABASE”] sal_server_username = os.environ[“SQL_SERVER_USERNAME”] sal_Server_pwd = os.environ[“SQL_SERVER_PASSWORD”] SQL_ODBC_DRIVER_PATH = os.environ[“SQL_ODBC_DRIVER_PATH”]
COSMOS_MONGO_CONNECTIONSTRING = os.environ[“COSMOS_MONGO_CONNECTIONSTRING”] COSMOS_MONGO_DBNAME = os.environ[“COSMOS_MONGO_DBNAME”] COSMOS_MONGO_CONTAINER = os.environ[“COSMOS_MONGO_CONTAINER”] COSMOS_MONGO_API = os.environ[“COSMOS_MONGO_API”] |
For this, we need the python library openai. Once installed, you can use the library by importing openai and your api secret key to run the following:
To create clients, we utilize AzureOpenAI from Openai.
|
client = openai.AzureOpenAI( base_url=f“{aoai_endpoint}/openai/deployments/{deployment_name}/”, api_key=aoai_api_key, api_version=“2023-12-01-preview” )
response = client.chat.completions.create( model=deployment_name, # The deployment name you chose when you deployed the ChatGPT or GPT-4 model.
messages=messages, temperature=0, max_tokens=2000
) |
After getting the client, the API ChatCompletions gets the user prompt and generates the SQL query for the natural language query
If you want, you can add more params based on the requirement.
Utilizing panda read sql (pandas.read_sql(sql, con)) to read the sql query or database table into a data frame and return the pandas data frame containing the results of the query run.
|
def run_sql_query(aoai_sqlquery): ”’ Function to run the generated SQL Query on SQL server and retrieve output. Input: AOAI completion (SQL Query) Output: Pandas dataframe containing results of the query run ”’ conn = connect_sql_server () df = pd.read_sql(aoai_sqlquery, conn) return df |
To create clients, we utilize AzureOpenAI from Openai.
|
client = openai.AzureOpenAI( base_url=f“{aoai_endpoint}/openai/deployments/{deployment_name}/extensions/”, api_key=aoai_api_key, api_version=aoai_api_version_For_COSMOS ) |
After getting the client, the API ChatCompletions gets the user prompt and generates the query along with the response for the natural language query. Adding params details based on the requirement.
Be sure to include the “extra_body” parameter when using Cosmos as the data source.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
return client. chat.completions.create( model=deployment_name, messages=[ {“role”: m[“role”], “content”: m[“content”]} for m in messages ], stream=True, extra_body={ “dataSources”: [ { “type”: “AzureCosmosDB”, “parameters”: { “connectionString”: COSMOS._MONGO_CONNECTIONSTRING, “indexName” : index_name, “containerName”: COSMOS_MONGO_CONTAINER, “databaseName”: COSMOS_MONGO_DBNAME, “fieldsMapping”: { “contentFieldsSeparator”: “\n”, “contentFields”: [“text”], “filepathField”: “id”, “titleField”: “description”, “urlField”: None, “vectorFields”: [“embedding”], }, “inScope”: “true”, “roleInformation”: “You are an AI assistant that helps people find information from retrieved data”, “embeddingEndpoint”: f“{aoai_endpoint}/openai/deployments/{aoai_embedding_deployment}/embeddings/”, “embeddingKey”: aoai_api_key, “strictness”: 3, ” topNDocuments”: 5, } |
This POC shows various implementation patterns for building chat interfaces using Azure AI services and Orchestrators on multiple data sources.
You can find the entire file for chatting with multiple data sources here.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
load_dotenv(“credentials.env”)
aoai_endpoint = os.environ[“AZURE_OPENAI_ENDPOINT”] aoai_api_key = os.environ[“AZURE_OPENAI_API_KEY”] deployment_name = os.environI “AZURE_OPENAI_DEPLOYMENT_NAME”] aoai_api_version = os.environI “AZURE_OPENAI_API_VERSION”]
sal_server_name = os.environ “SQL_SERVER_NAME”] sal_server_db = os.environ “SQL_SERVER_DATABASE”] sql_server_username = os. environ[” SQL_SERVER_USERNAME”] sql_Server_pwd = os.environ “SQL_SERVER_PASSWORD”]
blob_connection_string = os.environI “BLOB_CONNECTION_STRING”] blob_sas_token = os.environ[“BLOB_SAS_TOKEN”]
azure_cosmos_endpoint = os.environ[“AZURE_COSMOSDB_ENDPOINT”] azure_cosmos_db = os.environ[“AZURE_COSMOSDB_NAME”] azure_cosmos_container = os.environ[“AZURE_COSMOSD_CONTAINER”] azure_cosmos_connection = os.environ[“AZURE_COSMOSDB_CONNECTION_STRING”] |
For this, we need to import AzureChatOpenAI from langchain_openai and use the below params,
|
# Set LLM 11m = AzureChatOpenAI(azure_endpoint=aoai_endpoint, openai_api_key = aoai_api_key, azure_deployment=deployment_name, openai._api_version=aoai_api_version, streaming=False, temperature=0.5, max_tokens=1000 ,callback_manager=cb_manager ) |
All methods DocSearchAgent, BingSearchAgent, SQLSearchAgent, and ChatGPTTool are in utils calls.
|
# Initialize our Tools/Experts text_indexes = [azure_search_covid_index] #, “cogsrch-index-csv”] doc_search = DocSearchAgent(11m=11m, indexes=text_indexes, k=10, similarity_k=4, reranker_th=1, sas_token=blob_sas_token, name=“docsearch”, verbose=False) #callback_manager=cb_manager
www_search = BingSearchAgent(11m=11m, k=5,verbose=False) sal_search = SQLSearchAgent(11m=11m, k=10,verbose=False) chatgpt_search = ChatGPTTo01(11m=11m) #callback_manager=cb_manager tools = [www_search, sql_search, doc_search, chatgpt_search] #csv_search, |
For this, we need to import AgentExecutor, create_openai_tools_agent from langchain.agents
Also, to run the agent import RunnableWithMessageHistory from langchain_core.runnables.history
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
agent = create_openai_tools_agent(11m, tools, CUSTOM_CHATBOT_PROMPT) agent_executor = AgentExecutor(agent=agent, tools=tools) brain_agent_executor = RunnableWithMessageHistory( agent_executor, get_session_history, input_messages_key=“question”, history_messages_key=“history”, history_factory_config=[ ConfigurableFieldSpec( id=“user_id”, annotation=str, name=“User ID”, description=“Unique identifier for the user.”, default=“”, is_shared=True, ), ConfigurableFieldSpec( id=“session_id”, annotation=str, name=“Session ID”, description=“Unique identifier for the conversation.”, default=“”, is_shared=True, ), ], ) |
Utilizing create_openai_tools_agent to create an agent that uses OpenAItools and
Create an agent executor by passing in the agent and tools and run the agent using RunnableWithMessageHistory. RunnableWithMessageHistory must always be called with a config that contains the appropriate parameters for the chat message history factory.
|
response = brain_agent_executor.invoke({“question”: prompt},config=config)[ “output”] |
YOUTUBE.COM/THENEWSTACK
Tech moves fast, don’t miss an episode. Subscribe to our YouTube
channel to stream all our podcasts, interviews, demos, and more.
