-
|
Is that possible to build a Data Analyst Agent get data summary or data insights from pandas dataframe? Do you suggest any best practices? I am trying to simulate the below concept. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 6 replies
-
|
Yes, it is possible to build a Data Analyst Agent to get data summaries or insights from a pandas DataFrame. Here is a basic example using the OpenAI Function API and the
Here is a sample implementation: import pandas as pd
from llama_index.agent.openai import OpenAIAgent
from llama_index.llms.openai import OpenAI
from llama_index.experimental.query_engine.pandas import PandasQueryEngine
from llama_index.core.schema import QueryBundle
# Sample DataFrame
data = {
'Name': ['Alice', 'Bob', 'Charlie', 'David'],
'Age': [24, 30, 22, 35],
'Salary': [70000, 80000, 60000, 90000]
}
df = pd.DataFrame(data)
# Initialize the LLM
llm = OpenAI(model="gpt-3.5-turbo-0613")
# Initialize the PandasQueryEngine
pandas_query_engine = PandasQueryEngine(df=df, llm=llm, verbose=True)
# Define a tool for querying the DataFrame
def query_dataframe(query_str):
query_bundle = QueryBundle(query_str=query_str)
response = pandas_query_engine._query(query_bundle)
return response.response
# Define tools
tools = {
'query_dataframe': query_dataframe
}
# Initialize the OpenAI agent
agent = OpenAIAgent.from_tools(tools, llm=llm, verbose=True)
# Example task: Query the DataFrame
task = {
'tool': 'query_dataframe',
'params': {'query_str': 'What is the average age and salary?'}
}
# Execute the task
result = agent.query(task)
print(result)This setup allows the agent to provide data summaries or insights from the DataFrame effectively. Note that the For more advanced use cases and to build custom agents, LlamaIndex provides a comprehensive framework that includes components for query planning, tool use, and memory modules. You can explore these components and additional resources in the LlamaIndex documentation to enhance your agent's capabilities. For further details, you can refer to the following sources:
|
Beta Was this translation helpful? Give feedback.
-
|
@dosu still not working. Now I am getting str error. |
Beta Was this translation helpful? Give feedback.
-
|
The string type error you're encountering is likely due to the way the import pandas as pd
from llama_index.agent.openai import OpenAIAgent
from llama_index.llms.openai import OpenAI
from llama_index.experimental.query_engine.pandas import PandasQueryEngine
from llama_index.core.schema import QueryBundle
# Sample DataFrame
data = {
'Name': ['Alice', 'Bob', 'Charlie', 'David'],
'Age': [24, 30, 22, 35],
'Salary': [70000, 80000, 60000, 90000]
}
df = pd.DataFrame(data)
# Initialize the LLM
llm = OpenAI(model="gpt-3.5-turbo-0613")
# Initialize the PandasQueryEngine
pandas_query_engine = PandasQueryEngine(df=df, llm=llm, verbose=True)
# Define a tool for querying the DataFrame
def query_dataframe(query_str):
query_bundle = QueryBundle(query_str=query_str)
response = pandas_query_engine.query(query_bundle)
return response.response
# Define tools
tools = {
'query_dataframe': query_dataframe
}
# Initialize the OpenAI agent
agent = OpenAIAgent.from_tools(tools=tools, llm=llm, verbose=True)
# Example task: Query the DataFrame
task = {
'tool': 'query_dataframe',
'params': {'query_str': 'What is the average age and salary?'}
}
# Execute the task
result = agent.chat(task['params']['query_str'])
print(result)Key Points:
Additional Checks:
By ensuring these corrections and checks, the string type error should be resolved, and the task should execute successfully. If the issue persists, please provide the exact error message for further diagnosis.
|
Beta Was this translation helpful? Give feedback.
-
|
@dosu. Thank you. How I measure the token usage for above agent action? can you give me a sample code please? |
Beta Was this translation helpful? Give feedback.
-
|
To measure the token usage for the agent's actions in your current setup, you can integrate the import pandas as pd
from llama_index.agent.openai import OpenAIAgent
from llama_index.llms.openai import OpenAI
from llama_index.experimental.query_engine.pandas import PandasQueryEngine
from llama_index.core.schema import QueryBundle
from llama_index.core.callbacks import CallbackManager, TokenCountingHandler
import tiktoken
# Sample DataFrame
data = {
'Name': ['Alice', 'Bob', 'Charlie', 'David'],
'Age': [24, 30, 22, 35],
'Salary': [70000, 80000, 60000, 90000]
}
df = pd.DataFrame(data)
# Initialize the LLM
llm = OpenAI(model="gpt-3.5-turbo-0613")
# Initialize the token counter with the appropriate tokenizer
token_counter = TokenCountingHandler(
tokenizer=tiktoken.encoding_for_model("gpt-3.5-turbo").encode
)
# Set the global settings for the LLM and callback manager
Settings.llm = llm
Settings.callback_manager = CallbackManager([token_counter])
# Initialize the PandasQueryEngine
pandas_query_engine = PandasQueryEngine(df=df, llm=llm, verbose=True)
# Define a tool for querying the DataFrame
def query_dataframe(query_str):
query_bundle = QueryBundle(query_str=query_str)
response = pandas_query_engine.query(query_bundle)
return response.response
# Define tools
tools = {
'query_dataframe': query_dataframe
}
# Initialize the OpenAI agent
agent = OpenAIAgent.from_tools(tools=tools, llm=llm, verbose=True)
# Example task: Query the DataFrame
task = {
'tool': 'query_dataframe',
'params': {'query_str': 'What is the average age and salary?'}
}
# Execute the task
result = agent.chat(task['params']['query_str'])
print(result)
# Print the token usage
print(f"Total LLM Token Count: {token_counter.total_llm_token_count}")
print(f"Prompt LLM Token Count: {token_counter.prompt_llm_token_count}")
print(f"Completion LLM Token Count: {token_counter.completion_llm_token_count}")
print(f"Total Embedding Token Count: {token_counter.total_embedding_token_count}")This setup will allow you to measure the token usage for the agent's actions while querying the DataFrame.
|
Beta Was this translation helpful? Give feedback.
To measure the token usage for the agent's actions in your current setup, you can integrate the
TokenCountingHandlerfrom the LlamaIndex library. Here is the modified version of your code with theTokenCountingHandlerintegrated: