Skip to content

caesarHQ/textSQL

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

🚨 Check out the latest project from the creators of textSQL: Julius.ai 🚨

Natural Language → SQL

🌉 Demo on San Francisco City Data: SanFranciscoGPT.com

🇺🇸 Demo on US Census Data: CensusGPT.com

PRs Welcome Github Stars License GitHub commit activity

Welcome to textSQL, a project which uses LLMs to democratize access to data analysis. Example use cases of textSQL are San Francisco GPT and CensusGPT — natural language interfaces to public data (SF city data and US census data), enabling anyone to analyze and gain insights from the data.

Screenshot 2023-03-10 at 12 55 44 AM

🤔 How it works:

With CensusGPT, you can ask any question related to census data in natural language.

These natural language questions get converted to SQL using GPT-3.5 and are then used to query the census database.

Here are some examples:

Here is a similar example from sfGPT:

Diagram:

TextSQL diagram

🗺️ Roadmap:

We're splitting the roadmap for this project broadly into two categories:

1. Visualizations:

Currently, textSQL only supports visualizing zip codes and cities on an interactive map and bar chart using Mapbox + Plotly. But data can be visualized in other interesting ways such as Heatmaps and Pie charts. Not every kind of data can be (or should be) visualized on a map. For example, a query like "What percent of total crime in San Francisco is burglary vs in New York City" is perfect for visualizing as a stacked bar chart, but really hard to visualize on map.

Bar Chart:

Top 5 richest cities in Washington

[coming soon] Heatmap:

Screenshot 2023-03-10 at 12 58 33 AM

[coming soon] Visualization-GPT: A way to use natural language to create and iterate on data visualizations in natural language through a text-to-vega engine.

2. 🔌 Text-to-SQL BYOD (Bring Your Own Data) here

You can now connect your own database & datasets to textSQL and self-host the service. Our vision is to continue to modularize and improve this process.

Use cases

  • Public-facing interactive interfaces for data. Democratizing public data
  • Empowering researchers. Enabling journalists and other researchers to more easily explore data
  • Business intelligence. Reducing the burden on technical employees to build & run queries for non-technical

Setup instructions for BYOD are here.

📝 Additional Notes

Datasets:

A lot of the users of this project have asked for additional data for both CensusGPT and sfGPT — historical census data (trends), weather, health, transportation and real-estate data. Feel free to create a pull request, drop a link to your dataset in our Discord, or contribute data via our dedicated submission form.

More data → Better CensusGPT and sfGPT

Query Building:

Users build complex queries progressively. They start with a simple query like "Which neighborhoods in LA have the best schools?" and then progressively add details like "with median income that is under $100,000". One of the most powerful aspects of textSQL is enabling iterating on a query as a process of uncovering insights.

💻 How to Contribute:

Join our discord

ReadMe for the backend here

ReadMe for the frontend here

Note: Census data, like any other dataset, has its limitations and potential biases. Some data may not be collected or reported uniformly across different regions or time periods, which can affect the comparability of results. Users should keep these limitations in mind when interpreting the results of their queries and exercise caution when making decisions based on census data.