Skip to content

Updated project metadata #46

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 11, 2024
Merged

Updated project metadata #46

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Databricks Labs LSQL
[![PyPI - Version](https://img.shields.io/pypi/v/databricks-labs-lightsql.svg)](https://pypi.org/project/databricks-labs-lsql)
[![build](https://github.com/databrickslabs/ucx/actions/workflows/push.yml/badge.svg)](https://github.com/databrickslabs/lsql/actions/workflows/push.yml) [![codecov](https://codecov.io/github/databrickslabs/lsql/graph/badge.svg?token=p0WKAfW5HQ)](https://codecov.io/github/databrickslabs/ucx) [![lines of code](https://tokei.rs/b1/github/databrickslabs/lsql)]([https://codecov.io/github/databrickslabs/lsql](https://github.com/databrickslabs/lsql))

Execute SQL statements in a stateless manner.
[Lightweight](https://github.com/databrickslabs/lsql/blob/main/docs/comparison.md) execution of SQL queries through Databricks SDK for Python.

<!-- TOC -->
* [Databricks Labs LSQL](#databricks-labs-lsql)
Expand Down
40 changes: 23 additions & 17 deletions docs/comparison.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,9 @@ the stateful [Databricks SQL Connector for Python](https://docs.databricks.com/e
| Light-weight mocking | no | no | no | **yes** |
| Extended support for dataclasses | limited | no | no | **yes** |
| Strengths | almost Databricks Runtime, but locally | works with Python ecosystem | works with ODBC ecosystem | **tiny** |
| Compressed size | 60M | 51M (85%) | 44M (73.3%) | **0.8M (1.3%)** |
| Uncompressed size | 312M | 280M (89.7%) | ? | **30M (9.6%)** |
| Direct dependencies | 23 | 14 | 2 | **1** (Python SDK) |
| Compressed size | 60M | 51M (85%) | 44M (73.3%) | **2M (3.3%)** |
| Uncompressed size | 312M | 280M (89.7%) | ? | **43M (13.7%)** |
| Direct dependencies | 23 | 14 | 2 | **3** (Python SDK, SQLglot) |
| Unified Authentication | yes (via Python SDK) | no | no | **yes** (via Python SDK) |
| Works with | Databricks Clusters only | Databricks Clusters and Databricks SQL Warehouses | Databricks Clusters and Databricks SQL Warehouses | **Databricks SQL Warehouses only** |
| Full equivalent of Databricks Runtime | yes | no | no | **no** |
Expand All @@ -40,6 +40,7 @@ the stateful [Databricks SQL Connector for Python](https://docs.databricks.com/e
| Official | yes | yes | yes | **no** |
| Version checked | 14.0.1 | 2.9.3 | driver v2.7.5 | 0.1.0 |

[[back to top](#library-size-comparison)]

## Databricks Connect

Expand Down Expand Up @@ -106,6 +107,7 @@ urllib3-2.0.6
Direct dependencies 23
```

[[back to top](#library-size-comparison)]

## Databricks SQL Connector

Expand Down Expand Up @@ -172,13 +174,16 @@ thrift-0.16.0 typing-extensions-4.8.0 tzdata-2023.3 urllib3-2.0.6
Direct dependencies 14
```

[[back to top](#library-size-comparison)]

## Databricks Labs LightSQL

Compressed:

```shell
$ cd $(mktemp -d) && pip3 wheel databricks-sdk && echo "All wheels $(du -hs)" && echo "1Mb+ wheels: $(find . -type f -size +1M | xargs du -h | sort -h -r)" && cd -
All wheels 1.8M .
$ cd $(mktemp -d) && pip3 wheel databricks-labs-lsql && echo "All wheels $(du -hs)" && echo "1Mb+ wheels: $(find . -type f -size +1M | xargs du -h | sort -h -r)" && cd -
All wheels
2.0M .
1Mb+ wheels:
~
```
Expand All @@ -188,21 +193,22 @@ Uncompressed:
```shell
databricks-sdk $ python3 -m venv venv
databricks-sdk $ source ./venv/bin/activate
(venv) databricks-labs-lsql $ pip install databricks-sdk
(venv) databricks-labs-lsql $ pip install databricks-labs-lsql
...
Successfully installed certifi-2023.7.22 charset-normalizer-3.3.0 databricks-sdk-0.10.0 idna-3.4 requests-2.31.0
urllib3-2.0.6
Successfully installed cachetools-5.3.3 certifi-2024.2.2 charset-normalizer-3.3.2 databricks-labs-blueprint-0.2.5 databricks-labs-lsql-0.1.0 databricks-sdk-0.21.0 google-auth-2.28.2 idna-3.6 pyasn1-0.5.1 pyasn1-modules-0.3.0 requests-2.31.0 rsa-4.9 sqlglot-22.2.1
(venv) databricks-labs-lsql $ du -hs .
30M .
43M .
(venv) databricks-labs-lsql $ du -ha . | sort -h -r | head
30M ./venv/lib/python3.11/site-packages
30M ./venv/lib/python3.11
30M ./venv/lib
30M ./venv
30M .
43M ./venv/lib/python3.11/site-packages
43M ./venv/lib/python3.11
43M ./venv/lib
43M ./venv
43M .
16M ./venv/lib/python3.11/site-packages/pip
13M ./venv/lib/python3.11/site-packages/pip/_vendor
5.2M ./venv/lib/python3.11/site-packages/setuptools
4.4M ./venv/lib/python3.11/site-packages/databricks/sdk
4.4M ./venv/lib/python3.11/site-packages/databricks
6.3M ./venv/lib/python3.11/site-packages/databricks
5.9M ./venv/lib/python3.11/site-packages/databricks/sdk
5.3M ./venv/lib/python3.11/site-packages/setuptools
```

[[back to top](#library-size-comparison)]
9 changes: 4 additions & 5 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,16 +20,15 @@ classifiers = [
"Programming Language :: Python :: Implementation :: PyPy",
]
dependencies = [
"databricks-labs-blueprint~=0.2.5",
"databricks-labs-blueprint~=0.4.0",
"databricks-sdk~=0.21.0",
"PyYAML>=6.0.0,<7.0.0",
"sqlglot~=22.3.1"
]

[project.urls]
Documentation = "https://github.com/databrickslabs/databricks-labs-lsql#readme"
Issues = "https://github.com/databrickslabs/databricks-labs-lsql/issues"
Source = "https://github.com/databrickslabs/databricks-labs-lsql"
Documentation = "https://github.com/databrickslabs/lsql#readme"
Issues = "https://github.com/databrickslabs/lsql/issues"
Source = "https://github.com/databrickslabs/lsql"

[build-system]
requires = ["hatchling"]
Expand Down