Skip to content
Merged
23 changes: 16 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,20 +14,29 @@

## Documentation

For reference and tutorials, see the [Firebolt Python SDK reference](https://python-sdk.docs.firebolt.io/).
For reference and tutorials, see the [Firebolt Python SDK reference](https://python-sdk.docs.firebolt.io/en/0.x/).

## Connection parameters
These parameters are used to connect to a Firebolt database:
- **engine_url** - url for an engine to make requests to. Can be retrieved from Web UI, or from [engine](https://github.com/firebolt-db/firebolt-sdk/tree/main/src/firebolt/model/engine.py) attribute `endpoint`
- Username/Password authentication
- **username** - account username
- **password** - account password
- Service Account authentication
- **client_id** - service account client id
- **client_secret** - service account client secret
- **database** - name of the database to make queries to
- **username** - account username
- **password** - account password
- **engine_url** - url for an engine to make requests to. Can be retrieved from Web UI, or from [engine](https://github.com/firebolt-db/firebolt-sdk/tree/0.x/src/firebolt/model/engine.py) attribute `endpoint`
(Mutually exclusive with **engine_name**)
- **engine_name** - name of an engine to make requests to. Can be retrieved from Web UI, or from [engine](https://github.com/firebolt-db/firebolt-sdk/tree/0.x/src/firebolt/model/engine.py) attribute `name`
(Mutually exclusive with **engine_url**)

Optional parameters
- **account_name** - name of a Firebolt account to use. Default account is used if it's not provided
- **use_token_cache** - whether to store obtained authentication tokens in cache file. Defaults to True
- **api_endpoint** - api hostname for logging in. Defaults to `api.app.firebolt.io`.

## Examples
See [PEP-249](https://www.python.org/dev/peps/pep-0249) for the DB API reference and specifications. An example [jupyter notebook](https://github.com/firebolt-db/firebolt-sdk/tree/main/examples/dbapi.ipynb) is included to illustrate the use of the Firebolt API.
See [PEP-249](https://www.python.org/dev/peps/pep-0249) for the DB API reference and specifications. An example [jupyter notebook](https://github.com/firebolt-db/firebolt-sdk/tree/0.x/examples/dbapi.ipynb) is included to illustrate the use of the Firebolt API.

## Special considerations
### Cursor objects should not be shared between threads
Expand All @@ -39,7 +48,7 @@ By default, firebolt-sdk uses `datetime` module to parse date and datetime value

## Contributing

See: [CONTRIBUTING.MD](https://github.com/firebolt-db/firebolt-sdk/tree/main/CONTRIBUTING.MD)
See: [CONTRIBUTING.MD](https://github.com/firebolt-db/firebolt-sdk/tree/0.x/CONTRIBUTING.MD)

## License
The Firebolt DB API is licensed under the [Apache License Version 2.0](https://github.com/firebolt-db/firebolt-sdk/tree/main/LICENSE) software license.
The Firebolt DB API is licensed under the [Apache License Version 2.0](https://github.com/firebolt-db/firebolt-sdk/tree/0.x/LICENSE) software license.
21 changes: 8 additions & 13 deletions README_MANAGEMENT.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,20 @@
# Resource Management
### Usage

See: [examples/management.ipynb](https://github.com/firebolt-db/firebolt-sdk/tree/main/examples/management.ipynb).
See: [examples/management.ipynb](https://github.com/firebolt-db/firebolt-sdk/tree/0.x/examples/management.ipynb).

### Configuration

To use the SDK, you generally will want to set the following environment variables:
```
FIREBOLT_USER='email@domain.com'
FIREBOLT_PASSWORD='*****'
FIREBOLT_ACCOUNT='my_account'
FIREBOLT_SERVER='api.app.firebolt.io'
FIREBOLT_DEFAULT_REGION='us-east-1'
```

* You can store these in a `.env` file
* environment variables on your system always take precedence over those in `.env`

Once the environment variables are defined (either on your system or in `.env`),
Once the environment variables are defined,
you can initialize a ResourceManager with:

```python
Expand All @@ -29,23 +27,20 @@ print(rm.regions.default_region) # see your default region
Or you can configure settings manually:

```python
from firebolt.client import DEFAULT_API_URL
from firebolt.client.auth import UsernamePassword
from firebolt.service.manager import ResourceManager
from firebolt.common.settings import Settings
from pydantic import SecretStr

rm = ResourceManager(settings=Settings(
server="api.app.firebolt.io",
user="email@domain.com",
password=SecretStr("*****"),
auth=UsernamePassword("email@domain.com", "*****")
account_name="account", # Necessary if you have multiple accounts.
server=DEFAULT_API_URL,
default_region="us-east-1",
))
print(rm.client.account_id) # see your account id
```

Under the hood, configuration works via Pydantic,
see [here](https://pydantic-docs.helpmanual.io/usage/settings/).

### Contributing

See: [CONTRIBUTING.MD](https://github.com/firebolt-db/firebolt-sdk/tree/main/CONTRIBUTING.MD)
See: [CONTRIBUTING.MD](https://github.com/firebolt-db/firebolt-sdk/tree/0.x/CONTRIBUTING.MD)
142 changes: 69 additions & 73 deletions examples/dbapi.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
"source": [
"from firebolt.db import connect\n",
"from firebolt.client import DEFAULT_API_URL\n",
"from firebolt.client.auth import UsernamePassword, ServiceAccount\n",
"from datetime import datetime"
]
},
Expand All @@ -25,7 +26,7 @@
"id": "9cc3c3d4",
"metadata": {},
"source": [
"### Database credentials"
"### Connection parameters"
]
},
{
Expand All @@ -35,15 +36,29 @@
"metadata": {},
"outputs": [],
"source": [
"# Using username/password authentication\n",
"username = \"\"\n",
"password = \"\"\n",
"\n",
"# Using service account authentication\n",
"client_id = \"\"\n",
"client_secret = \"\"\n",
"\n",
"assert not (username and client_id), \"Specify only one set of credentials\"\n",
"if username and password:\n",
" auth = UsernamePassword(username, password)\n",
"else: # client_id and client_secret\n",
" auth = ServiceAccount(client_id, client_secret)\n",
"\n",
"# Only one of these two parameters should be specified\n",
"engine_url = \"\"\n",
"assert bool(engine_url) != bool(\n",
" engine_name\n",
"engine_name = \"\"\n",
"assert not (\n",
" engine_url and engine_name\n",
"), \"Specify only one of engine_name and engine_url\"\n",
"\n",
"database_name = \"\"\n",
"username = \"\"\n",
"password = \"\"\n",
"account_name = \"\"\n",
"api_endpoint = DEFAULT_API_URL"
]
},
Expand All @@ -64,11 +79,11 @@
"source": [
"# create a connection based on provided credentials\n",
"connection = connect(\n",
" auth=auth,\n",
" database=database_name,\n",
" engine_url=engine_url,\n",
" engine_name=engine_name,\n",
" database=database_name,\n",
" username=username,\n",
" password=password,\n",
" account_name=account_name,\n",
" api_endpoint=api_endpoint,\n",
")\n",
"\n",
Expand Down Expand Up @@ -238,7 +253,7 @@
"metadata": {},
"outputs": [],
"source": [
"from firebolt.async_db import connect"
"from firebolt.async_db import connect as async_connect"
]
},
{
Expand All @@ -256,30 +271,18 @@
"metadata": {},
"outputs": [],
"source": [
"async def async_connect():\n",
" # create a connection based on provided credentials\n",
" connection = await connect(\n",
" engine_url=engine_url,\n",
" engine_name=engine_name,\n",
" database=database_name,\n",
" username=username,\n",
" password=password,\n",
" api_endpoint=api_endpoint,\n",
" )\n",
"# create a connection based on provided credentials\n",
"async_connection = await async_connect(\n",
" auth=auth,\n",
" database=database_name,\n",
" engine_url=engine_url,\n",
" engine_name=engine_name,\n",
" account_name=account_name,\n",
" api_endpoint=api_endpoint,\n",
")\n",
"\n",
" # create a cursor for connection\n",
" cursor = connection.cursor()\n",
" return cursor"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "58885c37",
"metadata": {},
"outputs": [],
"source": [
"async_cursor = await async_connect()"
"# create a cursor for connection\n",
"async_cursor = async_connection.cursor()"
]
},
{
Expand All @@ -297,26 +300,15 @@
"metadata": {},
"outputs": [],
"source": [
"async def run_queries(cursor):\n",
" await cursor.execute(\n",
" \"create fact table if not exists test_table (id int, name text, dt datetime) primary index id\"\n",
" )\n",
" await cursor.execute(\n",
" \"insert into test_table values (1, 'hello', '2021-01-01 01:01:01'),\"\n",
" \"(2, 'world', '2022-02-02 02:02:02'),\"\n",
" \"(3, '!', '2023-03-03 03:03:03')\"\n",
" )\n",
" await cursor.execute(\"select * from test_table\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "768fc91a",
"metadata": {},
"outputs": [],
"source": [
"await run_queries(async_cursor)"
"await async_cursor.execute(\n",
" \"create fact table if not exists test_table (id int, name text, dt datetime) primary index id\"\n",
")\n",
"await async_cursor.execute(\n",
" \"insert into test_table values (1, 'hello', '2021-01-01 01:01:01'),\"\n",
" \"(2, 'world', '2022-02-02 02:02:02'),\"\n",
" \"(3, '!', '2023-03-03 03:03:03')\"\n",
")\n",
"await async_cursor.execute(\"select * from test_table\")"
]
},
{
Expand Down Expand Up @@ -353,20 +345,9 @@
"metadata": {},
"outputs": [],
"source": [
"async def print_results(cursor):\n",
" print(await cursor.fetchone())\n",
" print(await cursor.fetchmany(1))\n",
" print(await cursor.fetchall())"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "15f24a0a",
"metadata": {},
"outputs": [],
"source": [
"await print_results(async_cursor)"
"print(await async_cursor.fetchone())\n",
"print(await async_cursor.fetchmany(1))\n",
"print(await async_cursor.fetchall())"
]
},
{
Expand All @@ -386,27 +367,42 @@
"source": [
"# manually\n",
"connection.close()\n",
"print(connection.closed)\n",
"\n",
"await async_connection.aclose()\n",
"print(async_connection.closed)\n",
"\n",
"# using context manager\n",
"with connect(\n",
" engine_url=engine_url,\n",
" auth=auth,\n",
" account_name=account_name,\n",
" engine_name=engine_name,\n",
" database=database_name,\n",
" username=username,\n",
" password=password,\n",
" api_endpoint=api_endpoint,\n",
") as conn:\n",
" # create cursors, perform database queries\n",
" pass\n",
"conn.closed"
"print(conn.closed)\n",
"\n",
"# using context manager\n",
"async with await async_connect(\n",
" auth=auth,\n",
" account_name=account_name,\n",
" engine_name=engine_name,\n",
" database=database_name,\n",
" api_endpoint=api_endpoint,\n",
") as async_conn:\n",
" # create cursors, perform database queries\n",
" pass\n",
"async_conn.closed"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "python37",
"display_name": "Python 3 Trio",
"language": "python",
"name": "python37"
"name": "python3-trio"
},
"language_info": {
"codemirror_mode": {
Expand All @@ -418,7 +414,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.12"
"version": "3.9.16"
}
},
"nbformat": 4,
Expand Down
Loading