Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 27 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
# Changelog

## [0.6.0] -
## [0.6.0] - 2025-03-24

### New Features :magic_wand:

- Planetary computer credential provider by @kylebarron in https://github.com/developmentseed/obstore/pull/379

### Breaking changes :wrench:

Expand All @@ -10,12 +14,32 @@ No breaking changes.

#### Store constructors

- In the `AzureStore` constructor, the `container` positional argument was renamed to `container_name` to match the `container_name` key in `AzureConfig`.
- In the `AzureStore` constructor, the `container` positional argument was renamed to `container_name` to match the `container_name` key in `AzureConfig`. by @kylebarron in https://github.com/developmentseed/obstore/pull/380

This is a breaking change if you had been calling `AzureStore(container="my container name")`.

This is a breaking change if you had been calling `AzureStore(container="my container name")`. This is not breaking if you had been using it as a positional argument `AzureStore("my container name")` or if you had already been using `AzureStore(container_name="my container name")`.
This is **not** breaking if you had been using it as a positional argument `AzureStore("my container name")` or if you had already been using `AzureStore(container_name="my container name")`.

The idea here is that we want one and only one argument name for each underlying config parameter. Most of these breaking changes took place in 0.5.0, but this was overlooked.

### Bug fixes :bug:

- Fix import errors on Python 3.9:
- Fix azure auth import on Python 3.9 by @kylebarron in https://github.com/developmentseed/obstore/pull/378
- Fix `_buffered.pyi` for python 3.9 by @kylebarron in https://github.com/developmentseed/obstore/pull/381
- Define `__all__` to fix type checking import paths https://github.com/developmentseed/obstore/pull/389

### Documentation :book:

- Fix chunk_size typo by @kylebarron in https://github.com/developmentseed/obstore/pull/377
- Docs: Make integrations dropdown by @kylebarron in https://github.com/developmentseed/obstore/pull/382
- Docs: Use source order in credential provider docs by @kylebarron in https://github.com/developmentseed/obstore/pull/383

### Other

- Add typing extensions as runtime dependency by @kylebarron in https://github.com/developmentseed/obstore/pull/384

**Full Changelog**: https://github.com/developmentseed/obstore/compare/py-v0.5.1...py-v0.6.0

## [0.5.1] - 2025-03-17

Expand Down
2 changes: 1 addition & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 3 additions & 1 deletion docs/authentication.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,9 @@ Refer to [`obstore.auth.azure`](api/auth/azure.md).

#### Microsoft Planetary Computer

The Microsoft [Planetary Computer](https://planetarycomputer.microsoft.com/) hosts a multi-petabyte catalog of global environmental data. The contained data is publicly accessible, but requires the user to fetch [short-lived access tokens](https://planetarycomputer.microsoft.com/docs/concepts/sas/). But accessing and refreshing these tokens every hour can be confusing and annoying.
The [Microsoft Planetary Computer](https://planetarycomputer.microsoft.com/) hosts a multi-petabyte catalog of global environmental data.

The contained data is publicly accessible, but requires the user to fetch [short-lived access tokens](https://planetarycomputer.microsoft.com/docs/concepts/sas/). But accessing and refreshing these tokens every hour can be confusing and annoying.

The [`PlanetaryComputerCredentialProvider`][obstore.auth.planetary_computer.PlanetaryComputerCredentialProvider] **handles all token access and refresh automatically**.

Expand Down
128 changes: 128 additions & 0 deletions docs/blog/posts/obstore-0.6.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
---
draft: false
date: 2025-03-24
categories:
- Release
authors:
- kylebarron
links:
- CHANGELOG.md
---

# Releasing obstore 0.6!

Obstore is the simplest, highest-throughput Python interface to Amazon S3, Google Cloud Storage, and Azure Storage, powered by Rust.

This post gives an overview of what's new in obstore version 0.6.

<!-- more -->

Refer to the [changelog](../../CHANGELOG.md) for all updates.

## Easier access to Microsoft Planetary Computer

The [Microsoft Planetary Computer](https://planetarycomputer.microsoft.com/) hosts a multi-petabyte catalog of global environmental data.

The contained data is publicly accessible, but requires the user to fetch [short-lived access tokens](https://planetarycomputer.microsoft.com/docs/concepts/sas/). But accessing and refreshing these tokens every hour can be confusing and annoying.

Following up on the [addition in v0.5 of credential providers](obstore-0.5.md#credential-providers), this release adds [`PlanetaryComputerCredentialProvider`][obstore.auth.planetary_computer.PlanetaryComputerCredentialProvider], which **handles all token access and refresh automatically**.

As a quick example, we'll read data from the [NAIP dataset](https://planetarycomputer.microsoft.com/dataset/naip):

```py
from obstore.store import AzureStore
from obstore.auth.planetary_computer import PlanetaryComputerCredentialProvider

url = "https://naipeuwest.blob.core.windows.net/naip/v002/mt/2023/mt_060cm_2023/"

# Construct an AzureStore with this credential provider.
#
# The account, container, and container prefix are passed down to AzureStore
# automatically.
store = AzureStore(credential_provider=PlanetaryComputerCredentialProvider(url))
```

Then, for example, list some items in the container (the prefix `v002/mt/2023/mt_060cm_2023` was automatically set as the prefix on the `AzureStore`):

```py
items = next(store.list())
print(items[:2])
```

```py
[{'path': '44104/m_4410401_ne_13_060_20230811_20240103.200.jpg',
'last_modified': datetime.datetime(2025, 1, 13, 18, 18, 1, tzinfo=datetime.timezone.utc),
'size': 14459,
'e_tag': '0x8DD33FE9DB7A24D',
'version': None},
{'path': '44104/m_4410401_ne_13_060_20230811_20240103.tif',
'last_modified': datetime.datetime(2025, 1, 13, 16, 39, 6, tzinfo=datetime.timezone.utc),
'size': 400422790,
'e_tag': '0x8DD33F0CC1D1752',
'version': None}]
```

And we can fetch an image thumbnail:

```py
path = "44106/m_4410602_nw_13_060_20230712_20240103.200.jpg"
image_content = store.get(path).bytes()

# Write out the image content to a file in the current directory
with open("thumbnail.jpg", "wb") as f:
f.write(image_content)
```

And voilà:

![](../../assets/planetary-computer-naip-thumbnail.jpg)

### Using with the Planetary Computer STAC API

[STAC](https://stacspec.org/en) is a metadata specification for geospatial data. The Planetary Computer [provides a STAC API](https://planetarycomputer.microsoft.com/docs/quickstarts/reading-stac/) to help search and find data of interest.

The [`PlanetaryComputerCredentialProvider`][obstore.auth.planetary_computer.PlanetaryComputerCredentialProvider] includes a [`from_asset`][obstore.auth.planetary_computer.PlanetaryComputerCredentialProvider].from_asset] constructor to easily convey configuration.

```py
import pystac_client

from obstore.auth.planetary_computer import PlanetaryComputerCredentialProvider
from obstore.store import AzureStore

stac_url = "https://planetarycomputer.microsoft.com/api/stac/v1/"
# Open the STAC Catalog
catalog = pystac_client.Client.open(stac_url)

# Access a specific Collection and Asset
collection = catalog.get_collection("daymet-daily-hi")
asset = collection.assets["zarr-abfs"]

# Then we can pass this directly to `from_asset`
credential_provider = PlanetaryComputerCredentialProvider.from_asset(asset)

# Print objects at the root of this directory
store = AzureStore(credential_provider=credential_provider)
print(store.list_with_delimiter()["objects"])
```

```py
[{'path': '.zattrs',
'last_modified': datetime.datetime(2021, 6, 9, 15, 48, 6, tzinfo=datetime.timezone.utc),
'size': 402,
'e_tag': '0x8D92B5DF9186DED',
'version': None},
{'path': '.zgroup',
'last_modified': datetime.datetime(2021, 6, 9, 15, 45, 56, tzinfo=datetime.timezone.utc),
'size': 24,
'e_tag': '0x8D92B5DABDE919A',
'version': None},
{'path': '.zmetadata',
'last_modified': datetime.datetime(2021, 6, 9, 15, 46, 56, tzinfo=datetime.timezone.utc),
'size': 13479,
'e_tag': '0x8D92B5DCFE1B7CC',
'version': None}]
```

## All updates

Refer to the [changelog](../../CHANGELOG.md) for all updates.
2 changes: 1 addition & 1 deletion obstore/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "obstore"
version = "0.5.1"
version = "0.6.0"
authors = { workspace = true }
edition = { workspace = true }
description = "The simplest, highest-throughput interface to Amazon S3, Google Cloud Storage, Azure Blob Storage, and S3-compliant APIs like Cloudflare R2."
Expand Down
Loading