Skip to content

heiko1234/Dagster_Azurite

Repository files navigation

Dagster_Azurite

This Dagster Azurite Repo connects to the local running Azurite Blobstorage.

Dagster: Dagster Daemon and Dagit are running in its own container.

The User Sample Code (in its own container) will pull data from Blobstorage and will write it back into a separeted folder.

docker

To start the docker container

docker-compose up --build

And to remove the docker container

docker-compose down 

When executing the script to pull and push the data from azurite blobstorage the solid using the azurite resource fails.

error

Run it locally without docker set some variables for dagit and dagster daemon.

# in bash

export DAGSTER_HOME="/home/heiko/Repos/dagster_azurite/.dagster"
dagster-daemon run

commands

# in Powershell

Set-Variable -Name "DAGSTER_HOME" -value "./.dagster"
dagit -f repository.py

local_green_pipeline

Maybe helpful

Azurite connection: http://IP:Port/Account

https://blog.johnnyreilly.com/2021/05/15/azurite-and-table-storage-dev-container/

Azure/Azurite#379

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages