About dagster-libs-feedstock

Feedstock license: BSD-3-Clause

About dagster-libs

Home: https://github.com/dagster-io

Package license: Apache-2.0

Summary: The data orchestration platform built for productivity.

Documentation: https://docs.dagster.io

Dagster is a system for building modern data applications. Combining an elegant programming model and beautiful tools, Dagster allows infrastructure engineers, data engineers, and data scientists to seamlessly collaborate to process and produce the trusted, reliable data needed in today's world.

About dagster-airbyte

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/dagster-airbyte

Package license: Apache-2.0

Summary: Package for integrating Airbyte with Dagster.

About dagster-aws

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/dagster-aws

Package license: Apache-2.0

Summary: Package for AWS-specific Dagster framework solid and resource components.

About dagster-celery

Home: https://github.com/dagster-io/dagster/blob/master/python_modules/dagster-celery

Package license: Apache-2.0

Summary: Package for using Celery as Dagster's execution engine.

About dagster-census

Home: https://github.com/dagster-io/dagster/blob/master/python_modules/dagster-census

Package license: Apache-2.0

Summary: Package for integrating Census with Dagster.

About dagster-dask

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/dagster-dask

Package license: Apache-2.0

Summary: Package for using Dask as Dagster's execution engine.

About dagster-datadog

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-datadog

Package license: Apache-2.0

Summary: Package for datadog Dagster framework components.

About dagster-dbt

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/dagster-dbt

Package license: Apache-2.0

Summary: A Dagster integration for dbt

About dagster-deltalake

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/dagster-deltalake

Package license: Apache-2.0

Summary: Package for Deltalake-specific Dagster framework op and resource components.

About dagster-docker

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-docker

Package license: Apache-2.0

Summary: A Dagster integration for docker

About dagster-duckdb

Home: https://github.com/dagster-io/dagster/blob/master/python_modules/dagster-duckdb

Package license: Apache-2.0

Summary: Package for DuckDB-specific Dagster framework op and resource components.

About dagster-fivetran

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-fivetran

Package license: Apache-2.0

Summary: Package for integrating Fivetran with Dagster.

About dagster-github

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-github

Package license: Apache-2.0

Summary: Github plugin for Dagster

About dagster-k8s

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-k8s

Package license: Apache-2.0

Summary: A Dagster integration for k8s

About dagster-looker

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-looker

Package license: Apache-2.0

Summary: Looker with Dagster.

About dagster-managed-elements

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-managed-elements

Package license: Apache-2.0

Summary: Package for Managed elements with Dagster.

About dagster-mlflow

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-mlflow

Package license: Apache-2.0

Summary: Package for mlflow Dagster framework components

About dagster-msteams

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-msteams

Package license: Apache-2.0

Summary: A Microsoft Teams client resource for posting to Microsoft Teams

About dagster-mysql

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-mysql

Package license: Apache-2.0

Summary: A Dagster integration for MySQL

About dagster-openai

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-openai

Package license: Apache-2.0

Summary: A Open AI client resource for interacting with Open AI API

About dagster-pagerduty

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-pagerduty

Package license: Apache-2.0

Summary: Package for pagerduty Dagster framework components.

About dagster-pandas

Home: https://github.com/dagster-io/dagster

Package license: Apache-2.0

Summary: Utilities and examples for working with pandas and dagster, an opinionated framework for expressing data pipelines

About dagster-pandera

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-pandera

Package license: Apache-2.0

Summary: Integration layer for dagster and pandera.

About dagster-papertrail

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-papertrail

Package license: Apache-2.0

Summary: Package for papertrail Dagster framework components.

About dagster-polars

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-polars

Package license: Apache-2.0

Summary: A Dagster integration for polars

About dagster-postgres

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-postgres

Package license: Apache-2.0

Summary: A Dagster integration for postgres

About dagster-prometheus

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-prometheus

Package license: Apache-2.0

Summary: A Dagster integration for prometheus

About dagster-shell

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-shell

Package license: Apache-2.0

Summary: Package for Dagster shell ops.

About dagster-slack

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-slack

Package license: Apache-2.0

Summary: A Slack client resource for posting to Slack

About dagster-snowflake

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-snowflake

Package license: Apache-2.0

Summary: Package for Snowflake Dagster framework components.

About dagster-spark

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-spark

Package license: Apache-2.0

Summary: Package for Spark Dagster framework components.

About dagster-ssh

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-ssh

Package license: Apache-2.0

Summary: Package for ssh Dagster framework components.

About dagster-twilio

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-twilio

Package license: Apache-2.0

Summary: A Dagster integration for twilio

About dagster-wandb

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/dagster-wandb

Package license: Apache-2.0

Summary: Package for wandb Dagster components.

About dagstermill

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/dagstermill

Package license: Apache-2.0

Summary: A Dagster integration for papermill

About dagster-celery-docker

Home: https://github.com/dagster-io/dagster/blob/master/python_modules/dagster-celery-docker

Package license: Apache-2.0

Summary: A Dagster integration for celery-docker

About dagster-celery-k8s

Home: https://github.com/dagster-io/dagster/blob/master/python_modules/dagster-celery-k8s

Package license: Apache-2.0

Summary: A Dagster integration for celery-k8s-executor

About dagster-deltalake-pandas

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/dagster-deltalake-pandas

Package license: Apache-2.0

Summary: Package for storing Pandas DataFrames in Delta tables

About dagster-deltalake-polars

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/dagster-deltalake-polars

Package license: Apache-2.0

Summary: Package for storing Polars DataFrames in Delta tables.

About dagster-duckdb-pandas

Home: https://github.com/dagster-io/dagster/blob/master/python_modules/dagster-duckdb-pandas

Package license: Apache-2.0

Summary: Package for storing Pandas DataFrames in DuckDB.

About dagster-duckdb-polars

Home: https://github.com/dagster-io/dagster/blob/master/python_modules/dagster-duckdb-polars

Package license: Apache-2.0

Summary: Package for storing Polars DataFrames in DuckDB.

About dagster-duckdb-pyspark

Home: https://github.com/dagster-io/dagster/blob/master/python_modules/dagster-duckdb-pyspark

Package license: Apache-2.0

Summary: Package for storing PySpark DataFrames in DuckDB.

About dagster-gcp

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-gcp

Package license: Apache-2.0

Summary: Package for GCP-specific Dagster framework solid and resource components.

About dagster-ge

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-ge

Package license: Apache-2.0

Summary: Package for GE-specific Dagster framework solid and resource components.

About dagster-pyspark

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-pyspark

Package license: Apache-2.0

Summary: Package for PySpark Dagster framework components.

About dagster-snowflake-pandas

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-snowflake-pandas

Package license: Apache-2.0

Summary: Package for integrating Snowflake and Pandas with Dagster.

About dagster-snowflake-pyspark

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-snowflake-pyspark

Package license: Apache-2.0

Summary: Package for integrating Snowflake and PySpark with Dagster.

About dagster-gcp-pandas

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-gcp-pandas

Package license: Apache-2.0

Summary: Package for storing Pandas DataFrames in GCP.

About dagster-gcp-pyspark

Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-gcp-pyspark

Package license: Apache-2.0

Summary: Package for storing PySpark DataFrames in GCP

Current build status

All platforms:

Current release info

Name	Downloads	Version	Platforms

Installing dagster-libs

Installing dagster-libs from the conda-forge channel can be achieved by adding conda-forge to your channels with:

conda config --add channels conda-forge
conda config --set channel_priority strict

Once the conda-forge channel has been enabled, dagster-airbyte, dagster-aws, dagster-celery, dagster-celery-docker, dagster-celery-k8s, dagster-census, dagster-dask, dagster-datadog, dagster-dbt, dagster-deltalake, dagster-deltalake-pandas, dagster-deltalake-polars, dagster-docker, dagster-duckdb, dagster-duckdb-pandas, dagster-duckdb-polars, dagster-duckdb-pyspark, dagster-fivetran, dagster-gcp, dagster-gcp-pandas, dagster-gcp-pyspark, dagster-ge, dagster-github, dagster-k8s, dagster-looker, dagster-managed-elements, dagster-mlflow, dagster-msteams, dagster-mysql, dagster-openai, dagster-pagerduty, dagster-pandas, dagster-pandera, dagster-papertrail, dagster-polars, dagster-postgres, dagster-prometheus, dagster-pyspark, dagster-shell, dagster-slack, dagster-snowflake, dagster-snowflake-pandas, dagster-snowflake-pyspark, dagster-spark, dagster-ssh, dagster-twilio, dagster-wandb, dagstermill can be installed with conda:

conda install dagster-airbyte dagster-aws dagster-celery dagster-celery-docker dagster-celery-k8s dagster-census dagster-dask dagster-datadog dagster-dbt dagster-deltalake dagster-deltalake-pandas dagster-deltalake-polars dagster-docker dagster-duckdb dagster-duckdb-pandas dagster-duckdb-polars dagster-duckdb-pyspark dagster-fivetran dagster-gcp dagster-gcp-pandas dagster-gcp-pyspark dagster-ge dagster-github dagster-k8s dagster-looker dagster-managed-elements dagster-mlflow dagster-msteams dagster-mysql dagster-openai dagster-pagerduty dagster-pandas dagster-pandera dagster-papertrail dagster-polars dagster-postgres dagster-prometheus dagster-pyspark dagster-shell dagster-slack dagster-snowflake dagster-snowflake-pandas dagster-snowflake-pyspark dagster-spark dagster-ssh dagster-twilio dagster-wandb dagstermill

or with mamba:

mamba install dagster-airbyte dagster-aws dagster-celery dagster-celery-docker dagster-celery-k8s dagster-census dagster-dask dagster-datadog dagster-dbt dagster-deltalake dagster-deltalake-pandas dagster-deltalake-polars dagster-docker dagster-duckdb dagster-duckdb-pandas dagster-duckdb-polars dagster-duckdb-pyspark dagster-fivetran dagster-gcp dagster-gcp-pandas dagster-gcp-pyspark dagster-ge dagster-github dagster-k8s dagster-looker dagster-managed-elements dagster-mlflow dagster-msteams dagster-mysql dagster-openai dagster-pagerduty dagster-pandas dagster-pandera dagster-papertrail dagster-polars dagster-postgres dagster-prometheus dagster-pyspark dagster-shell dagster-slack dagster-snowflake dagster-snowflake-pandas dagster-snowflake-pyspark dagster-spark dagster-ssh dagster-twilio dagster-wandb dagstermill

It is possible to list all of the versions of dagster-airbyte available on your platform with conda:

conda search dagster-airbyte --channel conda-forge

or with mamba:

mamba search dagster-airbyte --channel conda-forge

Alternatively, mamba repoquery may provide more information:

# Search all versions available on your platform:
mamba repoquery search dagster-airbyte --channel conda-forge

# List packages depending on `dagster-airbyte`:
mamba repoquery whoneeds dagster-airbyte --channel conda-forge

# List dependencies of `dagster-airbyte`:
mamba repoquery depends dagster-airbyte --channel conda-forge

About conda-forge

conda-forge is a community-led conda channel of installable packages. In order to provide high-quality builds, the process has been automated into the conda-forge GitHub organization. The conda-forge organization contains one repository for each of the installable packages. Such a repository is known as a feedstock.

A feedstock is made up of a conda recipe (the instructions on what and how to build the package) and the necessary configurations for automatic building using freely available continuous integration services. Thanks to the awesome service provided by Azure, GitHub, CircleCI, AppVeyor, Drone, and TravisCI it is possible to build and upload installable packages to the conda-forge anaconda.org channel for Linux, Windows and OSX respectively.

To manage the continuous integration and simplify feedstock maintenance conda-smithy has been developed. Using the conda-forge.yml within this repository, it is possible to re-render all of this feedstock's supporting files (e.g. the CI configuration files) with conda smithy rerender.

For more information please check the conda-forge documentation.

Terminology

feedstock - the conda recipe (raw material), supporting scripts and CI configuration.

conda-smithy - the tool which helps orchestrate the feedstock. Its primary use is in the construction of the CI .yml files and simplify the management of many feedstocks.

conda-forge - the place where the feedstock and smithy live and work to produce the finished article (built conda distributions)

Updating dagster-libs-feedstock

If you would like to improve the dagster-libs recipe or build a new package version, please fork this repository and submit a PR. Upon submission, your changes will be run on the appropriate platforms to give the reviewer an opportunity to confirm that the changes result in a successful build. Once merged, the recipe will be re-built and uploaded automatically to the conda-forge channel, whereupon the built conda packages will be available for everybody to install and use from the conda-forge channel. Note that all branches in the conda-forge/dagster-libs-feedstock are immediately built and any created packages are uploaded, so PRs should be based on branches in forks and branches in the main repository should only be used to build distinct package versions.

In order to produce a uniquely identifiable distribution:

• If the version of a package is not being increased, please add or increase the build/number.
• If the version of a package is being increased, please remember to return the build/number back to 0.

Feedstock Maintainers

• @bollwyvl
• @conda-forge/dagster