target-s3-parquet

target-s3-parquet is a Singer target for S3Parquet.

Build with the Meltano Target SDK.

Configuration

Accepted Config Options

A full list of supported settings and capabilities for this target is available by running:

target-s3-parquet --about

Configure using environment variables

This Singer target will automatically import any environment variables within the working directory's .env if the --config=ENV is provided, such that config values will be considered if a matching environment variable is set either in the terminal context or in the .env file.

Source Authentication and Authorization

Usage

You can easily run target-s3-parquet by itself or in a pipeline using Meltano.

Executing the Target Directly

target-s3-parquet --version
target-s3-parquet --help
# Test using the "Carbon Intensity" sample:
tap-carbon-intensity | target-s3-parquet --config /path/to/target-s3-parquet-config.json

Developer Resources

Follow these instructions to contribute to this project.

Initialize your Development Environment

pipx install poetry
poetry install

Create and Run Tests

Create tests within the target_s3_parquet/tests subfolder and then run:

poetry run pytest

You can also test the target-s3-parquet CLI interface directly using poetry run:

poetry run target-s3-parquet --help

Testing with Meltano

Note: This target will work in any Singer environment and does not require Meltano. Examples here are for convenience and to streamline end-to-end orchestration scenarios.

Next, install Meltano (if you haven't already) and any needed plugins:

# Install meltano
pipx install meltano
# Initialize meltano within this directory
cd target-s3-parquet
meltano install

Now you can test and orchestrate using Meltano:

# Test invocation:
meltano invoke target-s3-parquet --version
# OR run a test `elt` pipeline with the Carbon Intensity sample tap:
meltano elt tap-carbon-intensity target-s3-parquet

SDK Dev Guide

See the dev guide for more instructions on how to use the Meltano Singer SDK to develop your own Singer taps and targets.