Skip to content

BZIP2 format not supported with COPY #110

@rectalogic

Description

@rectalogic

We use BZIP2 compression on our s3 files, it doesn't seem possible to specify this with the CopyCommand.
http://docs.aws.amazon.com/redshift/latest/dg/copy-parameters-data-format.html#copy-bzip2

>>> import sqlalchemy as sa
>>> from redshift_sqlalchemy.dialect import CopyCommand
>>> table = sa.table("foo")
>>> CopyCommand(table, "s3://bucket/data/", "XXXYYYZZZXXXYYYZZZ11", "XXXYYYZZZXXXYYYZZZ11XXXYYYZZZXXXYYYZZZ11", format="BZIP2")
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/virtualenv/local/lib/python2.7/site-packages/sqlalchemy_redshift/commands.py", line 362, in __init__
    self.formats)
ValueError: "format" parameter must be one of ['CSV', 'JSON', 'AVRO', None]
>>> CopyCommand(table, "s3://bucket/data/", "XXXYYYZZZXXXYYYZZZ11", "XXXYYYZZZXXXYYYZZZ11XXXYYYZZZXXXYYYZZZ11", compression="BZIP2")
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/virtualenv/local/lib/python2.7/site-packages/sqlalchemy_redshift/commands.py", line 368, in __init__
    self.compression_types
ValueError: "compression" parameter must be one of ['GZIP', 'LZOP']

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions