Skip to content

More CSV fixes#433

Merged
mildbyte merged 2 commits intomasterfrom
bugfix/more-robust-csvs
Apr 7, 2021
Merged

More CSV fixes#433
mildbyte merged 2 commits intomasterfrom
bugfix/more-robust-csvs

Conversation

@mildbyte
Copy link
Copy Markdown
Contributor

@mildbyte mildbyte commented Apr 7, 2021

Set default dialect to None (otherwise we always use 'excel' when not inferring the dialect). Also add a special ignore_decode_errors flag to ignore decoding errors and fall back to UTF-8 if chardet can't detect an encoding.

mildbyte added 2 commits April 7, 2021 21:40
… inferring the dialect). Also add a special `ignore_decode_errors` flag to ignore decoding errors and fall back to UTF-8 if chardet can't detect an encoding.
@mildbyte mildbyte merged commit de325d4 into master Apr 7, 2021
@mildbyte mildbyte deleted the bugfix/more-robust-csvs branch April 7, 2021 21:13
mildbyte added a commit that referenced this pull request Apr 14, 2021
  * Add customizable fetch size to the Snowflake data source (#434)
  * Fix issue with changing the engine password (#437)
  * Data source refactor (#438):
    * MySQL: parameter `remote_schema` has been renamed to `dbname`
    * Mongo: parameter `coll` has been renamed to `collection`; `db` to `database`
    * Table options are now a separate parameter that is passed to the
    * Introspection now returns a dictionary of tables and proposed table options OR error classes for tables that we weren't able to introspect (allowing for partial failures)
    * Mounting can now return a list of mount errors (caller can choose to ignore).
    * CSV data source: allow passing a partially initialized list of table options without a schema, making it introspect just those S3 keys and fill out the missing table options.
  * Postgres-level notices are now available in the `PsycopgEngine.notices` list after a `run_sql` invocation.
  * Multicorn: fix bug where server-level FDW options would override table-level FDW options.

Full set of changes: [`v0.2.12...v0.2.13`](v0.2.12...v0.2.13)
mildbyte added a commit that referenced this pull request Apr 14, 2021
  * Add customizable fetch size to the Snowflake data source (#434)
  * Fix issue with changing the engine password (#437)
  * Data source refactor (#438):
    * MySQL: parameter `remote_schema` has been renamed to `dbname`
    * Mongo: parameter `coll` has been renamed to `collection`; `db` to `database`
    * Table options are now a separate parameter that is passed to the
    * Introspection now returns a dictionary of tables and proposed table options OR error classes for tables that we weren't able to introspect (allowing for partial failures)
    * Mounting can now return a list of mount errors (caller can choose to ignore).
    * CSV data source: allow passing a partially initialized list of table options without a schema, making it introspect just those S3 keys and fill out the missing table options.
  * Postgres-level notices are now available in the `PsycopgEngine.notices` list after a `run_sql` invocation.
  * Multicorn: fix bug where server-level FDW options would override table-level FDW options.

Full set of changes: [`v0.2.12...v0.2.13`](v0.2.12...v0.2.13)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant