Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Disable the SDK from changing capitalization of output columns by default #917

Closed
pgzmnk opened this issue Sep 26, 2022 · 1 comment · Fixed by #1050
Closed

Disable the SDK from changing capitalization of output columns by default #917

pgzmnk opened this issue Sep 26, 2022 · 1 comment · Fixed by #1050
Assignees
@utkarsharma2
Labels
bug Something isn't working priority/critical Critical priority product/python-sdk Label describing products
Milestone
1.2.0

Comments

@pgzmnk
Copy link
Contributor

pgzmnk commented Sep 26, 2022 •
edited
Loading

Describe the bug

Users expressed confusion on why the column names for output tables are lowercased. It's due to this default setting.

Expected behavior

The default behavior should be to not change casing and it should be configureable with a global variable.

That way we can avoid hardcoding this setting into the Astro Build dagfile generator script.

Version
1.1.0

To Reproduce
Run this Pipeline.

Screenshots
image

Additional context
Slack thread from September 26 2022.
@kaxil kaxil added this to the 1.1.1 milestone Sep 26, 2022
@kaxil kaxil added product/python-sdk Label describing products bug Something isn't working priority/high High priority labels Sep 26, 2022
@utkarsharma2 utkarsharma2 mentioned this issue Sep 28, 2022
Closed
2 tasks
@utkarsharma2 utkarsharma2 self-assigned this Sep 28, 2022
@utkarsharma2 utkarsharma2 modified the milestones: 1.1.1, 1.2.0 Sep 29, 2022
@phanikumv phanikumv added priority/critical Critical priority and removed priority/high High priority labels Oct 12, 2022
@utkarsharma2 utkarsharma2 mentioned this issue Oct 12, 2022
Merged
2 tasks
@kaxil
Copy link
Collaborator

kaxil commented Oct 13, 2022

Some related snowflake issues:

utkarsharma2 added a commit that referenced this issue Oct 20, 2022
@utkarsharma2 @kaxil
Snowflake - Add quotes on col names to preserve the casing (#1050)
14fb31f 
# Description
## What is the current behavior?
Currently, we were changing the case of col names to lowercase for
snowflakes. Which trips user.


closes: #917 #1098 


## What is the new behavior?
We can keep the col names intact by passing them in quotes when creating
tables added quotes.
We no longer need the ColCapitalization feature. 

## Does this introduce a breaking change?
Yes, for pandas path, we will also load "data twice" - one during schema
detection (where we load and truncate) and second during actual load

### Checklist
- [ ] Created tests which fail without the change (if possible)
- [ ] Extended the README / documentation, if necessary

Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@utkarsharma2 utkarsharma2

Labels
bug Something isn't working priority/critical Critical priority product/python-sdk Label describing products
Projects
None yet
Milestone
1.2.0
4 participants
@kaxil @utkarsharma2 @pgzmnk @phanikumv