Smartsheets connector raw key:value pair data is scrambled. #8099
Comments
w-gitops
changed the title
|
Also seeing this issue as well. Syncing from Smart Sheet to Postgres Smart Sheet Row
_airbyte_data payload |
|
I did a basic map analysis, and it looks like it's 1 field off: This is also true in the raw data, so it's not a normalization problem: @zzstoatzz Any ideas? Thanks! Side note: I am also syncing to Postgres. Perhaps that's the common denominator? I'll try snowflake after the holiday break. |
|
@tehnorm |
|
@willsargent hey will! I'd really like to help resolve this, but I'm currently swamped. I believe what may have happened is that I fixed this myself and built a new image locally but forgot to go through with pushing this to airbyte master. This should be the source I built the working image from - I hope this is somewhat helpful, I'll return to this when I can! |
|
Thanks for this @zzstoatzz! I'll take a look at this and see what we can do on that front. Really appreciate your work on this! Have a great Holiday! |
|
@zzstoatzz Looks like that fixes it! I'll work through the PR process on this, all credit to you of course! Thanks again! |
|
@willsargent I tried this fix but in my case it still doesn't not work. I am running airbyte on docker container and I did the following
Did I do anything wrong? |
|
Hi Madhu,
I can help. let me get in the office tomorrow and i will write it up.
you don't need to use gradelw
…________________________________
From: Madhu Prabhakara ***@***.***>
Sent: Sunday, February 20, 2022 4:00 PM
To: airbytehq/airbyte
Cc: Will Sargent; Mention
Subject: Re: [airbytehq/airbyte] Smartsheets connector raw key:value pair data is scrambled. (Issue #8099)
@willsargent<https://github.com/willsargent> I tried this fix but in my case it still doesn't not work. I am running airbyte on docker container and I did the following
1. Overwrote the source.py for smartsheets with the one attached above
2. Compiled airbyte by running "./gradlew :airbyte-integrations:connectors:source_smartsheets:build"
3. docker-compose up
Did I do anything wrong?
—
Reply to this email directly, view it on GitHub<#8099 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABBKOO6MNIT2LANZH2WAJDDU4FI6ZANCNFSM5IJ6YGSQ>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
|
Thanks a lot @willsargent . I really appreciate the help! |
|
Hi @willsargent just circling back to check if you got a chance to look at this. Trying to resolve this for a client sorry to bother you. Thanks again! |
|
Marcos Marx commented: @tehnorm |
|
Deepest apologies, as this problem was traced, identified and solved with the code in the PR, but I messed up the PR with a bunch of files that should have been .ignored. I'll clean it up tomorrow python -m venv .venv Then, from the source connector page where you can select what version of the connector you want, use "dev", no quotes. Sorry for the abbreviated cheat sheet that makes a few assumptions, but no need to use gradle, just docker build. Hope this helps, and I'll get the PR's fixed because the second one adds some nice features as well. |
Enviroment
Current Behavior
In the raw data, The connector fetches key value pairs, but the key doesn't match the right value.
"key-a":"value-c","key-b":"value-a", etc..
As a result the DBT normalization fails, when a type like a date is fed a string.
This problem may not be apparent under some conditions, as numbers and strings are treated the same by smartsheets. So most sheets might not have a mixed type failure unless in this case a date is involved.
Expected Behavior
"KeyA":"ValueA", "KeyB":ValueB
Logs
LOG
Steps to Reproduce
Are you willing to submit a PR?
If I can get my dev environment up and running
The text was updated successfully, but these errors were encountered: