PLUGIN-1656 Added a field for datatype validation to avoid validation.#212
Conversation
vikasrathee-cs
requested review from
Aryan-Verma,
MrRahulSharma and
sau42shri
and removed request for
Aryan-Verma and
MrRahulSharma
vikasrathee-cs
changed the title
|
Is this a regression ? |
Yes, it was added to validate datatypes also along with name in the schema but previous users don't want this validation as plugin was working fine even if it get mapped with wrong data type. Added a comment in the JIRA for exact error that customer is facing. |
src/main/java/io/cdap/plugin/salesforce/plugin/sink/batch/SalesforceSinkConfig.java
Outdated
Show resolved
Hide resolved
vikasrathee-cs
force-pushed
the
validation_fix
branch
from
2784293 to
1e35b13
Compare
src/test/java/io/cdap/plugin/salesforce/connector/SalesforceConnectorTest.java
Outdated
Show resolved
Hide resolved
src/main/java/io/cdap/plugin/salesforce/plugin/sink/batch/SalesforceSinkConfig.java
Show resolved
Hide resolved
vikasrathee-cs
force-pushed
the
validation_fix
branch
from
1e35b13 to
0e78f04
Compare
| <artifactId>salesforce-plugins</artifactId> | ||
| <version>1.6.0-SNAPSHOT</version> | ||
| <version>1.7.0-SNAPSHOT</version> | ||
| <packaging>jar</packaging> |
There was a problem hiding this comment.
We do not require to do this change here.
| ConcurrencyMode.Parallel.name(), "1000000", "10000", "Fail on Error", | ||
| BaseSalesforceETLTest.SECURITY_TOKEN, | ||
| null, null); | ||
| null, null, false); |
There was a problem hiding this comment.
Please add a test for the true scenario as well.
There was a problem hiding this comment.
Unit test cases will be added separately as Mockito framework need some upgrade, separate ticket is added for that https://cdap.atlassian.net/browse/PLUGIN-1657
| "value": "false", | ||
| "label": "NO" | ||
| }, | ||
| "default": "true" |
There was a problem hiding this comment.
Shouldn't the default be false ?
There was a problem hiding this comment.
it should be true for new pipelines and false for upgrade pipelines. In case of upgrade pipeline, we are setting it as false in isDatatypeValidation method.
vikasrathee-cs
force-pushed
the
validation_fix
branch
from
0e78f04 to
8a9e8af
Compare
vikasrathee-cs
force-pushed
the
validation_fix
branch
from
8a9e8af to
7708dd2
Compare
4 participants
https://cdap.atlassian.net/browse/PLUGIN-1656
Added a field for datatype validation to avoid validation in Salesforce sink plugin. E.g. some existing users are providing date fields as a string in input schema and same was not getting validated and this process was working fine as while creating csv file these string were getting used as it as. We added validation on data types to avoid any wrong value in such fields but customer wants to use existing approach. So, we have added a flag datatype validation which will be false by default so that it will not impact existing users.