Skip to content

Fix TaskAttempt Context is null#266

Merged
Vipinofficial11 merged 1 commit intodata-integrations:developfrom
cloudsufi:refactorRecordReaders
Oct 23, 2024
Merged

Fix TaskAttempt Context is null#266
Vipinofficial11 merged 1 commit intodata-integrations:developfrom
cloudsufi:refactorRecordReaders

Conversation

@Vipinofficial11
Copy link
Contributor

@Vipinofficial11 Vipinofficial11 commented Oct 15, 2024

Changes include:

  1. Fixes in the SalesforceWideRecordReader partitioning logic to address API retrieve limit exceeded errors.
  2. Usage of default configurations in case of Non-CDAP calls

@Vipinofficial11 Vipinofficial11 changed the title Update Record Readers to accept Configurations from TaskAttemptContext and InputFormat Provider. Fix TaskAttempt Context is null Oct 17, 2024
@MrRahulSharma
Copy link
Contributor

Can we validate the WideRecordReader changes in CDAP/CDF today ?

InputSplit inputSplit, AuthenticatorCredentials credentials)
throws IOException, InterruptedException {
List<Map<String, ?>> fetchedIdList = fetchBulkQueryIds(inputSplit, null);
List<Map<String, ?>> fetchedIdList = fetchBulkQueryIds(inputSplit, credentials);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move the super.initialize call outside fetchBulkQueryIds and call it before the fetchBulkQueryIds call here.

Copy link
Contributor Author

@Vipinofficial11 Vipinofficial11 Oct 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved super.initialize before fetchBulkQueryIds and removed paramters from fetchBulkQueryIds as they are not required anymore in the method. [b5c9fe5]

Copy link
Contributor

@MrRahulSharma MrRahulSharma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Vipinofficial11 Vipinofficial11 merged commit 6165ee5 into data-integrations:develop Oct 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants