task/KBDEV-1194 get updated studies for last two weeks in clinicaltrials

[sshugsc]

KBDEV-1194

• update clinicaltrialsgov to load last updated data
• add maxRecords options to clinicaltrialsgov
• remove the out-of-dated rss/xml related code

[mathieulemieux]

@sshugsc, to be safe, testing with some other loaders via the CLI would be a must before merging.

[github-actions]

Unit Test Results

0 files  ±0  0 suites  ±0   0s ⏱️ ±0s
0 tests ±0  0 ✔️ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit 200296e. ± Comparison against base commit d1c3f6e.

[mathieulemieux]

@sshu, can you post here which loaders you tested (the command you used) so we may want to use the same ones for testing the CLI from now on? I'm curious; have you choose one file loader, one api, cosmic and civic, in conjunction with the maxRecords flag?

[sshugsc]

@sshu, can you post here which loaders you tested (the command you used) so we may want to use the same ones for testing the CLI from now on? I'm curious; have you choose one file loader, one api, cosmic and civic, in conjunction with the maxRecords flag?

I tested file loaders(ncit, refseq), api loaders(oncotree, docm), cosmic (resistance) and civic. They all work fine.

[mathieulemieux]

oh, that's a lot! Not the whole datasets I suppose; have you used the maxRecords flag?

[sshugsc]

oh, that's a lot! Not the whole datasets I suppose; have you used the maxRecords flag?

yeah, tried maxRecords flag for them. Work good on most of them, but oncotree didn't support it.