Skip to content

Implementing keyword search JS script#29

Merged
Suhyma merged 13 commits intomainfrom
suhyma
Feb 24, 2023
Merged

Implementing keyword search JS script#29
Suhyma merged 13 commits intomainfrom
suhyma

Conversation

@Suhyma
Copy link
Copy Markdown
Collaborator

@Suhyma Suhyma commented Feb 22, 2023

Currently I'm experiencing issues with keyword searching through the JSON file that combines BiorXiv & MedrXiv preprints from the past day. I'm getting an error that says "SyntaxError: Unexpected end of JSON input", I'll add more details through code comments.

@Suhyma Suhyma requested review from jvwong and maxkfranz February 22, 2023 17:46
Comment thread src/cli.js Outdated
export async function search (queryString, options) {
const searcher = new Search();

const articles = await getInput(options);
Copy link
Copy Markdown
Collaborator Author

@Suhyma Suhyma Feb 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GitHub isn't allowing me to comment on the collapsed parts of the code but the error would occur at line 74 within the getInput() function when executing JSON.parse(). It wasn't reading the combined JSON file with all the preprint data correctly.

Comment thread src/data-search.js Outdated
const combinedOptions = {
output: COMBINED_FILE
};
sendOutput(combinedData, combinedOptions);
Copy link
Copy Markdown
Collaborator Author

@Suhyma Suhyma Feb 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I fixed the issue, I realized I was missing await in front if the async function sendOutput() here. The error doesn't come up anymore, I'm assuming what happened was the JSON file that combined the BiorXiv and MedrXiv preprint info wasn't finished populating the data before the keyword search started.

@cloudflare-workers-and-pages
Copy link
Copy Markdown

cloudflare-workers-and-pages Bot commented Feb 22, 2023

Deploying with  Cloudflare Pages  Cloudflare Pages

Latest commit: 775d08a
Status:🚫  Build failed.

View logs

@codecov
Copy link
Copy Markdown

codecov Bot commented Feb 22, 2023

Codecov Report

Merging #29 (775d08a) into main (07ba9fa) will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff          @@
##            main     #29   +/-   ##
=====================================
  Coverage   0.00%   0.00%           
=====================================
  Files          2       2           
  Lines        106     106           
=====================================
  Misses       106     106           

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@Suhyma Suhyma changed the title Implementing keyword search through preprints Implementing keyword search JS script Feb 23, 2023
Comment thread src/data-search.js Outdated

// Getting all latest articles from BiorXiv
console.log(`Fetching from ${BIORXIV_SOURCE} between ${START_DATE} and ${END_DATE}`);
fs.open(BIORXIV_FILE, 'w', function (err, file) { // consider changing the callback function if needed
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of these fs.open calls (1) are never used (2) are passed a callback that doesn't do anything. Remove?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread src/data-search.js Outdated
if (err) throw err;
console.log('Saved!');
});
const combinedData = {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you're intending on concatenating two arrays bioData and medData this destructure isn't doing that. In fact it breaks the search, which expects an array.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe you meant [...a, ...b]

Comment thread src/data-search.js Outdated
};
console.log(`Searching for ${QUERY}`);
const searchHits = await search(QUERY, outputOptions);
const numSearchHits = Object.keys(searchHits).length;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

search looks like it returns an empty Promise - so I would expect searchHits to resolve to undefined.

@Suhyma Suhyma merged commit 6f0f746 into main Feb 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants