-
Notifications
You must be signed in to change notification settings - Fork 1
fix(scrape-single-url): return the only dataset item after the run is finished #3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix(scrape-single-url): return the only dataset item after the run is finished #3
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Small comments
components/apify/actions/scrape-single-url/scrape-single-url.mjs
Outdated
Show resolved
Hide resolved
components/apify/actions/scrape-single-url/scrape-single-url.mjs
Outdated
Show resolved
Hide resolved
| name: "Get Dataset Items", | ||
| description: "Returns data stored in a dataset. [See the documentation](https://docs.apify.com/api/v2/dataset-items-get)", | ||
| version: "0.0.2", | ||
| version: "0.0.3", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we really need to change version of all actions, when we are updating just scrape singe URL?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't get to the bottom of it, but this is the step I needed to take for PR checks to pass. I doesn't make much sense to me either, maybe this problem is caused by the fact, that this a PR from a different repo.
…t of terminal statuses to stop the loop
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please import consts from package otherwise fine, pre-approving. 👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just have one comment about the polling rates, other than that looks good
… in between calls
* feat(apify): Show list of built tags in actor run action * feat(apify): Allow selecting actor search source for run Actor action and change how Actor or task name is displayed * feat(apif): Remove wait for finish prop in run Actor action * fix(apify): Fix PR issues * fix(apif): Fix PR issues * fix(scrape-single-url): return the only dataset item after the run is finished (#3) * fix(scrape-single-url): return the only dataset item after the run is finished * fix(scrape-single-url): version up * fix(scrape-single-url): version up * fix(scrape-single-url): version up * fix(scrape-single-url): introduce a job status constant, expand a list of terminal statuses to stop the loop * fix(scrape-single-url): import constants from package, decrease delay in between calls * Migrate to use Apify client (#6) * feat(apify): Replace Axios with Apify client * fix(general): adding custom headers to client()=> preserve whole config to be passed to Axios later * feat(general): add linter script * fix(apify-get-dataset-items): a function for getting items and parsing of a result * fix(apify-run-actor): working sync and async, dynamic input schema injection, KVS output retrieval tested only string * fix(general): change maxResults for limit as an input field * fix(run-task-sync): move items retrieval to the component, add waitSecs determined by input or plan to prevent blunt timeout error, have the item retrieval logic be connected to run status, clean return value * fix(apify-scrape-single-url): incorporate timeouts, rework the whole API interaction logic * fix(apify-set-key-value-store-record): detection of content type, fixed API interaction * fix(apify-scrape-single-url): remove waiting timeout, return only dataset item, remove extra input fields connected to WCC run * fix(apify-run-actor): success message * fix(apify-run-task-synchronously): remove waiting for run to finish timeout * fix(app): remove paidPlan input filed config --------- Co-authored-by: Matyas Cimbulka <matyas.cimbulka@apify.com> * fix(apify-get-dataset-items) 6: change input parameters (#8) * chore(apify): Bump component versions * chore: Sync upstream repo (#9) * feat(apify): Prefill values from the input schema (#7) * Revert "chore: Sync upstream repo (#9)" This reverts commit cd804ba. * Revert "chore(apify): Bump component versions" This reverts commit 6040822 which for some reason bumped version of the wrong components. * fix(apify): Fix build tag * fix(apify): Address issues in run task synchronously action * feat(apify): Add default crawler type to scrape single url * fix(apify): Address issues from PR * chore(apify): Change component versions * fix(apify): Fix run Actor action * fix(apify): Fix run get dataset items action * fix(apify): Fix typos for PR --------- Co-authored-by: Oleksandra Valko <oleksandra.valko@apify.com>
WHY
Connected to this issue: https://github.com/orgs/apify/projects/19/views/1?pane=issue&itemId=118956431&issue=apify%7Cintegrations-team%7C4