Another fetchbatched fix

[tehpsalmist]

This introduces a breaking change to the fetchBatched method, in that the second param will be a full params object (which can included a filter, of course), instead of only a filter object. This allows the developer to customize important params like limit, attributes, and filter params (e.g. not-field123, contains-string, etc).

Another method was also added: fetchBatchedPaginated, which mirrors fetchBatched except that the return value will be wrapped in an Array, and each item in that Array will correspond to a page of data (also an Array), the length of which is defined by the limit param.

For both methods, if the limit param provided is not a valid number for that resource, the corrected limit that will be returned from Zengine is used to complete the rest of the recursive data-gathering, to ensure that each page is equal in length (except the last one) and all records are accurately obtained from the API.

One final note on the implementation: previously the fetchBatched method relied on Promise.all to get the remaining pages, but that is not very API friendly for large datasets, so the recursive pattern was used instead.

[coveralls]

Coverage remained the same at 100.0% when pulling b95c556 on another-fetchbatched-fix into e822b08 on master.

[tehpsalmist]

Thinking about releasing this as 2.0.0 because of the breaking change on that method, but maybe that's not how semver works. Anyone with better insight on that?

[AParks]

@tehpsalmist if you include the words "BREAKING CHANGE" in one of your commits, then the release script will know to do a major version bump - https://github.com/conventional-changelog/standard-version#commit-message-convention-at-a-glance

[tehpsalmist]

Thanks for the tip! So I guess you agree then that this should be a 2.0.0 release?

[AParks]

@tehpsalmist whats your use case for fetchBatchPaginated?

[tedkulp]

Since custom dev is going to start making heavy use of jobs/batching, the idea is that we can pull the records in a way that's already setup to create all the job's batches without having to recalc and split all the records into usable chunks.

[AParks]

ah makes sense thanks @tedkulp

[tehpsalmist]

What will it take to see this merged? I'm all ears.

[evert0n]

LGTM! 👍

[tehpsalmist]

Thanks y'all. @AParks I'll get some concurrency in those requests eventually, I promise.