Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support a maximum batch size per DataSource #76

Merged
1 commit merged into from
Nov 9, 2016
Merged

Conversation

peterneyens
Copy link
Contributor

  • Add the maxBatchSizePhase interpreter which splits fetches to data sources with a maximum batch size in to multiple sequential fetches. This required a change to Concurrent to return a InMemoryCache instead of a DataSourceCache making it possible to combine multiple caches for sequential batches.
  • Refactor the processConcurrent method which is used in the coreInterpreter (previously called interpreter) by using XorT instead of nested if / else and Option.fold and by extracting some helper methods.
  • Give some members of the Fetch constructors a better name (eg. FetchMany.as to FetchMany.ids)
  • Create the Fetch.multiple method which makes it possible to create a Fetch[List[...]] directly (using FetchMany underneath). This can currently (and previously) be done by using sequence or traverse, but multiple doesn't have the overhead of inspecting the Fetch structure to find independent fetches which can be parallelized.

I changed `Concurrent` to have a `NonEmptyList` instead of a `List`,
which led to some issues with the invariance of `NonEmptyList` and the
existential type `FetchQuery[_, _]`, for the moment I changed it to
`NonEmptyList[FetchQuery[Any, Any]]`.
@codecov-io
Copy link

Current coverage is 75.87% (diff: 92.04%)

Merging #76 into master will increase coverage by 6.13%

@@             master        #76   diff @@
==========================================
  Files             8          8          
  Lines           195        228    +33   
  Methods         190        226    +36   
  Messages          0          0          
  Branches          5          2     -3   
==========================================
+ Hits            136        173    +37   
+ Misses           59         55     -4   
  Partials          0          0          

Powered by Codecov. Last update 26f7b45...cb60e82

@ghost
Copy link

ghost commented Nov 9, 2016

Outstanding work, well done! :shipit:

@ghost ghost merged commit 58848d0 into master Nov 9, 2016
@ghost ghost deleted the peter-max-batch-size branch November 9, 2016 10:42
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants