Add integration crate for the warp framework by tomhoule · Pull Request #216 · graphql-rust/juniper

tomhoule · 2018-08-03T16:08:20Z

I think this still needs more testing but coming up with this integration was very smooth and it feels like idiomatic warp (I've only been programming with warp since its first public release, so yesterday :)).

I'll keep working on polishing this over the weekend (docs, tests) but feedback would still be appreciated.

This notably does not have support for batch requests. Should it be implemented in this PR or can it be done separately?

tomhoule · 2018-08-03T18:47:51Z

+) -> BoxedFilter<(impl warp::Reply,)>
+where
+    Context: Send + Sync + 'static,
+    // CtxRef: AsRef<Context> + Send + 'static,


this should be deleted

tomhoule · 2018-08-03T18:48:12Z

+    Context: Send + Sync + 'static,
+    // CtxRef: AsRef<Context> + Send + 'static,
+    Query: juniper::GraphQLType<Context = Context, TypeInfo = ()> + Send + Sync + 'static,
+    Mutation: juniper::GraphQLType<Context = Context, TypeInfo = ()> + Send + Sync + 'static,


I played with the design quite a bit, maybe not all of these bounds are needed anymore

LegNeato · 2018-08-06T18:58:35Z

This is looking great! I'm just coming up to speed, but if you are not aware we have an integration test harness you likely want to leverage. Here's Rocket using it: https://github.com/graphql-rust/juniper/blob/master/juniper_rocket/src/lib.rs#L453

tomhoule · 2018-08-06T19:14:18Z

I wasn't aware of that, thanks for the pointer!

tomhoule · 2018-08-07T05:50:17Z

+                .and(juniper_warp::graphiql_handler("/graphql"))
+                .or(homepage),
+        ).or(warp::path("graphql").and(graphql_filter))
+        .with(log),


with a few more days of warp experience, this doesn't look like the right way to do it, I'll have to improve this. Should this be a single file example by the way? I haven't worked with those but this one is meant to be run and used in a browser

tomhoule · 2018-08-12T20:20:19Z

Just a heads up: I am in vacations for the next 10 days so I will probably not work on this, but I intend to finish it after.

tomhoule · 2018-08-19T13:15:13Z

Small update on this: I spent some time setting up the framework integration harness, and GET query parameters handling is a bit weird when testing a filter directly, I may have to change the tests to spawn a whole server to test against instead.

edit: this is solved, now other tests are failing

tomhoule · 2018-08-19T14:26:42Z

Ok now all tests are passing locally (I haven't seen on CI yet). There is the small matter of error handling - I introduced failure as a dependency for juniper_warp since that's what I was able to implement a fast solution with, but we might want to have a custom error enum.

tomhoule · 2018-08-20T11:05:31Z

@LegNeato I think this is ready for review.

There is an issue with rust 1.21 and 1.22 in CI - some warp dependencies don't seem to compile with these versions. If latest stable is the oldest supported compiler for warp, we should probably have the same requirement for juniper-warp. I haven't looked into how that could be implemented yet. What do you think?

I'll squash before merging.

tomhoule · 2018-08-20T13:55:19Z

+/// let graphql_filter = make_graphql_filter(schema, context_extractor);
+///
+/// let graphql_endpoint = warp::path("graphql")
+///     .and(warp::post(graphql_filter));


this style of method filter is going to be deprecated, I'll change this next time I touch the code

LegNeato

Looks great! A couple of small things. I am not really qualified to review to warp portions so I will defer to you on those!

LegNeato · 2018-08-21T00:20:47Z

  "juniper_iron",
  "juniper_rocket",
+  "juniper_warp",
+  "juniper_warp/examples/warp_server",


Is this necessary? I don't believe we needed it for the other integrations.

LegNeato · 2018-08-21T00:23:52Z

+/// let graphql_filter = make_graphql_filter(schema, context_extractor);
+///
+/// let graphql_endpoint = warp::path("graphql")
+///     .and(warp::post(graphql_filter));


LegNeato · 2018-08-21T00:26:22Z

+
+            // if url.len() > 200 {
+            //     panic!("{:?}", url);
+            // }


LegNeato · 2018-08-21T00:26:58Z

+    impl HTTPIntegration for TestWarpIntegration {
+        fn get(&self, url: &str) -> TestResponse {
+            use percent_encoding::{percent_encode, DEFAULT_ENCODE_SET};
+            let url: String = percent_encode(url.replace("/?", "").as_bytes(), DEFAULT_ENCODE_SET)


This is fine for now, but I think we'll have the harness send valid encoded URLs so all these integrations don't have to deal with it themselves.

I'll check if it's possible to remove them, I remember I had to do the url encoding to make it work, but it could be a bug in warp (the framework is very young) - I'll try to open an issue on their side if that's the case.

Oh, I wasn't clear...I was just musing out loud! I don't think there is a bug in warp or you changes here, but it is actually an issue with the integration test harness. See #230 (comment) for the same thing with Hyper, and I had to fix it hackily in the iron integration a bit ago in #208.

Oh alright, sorry for the misunderstanding :)

LegNeato · 2018-08-21T00:31:47Z

No need to squash before merging, I can do that.

As for rust version, I'm not sure. We could not include juniper_warp in the workspace and test it separately on CI using only the most recent stable rust versions. We could probably support that at the CI config level, but we are also using cargo make so maybe that is a better spot for the split. It feels weird to bump juniper's requirement for the warp integration, so let's not do that.

I'm happy to poke at the build stuff if you don't want to...just let me know!

tomhoule · 2018-08-21T16:23:11Z

Agreed, the higher minimum version should only apply to juniper_warp. I'll try to make the CI setup adjustments and come back here if I run into trouble.

sagiegurari · 2018-08-22T07:06:01Z

you can take a look at the juniper rocket makefile.toml as an example

tomhoule · 2018-08-25T05:56:06Z

I'll try to get this done some time this weekend.

Just a note so we don't forget: we need some kind of thread pool, or document that this needs to be done by the user, since warp is async but juniper is not (yet).

LegNeato · 2018-08-25T06:51:39Z

@tomhoule Awesome, no rush! The Hyper integration (#230) includes a way to pass in afutures_cpupool::CpuPool so I would imagine we want to do the same. Not sure if there is a generic "pool" facade to be had or if one would even be necessary.

tomhoule · 2018-08-25T08:04:30Z

tokio currently has its own threadpool implementation that they claim is much faster than futures-cpupool. But mainly it is going to merge with the thread pool that will be built-in in the next major version of futures. This is probably what we will want to use eventually, but it's probably easier to use an external crate that does not depend on a particular version of futures (other than 0.1.*) or tokio at first (so futures-cpupool).

What do you think of making the thread pool implicit in make_graphql_filter (the filter makes one by default), and allow passing one in for users who want more control? The advantage of the implicit default thread pool is that the juniper filter appears as purely async from that angle (and less boilerplate).

edit: to make it more explicit, there would be two functions: make_graphql_filter and make_graphql_filter_with_thread_pool.

codecov-io · 2018-08-26T06:45:31Z

Codecov Report

Merging #216 into master will increase coverage by 0.58%.
The diff coverage is 74%.

@@            Coverage Diff             @@
##           master     #216      +/-   ##
==========================================
+ Coverage   86.98%   87.57%   +0.58%     
==========================================
  Files          96       98       +2     
  Lines       14196    16680    +2484     
==========================================
+ Hits        12349    14607    +2258     
- Misses       1847     2073     +226

Impacted Files	Coverage Δ
juniper_warp/examples/warp_server/src/main.rs	`3.03% <3.03%> (ø)`
juniper_warp/src/lib.rs	`94.01% <94.01%> (ø)`
juniper/src/macros/tests/object.rs	`85.71% <0%> (-4.1%)`	⬇️
juniper/src/macros/tests/scalar.rs	`73.21% <0%> (-2.69%)`	⬇️
juniper_tests/src/codegen/derive_object.rs	`85.71% <0%> (-2.22%)`	⬇️
juniper/src/macros/tests/union.rs	`80.29% <0%> (-2%)`	⬇️
juniper/src/macros/tests/interface.rs	`80.81% <0%> (-1.98%)`	⬇️
juniper/src/macros/tests/field.rs	`94.66% <0%> (-1.86%)`	⬇️
juniper/src/types/base.rs	`88.32% <0%> (-0.39%)`	⬇️
juniper/src/types/pointers.rs	`90.9% <0%> (-0.27%)`	⬇️
... and 29 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 45004f7...0a6b2ab. Read the comment docs.

LegNeato · 2018-08-27T23:26:37Z

FYI, we ended up dropping support for rust 1.21 to move our dep on regex to 1.0 in master...so we are fine requiring 1.22 here as well. If you rebase it should pass CI. Sorry for the churn and making you muck around with the build stuff!

tomhoule · 2018-08-28T05:33:15Z

I'll push my latest changes (small improvements and some cargo make things that may not be necessary anymore) and see if I can make the Makefile.toml simpler, or even remove it.

When depending on juniper through a path instead of the latest version (0.9.2) I get a test failure with batched queries (which aren't implemented), so that still needs to be done but I think it's the last missing feature.

[to be squashed] there is one failing test that still needs to be investigated

Otherwise it is impossible to write `graphql_filter.or(graphiql_filter).unify()`.

This reverts commit da59df8.

This reverts commit daf4722.

tomhoule · 2018-09-07T05:50:24Z

Just found time to work on this a bit - the new http integration tests are passing now :)

LegNeato

Awesome!!! I think we are ready pending deleting the extra file.

LegNeato · 2018-09-07T07:46:29Z

@@ -0,0 +1,9 @@
+# Exits successfully if the Rust version that is running is compatible with juniper_warp.


We don't need this file anymore, right?

Oh definitely, my bad. I'll also try to see if we can revert to using cargo make with the new feature.

We can do that in a follow-up, just delete this file for now.

LegNeato · 2018-09-07T21:30:06Z

🎉🎉🎉🍻 🎉 🍻

I am publishing a new juniper + all the integration crates next week...I'll publish juniper_warp and add you as an owner at that time.

tomhoule force-pushed the juniper-warp branch 2 times, most recently from cbc7241 to b7cdb76 Compare August 3, 2018 16:29

tomhoule changed the title ~~Add integration crate for the warp framework~~ WIP - Add integration crate for the warp framework Aug 3, 2018

tomhoule mentioned this pull request Aug 3, 2018

Build a warp integration? #215

Closed

tomhoule commented Aug 3, 2018

View reviewed changes

tomhoule commented Aug 7, 2018

View reviewed changes

tomhoule force-pushed the juniper-warp branch from 0b20f87 to c6ee983 Compare August 19, 2018 06:46

tomhoule force-pushed the juniper-warp branch 2 times, most recently from 40c9281 to d80c1b8 Compare August 19, 2018 14:24

tomhoule force-pushed the juniper-warp branch from d80c1b8 to ac39057 Compare August 20, 2018 10:29

tomhoule changed the title ~~WIP - Add integration crate for the warp framework~~ Add integration crate for the warp framework Aug 20, 2018

tomhoule commented Aug 20, 2018

View reviewed changes

LegNeato requested changes Aug 21, 2018

View reviewed changes

tomhoule force-pushed the juniper-warp branch from 742d7b2 to da59df8 Compare August 28, 2018 16:29

tomhoule added 23 commits September 7, 2018 06:49

Finish implementing GET requests handling

5e311e7

Fix warning and add #[deny(warnings)] in warp example

e1aabda

Disable CI on old stable compilers for juniper_warp only

e03ea0a

Implement passing/creating a thread pool to avoid blocking

dbc680b

[to be squashed] there is one failing test that still needs to be investigated

Properly configure CI for juniper_warp

a042eed

Document implicit thread pool creation on make_graphql_filter

eabdc23

Remove useless bounds

2d505e2

Add proper links in juniper_warp docs

1b07c0b

WIP - tweak Makefile.toml

d0033b3

Make the http response body bytes everywhere

7f288ee

Otherwise it is impossible to write `graphql_filter.or(graphiql_filter).unify()`.

Tweak Makefile.toml to make juniper_warp build

a6f179c

Implement batched queries

137c792

Try deleting the juniper_warp Makefile.toml

6fcb4cf

Revert "Try deleting the juniper_warp Makefile.toml"

985265c

This reverts commit da59df8.

Extract the cargo-make compatibility checking script

a5d1d6d

Remove juniper_warp/Makefile.toml

6fc423b

Revert "Remove juniper_warp/Makefile.toml"

f81d081

This reverts commit daf4722.

Put version script back in Makefile.toml

90e8dee

Remove cargo-make version filter script for juniper_warp

91de542

Remove Makefile.toml

16dd880

WIP - special-case juniper_warp in CI

10c08bb

Run juniper_warp CI outside of cargo-make

e11606b

Make juniper_warp pass new http harness tests

8c1aa7f

tomhoule force-pushed the juniper-warp branch from b395304 to 8c1aa7f Compare September 7, 2018 05:24

LegNeato requested changes Sep 7, 2018

View reviewed changes

Final cleanup

358ea91

LegNeato approved these changes Sep 7, 2018

View reviewed changes

LegNeato merged commit fd636e0 into graphql-rust:master Sep 7, 2018

		@@ -0,0 +1,9 @@
		# Exits successfully if the Rust version that is running is compatible with juniper_warp.

Conversation

tomhoule commented Aug 3, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LegNeato commented Aug 6, 2018

Uh oh!

tomhoule commented Aug 6, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tomhoule commented Aug 12, 2018

Uh oh!

tomhoule commented Aug 19, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tomhoule commented Aug 19, 2018

Uh oh!

tomhoule commented Aug 20, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LegNeato left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tomhoule Aug 21, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LegNeato commented Aug 21, 2018

Uh oh!

tomhoule commented Aug 21, 2018

Uh oh!

sagiegurari commented Aug 22, 2018

Uh oh!

tomhoule commented Aug 25, 2018

Uh oh!

LegNeato commented Aug 25, 2018

Uh oh!

tomhoule commented Aug 25, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov-io commented Aug 26, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

LegNeato commented Aug 27, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tomhoule commented Aug 28, 2018

Uh oh!

tomhoule commented Sep 7, 2018

Uh oh!

LegNeato left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

tomhoule commented Aug 3, 2018 •

edited

Loading

tomhoule commented Aug 19, 2018 •

edited

Loading

tomhoule Aug 21, 2018 •

edited

Loading

tomhoule commented Aug 25, 2018 •

edited

Loading

codecov-io commented Aug 26, 2018 •

edited

Loading

LegNeato commented Aug 27, 2018 •

edited

Loading