Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

grpc-netty-shaded is adding a lot of weight to apps using google-cloud-java #6425

Closed
clementdenis opened this issue Oct 3, 2019 · 5 comments
Labels
needs more info This issue needs more information from the customer to proceed. type: question Request for information or clarification. Not an issue.

Comments

@clementdenis
Copy link

The current version of some google-cloud-* libraries have grpc-netty-shaded as a dependency. This jar alone is almost 7 MB in size.

For environments with limited resources (App Engine, Cloud Functions), this can add precious seconds to startup time.

Could you consider changing the default transport?

There a way to reduce the size of the app by excluded the grpc-netty-shaded transport and replacing it with the much lighter grpc-okhttp. Could it be documented?

<dependency>
  <groupId>com.google.cloud</groupId>
  <artifactId>google-cloud-logging</artifactId>
  <exclusions>
    <exclusion>
      <groupId>io.grpc</groupId>
      <artifactId>grpc-netty-shaded</artifactId>
    </exclusion>
</exclusions>
</dependency>
<dependency>
  <groupId>io.grpc</groupId>
  <artifactId>grpc-okhttp</artifactId>
  <version>1.23.0</version>
</dependency>
@ejona86
Copy link

ejona86 commented Oct 3, 2019

gRPC team would still suggest using grpc-netty on those platforms. Most of the size of grpc-netty-shaded is for netty-tcnative which has multiple binaries for boringssl, one for each platform. App Engine Java 8 contains Conscrypt, which is an alternative, and with Java 11 neither is strictly necessary. Are you using App Engine Java 8 or 11?

Empirically, how much faster is your app startup by swapping to okhttp? Also mention how long startup is overall.

Using Conscrypt with Netty takes a little bit of runtime configuration at the moment. That could be fixed for App Engine Java 8.

Could you consider changing the default transport?

No. That doesn't help. We'd need to include Conscrypt for all the other platforms and that would increase the size back to where it is now. Also, Conscrypt is broken for OkHttp on Windows at present (google/conscrypt#444) (Netty works fine with it on Windows).

@chingor13 chingor13 added the type: question Request for information or clarification. Not an issue. label Oct 3, 2019
ejona86 added a commit to ejona86/grpc-java that referenced this issue Oct 3, 2019
This implicit loading is more conservative than the loading for
tcnative, as Conscrypt will only be implicitly loaded if there are no
other options. This means the Java 9+ JSSE is preferred over Conscrypt
without explicit user configuration.

While we would generally prefer Conscrypt over JSSE, we want to allow
the user to configure their security providers. There wasn't a good way
to do that with netty-tcnative and the performance of JSSE at the time
was abysmal.

While generally being a good way to allow adopting Conscrypt, this also
allows easily using App Engine Java 8's provided Conscrypt which can
substantially reduce binary size.

See googleapis/google-cloud-java#6425
@ejona86
Copy link

ejona86 commented Oct 3, 2019

Actually... looking at OkHttp more it doesn't implicitly use Conscrypt. I thought Conscrypt wasn't in the default providers list in App Engine, but maybe that changed. The other option is @clementdenis is on App Engine Java 11.

@clementdenis
Copy link
Author

Hi,

I'm running on Java 11 indeed, which requires google-cloud-logging to get the custom log appender that can send the logs to Stackdriver in a way that allows consolidation under the request log entry (BTW, this is a LOT of dependencies for getting a feature that was built-in on App Engine Java 8 ...).

I did not time the improvement of using okhttp over the default provider, but we noticed that startup time on App Engine 11 increases significantly with the number / size of dependencies. So If we can avoid a 7MB jar, that's always a win.

@ejona86
Copy link

ejona86 commented Oct 4, 2019

Okay, so for Java 11 you can just use grpc-netty instead of grpc-netty-shaded. No need to use okhttp.

That has been the expectation for a while: we use grpc-netty-shaded for most users since it "just works" and users that are more concerned about dependency size use grpc-netty directly. I would love a world where we are all on Java 11, but here we are still supporting Java 7.

ejona86 added a commit to grpc/grpc-java that referenced this issue Oct 7, 2019
This implicit loading is more conservative than the loading for
tcnative, as Conscrypt will only be implicitly loaded if there are no
other options. This means the Java 9+ JSSE is preferred over Conscrypt
without explicit user configuration.

While we would generally prefer Conscrypt over JSSE, we want to allow
the user to configure their security providers. There wasn't a good way
to do that with netty-tcnative and the performance of JSSE at the time
was abysmal.

While generally being a good way to allow adopting Conscrypt, this also
allows easily using App Engine Java 8's provided Conscrypt which can
substantially reduce binary size.

See googleapis/google-cloud-java#6425
@codyoss
Copy link
Member

codyoss commented Dec 4, 2019

@clementdenis is this still an issue for you? Or did the suggestion above solve this for you?

@codyoss codyoss added the needs more info This issue needs more information from the customer to proceed. label Dec 6, 2019
dfawley pushed a commit to dfawley/grpc-java that referenced this issue Jan 15, 2021
This implicit loading is more conservative than the loading for
tcnative, as Conscrypt will only be implicitly loaded if there are no
other options. This means the Java 9+ JSSE is preferred over Conscrypt
without explicit user configuration.

While we would generally prefer Conscrypt over JSSE, we want to allow
the user to configure their security providers. There wasn't a good way
to do that with netty-tcnative and the performance of JSSE at the time
was abysmal.

While generally being a good way to allow adopting Conscrypt, this also
allows easily using App Engine Java 8's provided Conscrypt which can
substantially reduce binary size.

See googleapis/google-cloud-java#6425
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs more info This issue needs more information from the customer to proceed. type: question Request for information or clarification. Not an issue.
Projects
None yet
Development

No branches or pull requests

5 participants