Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

apache-kafka: replace ParNew to G1 YoungGeneration #100115

Closed
wants to merge 2 commits into from

Conversation

@GTrunSec
Copy link
Contributor

@GTrunSec GTrunSec commented Oct 9, 2020

Motivation for this change
  1. the flag of UseParNewGC has been deprecated in JDK 9 and removed in JDK 10.

you can find info from here:

  1. remove CMS flags
    Dropping support for CMS and then removing the CMS code, or at least more thoroughly segregating it, will reduce the maintenance burden of the GC code base and accelerate new development. The G1 garbage collector is intended, in the long term, to be a replacement for most uses of CMS.

the issue output will be like this:

Unrecognized VM option 'UseParNewGC'
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
Things done
  • test flags
 more /nix/store/bq7qd8mmm63rwn1kzbw9rsfprnpa7q10-run-kafka/nix/store/y853yiia1ky7aipp73fv08xd2n6y3jyd-openjdk-14.0.2-ga/bin/java \
  -cp "/nix/store/kvq4jw9d9kzigdik4n2wrx1q9mzgapc2-apache-kafka-2.13-2.5.0/libs/*" \
  -Dlog4j.configuration=file:/nix/store/7qq35w8373pvbn9av0xk9c2rmwbas20m-log4j.properties \
  -server -Xmx1G -Xms1G -XX:+UseCompressedOops -XX:+UseG1GC -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled -XX:+CMSScavengeBeforeR
emark -XX:+DisableExplicitGC -Djava.awt.headless=true -Djava.net.preferIPv4Stack=true \
  kafka.Kafka \
  /nix/store/ys10abnr79brsqh9ys1mxzx19jwmcd59-server.properties
  • output
    NOTICE: the ignoring warning will be sloved in second commit.
/nix/store/2jysm3dfsgby5sw5jgj43qjrb5v79ms9-bash-4.4-p23/bin/bash /nix/store/bq7qd8mmm63rwn1kzbw9rsfprnpa7q10-run-kafka
OpenJDK 64-Bit Server VM warning: Ignoring option UseConcMarkSweepGC; support was removed in 14.0
OpenJDK 64-Bit Server VM warning: Ignoring option CMSClassUnloadingEnabled; support was removed in 14.0
OpenJDK 64-Bit Server VM warning: Ignoring option CMSScavengeBeforeRemark; support was removed in 14.0
[2020-10-10 07:01:56,191] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)
[2020-10-10 07:01:56,444] INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util)
[2020-10-10 07:01:56,491] INFO Registered signal handlers for TERM, INT, HUP (org.apache.kafka.common.utils.LoggingSignalHandler)
[2020-10-10 07:01:56,495] INFO starting (kafka.server.KafkaServer)
[2020-10-10 07:01:56,496] INFO Connecting to zookeeper on localhost:2181 (kafka.server.KafkaServer)
[2020-10-10 07:01:56,517] INFO [ZooKeeperClient Kafka server] Initializing a new session to localhost:2181. (kafka.zookeeper.ZooKeeperClient)
[2020-10-10 07:01:56,525] INFO Client environment:zookeeper.version=3.5.7-f0fdd52973d373ffd9c86b81d99842dc2c7f660e, built on 02/10/2020 11:30 GMT (org.apache.zookeeper.ZooKeeper)
[2020-10-10 07:01:56,525] INFO Client environment:host.name=gtcompile (org.apache.zookeeper.ZooKeeper)
[2020-10-10 07:01:56,525] INFO Client environment:java.version=14.0.2-internal (org.apache.zookeeper.ZooK
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
"-XX:+UseConcMarkSweepGC"
"-XX:+CMSClassUnloadingEnabled"
"-XX:+CMSScavengeBeforeRemark"
"-XX:+UseG1GC"
Copy link
Contributor

@asbachb asbachb Oct 12, 2020

I wonder if there's a need to be explicit since G1 should be default GC since java 9.

Copy link
Contributor Author

@GTrunSec GTrunSec Oct 12, 2020

https://github.com/apache/kafka/blob/aba9036eb681597911343510e246de4fc547a479/bin/kafka-run-class.sh#L257. I think, the option of default should be matched with this line of the shell. otherwise, I can remove the default flags to list of example such as XX:+DisableExplicitGC was replaced by -XX:+ExplicitGCInvokesConcurrent in the default JVM settings. may this will be a good way to comment for users. it does not matter the flag is default setting or not. Or set the default list to empty(the example list as explain option).

@GTrunSec GTrunSec force-pushed the apache-kafka-gc branch from 2ca24ff to 6cc8a8e Oct 12, 2020
@GTrunSec GTrunSec closed this Oct 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants