[#3750] feat: Support running in frontend #3749
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Good point! |
pan3793
changed the title
|
Also this document needs to be modified, thank you. |
mchades
approved these changes
Jun 4, 2024
diqiu50
pushed a commit
to diqiu50/gravitino
that referenced
this pull request
Jun 13, 2024
### What changes were proposed in this pull request? `bin/gravitino.sh` support running on frontend. ### Why are the changes needed? For dev cases, it's easy to start/stop and watch logs. For production environments where services are managed by `supervisord` or `systemd`, we usually run the service in frontend and print logs to stdout/stderr. Fix: apache#3750 ### Does this PR introduce _any_ user-facing change? Yes, users can run `bin/gravitino.sh run` to run Gravitino in frontend, and press `Ctrl + C` to terminate the process. ### How was this patch tested? Manuel test. ``` $ ./gradlew clean compileDistribution -x test $ cd distribution/package $ echo > conf/log4j2.properties <<EOF rootLogger.level = info rootLogger.appenderRef.rolling.ref = console appender.console.type = Console appender.console.name = console appender.console.target = SYSTEM_ERR appender.console.layout.type = PatternLayout appender.console.layout.pattern = %d{yy/MM/dd HH:mm:ss} %p %c{1}: %m%n%ex EOF $ bin/gravitino.sh run 24/06/04 17:53:58 INFO GravitinoServer: Starting Gravitino Server 24/06/04 17:53:58 INFO GravitinoEnv: Initializing Gravitino Environment... 24/06/04 17:53:58 INFO MetricsSystem: Register jvm to metrics system 24/06/04 17:53:58 INFO RocksDBKvBackend: Rocksdb storage directory:/Users/chengpan/Projects/gravitino/distribution/package/data/rocksdb/instance 24/06/04 17:53:59 INFO AuxiliaryServiceManager: AuxService name:iceberg-rest, config:{classpath=catalogs/lakehouse-iceberg/libs, catalogs/lakehouse-iceberg/conf, httpPort=9001, host=0.0.0.0}, valid classpath:[/Users/chengpan/Projects/gravitino/distribution/package/catalogs/lakehouse-iceberg/libs, /Users/chengpan/Projects/gravitino/distribution/package/catalogs/lakehouse-iceberg/conf] 24/06/04 17:53:59 INFO AuxiliaryServiceManager: AuxService:iceberg-rest registered successfully 24/06/04 17:53:59 INFO log: Logging initialized @767ms to org.eclipse.jetty.util.log.Slf4jLog 24/06/04 17:53:59 INFO MetricsSystem: Register iceberg-rest-server to metrics system 24/06/04 17:53:59 INFO IcebergCatalogUtil: Load catalog backend of memory 24/06/04 17:53:59 INFO IcebergTableOps: Load Iceberg metrics store: com.datastrato.gravitino.catalog.lakehouse.iceberg.web.metrics.DummyMetricsStore. 24/06/04 17:53:59 INFO IcebergRESTService: Iceberg REST service inited 24/06/04 17:53:59 INFO GravitinoEnv: Gravitino Environment is initialized. 24/06/04 17:53:59 INFO LockManager: Start to check the dead lock... 24/06/04 17:53:59 INFO LockManager: Finish to check the dead lock... 24/06/04 17:53:59 INFO JettyServer: Gravitino Webapp path: /var/folders/v5/tw7cy5px1xjbt1pt77x70m980000gp/T/GravitinoWar17320525550260632159 24/06/04 17:53:59 INFO MetricsSystem: Register gravitino-server to metrics system 24/06/04 17:53:59 INFO Server: jetty-9.4.51.v20230217; built: 2023-02-17T08:19:37.309Z; git: b45c405e4544384de066f814ed42ae3dceacdd49; jvm 17.0.8+7-LTS Jun 04, 2024 5:53:59 PM org.glassfish.jersey.server.wadl.WadlFeature configure WARNING: JAXBContext implementation could not be found. WADL feature is disabled. 24/06/04 17:53:59 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@3ac3f6f{/,null,AVAILABLE} 24/06/04 17:53:59 INFO AbstractConnector: Started ServerConnector@7343922c{HTTP/1.1, (http/1.1)}{0.0.0.0:9001} 24/06/04 17:53:59 INFO Server: Started @1172ms 24/06/04 17:53:59 WARN JettyServer: Users would better use HTTPS to avoid token data leak. 24/06/04 17:53:59 INFO JettyServer: iceberg-rest web server started on host 0.0.0.0 port 9001. 24/06/04 17:53:59 INFO IcebergRESTService: Iceberg REST service started 24/06/04 17:53:59 INFO Server: jetty-9.4.51.v20230217; built: 2023-02-17T08:19:37.309Z; git: b45c405e4544384de066f814ed42ae3dceacdd49; jvm 17.0.8+7-LTS 24/06/04 17:53:59 INFO StandardDescriptorProcessor: NO JSP Support for /, did not find org.eclipse.jetty.jsp.JettyJspServlet 24/06/04 17:53:59 INFO session: DefaultSessionIdManager workerName=node0 24/06/04 17:53:59 INFO session: No SessionScavenger set, using defaults 24/06/04 17:53:59 INFO session: node0 Scavenging every 660000ms Jun 04, 2024 5:53:59 PM org.glassfish.jersey.server.wadl.WadlFeature configure WARNING: JAXBContext implementation could not be found. WADL feature is disabled. 24/06/04 17:53:59 INFO ContextHandler: Started o.e.j.w.WebAppContext@9f674ac{gravitino-web,/,jar:file:///Users/chengpan/Projects/gravitino/distribution/package/web/gravitino-web-0.6.0-SNAPSHOT.war!/,AVAILABLE}{/Users/chengpan/Projects/gravitino/distribution/package/web/gravitino-web-0.6.0-SNAPSHOT.war} 24/06/04 17:53:59 INFO AbstractConnector: Started ServerConnector@1da4b3f9{HTTP/1.1, (http/1.1)}{0.0.0.0:8090} 24/06/04 17:53:59 INFO Server: Started @1307ms 24/06/04 17:53:59 WARN JettyServer: Users would better use HTTPS to avoid token data leak. 24/06/04 17:53:59 INFO JettyServer: Gravitino-webserver web server started on host 0.0.0.0 port 8090. 24/06/04 17:53:59 INFO GravitinoServer: Done, Gravitino server started. <Press CTRL+C> ^C24/06/04 17:54:03 INFO AbstractConnector: Stopped ServerConnector@7343922c{HTTP/1.1, (http/1.1)}{0.0.0.0:9001} 24/06/04 17:54:03 INFO ContextHandler: Stopped o.e.j.s.ServletContextHandler@3ac3f6f{/,null,STOPPED} 24/06/04 17:54:03 INFO AbstractConnector: Stopped ServerConnector@1da4b3f9{HTTP/1.1, (http/1.1)}{0.0.0.0:8090} 24/06/04 17:54:03 INFO session: node0 Stopped scavenging 24/06/04 17:54:03 INFO ContextHandler: Stopped o.e.j.w.WebAppContext@9f674ac{gravitino-web,/,null,STOPPED}{/Users/chengpan/Projects/gravitino/distribution/package/web/gravitino-web-0.6.0-SNAPSHOT.war} 24/06/04 17:54:03 INFO GravitinoServer: Shutting down Gravitino Server ... 24/06/04 17:54:03 INFO JettyServer: Gravitino-webserver web server stopped on host 0.0.0.0 port 8090. 24/06/04 17:54:03 INFO GravitinoEnv: Shutting down Gravitino Environment... 24/06/04 17:54:03 INFO JettyServer: iceberg-rest web server stopped on host 0.0.0.0 port 9001. 24/06/04 17:54:03 INFO IcebergRESTService: Iceberg REST service stopped 24/06/04 17:54:03 WARN IcebergTableOps: Iceberg Metrics writer thread is interrupted. 24/06/04 17:54:03 INFO GravitinoEnv: Gravitino Environment is shut down. 24/06/04 17:54:03 INFO GravitinoServer: Gravitino Server has shut down. ```
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
bin/gravitino.shsupport running on frontend.Why are the changes needed?
For dev cases, it's easy to start/stop and watch logs.
For production environments where services are managed by
supervisordorsystemd, we usually run the service in frontend and print logs to stdout/stderr.Does this PR introduce any user-facing change?
Yes, users can run
bin/gravitino.sh runto run Gravitino in frontend, and pressCtrl + Cto terminate the process.How was this patch tested?
Manuel test.