-
Notifications
You must be signed in to change notification settings - Fork 288
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[#3750] feat: Support running in frontend #3749
Conversation
Good point! |
Also this document needs to be modified, thank you. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Thank you for the report and fix.
### What changes were proposed in this pull request? `bin/gravitino.sh` support running on frontend. ### Why are the changes needed? For dev cases, it's easy to start/stop and watch logs. For production environments where services are managed by `supervisord` or `systemd`, we usually run the service in frontend and print logs to stdout/stderr. Fix: apache#3750 ### Does this PR introduce _any_ user-facing change? Yes, users can run `bin/gravitino.sh run` to run Gravitino in frontend, and press `Ctrl + C` to terminate the process. ### How was this patch tested? Manuel test. ``` $ ./gradlew clean compileDistribution -x test $ cd distribution/package $ echo > conf/log4j2.properties <<EOF rootLogger.level = info rootLogger.appenderRef.rolling.ref = console appender.console.type = Console appender.console.name = console appender.console.target = SYSTEM_ERR appender.console.layout.type = PatternLayout appender.console.layout.pattern = %d{yy/MM/dd HH:mm:ss} %p %c{1}: %m%n%ex EOF $ bin/gravitino.sh run 24/06/04 17:53:58 INFO GravitinoServer: Starting Gravitino Server 24/06/04 17:53:58 INFO GravitinoEnv: Initializing Gravitino Environment... 24/06/04 17:53:58 INFO MetricsSystem: Register jvm to metrics system 24/06/04 17:53:58 INFO RocksDBKvBackend: Rocksdb storage directory:/Users/chengpan/Projects/gravitino/distribution/package/data/rocksdb/instance 24/06/04 17:53:59 INFO AuxiliaryServiceManager: AuxService name:iceberg-rest, config:{classpath=catalogs/lakehouse-iceberg/libs, catalogs/lakehouse-iceberg/conf, httpPort=9001, host=0.0.0.0}, valid classpath:[/Users/chengpan/Projects/gravitino/distribution/package/catalogs/lakehouse-iceberg/libs, /Users/chengpan/Projects/gravitino/distribution/package/catalogs/lakehouse-iceberg/conf] 24/06/04 17:53:59 INFO AuxiliaryServiceManager: AuxService:iceberg-rest registered successfully 24/06/04 17:53:59 INFO log: Logging initialized @767ms to org.eclipse.jetty.util.log.Slf4jLog 24/06/04 17:53:59 INFO MetricsSystem: Register iceberg-rest-server to metrics system 24/06/04 17:53:59 INFO IcebergCatalogUtil: Load catalog backend of memory 24/06/04 17:53:59 INFO IcebergTableOps: Load Iceberg metrics store: com.datastrato.gravitino.catalog.lakehouse.iceberg.web.metrics.DummyMetricsStore. 24/06/04 17:53:59 INFO IcebergRESTService: Iceberg REST service inited 24/06/04 17:53:59 INFO GravitinoEnv: Gravitino Environment is initialized. 24/06/04 17:53:59 INFO LockManager: Start to check the dead lock... 24/06/04 17:53:59 INFO LockManager: Finish to check the dead lock... 24/06/04 17:53:59 INFO JettyServer: Gravitino Webapp path: /var/folders/v5/tw7cy5px1xjbt1pt77x70m980000gp/T/GravitinoWar17320525550260632159 24/06/04 17:53:59 INFO MetricsSystem: Register gravitino-server to metrics system 24/06/04 17:53:59 INFO Server: jetty-9.4.51.v20230217; built: 2023-02-17T08:19:37.309Z; git: b45c405e4544384de066f814ed42ae3dceacdd49; jvm 17.0.8+7-LTS Jun 04, 2024 5:53:59 PM org.glassfish.jersey.server.wadl.WadlFeature configure WARNING: JAXBContext implementation could not be found. WADL feature is disabled. 24/06/04 17:53:59 INFO ContextHandler: Started o.e.j.s.ServletContextHandler@3ac3f6f{/,null,AVAILABLE} 24/06/04 17:53:59 INFO AbstractConnector: Started ServerConnector@7343922c{HTTP/1.1, (http/1.1)}{0.0.0.0:9001} 24/06/04 17:53:59 INFO Server: Started @1172ms 24/06/04 17:53:59 WARN JettyServer: Users would better use HTTPS to avoid token data leak. 24/06/04 17:53:59 INFO JettyServer: iceberg-rest web server started on host 0.0.0.0 port 9001. 24/06/04 17:53:59 INFO IcebergRESTService: Iceberg REST service started 24/06/04 17:53:59 INFO Server: jetty-9.4.51.v20230217; built: 2023-02-17T08:19:37.309Z; git: b45c405e4544384de066f814ed42ae3dceacdd49; jvm 17.0.8+7-LTS 24/06/04 17:53:59 INFO StandardDescriptorProcessor: NO JSP Support for /, did not find org.eclipse.jetty.jsp.JettyJspServlet 24/06/04 17:53:59 INFO session: DefaultSessionIdManager workerName=node0 24/06/04 17:53:59 INFO session: No SessionScavenger set, using defaults 24/06/04 17:53:59 INFO session: node0 Scavenging every 660000ms Jun 04, 2024 5:53:59 PM org.glassfish.jersey.server.wadl.WadlFeature configure WARNING: JAXBContext implementation could not be found. WADL feature is disabled. 24/06/04 17:53:59 INFO ContextHandler: Started o.e.j.w.WebAppContext@9f674ac{gravitino-web,/,jar:file:///Users/chengpan/Projects/gravitino/distribution/package/web/gravitino-web-0.6.0-SNAPSHOT.war!/,AVAILABLE}{/Users/chengpan/Projects/gravitino/distribution/package/web/gravitino-web-0.6.0-SNAPSHOT.war} 24/06/04 17:53:59 INFO AbstractConnector: Started ServerConnector@1da4b3f9{HTTP/1.1, (http/1.1)}{0.0.0.0:8090} 24/06/04 17:53:59 INFO Server: Started @1307ms 24/06/04 17:53:59 WARN JettyServer: Users would better use HTTPS to avoid token data leak. 24/06/04 17:53:59 INFO JettyServer: Gravitino-webserver web server started on host 0.0.0.0 port 8090. 24/06/04 17:53:59 INFO GravitinoServer: Done, Gravitino server started. <Press CTRL+C> ^C24/06/04 17:54:03 INFO AbstractConnector: Stopped ServerConnector@7343922c{HTTP/1.1, (http/1.1)}{0.0.0.0:9001} 24/06/04 17:54:03 INFO ContextHandler: Stopped o.e.j.s.ServletContextHandler@3ac3f6f{/,null,STOPPED} 24/06/04 17:54:03 INFO AbstractConnector: Stopped ServerConnector@1da4b3f9{HTTP/1.1, (http/1.1)}{0.0.0.0:8090} 24/06/04 17:54:03 INFO session: node0 Stopped scavenging 24/06/04 17:54:03 INFO ContextHandler: Stopped o.e.j.w.WebAppContext@9f674ac{gravitino-web,/,null,STOPPED}{/Users/chengpan/Projects/gravitino/distribution/package/web/gravitino-web-0.6.0-SNAPSHOT.war} 24/06/04 17:54:03 INFO GravitinoServer: Shutting down Gravitino Server ... 24/06/04 17:54:03 INFO JettyServer: Gravitino-webserver web server stopped on host 0.0.0.0 port 8090. 24/06/04 17:54:03 INFO GravitinoEnv: Shutting down Gravitino Environment... 24/06/04 17:54:03 INFO JettyServer: iceberg-rest web server stopped on host 0.0.0.0 port 9001. 24/06/04 17:54:03 INFO IcebergRESTService: Iceberg REST service stopped 24/06/04 17:54:03 WARN IcebergTableOps: Iceberg Metrics writer thread is interrupted. 24/06/04 17:54:03 INFO GravitinoEnv: Gravitino Environment is shut down. 24/06/04 17:54:03 INFO GravitinoServer: Gravitino Server has shut down. ```
What changes were proposed in this pull request?
bin/gravitino.sh
support running on frontend.Why are the changes needed?
For dev cases, it's easy to start/stop and watch logs.
For production environments where services are managed by
supervisord
orsystemd
, we usually run the service in frontend and print logs to stdout/stderr.Does this PR introduce any user-facing change?
Yes, users can run
bin/gravitino.sh run
to run Gravitino in frontend, and pressCtrl + C
to terminate the process.How was this patch tested?
Manuel test.