Skip to content

Latest commit

 

History

History
192 lines (144 loc) · 41.3 KB

installation.md

File metadata and controls

192 lines (144 loc) · 41.3 KB
Raw

Installation Guide

Prerequisite

  • Scavenger is a combination of various server components(API, Frontend, Collector) and agent component. To install scavenger, you need JDK 17 or latest version.
  • Scavenger uses 2 ports. if some ports are blocked by your firewall, you should ask the network admin to open the following ports in the firewall. These ports can be configured by the configuration.
    • Agent: Any ===> Collector: This is dependent on your tomcat configuration. By default, it is set as 8080.
    • Agent: Any ===> Collector: This is dependent on your grpc configuration. By default, it is set as 8080.
    • API: This is dependent on your tomcat configuration. By default, it is set as 8081.

Install Collector

Build Collector

Download Collector from the following link. https://github.com/naver/scavenger/releases

Or, You can also build it by yourself.

  1. Build Scavenger manually from the Git clone.
  2. Run ./gradlew assemble -p scavenger-collector

Installation

Start Collector using the following command.

  • e.g) java -jar scavenger-collector.jar

Configuration

You can override any configuration values with -D option.

  • java -Darmeria.port=8080 -jar scavenger-collector.jar

To change the http(default 8080) and grpc(default 8080) ports used by Collector, you can change the settings below.

  • -Darmeria.port: http, grpc uses 8080 by default

To change the max message size of gRPC in Collector, change the setting below.

  • -Darmeria.max-message-size: 10MB by default
    • Supports B, KB, MB, GB, and TB units.

The collector provides two profiles: h2 and local(default)
The profile can be enabled with spring.profiles.active configuration.

The collector uses in-memory H2 in local profile.
This means that the data is initialised every time the collector application starts.
If you want to work around this using file mode, you can enable the h2 profile.

  • java -Dspring.profiles.active=h2 -jar scavenger-collector.jar

If you've built your own MySQL, follow the steps below.

  1. Install MySQL Server using the official MySQL guide.
  2. Log into MySQL Server.
  3. Create MySQL Database. 
    CREATE DATABASE scavenger;
  4. Use the -D option to set the following four properties.
    • spring.profiles.active=mysql
    • spring.datasource.url={mysql url}
    • spring.datasource.username={mysql user name}
    • spring.datasource.password={mysql password}
    • spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
When Building Manually

You can modify default configuration values or add new profiles under scavenger-collector/src/main/resources/.

Install API

Build API

Download API from the following link. https://github.com/naver/scavenger/releases

Or, You can also build it by yourself.

  1. Build Scavenger manually from the Git clone.
  2. Run ./gradlew assemble -p scavenger-api

Installation

Start API using the following command. When you start the API, you need to set the value scavenger.collector-server-url with the -D option so that the API knows the collector's url.

  • e.g) java -Dscavenger.collector-server-url=http://localhost:8080 -jar scavenger-api-boot.jar

Configuration

You can override any configuration values with -D option.

  • java -jar scavenger-api-boot.jar

To change the http(default 8081) ports used by API, you can change the settings below.

  • server.port: Tomcat uses 8081 by default

The API provides two profiles: h2 and local(default)
The profile can be enabled with spring.profiles.active configuration.

The API uses in-memory H2 in local profile.
This means that the data is initialised every time the API application starts, and the data is independent of the collector because it is using a different h2 than the collector.
If you have enabled the h2 profile on the collector to work around this, you can enable the h2 profile on the API as well.

  • java -Dspring.profiles.active=h2 -jar scavenger-api-boot.jar

If you've built your own MySQL, follow the steps below.

If you have already deployed MySQL when installing collector, you can skip ahead to step 4.

  1. Install MySQL Server using the official MySQL guide.
  2. Log into MySQL Server.
  3. Create MySQL Database. 
    CREATE DATABASE scavenger;
  4. Use the -D option to set the following four properties.
    • spring.profiles.active=mysql
    • spring.datasource.url={mysql url}
    • spring.datasource.username={mysql user name}
    • spring.datasource.password={mysql password}
    • spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
When Building Manually

You can modify default configuration values or add new profiles under scavenger-api/src/main/resources/.

Install Agent

Build Agent

You can also build it yourself.

  1. Build Scavenger manually from the Git clone.
  2. Run ./gradlew assemble -p scavenger-agent-java

Configuration

  • By default, this is done using the scavenger.conf file.
  • It can also be configured via Java system properties and environment variables, but the agent is developed in such a way that it will not work if the scavenger.conf file does not exist.
  • Therefore, it is recommended to use scavenger.conf as the default and only to override settings.
  • The scavenger.conf can be set and downloaded via the UI.

scavenger-conf.png

  • Example scavenger.conf
apiKey=eb99ff0f-aaaa-bbbb-cccc-5d1ec81f6183
serverUrl=http://10.106.93.41:8081
environment=dev
appName=apiserver
codeBase=webapps/ROOT/WEB-INF
packages=com.navercorp
  • Scavenger configuration
configuration default Description
enabled true
  • Option to enable or disable Scavenger.
    		•
    apiKey (*)
  • The key used to identify the project.
  • ex) ec69466e-aaaa-bbbb-cccc-3021386f997e
    • serverUrl (*)
  • URL of the Scavenger collector to send collection data to.
  • ex) http://collector-test.scavenger.navercorp.com
    • appName (*)
  • A name for the application.
  • ex) foo_apiserver
    • packages (*)
  • a list of packages to scan. Packages are separated by , and all subpackages are scanned.
  • ex) io.scavenger.javaagent, io.scavenger.dashboard
    		•
    codeBase
  • In Scavenger, a codebase is a representation of the code being scaned at the time of collection. If you change the content of the code you're scaning, or change settings such as which packages are scaned, Scavenger will treat it as a new codebase and upload it. When a new codebase is uploaded, new methods are inserted and methods from the old codebase are removed by a garbage collector that runs at regular intervals. Each method is identified using a signature, which is a function signal Call information is also recorded dependent on the method in each codebase, so call information for methods whose signatures have not changed is preserved between codebase uploads.
  • This option specifies the location of the codebase. If not specified, the Scavenger agent will use the default classpath of the class loader to scan for classes. If your application loads classes on its own from a path other than the default classpath, such as Tomcat, you must explicitly specify this option as shown in the example below for proper ingestion. In your respective deployment environment, you can set this option as follows.
    1. jar file(executable jar) - Specify a directory containing a single JAR file, or specify the JAR file itself. ex) path/to/example.jar
    2. extracted jar(layertools) - Specify /lib, /classes in BOOT-INF ex) BOOT-INF/lib, BOOT-INF/classes
    3. war file(web archive) - Specify the WEB-INF directory. ex) path/to/WEB-INF
    4. Gradle worksapce - build/libs/
    5. Maven workspace - target/
  • If you're using Tomcat, you can specify the path to the webapp where the code you want to scan resides, such as codeBase = /home1/irteam/service/tomcat/webapps/ROOT/WEB-INF.
    		•
    excludePackages
  • Specifies which of the subpackages specified in packages you want to exclude from tracking.
  • ex) foo.bar, baz.qux
    		•
    annotations
  • Target sub methods of the annotated class to scan.
  • ex) @org.springframework.web.bind.annotation.RestController, @org.springframework.stereotype.Controller
    		•
    additionalPackages
  • Specifies the packages excluded by annotations that you want to include in the scan.
  • ex) io.scavenger.javaagent, io.scavenger.dashboard
    		•
    methodVisibility protected
  • Filter what to scan based on the access controller of the method. Values are public, protected, package-private, and private, with the cost increasing the further back you go. Package-private and private are unlikely to be used in practice, as they can be easily removed using static code analysis in the IDE.
    		•
    excludeConstructors false
  • Specifies whether constructors should be excluded from scaning. For Spring applications, we recommend setting this to true to exclude constructors of beans from collection because they are called automatically.
    		•
    excludeGetterSetter false
  • Specifies whether getters and setters should be excluded from scaning. Methods in the form of componentN() that are created by the data class of your Kotlin are also excluded. These are often automatically generated by libraries like Lombok, so we recommend setting them to true to exclude them from collection.
    		•
    asyncCodeBaseScanMode false
  • Scavenger first aggregates the codebase before instrumentation. This causes your application to be delayed from booting by Scavenger. We recommend setting this to true after you have verified that your Scavenger settings are working properly.
    		•
    environment
  • Specifies the application environment values. This can be used if you maintain different codebases for different deployment stages, as they are collected as if they were separate codebases based on that value.
  • ex) dev / alpha / beta / real
  • In the Scavenger UI, data collected in separate environments can be specified multiple times when taking a snapshot to check for dead code all at once.
    		•
    appVersion unspecified
  • Specify the application version. This option does not affect collection. Instead, it is useful if you want to specify which version of the application you are scaning.
  • ex) 1.0.0
    		•
    hostname
  • Specifies that it should be recorded differently from the existing hostname. This option does not affect ingestion, but can be useful when determining which host the agent ran on.
    		•
    debugMode false
  • You can check the logs to see which methods are being traced and whether they are being called. Enabling this mode will have a serious impact on the performance of your application, and we do not recommend using it outside of a development environment.
    		•
    maxMethodsCount 100000
  • Sets a maximum limit on the number of methods that can be tracked.
    		•

    Installation

    • Setting up the agent via -javaagent:path/to/scavenger-agent.jar
    • Specify the config file via -Dscavenger.configuration=path/to/scavenger.conf
    • To change the max message size of gRPC in agent, change the setting below.
    • -Dscavenger.max-message-size: 10MB by default
      • Supports B, KB, MB, GB, and TB units.

    We recommend tuning your codeBase configuration for minimizing tracing ranges before changing scavenger.max-message-size.

    export JAVA_OPTS="$JAVA_OPTS -Dscavenger.configuration=$CATALINA_BASE/conf/scavenger.conf"
export JAVA_OPTS="$JAVA_OPTS -javaagent:/home1/irteam/scavenger-agent.jar"