Skip to content

Commit

Permalink
Support running HA JobMaster process in intellij
Browse files Browse the repository at this point in the history
### What changes are proposed in this pull request?
Add tools for intellij to start local HA JobMaster process., this pr is
a Is a supplement to pr #14042
### Why are the changes needed?
It is convenient to debug and test job master in intellij.
Please clarify why the changes are needed. For instance,
1. Add AlluxioJobMaster xml file to dev/intellij directory
2. Update the doc.
### Does this PR introduce any user facing changes?
No
Please list the user-facing changes introduced by your change, including
No

pr-link: #14120
change-id: cid-e55be3c958b813c2310de6e509d867a0018a4b21
  • Loading branch information
bzheng888 committed Sep 22, 2021
1 parent 3a5bd25 commit 0a5669c
Show file tree
Hide file tree
Showing 4 changed files with 98 additions and 0 deletions.
28 changes: 28 additions & 0 deletions dev/intellij/runConfigurations/AlluxioJobMaster_0.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
<!--
The Alluxio Open Foundation licenses this work under the Apache License, version 2.0
(the "License"). You may not use this work except in compliance with the License, which is
available at www.apache.org/licenses/LICENSE-2.0
This software is distributed on an "AS IS" basis, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
either express or implied, as more fully set forth in the License.
See the NOTICE file distributed with this work for information regarding copyright ownership.
-->
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="AlluxioJobMaster-0" type="Application" factoryName="Application">
<option name="MAIN_CLASS_NAME" value="alluxio.master.AlluxioJobMaster" />
<module name="alluxio-job-server" />
<option name="VM_PARAMETERS" value="-Dalluxio.job.master.rpc.port=20001 -Dalluxio.job.master.web.port=20002 -Dalluxio.job.master.embedded.journal.port=20003 -Dalluxio.work.dir=/tmp/alluxio-tmp/alluxio-0 -Xbootclasspath/a:$PROJECT_DIR$/conf -Dalluxio.logger.type=Console" />
<extension name="coverage">
<pattern>
<option name="PATTERN" value="alluxio.master.*" />
<option name="ENABLED" value="true" />
</pattern>
</extension>
<method v="2">
<option name="Make" enabled="true" />
</method>
</configuration>
</component>
28 changes: 28 additions & 0 deletions dev/intellij/runConfigurations/AlluxioJobMaster_1.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
<!--
The Alluxio Open Foundation licenses this work under the Apache License, version 2.0
(the "License"). You may not use this work except in compliance with the License, which is
available at www.apache.org/licenses/LICENSE-2.0
This software is distributed on an "AS IS" basis, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
either express or implied, as more fully set forth in the License.
See the NOTICE file distributed with this work for information regarding copyright ownership.
-->
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="AlluxioJobMaster-1" type="Application" factoryName="Application">
<option name="MAIN_CLASS_NAME" value="alluxio.master.AlluxioJobMaster" />
<module name="alluxio-job-server" />
<option name="VM_PARAMETERS" value="-Dalluxio.job.master.rpc.port=20011 -Dalluxio.job.master.web.port=20012 -Dalluxio.job.master.embedded.journal.port=20013 -Dalluxio.work.dir=/tmp/alluxio-tmp/alluxio-1 -Xbootclasspath/a:$PROJECT_DIR$/conf -Dalluxio.logger.type=Console " />
<extension name="coverage">
<pattern>
<option name="PATTERN" value="alluxio.master.*" />
<option name="ENABLED" value="true" />
</pattern>
</extension>
<method v="2">
<option name="Make" enabled="true" />
</method>
</configuration>
</component>
28 changes: 28 additions & 0 deletions dev/intellij/runConfigurations/AlluxioJobMaster_2.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
<!--
The Alluxio Open Foundation licenses this work under the Apache License, version 2.0
(the "License"). You may not use this work except in compliance with the License, which is
available at www.apache.org/licenses/LICENSE-2.0
This software is distributed on an "AS IS" basis, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
either express or implied, as more fully set forth in the License.
See the NOTICE file distributed with this work for information regarding copyright ownership.
-->
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="AlluxioJobMaster-2" type="Application" factoryName="Application">
<option name="MAIN_CLASS_NAME" value="alluxio.master.AlluxioJobMaster" />
<module name="alluxio-job-server" />
<option name="VM_PARAMETERS" value="-Dalluxio.job.master.rpc.port=20021 -Dalluxio.job.master.web.port=20022 -Dalluxio.job.master.embedded.journal.port=20023 -Dalluxio.work.dir=/tmp/alluxio-tmp/alluxio-2 -Xbootclasspath/a:$PROJECT_DIR$/conf -Dalluxio.logger.type=Console" />
<extension name="coverage">
<pattern>
<option name="PATTERN" value="alluxio.master.*" />
<option name="ENABLED" value="true" />
</pattern>
</extension>
<method v="2">
<option name="Make" enabled="true" />
</method>
</configuration>
</component>
14 changes: 14 additions & 0 deletions docs/en/contributor/Contributor-Tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,20 @@ AVAILABLE | 0 | localhost:19200
AVAILABLE | 0 | localhost:19201
AVAILABLE | 0 | localhost:19202
```
In addition, you can also start a High Availability (HA) JobMaster process on this basis.

11. Edit `conf/alluxio-site.properties` and add lines
```console
alluxio.job.master.rpc.addresses=localhost:20001,localhost:20011,localhost:20021
alluxio.job.master.embedded.journal.addresses=localhost:20003,localhost:20013,localhost:20023
```
12. In Intellij, start Alluxio master 0 process by selecting `Run > Run > AlluxioJobMaster-0`
13. In Intellij, start Alluxio master 1 process by selecting `Run > Run > AlluxioJobMaster-1`
14. In Intellij, start Alluxio master 2 process by selecting `Run > Run > AlluxioJobMaster-2`
15. Verify the HA JobMaster process is up, Run`bin/alluxio job leader`, and you will get message like this:
```console
localhost
```

## Maven Targets and Plugins

Expand Down

0 comments on commit 0a5669c

Please sign in to comment.