Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1 +1,3 @@
/.idea/
.idea
.DS_Store

309 changes: 309 additions & 0 deletions website/blog/2022-12-18-Introduction_of_Metadata/index.md

Large diffs are not rendered by default.

5 changes: 5 additions & 0 deletions website/blog/authors.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,8 @@ xxyu:
title: PMC member and Release Manager of Apache Kylin
url: https://github.com/hit-lacus
image_url: https://github.com/hit-lacus.png

pfzhan:
name: pfzhan
url: https://github.com/dethrive
image_url: https://github.com/dethrive.png
16 changes: 8 additions & 8 deletions website/docs/development/coding_convention.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,15 +16,15 @@ last_update:
author: Tengting Xu
---

Coding convention is very important for teamwork. Not only it keeps code neat and tidy, it saves a lot of work too. Different coding convention (and auto formatter) will cause unnecessary code changes that requires more effort at code review and code merge.
Coding convention is very important for teamwork. Not only it keeps code neat and tidy, but it also saves a lot of work too. Different coding convention (and auto formatter) will cause unnecessary code changes that require more effort at code review and code merge.

## Setup IDE code formatter

For Java code, we use Eclipse default formatter setting, with one change that to allow long lines.
For Java code, we use Eclipse's default formatter setting, with one change that allows long lines.

- For Eclipse developers, no manual setting is required. Code formatter configurations `src/core-common/.settings/org.eclipse.jdt.core.prefs` is on git repo. Your IDE should be auto configured when the projects are imported.
- For Eclipse developers, no manual setting is required. Code formatter configurations `src/core-common/.settings/org.eclipse.jdt.core.prefs` are on the git repo. Your IDE should be auto-configured when the projects are imported.

- For intellij IDEA developers, you need to install `Eclipse Code Formatter` and load the Eclipse formatter settings into your IDE manually.
- For IntelliJ IDEA developers, you need to install `Eclipse Code Formatter` and load the Eclipse formatter settings into your IDE manually.

you have to do a few more steps:

Expand All @@ -36,7 +36,7 @@ For Java code, we use Eclipse default formatter setting, with one change that to

![](images/coding_convention/coding_convention_2.png)

3. Disable intellij IDEA’s `Optimize imports on the fly`
3. Disable IntelliJ IDEA’s `Optimize imports on the fly`

![](images/coding_convention/coding_convention_3.png)

Expand Down Expand Up @@ -64,7 +64,7 @@ See the License for the specific language governing permissions and
limitations under the License.
```

The checkstyle plugin will check the header rule when packaging also. The license file locates under `dev-support/checkstyle-apache-header.txt`. To make it easy for developers, please add the header as Copyright Profile and set it as default for Kylin project.
The checkstyle plugin will check the header rule when packaging also. The license file locates under `dev-support/checkstyle-apache-header.txt`. To make it easy for developers, please add the header as Copyright Profile and set it as default for the Kylin project.

![](images/coding_convention/coding_convention_4.png)

Expand All @@ -80,8 +80,8 @@ The checkstyle plugin will check the header rule when packaging also. The licens

5. Add a new test or modified a test.

1) Please using the `junit5` instead of `junit4`. Example, Using the annotation of `org.junit.jupiter.api.Test` instead of `org.junit.Test`.
1) Please use the `junit5` instead of `junit4`. For example, Using the annotation of `org.junit.jupiter.api.Test` instead of `org.junit.Test`.

2) A test case which extends from `NLocalFileMetadataTestCase` need to change with annotation `@MetadataInfo` and remove the `extend`.
2) A test case that extends from `NLocalFileMetadataTestCase` needs to change with annotation `@MetadataInfo` and remove the `extend`.

> Example: org.apache.kylin.junit.MetadataExtension, org.apache.kylin.metadata.epoch.EpochManagerTest
30 changes: 15 additions & 15 deletions website/docs/development/how_to_debug_kylin_in_ide.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,55 +18,55 @@ last_update:

### Background
#### Why debug Kylin in IDEA using docker
This article aims to introduce a simple and useful way to develop and debug Kylin for developer, and provided similar deployment to user's real scenario.
This article aims to introduce a simple and useful way to develop and debug Kylin for developers and provided a similar deployment to the user's real scenario.

#### Deployment architecture
Following is architecture of current deployment.
Following is the architecture of the current deployment.

![debug_in_laptop](images/debug_kylin_by_docker_compose.png)

This guide **assumes** you have prepared the following things:

- [ ] A **laptop** with MacOS installed to do development work (Windows is not verified at the moment)
- [ ] A **remote linux server** for testing and deployment purpose(if you do not prepare remote linux server, you will deploy Hadoop on your laptop)
- [ ] kylin's source code is cloned into some directory in your laptop
- [ ] A **laptop** with macOS installed to do development work (Windows is not verified at the moment)
- [ ] A **remote Linux server** for testing and deployment purposes (if you do not prepare a remote Linux server, you will deploy Hadoop on your laptop)
- [ ] Kaylin's source code is cloned into some directory on your laptop

:::info For Windows Dev Machine
For Windows dev machine, setup the Kylin dev env in [Windows Subsystem for Linux](https://learn.microsoft.com/en-us/windows/wsl/about) is the best option. Follow this guide on [how to install WSL with GUI](https://learn.microsoft.com/en-us/windows/wsl/tutorials/gui-apps), and install both Kylin code and your favorite IDE (but not the docker) in WSL for best performance.
For Windows dev machine, setup the Kylin dev env in [Windows Subsystem for Linux](https://learn.microsoft.com/en-us/windows/wsl/about) is the best option. Follow this guide on [how to install WSL with GUI](https://learn.microsoft.com/en-us/windows/wsl/tutorials/gui-apps), and install both the Kylin code and your favorite IDE (but not the docker) in WSL for best performance.
:::

### Prepare IDEA and build source code

#### Step 1: Check Software Requirement

Please visit [software_requirement](how_to_package#software_reqiurement), make sure your laptop has meet the requirement.
Please visit [software_requirement](how_to_package#software_reqiurement), and make sure your laptop has met the requirement.

#### Step 2: Build source code
- Build back-end source code before your start debug.
- Build backend source code before your start debugging.
```shell
cd <path-to-kylin-source>
mvn clean install -DskipTests
```

- Build front-end source code.
(Please use node.js **v12.14.0**, for how to use specific version of node.js, please check [how to switch to specific node js](how_to_package#install_older_node) )
(Please use node.js **v12.14.0**, for how to use a specific version of node.js, please check [how to switch to a specific node js](how_to_package#install_older_node) )
```shell
cd kystudio
npm install
```

#### Step 3: Install IntelliJ IDEA and build the source
1. Install IDEA Community edition (Ultimate edition is ok too).
2. Import the source code into IDEA. Click the **Open**, and choose the directory of **kylin source code**.
![](images/OPEN_KYLIN_PROJECT.png)
1. Install the IDEA Community edition (the Ultimate edition is ok too).
2. Import the source code into IDEA. Click the **Open**, and choose the directory of **Kylin source code**.
![](images/OPEN_KYLIN_PROJECT.png)

3. Install scala plugin and restart
3. Install the scala plugin and restart
![](images/IDEA_Install_Scala_plugin.png)

4. Configure SDK(JDK and Scala), make sure you use **JDK 1.8.X** and **Scala 2.12.X**.
4. Configure SDK(JDK and Scala), and make sure you use **JDK 1.8.X** and **Scala 2.12.X**.
![](images/IDEA_Notify_Install_SDK.png)

5. Reload maven projects, and directory `scala` will be marked as source root(in blue color).
5. Reload maven projects, and the `scala` directory will be marked as source root(in blue color).
![](images/IDEA_RELOAD_ALL_MAVEN_PROJECT.png)

6. Build the projects.(make sure you have executed `mvn clean package -DskipTests`, otherwise some source code is not generated by maven javacc plugin)
Expand Down
24 changes: 12 additions & 12 deletions website/docs/development/how_to_debug_kylin_in_local.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,32 +19,32 @@ last_update:
### Background

#### Why debug Kylin in IDEA without Hadoop
This article aims to introduce a simple and useful way to develop and debug Kylin for developer.
This article aims to introduce a simple and useful way to develop and debug Kylin for developers.

#### Deployment architecture

Following is architecture of current deployment.
Following is the architecture of the current deployment.

![debug_in_laptop](./images/how_to_debug_kylin_in_local/laptop.png)

This guide **assumes** you have prepared the following things:

- [X] A **laptop** with MacOS installed to do development work (Windows is not verified at the moment)
- [X] kylin's source code is cloned into some directory in your laptop
- [X] A **laptop** with macOS installed to do development work (Windows is not verified at the moment)
- [X] Kylin's source code is cloned into some directory on your laptop

:::info For Windows Dev Machine
For Windows dev machine, setup the Kylin dev env in [Windows Subsystem for Linux](https://learn.microsoft.com/en-us/windows/wsl/about) is the best option. Follow this guide on [how to install WSL with GUI](https://learn.microsoft.com/en-us/windows/wsl/tutorials/gui-apps), and install both Kylin code and your favorite IDE (but not the docker) in WSL for best performance.
For Windows dev machine, setup the Kylin dev env in [Windows Subsystem for Linux](https://learn.microsoft.com/en-us/windows/wsl/about) is the best option. Follow this guide on [how to install WSL with GUI](https://learn.microsoft.com/en-us/windows/wsl/tutorials/gui-apps), and install both the Kylin code and your favorite IDE (but not the docker) in WSL for best performance.
:::

### Prepare IDEA and build source code

#### Step 1: Check Software Requirement

Please visit [Software Requirement](how_to_package#software_reqiurement), make sure your laptop has meet the requirement.
Please visit [Software Requirement](how_to_package#software_reqiurement), and make sure your laptop has met the requirement.

#### Step 2: Build source code

- Build back-end source code before your start debug.
- Build backend source code before your start debugging.

```shell
cd <path-to-kylin-source>
Expand All @@ -53,26 +53,26 @@ Please visit [Software Requirement](how_to_package#software_reqiurement), make s

- Build front-end source code.

(Please use node.js **v12.14.0**, for how to use specific version of node.js, please check [how to switch to specific node js](how_to_package#install_older_node) )
(Please use node.js **v12.14.0**, for how to use a specific version of node.js, please check [how to switch to a specific node js](how_to_package#install_older_node) )

```shell
cd kystudio
npm install
```
#### Step 3: Install IntelliJ IDEA and build the source

1. Install IDEA Community edition (Ultimate edition is ok too).
1. Install the IDEA Community edition (the Ultimate edition is ok too).

2. Import the source code into IDEA. Click the **Open**, and choose the directory of **kylin source code**.
2. Import the source code into IDEA. Click the **Open**, and choose the directory of **Kylin source code**.
![](images/how_to_debug_kylin_in_local/OPEN_KYLIN_PROJECT.png)

3. Install scala plugin and restart
3. Install the scala plugin and restart
![](images/how_to_debug_kylin_in_local/IDEA_Install_Scala_plugin.png)

4. Configure SDK(JDK and Scala), make sure you use **JDK 1.8.X** and **Scala 2.12.X**.
![](images/how_to_debug_kylin_in_local/IDEA_Notify_Install_SDK.png)

5. Reload maven projects, and directory `scala` will be marked as source root(in blue color).
5. Reload maven projects, and the `scala` directory will be marked as source root(in blue color).
![](images/how_to_debug_kylin_in_local/IDEA_RELOAD_ALL_MAVEN_PROJECT.png)

6. Build the projects.(make sure you have executed `mvn clean package -DskipTests`, otherwise some source code is not generated by maven javacc plugin)
Expand Down
38 changes: 19 additions & 19 deletions website/docs/development/how_to_package.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,11 @@ last_update:
| Software | Comment | Version | Download Link |
|---------------| ---------------------------------------------|----------------|--------------------|
| Git | Fetch branch name and hash of latest commit | latest | https://git-scm.com/book/en/v2/Getting-Started-Installing-Git |
| Apache Maven | Build Java and Scala source code | 3.8.2 or latest| https://maven.apache.org/download.cgi |
| Apache Maven | Build Java and Scala source code | 3.8.2 or latest| https://maven.apache.org/download.cgi |
| Node.js | Build front end | 12.14.0 is recommended ( or 12.x ~ 14.x) | [How to switch to older node.js](development/how_to_package.md#install_older_node)|
| JDK | Java Compiler and Development Tools | JDK 1.8.x | https://www.oracle.com/java/technologies/javase/javase8u211-later-archive-downloads.html |

After installed above software, please do verify **software requirement** by following commands:
After installing the above software, please verify **software requirements** by following commands:

```shell
$ java -version
Expand All @@ -49,14 +49,14 @@ git version 2.30.1 (Apple Git-130)
```
### Options for Packaging Script

| Option | Comment |
|-------------------- | ---------------------------------------------------|
| -official | If add this option, package name won't contain timestamp|
| -noThirdParty | If add this option, third party binary won't be packaging into binary, current they are influxdb,grafana and postgresql |
| -noSpark | If add this option, spark won't packaging into Kylin binary |
| -noHive1 | By default kylin 5.0 will support Hive 1.2, if add this option, this binary will support Hive 2.3+ |
| -skipFront | If add this option, front-end won't be build and packaging |
| -skipCompile | Add this option will assume java source code no need be compiled again |
| Option | Comment |
| ------------- | ------------------------------------------------------------ |
| -official | If adding this option, the package name won't contain the timestamp |
| -noThirdParty | If adding this option, third-party binary won't be packaged into binary, current they are influxdb,grafana and PostgreSQL |
| -noSpark | If adding this option, spark won't be packaged into the Kylin binary |
| -noHive1 | By default Kylin 5.0 will support Hive 1.2, if add this option, this binary will support Hive 2.3+ |
| -skipFront | If add this option, the front-end won't be built and packaged |
| -skipCompile | Add this option will assume java source code no need to be compiled again |

### Other Options for Packaging Script
| Option | Comment |
Expand All @@ -79,11 +79,11 @@ For example, an unofficial package could be `apache-kylin-5.0.0-SNAPSHOT.2022081

```shell

## Case 1: For developer who want to package for testing purpose
## Case 1: For the developer who wants to package for testing purposes
./build/release/release.sh

## Case 2: Official apache release, kylin binary for deploy on Hadoop3+ and Hive2.3+,
# and third party cannot be distributed because of apache distribution policy(size and license)
## Case 2: Official apache release, Kylin binary for deployment on Hadoop3+ and Hive2.3+,
# and the third party cannot be distributed because of apache distribution policy(size and license)
./build/release/release.sh -noSpark -official

## Case 3: A package for Apache Hadoop 3 platform
Expand All @@ -92,31 +92,31 @@ For example, an unofficial package could be `apache-kylin-5.0.0-SNAPSHOT.2022081

### <span id="install_older_node">How to install older node.js</span>

1. Please visit https://nodejs.org/en/download/ to download and install the latest node.js . After installed, you may use follow command to verify if the latest node.js is in use:
1. Please visit https://nodejs.org/en/download/ to download and install the latest node.js. After installed, you may use the following command to verify if the latest node.js is in use:
```shell
$ node -v
v16.17.0
```

2. Use some tools like https://github.com/nvm-sh/nvm to install specific version of node.js
2. Use some tools like https://github.com/nvm-sh/nvm to install a specific version of node.js

```shell
## Switch to specific version using nvm
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash
nvm install 12.14.0

## Before packaging, please switch to specific version
## Before packaging, please switch to a specific version
nvm use 12.14.0
```

You may use follow command to verify if older node.js is in use:
You may use the following command to verify if older node.js is in use:
```shell
$ node -v
v12.14.0
```

3. Switch to latest node.js
3. Switch to the latest node.js
```shell
## switch to original version
## switch to the original version
nvm use system
```
14 changes: 7 additions & 7 deletions website/docs/development/how_to_subscribe_mailing_list.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ last_update:
---

### Mailing List Table
These are the mailing lists that have been established for kylin project. For each list, there is a subscribe, unsubscribe, and an archive link.
These are the mailing lists that have been established for the Kylin project. For each list, there is a subscribe, unsubscribe, and archive link.

| Mailing List | Subscribe Link | Unsubscribe Link | Archive Link |
|:------------:|:----------------------------------------------------:|:--------------------------------------------------------:|:-------------------------------------------------------------------------:|
Expand All @@ -25,23 +25,23 @@ These are the mailing lists that have been established for kylin project. For ea

### <span id="mailing_list">Subscribe mailing list</span>

#### Step 1: Send subscribe request
You click the _Subscribe Link_ in above table, and send the simple request(it is ok to leave the title and body with a very short sentence) to the appropriate address .
#### Step 1: Send subscription request
You click the _Subscribe Link_ in the above table and send the simple request(it is ok to leave the title and body with a very short sentence) to the appropriate address.
![](images/subscribe_mailing_list_1.jpg)

#### Step 2: Receive confirmation reply from ezmlm
The mailing list management program, [ezmlm](http://untroubled.org/ezmlm/), will send you a reply in 2-10 minutes, ask you to **confirm your subscription**.
The mailing list management program, [ezmlm](http://untroubled.org/ezmlm/), will send you a reply in 2-10 minutes, asking you to **confirm your subscription**.

Following is a successful case.

![](images/subscribe_mailing_list_2.jpg)

#### Step 3: Send confirmation request by replying to previous email
#### Step 3: Send a confirmation request by replying to the previous email
Reply to the previous email(it is ok to leave the title and body with a short sentence)
![](images/subscribe_mailing_list_3.jpg)

#### Step 4: ezmlm acknowledge your confirmation request
You will receive "Welcome to user/dev@kylin.apache.org" in 2-10 minutes. From now, you have right to send and receive mails from all subscribers of current mailing list.
#### Step 4: ezmlm acknowledges your confirmation request
You will receive "Welcome to user/dev@kylin.apache.org" in 2-10 minutes. From now, you have the right to send and receive mail from all subscribers of the current mailing list.

Following is a successful case.

Expand Down
Loading