Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions AsLib/LibSrc/.idea/workspace.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions AsLib/LibSrc/cmake-build-debug/Testing/Temporary/LastTest.log
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
Start testing: Mar 26 11:37 ?D1��������?����??
Start testing: Apr 09 14:04 ?D1��������?����??
----------------------------------------------------------
End testing: Mar 26 11:37 ?D1��������?����??
End testing: Apr 09 14:04 ?D1��������?����??
Binary file modified AsLib/libBeardedManZhao.dll
Binary file not shown.
15 changes: 9 additions & 6 deletions KnowledgeDocument/Operands-Chinese.md
Original file line number Diff line number Diff line change
Expand Up @@ -413,14 +413,16 @@ Table 是AS库中用于进行数据分析的数据对象,其表现形式属于

### DataFrameBuilder 与 DataFrame

DataFrameBuilder 与 DataFrame 分别用于数据的加载与数据的分析操作,在数据的加载过程中可以通过 DataFameBuilder 数据对象快捷同时容易理解的函数构造出一个DataFrame,并使用DataFrame进行数据的处理。
DataFrameBuilder 与 DataFrame 分别用于数据的加载与数据的分析操作,在数据的加载过程中可以通过 DataFameBuilder
数据对象快捷同时容易理解的函数构造出一个DataFrame,并使用DataFrame进行数据的处理。

DataFrame 简称 "DF" 在数据的处理阶段,诸多函数采用SQL风格设计,能够有效降低学习成本,使得精力专注于更重要的事情上,接下来展示下 DataFrameBuilder 的基本使用。

#### 使用 FDataFrame 加载数据
- 读取数据库
在AS库中您可以将数据加载成为 FDataFrame 数据对象,该对象能够实现基本的数据读取与数据处理功能,能够实现有效的数据管控,您可以将数据库中的数据加载成为一个 FDataFrame ,接下来就是有关数据库数据加载的代码示例。
- 需要注意的是,在读取数据库的时候请在项目中导入 JDBC 的驱动类。

- 读取数据库 在AS库中您可以将数据加载成为 FDataFrame 数据对象,该对象能够实现基本的数据读取与数据处理功能,能够实现有效的数据管控,您可以将数据库中的数据加载成为一个 FDataFrame
,接下来就是有关数据库数据加载的代码示例。
- 需要注意的是,在读取数据库的时候请在项目中导入 JDBC 的驱动类。

```java
package zhao.algorithmMagic;
Expand All @@ -447,8 +449,8 @@ public class MAIN1 {
}
}
```
- 读取文件系统
针对文件系统的读取,FDataFrame 是可以轻松做到本地文件系统读取的,不需要依赖任何的第三方库就可以实现文件系统的读取,接下来就实现一下具体的步骤!

- 读取文件系统 针对文件系统的读取,FDataFrame 是可以轻松做到本地文件系统读取的,不需要依赖任何的第三方库就可以实现文件系统的读取,接下来就实现一下具体的步骤!

```java
package zhao.algorithmMagic;
Expand Down Expand Up @@ -481,6 +483,7 @@ public class MAIN1 {
}
}
```

#### 综合案例

```java
Expand Down
26 changes: 18 additions & 8 deletions KnowledgeDocument/Operands.md
Original file line number Diff line number Diff line change
Expand Up @@ -427,19 +427,27 @@ public class MAIN1 {
}
}
```

## Table

Table is the data object used for data analysis in the AS database. Its representation is a table with row and column indexes, which can realize convenient data processing tasks. Data can be loaded and processed in the AS database through the DataFrame object.
Table is the data object used for data analysis in the AS database. Its representation is a table with row and column
indexes, which can realize convenient data processing tasks. Data can be loaded and processed in the AS database through
the DataFrame object.

### DataFrameBuilder & DataFrame

DataFrameBuilder and DataFrame are used for data loading and data analysis respectively. In the process of data loading, a DataFrame can be constructed through DataFameBuilder data object fast and easy to understand functions, and the DataFrame can be used for data processing.
DataFrame is called "DF" for short. In the data processing stage, many functions are designed in SQL style, which can effectively reduce learning costs and focus on more important things. Next, we will show the basic use of DataFrameBuilder.
DataFrameBuilder and DataFrame are used for data loading and data analysis respectively. In the process of data loading,
a DataFrame can be constructed through DataFameBuilder data object fast and easy to understand functions, and the
DataFrame can be used for data processing. DataFrame is called "DF" for short. In the data processing stage, many
functions are designed in SQL style, which can effectively reduce learning costs and focus on more important things.
Next, we will show the basic use of DataFrameBuilder.

#### Load data using FDataFrame
-Read Database
In the AS database, you can load data into an FDataFrame data object, which can realize basic data reading and data processing functions, and effective data control. You can load data in the database into an FDataFrame. Next is the code example about database data loading.
-It should be noted that when reading the database, please import the JDBC driver class in the project.

-Read Database In the AS database, you can load data into an FDataFrame data object, which can realize basic data
reading and data processing functions, and effective data control. You can load data in the database into an FDataFrame.
Next is the code example about database data loading. -It should be noted that when reading the database, please import
the JDBC driver class in the project.

```java
package zhao.algorithmMagic;
Expand All @@ -466,8 +474,9 @@ public class MAIN1 {
}
}
```
-Read file system
For the reading of the file system, FDataFrame can easily read the local file system without relying on any third-party library. Next, we will implement the specific steps!

-Read file system For the reading of the file system, FDataFrame can easily read the local file system without relying
on any third-party library. Next, we will implement the specific steps!

```java
package zhao.algorithmMagic;
Expand Down Expand Up @@ -500,6 +509,7 @@ public class MAIN1 {
}
}
```

#### Comprehensive case

```java
Expand Down
22 changes: 21 additions & 1 deletion README-Chinese.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,16 +18,19 @@
<dependency>
<groupId>io.github.BeardedManZhao</groupId>
<artifactId>algorithmStar</artifactId>
<version>1.17</version>
<version>1.18</version>
</dependency>
</dependencies>
```

### AS库的所需依赖

在1.17版本之后,AS库的所有依赖被剥离,更好避免依赖的捆绑问题,减少项目发生冲突的可能性,同时也可以按照开发者的需求使用更加适合的依赖配置项,您可以在这里查看到AS库所依赖的第三方库依赖。

#### 必选依赖项

AS库在进行诸多计算函数的时候会产生一些日志数据,因此AS库的使用需要导入日志依赖项,这个依赖项是必不可少的,请按照如下的方式导入依赖。

```xml
<dependencies>
<!-- 使用 log4j2 的适配器进行绑定 -->
Expand All @@ -54,9 +57,11 @@ AS库在进行诸多计算函数的时候会产生一些日志数据,因此AS
</dependency>
</dependencies>
```

#### 可选依赖项

AS库在针对数据库,Spark等各种平台对接的时候,需要使用到第三方依赖程序包,这些包是可选的,如果您不需要使用这些功能,您可以不去导入依赖,如果您需要,可以参考下面的配置。

```xml
<dependencies>
<!-- MySQL数据库连接驱动 如果您需要连接的关系型数据库是其它类型,这里也可以随之修改 -->
Expand All @@ -82,6 +87,21 @@ AS库在针对数据库,Spark等各种平台对接的时候,需要使用到
<artifactId>spark-mllib_2.12</artifactId>
<version>3.1.3</version>
</dependency>

<!-- 摄像头依赖库,如果您有需要通过摄像头获取数据对象的需求,可以引入本库 -->
<dependency>
<groupId>com.github.sarxos</groupId>
<artifactId>webcam-capture</artifactId>
<version>0.3.12</version>
</dependency>

<!-- HDFS 输入输出设备依赖库,如果您有需要通过HDFS分布式存储平台进行数据读写的需求,可以引入本库 -->
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client</artifactId>
<version>3.3.1</version>
</dependency>

</dependencies>
```

Expand Down
41 changes: 34 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,18 +21,25 @@ can add it to your maven project, or you can download it from Releases and manua
<dependency>
<groupId>io.github.BeardedManZhao</groupId>
<artifactId>algorithmStar</artifactId>
<version>1.17</version>
<version>1.18</version>
</dependency>
</dependencies>
```

### Required dependencies of the AS library

After version 1.17, all dependencies of the AS library have been stripped to better avoid binding dependencies and reduce the possibility of project conflicts. At the same time, more suitable dependency configuration items can be used according to the needs of developers. You can view third-party library dependencies on which the AS library depends here.
After version 1.17, all dependencies of the AS library have been stripped to better avoid binding dependencies and
reduce the possibility of project conflicts. At the same time, more suitable dependency configuration items can be used
according to the needs of developers. You can view third-party library dependencies on which the AS library depends
here.

#### Required Dependencies
The AS library generates some log data when performing many calculation functions. Therefore, the use of the AS library requires importing log dependencies, which are essential. Please import the dependencies as follows.

The AS library generates some log data when performing many calculation functions. Therefore, the use of the AS library
requires importing log dependencies, which are essential. Please import the dependencies as follows.

```xml

<dependencies>
<!-- Binding using the adapter of log4j2 -->
<dependency>
Expand All @@ -58,18 +65,23 @@ The AS library generates some log data when performing many calculation function
</dependency>
</dependencies>
```

#### Optional Dependencies

When interfacing with various platforms such as databases and Sparks, the AS library needs to use third-party dependency packages, which are optional. If you do not need to use these functions, you may not need to import dependencies. If you need to, you can refer to the following configuration.
When interfacing with various platforms such as databases and Sparks, the AS library needs to use third-party dependency
packages, which are optional. If you do not need to use these functions, you may not need to import dependencies. If you
need to, you can refer to the following configuration.

```xml

<dependencies>
<!-- MySQL database connection driver If the relational database you want to connect to is of another type, you can also modify it here -->
<!-- MySQL database connection driver If the relational database you want to connect to is of another type, you can also modify it here -->
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>8.0.30</version>
</dependency>
<!-- The dependency development package for the three major Spark modules can also be imported if you need to use it here, or not if you don't need it -->
<!-- The dependency development package for the three major Spark modules can also be imported if you need to use it here, or not if you don't need it -->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.12</artifactId>
Expand All @@ -80,12 +92,27 @@ When interfacing with various platforms such as databases and Sparks, the AS lib
<artifactId>spark-sql_2.12</artifactId>
<version>3.1.3</version>
</dependency>

<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-mllib_2.12</artifactId>
<version>3.1.3</version>
</dependency>

<!-- Camera device dependency library. If you have a need to obtain data objects through the camera, you can import this dependency. -->
<dependency>
<groupId>com.github.sarxos</groupId>
<artifactId>webcam-capture</artifactId>
<version>0.3.12</version>
</dependency>

<!-- HDFS input/output device dependency library. If you have a need for data reading and writing through the HDFS distributed storage platform, you can introduce this library. -->
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client</artifactId>
<version>3.3.1</version>
</dependency>

</dependencies>
```

Expand Down
Loading