-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[Remote table driver implementation and leveraging it in HTTP service]
[#59][switched internal communication to gRPC]
- Loading branch information
Showing
73 changed files
with
2,433 additions
and
1,265 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| Settting up environment | ||
| ======= | ||
|
|
||
| The project is developed using Intelliji Idea. Community Edition should work fine. All further steps assume you have it already installed. | ||
|
|
||
| Preliminary steps | ||
| ---- | ||
|
|
||
| * Make sure you have Java 8 installed (the JDK version is as an example, Oracle JDK will do either): | ||
|
|
||
| ```bash | ||
| > java -version | ||
| openjdk version "1.8.0_252" | ||
| OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_252-b09) | ||
| OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.252-b09, mixed mode) | ||
| ``` | ||
|
|
||
| * Clone repository | ||
|
|
||
| ```bash | ||
| git clone git@github.com:WaveBeans/wavebeans.git | ||
| ``` | ||
|
|
||
| * Make your first build and wait test to pass. No special requirements here. | ||
|
|
||
| ```bash | ||
| ./gradlew build | ||
| ``` | ||
|
|
||
| Install plugins | ||
| ---- | ||
|
|
||
| A few extra plugins you need to make sure you have installed. | ||
|
|
||
| * To run tests using IDE: [Spek Framework plugin](https://plugins.jetbrains.com/plugin/10915-spek-framework). | ||
| * For protobuf support: [Protobuf support plugin](https://plugins.jetbrains.com/plugin/8277-protobuf-support) | ||
|
|
||
| Import project | ||
| ---- | ||
|
|
||
| To import the project just open via `File > Open` in IDE the `build.gradle.kts` file in the root directory of the project, select `Open as project`. Select to use gradle wrapper. | ||
|
|
||
| Setting up build configurations | ||
| ---- | ||
|
|
||
| A few basic configurations are handy to create. Especially for running tests. As by the time of writing this there was no way found to run all tests at once, as well as using Spek plugin. So you need to cre JUnit runner and repeat steps for all projects (`lib`, `exe`, `cli`, `http`). | ||
|
|
||
| * Create configuration, then select `JUnit`. | ||
| * Name configuration, for example `LIB tests`. | ||
| * Select `Test kind: All in package`. | ||
| * Select `Search for tests: In single module`. | ||
| * Select `Use classpath or module: io.wavebeans.lib.test` (or corresponding to the project). | ||
| * Add additional VM option `-DSPEK_TIMEOUT=0` as some tests are taking more than default 10 seconds timeout and failing weirdly. | ||
|
|
||
| While working on Protobuf stuff | ||
| ---- | ||
|
|
||
| IDE doesn't currently build automatically proto-files, but all source folders are configured correctly. Every time you change them (or cleaned the project) regenerate them by running gradle command: | ||
|
|
||
| ```bash | ||
| ./gradlew clean generateProto | ||
| ``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
48 changes: 48 additions & 0 deletions
48
exe/src/main/kotlin/io/wavebeans/execution/SerializationUtils.kt
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,48 @@ | ||
| package io.wavebeans.execution | ||
|
|
||
| import io.wavebeans.lib.BeanParams | ||
| import io.wavebeans.lib.NoParams | ||
| import io.wavebeans.lib.io.* | ||
| import io.wavebeans.lib.stream.* | ||
| import io.wavebeans.lib.stream.fft.FftStreamParams | ||
| import io.wavebeans.lib.stream.window.WindowStreamParams | ||
| import io.wavebeans.lib.stream.window.WindowStreamParamsSerializer | ||
| import io.wavebeans.lib.table.* | ||
| import kotlinx.serialization.json.Json | ||
| import kotlinx.serialization.json.JsonConfiguration | ||
| import kotlinx.serialization.modules.EmptyModule | ||
| import kotlinx.serialization.modules.SerialModule | ||
| import kotlinx.serialization.modules.SerializersModuleBuilder | ||
|
|
||
| fun jsonCompact(paramsModule: SerialModule? = null) = Json(context = paramsModule ?: EmptyModule) | ||
| fun jsonPretty(paramsModule: SerialModule? = null) = Json(context = paramsModule | ||
| ?: EmptyModule, configuration = JsonConfiguration.Stable.copy(prettyPrint = true)) | ||
|
|
||
| fun SerializersModuleBuilder.tableQuery() { | ||
| polymorphic(TableQuery::class) { | ||
| TimeRangeTableQuery::class with TimeRangeTableQuery.serializer() | ||
| LastIntervalTableQuery::class with LastIntervalTableQuery.serializer() | ||
| ContinuousReadTableQuery::class with ContinuousReadTableQuery.serializer() | ||
| } | ||
| } | ||
|
|
||
| fun SerializersModuleBuilder.beanParams() { | ||
| polymorphic(BeanParams::class) { | ||
| ChangeAmplitudeSampleStreamParams::class with ChangeAmplitudeSampleStreamParams.serializer() | ||
| SineGeneratedInputParams::class with SineGeneratedInputParams.serializer() | ||
| NoParams::class with NoParams.serializer() | ||
| TrimmedFiniteSampleStreamParams::class with TrimmedFiniteSampleStreamParams.serializer() | ||
| CsvStreamOutputParams::class with CsvWindowStreamOutputParamsSerializer | ||
| BeanGroupParams::class with BeanGroupParams.serializer() | ||
| CsvFftStreamOutputParams::class with CsvFftStreamOutputParams.serializer() | ||
| FftStreamParams::class with FftStreamParams.serializer() | ||
| WindowStreamParams::class with WindowStreamParamsSerializer | ||
| ProjectionBeanStreamParams::class with ProjectionBeanStreamParams.serializer() | ||
| MapStreamParams::class with MapStreamParamsSerializer | ||
| InputParams::class with InputParamsSerializer | ||
| FunctionMergedStreamParams::class with FunctionMergedStreamParamsSerializer | ||
| ListAsInputParams::class with ListAsInputParamsSerializer | ||
| TableOutputParams::class with TableOutputParamsSerializer | ||
| TableDriverStreamParams::class with TableDriverStreamParams.serializer() | ||
| } | ||
| } |
18 changes: 18 additions & 0 deletions
18
exe/src/main/kotlin/io/wavebeans/execution/TableQuerySerializer.kt
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| package io.wavebeans.execution | ||
|
|
||
| import io.wavebeans.lib.table.TableQuery | ||
| import kotlinx.serialization.PolymorphicSerializer | ||
| import kotlinx.serialization.json.Json | ||
| import kotlinx.serialization.modules.SerializersModule | ||
|
|
||
| object TableQuerySerializer { | ||
| val paramsModule = SerializersModule { | ||
| tableQuery() | ||
| } | ||
|
|
||
| private val json = jsonCompact(paramsModule) | ||
|
|
||
| fun deserialize(query: String): TableQuery = json.parse(PolymorphicSerializer(TableQuery::class), query) | ||
|
|
||
| fun serialize(query: TableQuery, json: Json = this.json): String = json.stringify(PolymorphicSerializer(TableQuery::class), query) | ||
| } |
42 changes: 5 additions & 37 deletions
42
exe/src/main/kotlin/io/wavebeans/execution/TopologySerializer.kt
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,50 +1,18 @@ | ||
| package io.wavebeans.execution | ||
|
|
||
| import io.wavebeans.execution.distributed.AnySerializer | ||
| import kotlinx.serialization.json.Json | ||
| import kotlinx.serialization.json.JsonConfiguration | ||
| import kotlinx.serialization.modules.SerializersModule | ||
| import io.wavebeans.lib.BeanParams | ||
| import io.wavebeans.lib.NoParams | ||
| import io.wavebeans.lib.io.* | ||
| import io.wavebeans.lib.stream.* | ||
| import io.wavebeans.lib.stream.fft.FftStreamParams | ||
| import io.wavebeans.lib.stream.window.WindowStreamParams | ||
| import io.wavebeans.lib.stream.window.WindowStreamParamsSerializer | ||
| import io.wavebeans.lib.table.* | ||
|
|
||
| object TopologySerializer { | ||
|
|
||
| val paramsModule = SerializersModule { | ||
| polymorphic(BeanParams::class) { | ||
| ChangeAmplitudeSampleStreamParams::class with ChangeAmplitudeSampleStreamParams.serializer() | ||
| SineGeneratedInputParams::class with SineGeneratedInputParams.serializer() | ||
| NoParams::class with NoParams.serializer() | ||
| TrimmedFiniteSampleStreamParams::class with TrimmedFiniteSampleStreamParams.serializer() | ||
| CsvStreamOutputParams::class with CsvWindowStreamOutputParamsSerializer | ||
| BeanGroupParams::class with BeanGroupParams.serializer() | ||
| CsvFftStreamOutputParams::class with CsvFftStreamOutputParams.serializer() | ||
| FftStreamParams::class with FftStreamParams.serializer() | ||
| WindowStreamParams::class with WindowStreamParamsSerializer | ||
| ProjectionBeanStreamParams::class with ProjectionBeanStreamParams.serializer() | ||
| MapStreamParams::class with MapStreamParamsSerializer | ||
| InputParams::class with InputParamsSerializer | ||
| FunctionMergedStreamParams::class with FunctionMergedStreamParamsSerializer | ||
| ListAsInputParams::class with ListAsInputParamsSerializer | ||
| TableOutputParams::class with TableOutputParamsSerializer | ||
| TableDriverStreamParams::class with TableDriverStreamParams.serializer() | ||
| } | ||
| polymorphic(TableQuery::class) { | ||
| TimeRangeTableQuery::class with TimeRangeTableQuery.serializer() | ||
| LastIntervalTableQuery::class with LastIntervalTableQuery.serializer() | ||
| } | ||
| beanParams() | ||
| tableQuery() | ||
| } | ||
|
|
||
| val jsonCompact = Json(context = paramsModule) | ||
| private val json = jsonCompact(paramsModule) | ||
|
|
||
| val jsonPretty = Json(context = paramsModule, configuration = JsonConfiguration.Stable.copy(prettyPrint = true)) | ||
| fun deserialize(topology: String): Topology = json.parse(Topology.serializer(), topology) | ||
|
|
||
| fun deserialize(topology: String): Topology = jsonCompact.parse(Topology.serializer(), topology) | ||
|
|
||
| fun serialize(topology: Topology, json: Json = jsonCompact): String = json.stringify(Topology.serializer(), topology) | ||
| fun serialize(topology: Topology, json: Json = this.json): String = json.stringify(Topology.serializer(), topology) | ||
| } |
Oops, something went wrong.