Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP - Refactoring for release 0.1 #156

Merged
merged 37 commits into from
Feb 15, 2019
Merged

WIP - Refactoring for release 0.1 #156

merged 37 commits into from
Feb 15, 2019

Conversation

wzorgdrager
Copy link
Collaborator

@wzorgdrager wzorgdrager commented Feb 11, 2019

Related issue(s): #163

Description of changes

This PR covers some refactoring for release 0.1.

Refactoring changelog

  • Rename PipelineObject to Stage. This name is more intuitive given we are using (Input|Transform|Output)Stages. Consequently, PipelineObjectList is also renamed to StageList.
  • Added appendSource(StreamExecutionEnvironment => DataStream[T]) to create an anonymous InputStage (append already exists for both Transform and Output stages).
  • PipelineBuilder allows for setting the StreamTimeCharacteristic instead of always setting it to EventTime.
  • Custom serializer functionality (see Make Serializers external #163)
  • Added ScalaDoc to all core classes.
  • Minor bugfixes to all core classes.
  • Minor renamings/refactorings to all core classes`.

Check all classes and tests

  • Correct Scaladoc
  • Correct formatting and style
Core
  • Review classes and tests
  • Fix KafkaClient bug in tests (connection time-out)

Check-list

Please make sure to review and check all of these items:

Core

Buffer

  • Buffer
  • BufferFactory
  • BufferType
  • Exceptions
  • KafkaBuffer
  • RabbitMQBuffer

Serialization

  • AbstractSerde
  • BsonSerde
  • JSONSerde
  • KryoSerde
  • Serializer

Schema Exposure

  • SchemaExposer
  • ZookeeperSchemaExposer
  • RedisSchemaExposer

Keymanager

  • KeyManager
  • StaticKeyManager

Redis

  • RedisKeyManager

Pipeline

  • DirectedAcyclicGraph
  • Exceptions
  • Pipeline
  • PipelineBuilder
  • PipelineObject -> Stageutilties
  • PipelineObjectList -> StageList
  • PipelineObjectN -> StageN
  • Types

Stages

  • InputStage
  • OutputStage
  • TransformStage
  • StageAttributes

Kafka

  • KafkaInput
  • KafkaOutput

RabbitMQ

  • RabbitMQInput
  • RabbitMQOutput

Utilities

  • DuplicateService
  • FiniteQueue
  • HttpRequester
  • JsonPrinterOutput
  • PrinterOutput
  • SeqInput
  • StringInput
  • Properties

NOTE: these things are not required to open a PR and can be done
afterwards / while the PR is open.

@wzorgdrager wzorgdrager changed the title Prepare release 1.0 Prepare release 0.1 Feb 13, 2019
@wzorgdrager wzorgdrager changed the title Prepare release 0.1 WIP - Prepare release 0.1 Feb 14, 2019
@wzorgdrager wzorgdrager added the not ready This branch is not ready to merge. label Feb 14, 2019
@wzorgdrager wzorgdrager self-assigned this Feb 14, 2019
@wzorgdrager wzorgdrager changed the title WIP - Prepare release 0.1 WIP - Refactoring for release 0.1 Feb 15, 2019
@wzorgdrager wzorgdrager added the v0.1 Should be fixed for version 1.0. label Feb 15, 2019
@wzorgdrager
Copy link
Collaborator Author

Instead of creating this huge PR, I will merge and close this one soon (mostly refactoring). To split the other work into smaller PR's. That will be way more clear.

@wzorgdrager wzorgdrager removed the not ready This branch is not ready to merge. label Feb 15, 2019
@wzorgdrager wzorgdrager merged commit 7016378 into develop Feb 15, 2019
@wzorgdrager wzorgdrager deleted the prepare_release_1.0 branch February 15, 2019 22:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
v0.1 Should be fixed for version 1.0.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant