Skip to content

New OAP annotations, they are friendly for streaming loader.#2670

Merged
wu-sheng merged 14 commits into
apache:masterfrom
peng-yongsheng:feature/annotations
May 15, 2019
Merged

New OAP annotations, they are friendly for streaming loader.#2670
wu-sheng merged 14 commits into
apache:masterfrom
peng-yongsheng:feature/annotations

Conversation

@peng-yongsheng
Copy link
Copy Markdown
Member

  1. Stream annotation
    This is the most common metrics entity definition with the stream annotations.

@MetricsType
@StreamData
@StorageEntity(name = ServiceRelationClientSideMetrics.INDEX_NAME, builder = ServiceRelationClientSideMetrics.Builder.class, sourceScopeId = DefaultScopeDefine.SERVICE_RELATION)

@MetricsType: What model is streaming processing use? Other types are @RecordType, @InventoryType, @TopNType.
@StreamData: Register this entity class into the remote module that makes this entity could transfer between OAP server's nodes.
@StorageEntity: Identify the storage entity name, builder class for serialization or deserialization this entity between memory and database.

Those three annotations look like having clear responsibilities, but if you spend time to reflect on them, you will find that the @xxtype's level is higher than others.

  • @MetricsType and @InventoryType are must include @StreamData and @StorageEntity.
  • @RecordType and @TopNType are must exclude @StreamData.

So, I suggest that let the @StreamData and @StorageEntity to be the parameter of the @xxtype.

For example:

@stream(name = ServiceRelationClientSideMetrics.INDEX_NAME, scopeId = DefaultScopeDefine.SERVICE_RELATION, storage = @Storage(builder = ServiceRelationClientSideMetrics.Builder.class), processor = MetricsStreamProcessor.class)
public class ServiceRelationClientSideMetrics extends Metrics {

  1. Processors
  • MetricsStreamProcessor.class
  • RecordStreamProcessor.class
  • TopNStreamProcessor.class
  • InventoryStreamProcessor.class

Make those processors connect with the stream annotation directly.

- Inventory annotation example:
@stream(name = ServiceInventory.INDEX_NAME, scopeId = DefaultScopeDefine.SERVICE_INVENTORY, storage = @Storage(builder = ServiceInventory.Builder.class, deleteHistory = false), kind = StreamKind.Inventory)

- Metrics annotation example:
@stream(name = ServiceRelationClientSideMetrics.INDEX_NAME, scopeId = DefaultScopeDefine.SERVICE_RELATION, storage = @Storage(builder = ServiceRelationClientSideMetrics.Builder.class), kind = StreamKind.Metrics)

- Record annotation example:
@stream(name = SegmentRecord.INDEX_NAME, scopeId = DefaultScopeDefine.SEGMENT, storage = @Storage(builder = SegmentRecord.Builder.class), kind = StreamKind.Record)

- TopN annotation example:
@stream(name = TopNDatabaseStatement.INDEX_NAME, scopeId = DefaultScopeDefine.DATABASE_SLOW_STATEMENT, storage = @Storage(builder = TopNDatabaseStatement.Builder.class), kind = StreamKind.TopN)
@peng-yongsheng peng-yongsheng added backend OAP backend related. feature New feature labels May 14, 2019
@peng-yongsheng peng-yongsheng added this to the 6.2.0 milestone May 14, 2019
@coveralls
Copy link
Copy Markdown

coveralls commented May 15, 2019

Coverage Status

Coverage increased (+0.009%) to 16.545% when pulling ca2b994 on peng-yongsheng:feature/annotations into 1b6bd5f on apache:master.

Copy link
Copy Markdown
Member

@wu-sheng wu-sheng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only concern is rename of MODEL_NAME to INDEX_NAME. This is used in different storage implementations. I suggest we don't change that.

@peng-yongsheng
Copy link
Copy Markdown
Member Author

The only concern is rename of MODEL_NAME to INDEX_NAME. This is used in different storage implementations. I suggest we don't change that.

This name in inventory entity is different from metrics entity, record entity, topn entity.

@wu-sheng wu-sheng merged commit 63fb961 into apache:master May 15, 2019
@peng-yongsheng peng-yongsheng deleted the feature/annotations branch May 15, 2019 14:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend OAP backend related. feature New feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants