Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Device state rule node; device state service improvements #9030

Merged
merged 44 commits into from
Feb 16, 2024
Merged

Device state rule node; device state service improvements #9030

merged 44 commits into from
Feb 16, 2024

Conversation

dskarzh
Copy link
Contributor

@dskarzh dskarzh commented Aug 2, 2023 •
edited
Loading

Pull Request description

Case: data must be pulled from external REST API or new data is computed within the rule chain, device is being created inside the rule chain using relation node, for example etc.

Problem: device state service doesn’t know about artificially created device and considers it inactive, when in fact it is active.

Workaround: using integrations or custom-made rule chain just to handle this case is messy and less performant

Solution: add rule node for this case that will directly interact with device state service

Scope of changes: added device state rule node and tests for it; device state service: fixed concurrency bug, minor refactoring of some parts of service code, added tests to cover affected logic

PE PR
UI changes PR
Documentation PR WIP

Please close outdated PR and issue for this feature by @YuriyLytvynchuk

General checklist

  • You have reviewed the guidelines document.
  • Labels that classify your pull request have been added.
  • The milestone is specified and corresponds to fix version.
  • Description references specific issue.
  • Description contains human-readable scope of changes.
  • Description contains brief notes about what needs to be added to the documentation.
  • No merge conflicts, commented blocks of code, code formatting issues.
  • Changes are backward compatible or upgrade script is provided.
  • Similar PR is opened for PE version to simplify merge. Crosslinks between PRs added. Required for internal contributors only.

Back-End feature checklist

  • Added corresponding unit and/or integration test(s). Provide written explanation in the PR description if you have failed to add tests.
  • If new dependency was added: the dependency tree is checked for conflicts.
  • If new service was added: the service is marked with corresponding @TbCoreComponent, @TbRuleEngineComponent, @TbTransportComponent, etc.
  • If new REST API was added: the RestClient.java was updated, issue for Python REST client is created.

@dskarzh dskarzh changed the title Feature/device state node Feature: device state node Aug 2, 2023
@ShvaykaD ShvaykaD added this to the 3.5.2 milestone Aug 3, 2023
@ShvaykaD ShvaykaD added feature enhancement Rule Engine Changes to Rule Engine Core Changes to Core labels Aug 3, 2023
@dskarzh dskarzh changed the title Feature: device state node [3.5.2] Feature: device state node Aug 3, 2023
@dskarzh dskarzh changed the title [3.5.2] Feature: device state node [3.5.2] feature: device state node; device state service improvements Aug 3, 2023
@ashvayka
Copy link
Member

ashvayka commented Aug 3, 2023

The idea is great, but the implementation will not work in a microservices mode. Please note that DefaultDeviceStateService is marked with the @TbCoreComponent. It is unavailable in the rule engine when launched as a separate microservice. Therefore, you should use TbClusterService.pushMsgToCore or a similar method to deliver the activity event to the corresponding DeviceStateService.

@dskarzh dskarzh changed the title [3.5.2] feature: device state node; device state service improvements [WIP] [3.5.2] feature: device state node; device state service improvements Aug 8, 2023
@dskarzh dskarzh changed the title [WIP] [3.5.2] feature: device state node; device state service improvements [WIP] feature: device state node; device state service improvements Aug 15, 2023
@kalutkaz kalutkaz mentioned this pull request Aug 15, 2023
Merged
@ashvayka ashvayka modified the milestones: 3.6, 3.6.1 Sep 6, 2023
@dskarzh dskarzh closed this Sep 8, 2023
@dskarzh dskarzh reopened this Sep 8, 2023
@dskarzh
Copy link
Contributor Author

dskarzh commented Sep 8, 2023 •
edited
Loading

Closed by accident

@dskarzh dskarzh changed the title [WIP] feature: device state node; device state service improvements [3.6.1] feature: device state node; device state service improvements Sep 8, 2023
@dskarzh dskarzh changed the title Device state rule node; device state service improvements. [3.6.3] Device state rule node; device state service improvements Feb 2, 2024
@ashvayka ashvayka modified the milestones: 3.6.1, 3.6.3 Feb 8, 2024
@ashvayka ashvayka changed the title [3.6.3] Device state rule node; device state service improvements Device state rule node; device state service improvements Feb 8, 2024
ashvayka
ashvayka reviewed Feb 13, 2024
View reviewed changes
...on/src/main/java/org/thingsboard/server/service/state/LocalRuleEngineDeviceStateManager.java Outdated
@Slf4j
@Service
@Primary
@TbCoreComponent
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use partitionService.isMyPartition to detect routing strategy

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use

public void pushMsgToCore(TopicPartitionInfo tpi, UUID msgId, ToCoreMsg msg, TbQueueCallback callback)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see c5c1820

ashvayka
ashvayka reviewed Feb 13, 2024
View reviewed changes
...le-engine-components/src/main/java/org/thingsboard/rule/engine/action/TbDeviceStateNode.java Outdated

Duration elapsedSinceLastEventSent = now.minus(lastEventTs);
if (elapsedSinceLastEventSent.compareTo(ONE_SECOND) < 0) {
ctx.tellSuccess(msg);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rateLimit + add to annotation the corresponding relation types

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see e55d143

ashvayka
ashvayka reviewed Feb 13, 2024
View reviewed changes
...le-engine-components/src/main/java/org/thingsboard/rule/engine/action/TbDeviceStateNode.java Outdated
private static final Duration ENTRY_CLEANUP_PERIOD = Duration.ofHours(1L);

private Stopwatch stopwatch;
private ConcurrentMap<DeviceId, Duration> lastActivityEventTimestamps;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ConcurrentReferenceHashMap<DeviceId, TbRateLimits> with SOFT
Make the rate limit configuration an env parameter. Default value: 1:1,30:60,60:3600.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see e55d143

@ashvayka ashvayka merged commit d5da538 into thingsboard:master Feb 16, 2024
3 checks passed
@dskarzh dskarzh deleted the feature/device-state-node branch March 27, 2024 15:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ashvayka ashvayka ashvayka left review comments

@ShvaykaD ShvaykaD ShvaykaD left review comments

@smatvienko-tb smatvienko-tb Awaiting requested review from smatvienko-tb

Assignees
No one assigned
Labels
Core Changes to Core enhancement feature Rule Engine Changes to Rule Engine
Projects
None yet
Milestone
3.6.3
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@dskarzh @ashvayka @ShvaykaD @smatvienko-tb