Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Device manager improvements #69

Merged
merged 3 commits into from
Oct 25, 2023
Merged

Conversation

MagdaPaj
Copy link
Contributor

@MagdaPaj MagdaPaj commented Oct 24, 2023

  • Setup the Device Manager to run as a generic host (there is many things provided out-of-the-box, so the setup is simpler)
  • Configure EventHubReceiverService as IHostedService so StartUp and Shutdown are handled by the Host.
  • Some small updates to EventHubReceiverService for fixing warnings or improving readability.

As a result of that:

  • we don't see any more the Device Manager pod being restarted every 5 min
  • there is still a weird behavior in the outgoing messages metric reported by Event Hub after stopping the Device Manager and starting it, but it comes back to normal
    image
  • there are still some errors reported from the BlobStorage client (404) but it happens less frequently as the application is not restarted every 5 min
    image
  • I also validated checkpointing and it behaves correctly, it starts where it previously finished. And the number of calls to the checkpoint container matches the number of requests from the Device Manager to the Device API (though there is a bug and all telemetry reporting HTTP requests are duplicated, but if we write a query correctly then we can see it matches
    image

image

  • I found out that the huge amount of dependencies calls comes from the EventProcessorClient that uses Azure Storage blob to manage ownership. There is one PUT and one GET request every 30 seconds to the ownership Blob container, even when we're not processing any messages. Originally I thought that we were making so many calls to checkpoint Blob container, but that's not the case.
    image

This doesn't solve but decreases the number of errors from this ticket.

Copy link
Contributor

@MahrRah MahrRah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@MagdaPaj MagdaPaj merged commit 391ee36 into main Oct 25, 2023
@MagdaPaj MagdaPaj deleted the mbaran/device-manager-improvements branch October 25, 2023 13:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants