Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1436811: Send reports on the interval per destination #59

Merged
merged 6 commits into from Apr 3, 2017

Conversation

@cnsnyder
Copy link
Member

commented Mar 31, 2017

This set of changes allows virt-who to send all reports (batched where possible) on the interval for each destination. Reports for different destinations no longer block each other and cause check in times to increase. This allows virt-who to be set up more effectively for large organizations having some set of hypervisors that need to be reported to more than one organization or environment or even more than one satellite!

cnsnyder added 6 commits Jan 27, 2017
Move from using processes to threads
This is a part of the interval design.
As virt-who does not need to take advantage of multiple physical cores,
we have decided to move from using processes to threads.
We consider it more maintainable and reliable to get threads to stop
than processes.

Update tests to use threading where necessary

Fix patching to refer to the right Queue class

Update assertions to ensure the proper termination method is called
Implemements a threadsafe datastore
The implemented threadsafe datastore uses maintains a dictionary of keys and
values. Modifying the dictionary is done exclusively post lock aquisition and
prior to lock release. Every item is pickled on the way in and unpickled on
the way out of the underlying dictionary to ensure no two threads share
references to the same objects.

Use pickle for serialization and deserialization

Use cPickle if available, else use pickle
cPickle is upto 1000 times faster, but not always available.
Update ConfigManager to produce destination and source mappings.
This is step 3 of the interval design. The configuration is now
parsed into unique destinations and sources. Historical Config
objects are treated as sources and the destination information
is pulled out of them and tracked. In this way the main thread
will know how many destination/source threads (and of what type) to
start.
Adds IntervalThread base class and refactors Virt classes
This also adds a new class DestinationThread. The new class is
able to make connections to various destinations and submits
reports gathered from varying sources.

For future reference this is part of the interval design.

Create IntervalThread base class (for virt and destinations) update tests

Add DestinationThread to handle batching and sending of reports to using any Manager

Add tests for DestinationThread
DestinationThreads now send all reports
This step is the last of the mvp of the interval design.
Herein the main thread no longer sends reports to any
destinations, that is now the job of a set of
DestinationThreads. Below are additional changes in this
commit.

Updating tests to account for new design

Separate out oneshot, print mode from regular mode

Removes obsolete tests, adds framework for additional testing

Ensure env is free of all VIRTWHO* variables for subsequent tests

rhsm username and password are optional, add default dest if none found
@pcreech

This comment has been minimized.

Copy link
Contributor

commented Mar 31, 2017

LGTM!

@pcreech pcreech merged commit dac0263 into master Apr 3, 2017

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.