Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fixes #7291: Exit sending inventory to rudder-webapp if the queue is full #905

Merged
merged 1 commit into from
Apr 6, 2016
Merged

Fixes #7291: Exit sending inventory to rudder-webapp if the queue is full #905

merged 1 commit into from
Apr 6, 2016

Conversation

amousset
Copy link
Member

@amousset amousset commented Apr 6, 2016

amousset
amousset reviewed Apr 6, 2016
View reviewed changes
initial-promises/node-server/distributePolicy/1.0/propagatePromises.cf
classes => if_else("inventory_sent", "rudder_send_inventory_to_cmdb_cant_send_inventory"),
ifvarclass => "!rudder_send_inventory_to_cmdb_cant_send_inventory",
comment => "Sending the inventory to the cmdb";

# NEED TO DO THE RSYNC PART
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Off topic, but do you know what it refers to?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it's really old, when we were considering rsync for copying promises from main server to relay servers, IIRC. You can remove it :)

@amousset amousset force-pushed the ust_7291/exit_sending_inventory_to_rudder_webapp_if_the_queue_is_full branch from 5c041d1 to cf0c529 Compare April 6, 2016 10:51
@jooooooon
Copy link
Member

This looks great! Can you just confirm you have tested it?

@amousset amousset force-pushed the ust_7291/exit_sending_inventory_to_rudder_webapp_if_the_queue_is_full branch from cf0c529 to 6e459a2 Compare April 6, 2016 11:07
@amousset
Copy link
Member Author

amousset commented Apr 6, 2016

Tested with broken inventories and with too many inventories:

...
2016-04-06T13:00:50+0200     info: /default/sendInventoryToCmdb/files/'/var/rudder/inventories/incoming/inv246.ocs'[163]:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
2016-04-06T13:00:50+0200     info: /default/sendInventoryToCmdb/files/'/var/rudder/inventories/incoming/inv246.ocs'[163]:                                  Dload  Upload   Total   Spent    Left  Speed
100  328k    0    62  100  328k    190  1009k --:--:-- --:--:-- --:--:-- 1012kdder/inventories/incoming/inv246.ocs'[163]: 
2016-04-06T13:00:50+0200     info: /default/sendInventoryToCmdb/files/'/var/rudder/inventories/incoming/inv246.ocs'[163]: Transformer '/var/rudder/inventories/incoming/inv246.ocs' => '/var/rudder/tools/send-clean.sh http://localhost:8080/endpoint/upload/ /var/rudder/inventories/incoming/inv246.ocs /var/rudder/inventories/received/ /var/rudder/inventories/failed/' seemed to work ok
2016-04-06T13:00:50+0200     info: /default/sendInventoryToCmdb/files/'/var/rudder/inventories/incoming/inv247.ocs'[164]: Transforming '/var/rudder/tools/send-clean.sh http://localhost:8080/endpoint/upload/ /var/rudder/inventories/incoming/inv247.ocs /var/rudder/inventories/received/ /var/rudder/inventories/failed/' 
2016-04-06T13:00:50+0200     info: /default/sendInventoryToCmdb/files/'/var/rudder/inventories/incoming/inv247.ocs'[164]:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
2016-04-06T13:00:50+0200     info: /default/sendInventoryToCmdb/files/'/var/rudder/inventories/incoming/inv247.ocs'[164]:                                  Dload  Upload   Total   Spent    Left  Speed
100  328k    0    62  100  328k    175   930k --:--:-- --:--:-- --:--:--  962kdder/inventories/incoming/inv247.ocs'[164]: 
2016-04-06T13:00:51+0200     info: /default/sendInventoryToCmdb/files/'/var/rudder/inventories/incoming/inv247.ocs'[164]: Transformer '/var/rudder/inventories/incoming/inv247.ocs' => '/var/rudder/tools/send-clean.sh http://localhost:8080/endpoint/upload/ /var/rudder/inventories/incoming/inv247.ocs /var/rudder/inventories/received/ /var/rudder/inventories/failed/' seemed to work ok
2016-04-06T13:00:51+0200     info: /default/sendInventoryToCmdb/files/'/var/rudder/inventories/incoming/inv248.ocs'[165]: Transforming '/var/rudder/tools/send-clean.sh http://localhost:8080/endpoint/upload/ /var/rudder/inventories/incoming/inv248.ocs /var/rudder/inventories/received/ /var/rudder/inventories/failed/' 
2016-04-06T13:00:51+0200     info: /default/sendInventoryToCmdb/files/'/var/rudder/inventories/incoming/inv248.ocs'[165]:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
2016-04-06T13:00:51+0200     info: /default/sendInventoryToCmdb/files/'/var/rudder/inventories/incoming/inv248.ocs'[165]:                                  Dload  Upload   Total   Spent    Left  Speed
100  328k    0     0  100  328k      0   936k --:--:-- --:--:-- --:--:--  940kdder/inventories/incoming/inv248.ocs'[165]: 
2016-04-06T13:00:51+0200     info: /default/sendInventoryToCmdb/files/'/var/rudder/inventories/incoming/inv248.ocs'[165]: curl: (22) The requested URL returned error: 503
2016-04-06T13:00:51+0200    error: /default/sendInventoryToCmdb/files/'/var/rudder/inventories/incoming/inv248.ocs'[165]: Finished command related to promiser '/var/rudder/inventories/incoming/inv248.ocs' -- an error occurred, returned 22
2016-04-06T13:00:51+0200    error: /default/sendInventoryToCmdb/files/'/var/rudder/inventories/incoming/inv248.ocs'[165]: Transformer '/var/rudder/inventories/incoming/inv248.ocs' => '/var/rudder/tools/send-clean.sh http://localhost:8080/endpoint/upload/ /var/rudder/inventories/incoming/inv248.ocs /var/rudder/inventories/received/ /var/rudder/inventories/failed/' returned error
2016-04-06T13:00:51+0200     info: /default/sendInventoryToCmdb/files/'/var/rudder/inventories/incoming/inv248.ocs'[165]: WARNING: Unable to send /var/rudder/inventories/incoming/inv248.ocs, inventory endpoint is temporarily unavailable, will retry later
2016-04-06T13:00:51+0200     info: /default/sendInventoryToCmdb/files/'/var/rudder/inventories/incoming/inv248.ocs'[165]: This often happens due to rate-throttling in the endpoint to save on memory consumption. This is standard behavior.
2016-04-06T13:00:51+0200     info: /default/sendInventoryToCmdb/files/'/var/rudder/inventories/incoming/inv248.ocs'[165]: Transforming '/var/rudder/tools/send-clean.sh http://localhost:8080/endpoint/upload/ /var/rudder/inventories/incoming/inv248.ocs /var/rudder/inventories/received/ /var/rudder/inventories/failed/' 
error    DistributePolicy          Send inventories to CMDB                     Some inventories failed to add successfully to Rudder

@jooooooon jooooooon merged commit c0ea52a into Normation:branches/rudder/2.11 Apr 6, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@amousset @jooooooon