Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New python service #8455

Merged
merged 26 commits into from Jun 30, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
6cd6fe1
feature: new python service
murrant Jul 11, 2017
27c9969
New python service (#9)
TheMysteriousX Apr 8, 2018
878c8b1
bug: restore poller stats
TheMysteriousX Apr 15, 2018
9441d05
bug: Groups can be comma seperated strings
TheMysteriousX Apr 15, 2018
0ccdb97
Verify that the work has been pulled from the correct queue before ac…
TheMysteriousX Apr 18, 2018
e6132a0
bug: Make SQL databse connections multi threaded
TheMysteriousX Apr 18, 2018
b548cbb
Doc updates
TheMysteriousX Apr 18, 2018
5000eb7
Instrument queue assignment
TheMysteriousX Apr 18, 2018
3b0c9f6
bug: queue assignment was terminally broken
TheMysteriousX Apr 18, 2018
4686700
Suppress timestamps unless requested
TheMysteriousX Apr 18, 2018
0757e49
Restore python3.4 compatibility
TheMysteriousX Apr 19, 2018
d996938
Update config_to_json.php with recent config refactor (#12)
TheMysteriousX Apr 20, 2018
e80232b
New python service fixedgit (#13)
TheMysteriousX Jun 4, 2018
d078255
Fix off by 1 error
TheMysteriousX Jun 5, 2018
0e4c0fb
Schema update
TheMysteriousX Jun 5, 2018
ba06209
* Remove implicit queue creation (#15)
TheMysteriousX Jun 14, 2018
6b5df23
Use .env config (requires python-dotenv, docs updated)
murrant Jun 15, 2018
707eb1f
Update schema
murrant Jun 15, 2018
4452dce
fix issue where discovery is run twice.
murrant Jun 16, 2018
df37d8b
bugfix: Force expiration to be at least 1 bugfix: typo (#16)
TheMysteriousX Jun 20, 2018
47e79d0
Run populateEnv() every composer install run (makes sure NODE_ID is set)
murrant Jun 25, 2018
fabb040
Add Node ID column and make it red if empty
murrant Jun 25, 2018
4e8a110
Consider clustered polling when validating (#17)
TheMysteriousX Jun 26, 2018
e1897dc
Fatal .env loading and SQL query fix (#18)
TheMysteriousX Jun 28, 2018
6a78fd5
add python requirements.txt
murrant Jun 28, 2018
0ac18bf
Added Status: BETA to Poller-service.md
laf Jun 30, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Expand Up @@ -13,6 +13,8 @@

# Others #
##########
__pycache__
*.pyc
*.swp
Thumbs.db
config.php
Expand Down
68 changes: 37 additions & 31 deletions LibreNMS/ComposerHelper.php
Expand Up @@ -42,8 +42,9 @@ public static function postInstall(Event $event)
{
if (!file_exists('.env')) {
self::setPermissions();
self::populateEnv();
}

self::populateEnv();
}

public static function preUpdate(Event $event)
Expand Down Expand Up @@ -80,34 +81,34 @@ private static function populateEnv()
if (!file_exists('.env')) {
copy('.env.example', '.env');
self::exec('php artisan key:generate');

$config = [
'db_host' => '',
'db_port' => '',
'db_name' => '',
'db_user' => '',
'db_pass' => '',
'db_socket' => '',
'base_url' => '',
'user' => '',
'group' => '',
];

@include 'config.php';

self::setEnv([
'NODE_ID' => uniqid(),
'DB_HOST' => $config['db_host'],
'DB_PORT' => $config['db_port'],
'DB_USERNAME' => $config['db_user'],
'DB_PASSWORD' => $config['db_pass'],
'DB_DATABASE' => $config['db_name'],
'DB_SOCKET' => $config['db_socket'],
'APP_URL' => $config['base_url'],
'LIBRENMS_USER' => $config['user'],
'LIBRENMS_GROUP' => $config['group'],
]);
}

$config = [
'db_host' => '',
'db_port' => '',
'db_name' => '',
'db_user' => '',
'db_pass' => '',
'db_socket' => '',
'base_url' => '',
'user' => '',
'group' => '',
];

@include 'config.php';

self::setEnv([
'NODE_ID' => uniqid(),
'DB_HOST' => $config['db_host'],
'DB_PORT' => $config['db_port'],
'DB_USERNAME' => $config['db_user'],
'DB_PASSWORD' => $config['db_pass'],
'DB_DATABASE' => $config['db_name'],
'DB_SOCKET' => $config['db_socket'],
'APP_URL' => $config['base_url'],
'LIBRENMS_USER' => $config['user'],
'LIBRENMS_GROUP' => $config['group'],
]);
}

/**
Expand All @@ -118,8 +119,10 @@ private static function populateEnv()
*/
private static function setEnv($settings, $file = '.env')
{
$content = file_get_contents($file);
if (substr($content, -1) !== "\n") {
$original_content = $content = file_get_contents($file);

// ensure trailing line return
if (substr($content, -1) !== PHP_EOL) {
$content .= PHP_EOL;
}

Expand All @@ -143,7 +146,10 @@ private static function setEnv($settings, $file = '.env')
}
}

file_put_contents($file, $content);
// only write if the content has changed
if ($content !== $original_content) {
file_put_contents($file, $content);
}
}

private static function setPermissions()
Expand Down
12 changes: 12 additions & 0 deletions LibreNMS/Config.php
Expand Up @@ -314,6 +314,18 @@ public static function has($key)
return is_array($curr) && isset($curr[$last]);
}

/**
* Serialise the whole configuration to json for use in external processes.
*
* @return string
*/
public static function json_encode()
{
global $config;

return json_encode($config);
}

/**
* merge the database config with the global config
* Global config overrides db
Expand Down
9 changes: 9 additions & 0 deletions LibreNMS/Validations/Poller.php
Expand Up @@ -90,6 +90,15 @@ private function checkLastPolled(Validator $validator)
$validator->fail("The poller ($poller) has not completed within the last 5 minutes, check the cron job.");
}
}
} elseif (dbFetchCell('SELECT COUNT(*) FROM `poller_cluster`')) {
$sql = "SELECT `node_id` FROM `poller_cluster` WHERE `last_report` <= DATE_ADD(NOW(), INTERVAL - 5 MINUTE)";

$pollers = dbFetchColumn($sql);
if (count($pollers) > 0) {
foreach ($pollers as $poller) {
$validator->fail("The poller cluster member ($poller) has not checked in within the last 5 minutes, check that it is running and healthy.");
}
}
} else {
$validator->fail('The poller has never run or you are not using poller-wrapper.py, check the cron job.');
}
Expand Down