Skip to content

Commit

Permalink
Dev v3 (#971)
Browse files Browse the repository at this point in the history
* fixed issue with copying cx-freeze libs

* remove trailing whitespace

* Remove trailing whitespace

* fixed issue with copying cx-freeze libs

* fixed typo

* fixed gitignore

* fixed sed for getting major version

* reverted change to bin_includes in setup.py

* remove python executable from pkg. It used to need this to work, but doesn't any longer.

* made it so py and ssl version are printed at info level of logging

* added full path to libssletc. in setup

* another update centos 7 epel install

* don't include python executable in pkg.

* added openssl version check for deb distros

* Added stuff to gitignore

* reverted prev chg which added path to bin files - didn't help

* skip openSSL install, fix epel repos for CentOS 7

* adjust epel processing

* Fixed cx lib link conditional

* Updated script init promp info

* remove obsolete yul epel install line

* Minor improvements to source install functions. Moved some functions from setup.sh

* Consolidated literals (version numbers), modified package installatoins, add OpenSSL source installation.

* Changed function call to corrrect name

* add scripts for openSSL install

* Numerous updates to OpenSSL and Python installers. Still does not work correctly

* Source installers work on cent8 and ubuntu 20. For testing running as standalone, install some of ffi, sqlite reprequisites normally installed by setup script.

* Updated ncpa to print OpenSSL version in debug logging mode

* Updated installers to use either passed params or globals. Updated Py installer to configure source without flag pointing to custom OpenSSL, if none is installed.

* bug: openssl version sed - OS compatibility

* Print Py version in debug mode

* Print out OpenSSL path after install, allow passsing new SSL path to python as arg for runing as stand-alone

* Bring OpenSSL install build into ncpa build folder

* Bring OpenSSL install build into ncpa build folder

* install openssl to generic folder name in build/resources. Update setup.py to try to include libssl etc.

* dbg: echo PYTHONBIN after install.

* updated to put py and ssl version in log

* added bin paths to setup.py

* Added/improved comments for installers.sh

* Added code for copying _ssl if needed

* fixed typo in openssl library linking

* add --disable-shared option to python configure

* Reverted to no --disable-shared python  compile options

* sym-linked cx_freeze lib-dynload directory to instealled Python. Can't seem to find it by itself. TODO-Ph: see if there s a way to make cx-freeze copy from this dir

* setup.sh - skip openssl pkg install if we are going to build from source.

* added *.rpm to git ignore

* setup.sh fixed some info output

* update to find original libssl.so on ubuntu 16

* fixed typo

* add --allow-unauthenticated flag to linux set up apt commands so they will work on older distros

* removed unecessary sudo cmmands that cause trouble on debian 8. You nned to be root to run the script anyway.

* added apt-get update - helps install complete on lod distros.

* Update installer to support some more distros

* fixed opensuse zypper options

* reverted bin_path settings that never worked

* updated setup.sh so suse distros take openssl building into account

* suppressed install of epel on non-centos because it fails on rhel8, and isn't needed.

* Minor changes to resolve issues from merge of windows code

* fixed permissoins issue on log files

* updated NSIS script for new install/management process

* added empty logs to repo

* removed custom launcher that I was testing

* cleaning up and increasing clarity

* adjusted log message

* updated function name for clarity

* updating comments and increasing clarity

* merge build.sh from dev-v3-app-changes

* updated default python launcher to py (python3 is usually py on windows)

* updating BUILDING.rst

* updating to use parameters on windows

* update gitignore to include log files (should be empty)

* Fix some issues in ncpa.cfg and make sample match

* REmoved ncpa.log and updated to cinlude empty logs for both processes.

* merge build.sh from app-changes branch

* added message about not using setup_config.py anymore (cx_Freeze service config file)

* Minor bash changes to make Solaris compatible

* Update so cx_freeze libs that aren't included in python are moved into python lib-dynload, before linking to it.

* Only load windows only modules on Windows

* simplified lib-dynload udpate

* simplified cx dynlib so it no longer uses linking

* wrap WinService class definition in conditional, so it doesn't break linux

* added initial log set up code to fix permissions issue, but now log entries are doubled still needs some work

* reimplementing last formatting change for setup.py

* fixing some errors with passive and NRDP code

* updated NSIS script to ask user if they want to save their configuration files when deleting NCPA

* updated NSIS script

* added delay so the user doesn't have to hit retry

* organizing pywin32 imports

* commented out option to not remove configs in uninstaller -- not working

* chown installed passive log file before daemon runs to avoid permission error

* Add .deb creation when building on CentOS 7.

* updating building instructions

* Update CHANGES.rst

* fixing kafka

* update service shutdown for older distros during upgrade

* updated linux install service handling to unload v2.4.1 services

* updated build_windows.py to write git hash (also added some coloration to the build and a build-complete message

* adding logging to debug plugins on linux

* testing and debugging

* restore listener/pluginnodes.py

* got kafka working on linux

* updated NSIS script to set NCPA service startup to auto instead of delayed-auto

* updated install script to remove NCPA 2 services

* updated NSIS installer to install into the Program Files folder since the binary is a 64-bit program. This may break 32-bit compatibility, but nothing is 32-bit anymore so it shouldn't be a problem.

* Fixed some default config issues

* Added Python and OpenSSL version to listener UI dashboard

* Refactored Listener UI Admin section to pull values from actual configuration rather than making new assignments with a separeate set of defaults to maintain. Also, updated to match current configuration options.

* test change to admin/kafkaproducers

* Brought changes.rst into sync with v2.4.1, and added v3 info

* Add'l changes to CHANGES

* fised typo in unsinstall for linux

* added git hash to ship with build

* decode bytes into string so string split can process

* Set passive process loop dlay back to 1 sec

* Updated changes.rst

* update readme to v2.4.1 version

* Update README.rst for v3.0.0

* Updated BUILDING.rst for v3

* Removed bash color changes - they conflict with non-black backgrounds.

* Updates to incorporate ms vc runtime.

* Updated Windows installer to reflect current config

* Changed nrpd default hostname from NCPA 2 to NCPA. This shouldn't have version info in it.

* Fix grammar ran -> run

* Further clean up of config to match v3, and make some stuff more clear.

* Removed change log section and replaced with link to change log. No need to duplicate this info here, and it is very difficult to maintain.

* Fix grammar. ran -> run.

* Changed nrpd default hostname from NCPA 2 to NCPA. This shouldn't have version info in it.

* Clean trailing whitespace

* Update Listener UI Help sectoin to match current release.

* Added include file to resolve late monkey loading issue in unit tests

* updated certificate.py and its unit test to close open cert files that generate confusing  warnings

* Updated ncpa.py to make passive logger declaratoin global for accessibility by other files.
added passive logger include to files that are primariliy used by passive process.

* added .vscode folder to git ignore

* added .vscode folder to git ignore

* Added server headers X-content nosniff, Strict-Transport-Security, and disallowed OPTIONS verb for all routes

* removed errant quotes arond __NAME__

* update signal handler to intercept CTL-C (SIGINT), and now interupts main loop before exiting for a more graceful shutdown

* Updated main loop try-catch structure, and fall through for post interupt

* Imporved Daemon class stop() to make more logical and reduce unnecessary retries

* Wrapped Daemon:daemonize() call in try/catch"

* Updated some log levels to more appropriate values

* Making it so VERSION only has to be entered in one ncpa.py. Making it clearer what si shared from ncpa.py

* created tests/unused folder and moved obsolete nrds test into it

* added handling for bad nrdp address (http://ip_address without /nrdp gives XI page and XML parse error)

* restore colors to windows build script (and added restoring colors to pre-build-script colors)

* updated installer directory in message

* update release in .spec file

* Update remove nrdp hostname placeholder to allow app to guess hostname. TODO: udpate Windows installer to add info

* replicate cfg changes in sample

* Update to reflecet removal of nrdp hostname value form config.

* Shortened text in win installer

* changed github action to build dev-v3-beta02 macos

* changed instances of osx to macos, added macosreadme.txt, tweeked macosinstall and set build.yml to use this branch

* more osx to macos renames.

* updated mac to python 3.11

* made add'l change to cinclude macosreadme

* updated python version for mac in another place

* Minor changes to Mac install script output. Added restart instructions to mac readme.

* Added release # to MacOS package.sh, and changed to auto-detect architecture

* REmoved sleep from MacOS install

* Fixed issue with including release in file name.

* change buld.yml to use dev-v3-beta02 branch

* add MacOS build-install dierectory exclusion to .gitignore

* add MacOS .dmg exclusion to .gitignore

* added sudo back in for rm and chown to work on MacOS

* Make some dylib dependency paths relative

* Moved global version definitions into build.sh

* Moved global version definitions into build.sh

* Minor udpates, echo stmts

* Updated macos set-up to install homebrew, openssl and python

* Deleted obsolete update_py_packages fncn

* Made functions to check for openssl and python

* Add checks for openssl and python Only install if necessary.

* remove redundant py  mods update

* misc fixes to macos installers

* Removed SSL install for MAc - it is included with python install.

* Commented out some code used for debugging.

* IMproved output bundled all brew pkg inst with brew install

* update .gitignore to ignore macOS build artifacts

* Fix issue in which old and new version RPMs were created if built on same machine

* Fixed typo in macos package.sh, and improved output

* fixed bad command

* Add libffi.dylilb

* Shell script to convert python dylib dependencies frmo relative to absolute paths

* include dependency fixing scripts

* made relative dependency lib fix only for MacOS 11+

* added code to fix dylib dependencies

* Add ZLib version to log and interface

* Only copy libffi for MacOS v10

* Add messaging for building binaries so it doesn't seem like its hung

* fix message

* Fix packaging to handle version dependent libffi inclusion

* clean up output

* Fixed MacOs major version detection, moved be pkg installs out of brew install conditional.

* typo in message

* Fix zlib spelling, remove CRs.

* fixed CRs

---------

Co-authored-by: bb <114498860+ne-bbahn@users.noreply.github.com>
Co-authored-by: ne-bbahn <bbahnernagios@gmail.com>
  • Loading branch information
3 people committed Aug 25, 2023
1 parent b9a5c51 commit c5cd0b7
Show file tree
Hide file tree
Showing 36 changed files with 837 additions and 314 deletions.
7 changes: 4 additions & 3 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: Build

on:
push:
branches: [ 'dev-v3-app-changes' ]
branches: [ 'dev-v3' ]

jobs:

Expand All @@ -13,14 +13,15 @@ jobs:
matrix:
# os: [ubuntu-latest, macos-latest, windows-latest]
# os: [ubuntu-latest, macos-latest]
os: [ubuntu-20.04]
# os: [ubuntu-20.04]
os: [macos-latest]

steps:
- uses: actions/checkout@v2
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: '3.9'
python-version: '3.11'
- name: Check system platform
run: python -c "import sys; print(sys.platform)"
- name: Install Python dependencies
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,9 @@ build/ncpa.spec
build/ncpa
build/ncpa-3*/
build/ncpa-3*/
build/NCPA-INSTALL-*/
build/*.deb
build/*.rpm
build/*.dmg
build/share
build/NCPA-INSTALL-*/*
6 changes: 4 additions & 2 deletions agent/etc/ncpa.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -254,9 +254,11 @@ token =

#
# The hostname that will replace %HOSTNAME% in the check definitions and will be
# sent to NRDP with the check name as the service description (service name)
# sent to NRDP with the check name as the service description (service name).
# If none is provided, NCPA will assume it is the node name defined for the server.
# E.g. what is returned by linux hostname command, or by python platform.node().
#
hostname = NCPA
hostname =

#
# Connection timeout in seconds that will be used for transmitting the
Expand Down
6 changes: 4 additions & 2 deletions agent/etc/ncpa.cfg.sample
Original file line number Diff line number Diff line change
Expand Up @@ -254,9 +254,11 @@ token =

#
# The hostname that will replace %HOSTNAME% in the check definitions and will be
# sent to NRDP with the check name as the service description (service name)
# sent to NRDP with the check name as the service description (service name).
# If none is provided, NCPA will assume it is the node name defined for the server.
# E.g. what is returned by linux hostname command, or by python platform.node().
#
hostname = NCPA
hostname =

#
# Connection timeout in seconds that will be used for transmitting the
Expand Down
17 changes: 12 additions & 5 deletions agent/listener/certificate.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ def remove_empty_file(file):
return False

def create_self_signed_cert(cert_dir, cert_file, key_file):

# Cert files
target_cert = os.path.join(cert_dir, cert_file)
target_key = os.path.join(cert_dir, key_file)
Expand All @@ -36,17 +36,24 @@ def create_self_signed_cert(cert_dir, cert_file, key_file):
cert.get_subject().O = "Nagios Enterprises, LLC"
cert.get_subject().OU = "Development"
cert.get_subject().CN = socket.gethostname()

sn = int(time.time())
cert.set_serial_number(sn)

cert.gmtime_adj_notBefore(0)
cert.gmtime_adj_notAfter(10 * 365 * 24 * 60 * 60)
cert.set_issuer(cert.get_subject())
cert.set_pubkey(k)
cert.sign(k, 'sha256')

open(target_cert, "wb").write(crypto.dump_certificate(crypto.FILETYPE_PEM, cert))
open(target_key, "wb").write(crypto.dump_privatekey(crypto.FILETYPE_PEM, k))
# open(target_cert, "wb").write(crypto.dump_certificate(crypto.FILETYPE_PEM, cert))
cfh = open(target_cert, "wb")
cfh.write(crypto.dump_certificate(crypto.FILETYPE_PEM, cert))
cfh.close()

# open(target_key, "wb").write(crypto.dump_privatekey(crypto.FILETYPE_PEM, k))
kfh = open(target_key, "wb")
kfh.write(crypto.dump_privatekey(crypto.FILETYPE_PEM, k))
kfh.close()

return target_cert, target_key
10 changes: 6 additions & 4 deletions agent/listener/psapi.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@

importables = ("windowscounters", "windowslogs")

__SYSTEM__ = os.name
__VERSION__ = ncpa.__VERSION__

def get_uptime():
current_time = time.time()
Expand Down Expand Up @@ -94,7 +96,7 @@ def make_mountpoint_nodes(partition_name):
]

# Unix specific inode counter ~ sorry Windows! :'(
if ncpa.__SYSTEM__ != "nt":
if __SYSTEM__ != "nt":
try:
st = os.statvfs(mountpoint)
iu = st.f_files - st.f_ffree
Expand Down Expand Up @@ -171,7 +173,7 @@ def make_if_nodes(if_name):

def get_timezone():
zones = time.tzname
if ncpa.__SYSTEM__ == "nt":
if __SYSTEM__ == "nt":
zones = [x for x in zones]
return zones, ""

Expand All @@ -184,7 +186,7 @@ def get_system_node():
sys_machine = RunnableNode("machine", method=lambda: (platform.uname()[4], ""))
sys_processor = RunnableNode("processor", method=lambda: (platform.uname()[5], ""))
sys_uptime = RunnableNode("uptime", method=get_uptime)
sys_agent = RunnableNode("agent_version", method=lambda: (ncpa.__VERSION__, ""))
sys_agent = RunnableNode("agent_version", method=lambda: (__VERSION__, ""))
sys_time = RunnableNode("time", method=lambda: (time.time(), ""))
sys_timezone = RunnableNode("timezone", method=get_timezone)
return ParentNode(
Expand Down Expand Up @@ -421,7 +423,7 @@ def get_root_node(config):

children = [cpu, memory, disk, interface, plugins, user, system, service, process]

if ncpa.__SYSTEM__ == "nt":
if __SYSTEM__ == "nt":
for importable in importables:
try:
relative_name = "listener." + importable
Expand Down
Loading

0 comments on commit c5cd0b7

Please sign in to comment.