Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for running as an API server. #35

Merged
merged 19 commits into from
Dec 24, 2021

Commits on Nov 30, 2021

  1. Created separate file for fetch status functions.

    Cleaned up imports.
    
    Signed-off-by: KoleBarnes <kbarnes261@gmail.com>
    KoleBarnes committed Nov 30, 2021
    Configuration menu
    Copy the full SHA
    47e5212 View commit details
    Browse the repository at this point in the history
  2. * Fix pipe to file issue on Windows

    - Only use `winpty` when running interactively on Windows.
    - This fixes the `stdout is not a tty` issue seen on Windows when piping the output of the script to a file.
      - For example; `./run.sh --net smn --status > status.json`
    
    Removed all trailing white-space from network names.
    Added network name and consistent timestamp to node summary.
    
    - Add the network name to the node summary so monitoring results can be filtered and grouped by network.
    - Ensure there is a consistent (UTC) timestamp on the results that can be used as the time series for the collected data.  This ensures events can be compared across nodes regardless of any clock skew on the nodes themselves.
    
    Updated run script.
    
    - Add support for specifying the number of samples to collect via an environment variable (default 1); `SAMPLES`.
      - Samples are taken one minute apart as the Nodes refresh their own data every minute.
    - Add support for logging results directly to a timestamped file via an environment variable; `LOG`.
    
    Full Example:
    `SAMPLES=20 LOG=/c/tick-sandbox/telegraf/indy-node-monitor/BuilderNet/BuilderNet-AllNodes.json ./run.sh --net sbn --seed <seed>`
    - Runs the `fetch_status` process 20 times (once every minute) producing a set of files in `/c/tick-sandbox/telegraf/indy-node-monitor/BuilderNet/` with a timestamped filename that looks like this; `BuilderNet-AllNodes_2021-05-23_06-48-20.json`.
    
    Signed-off-by: KoleBarnes <kbarnes261@gmail.com>
    KoleBarnes committed Nov 30, 2021
    Configuration menu
    Copy the full SHA
    8f48087 View commit details
    Browse the repository at this point in the history
  3. Added requirements.txt

    Signed-off-by: KoleBarnes <kbarnes261@gmail.com>
    KoleBarnes committed Nov 30, 2021
    Configuration menu
    Copy the full SHA
    87ff7ff View commit details
    Browse the repository at this point in the history

Commits on Dec 20, 2021

  1. General refactoring.

    - Added function init_network_args_test. This function returns a tuple with the network info.
    - Added more logging to pool connection.
    - Added asyncio to requirements.txt
    - Minor clean-up.
    - Removed Commented code.
    
    Signed-off-by: KoleBarnes <kbarnes261@gmail.com>
    KoleBarnes committed Dec 20, 2021
    Configuration menu
    Copy the full SHA
    8744ade View commit details
    Browse the repository at this point in the history
  2. Created rest_api.py

    fetch_status.py
     - Added parser arg to start Fast API
     - Moved default for genesis_path to fetch_status_library.py in init_network_args so it can be used by the command line and REST API
    
    fetch_status_library.py
     - Added return to fetch_status
     - Added default for genesis_path
    
    run.sh
     - Opened port
    
    Added requirements
    Cleaned up imports.
    Fixed fetch_status_library.py verbose not being enabled when using REST API.
    
    Signed-off-by: KoleBarnes <kbarnes261@gmail.com>
    KoleBarnes committed Dec 20, 2021
    Configuration menu
    Copy the full SHA
    1cd82ea View commit details
    Browse the repository at this point in the history
  3. Sorted fetch_status.py

    Created did helper function.
    Created separate instances for plug-ins in REST API. One to set default args and one for a separate user instance.
    Added and fixed logging.
    Fixed results being printed to console when running REST API.
    
    Signed-off-by: KoleBarnes <kbarnes261@gmail.com>
    KoleBarnes committed Dec 20, 2021
    Configuration menu
    Copy the full SHA
    5bbb1ae View commit details
    Browse the repository at this point in the history
  4. Node Monitor Refactoring

     - Fetch status renamed to main.
     - Fetch status library renamed to fetch status.
     - Fetching of pool connection and fetch status are now classes.
     - Created util file for helper functions.
     - Fixed and added logging.
     - Removed some imports and requirements.
    
    Signed-off-by: KoleBarnes <kbarnes261@gmail.com>
    KoleBarnes committed Dec 20, 2021
    Configuration menu
    Copy the full SHA
    8c9d0ec View commit details
    Browse the repository at this point in the history
  5. Added caching of pool connections with async thread lock to avoid mul…

    …ti request errors when running REST API.
    
    Added Singleton meta class to Pool class to have a global pool cache.
    Added refactoring Notes.
    
    Signed-off-by: KoleBarnes <kbarnes261@gmail.com>
    KoleBarnes committed Dec 20, 2021
    Configuration menu
    Copy the full SHA
    93d2fa5 View commit details
    Browse the repository at this point in the history
  6. FetchStatus Class refactor.

    Fixed import in pool.py
    
    Signed-off-by: KoleBarnes <kbarnes261@gmail.com>
    KoleBarnes committed Dec 20, 2021
    Configuration menu
    Copy the full SHA
    3a3a334 View commit details
    Browse the repository at this point in the history
  7. Fetch status class has been made a singleton.

    Singleton meta class now has its own file.
    Changed string manipulation in fetch pool connection for something better.
    Added genesis path and url to fetch status class.
    Added comments.
    
    Signed-off-by: KoleBarnes <kbarnes261@gmail.com>
    KoleBarnes committed Dec 20, 2021
    Configuration menu
    Copy the full SHA
    65cc9ef View commit details
    Browse the repository at this point in the history
  8. Refactor PoolCollection

    - Pull the known network related code out into it's own class; Networks
    - Refactor network and pool resolution to use network ids.
    - Update dependent code.  Testing is not complete
    
    Signed-off-by: Wade Barnes <wade@neoterictech.ca>
    WadeBarnes authored and KoleBarnes committed Dec 20, 2021
    Configuration menu
    Copy the full SHA
    f5808ba View commit details
    Browse the repository at this point in the history
  9. Made seed_as_bytes an instance method in DidKey.

    Renamed Networks.get_names to Networks.get_ids, and Networks.get_all to Networks.get_details.
    Renamed status_test to node_info and cleaned up global variables in rest_api.py.
    Fixed comments and logs.
    
    Signed-off-by: KoleBarnes <kbarnes261@gmail.com>
    KoleBarnes committed Dec 20, 2021
    Configuration menu
    Copy the full SHA
    c659c67 View commit details
    Browse the repository at this point in the history
  10. API refactoring.

    - Created an exception for when a requested node is not found on the requested network.
    - Refactored Pool.resolve to deal with genesis url's using the new Networks class.
    - Cleaned up pool cacheing.
    - Clean-up variable names and logging messages.
    - Add redirect to API, from `/` to `/docs`.
    
    Signed-off-by: Wade Barnes <wade@neoterictech.ca>
    Signed-off-by: KoleBarnes <kbarnes261@gmail.com>
    KoleBarnes committed Dec 20, 2021
    Configuration menu
    Copy the full SHA
    86128a4 View commit details
    Browse the repository at this point in the history
  11. Moved Networks class into its own file.

    - Removed import argparse ArgumentError.
    
    Signed-off-by: KoleBarnes <kbarnes261@gmail.com>
    KoleBarnes committed Dec 20, 2021
    Configuration menu
    Copy the full SHA
    d449280 View commit details
    Browse the repository at this point in the history
  12. Added debug and production HTTP servers.

     - uvicorn for debug.
     - gunicorn for production.
    
    Signed-off-by: KoleBarnes <kbarnes261@gmail.com>
    KoleBarnes committed Dec 20, 2021
    Configuration menu
    Copy the full SHA
    77ae744 View commit details
    Browse the repository at this point in the history
  13. Added documentation on how to use the REST API.

    Added code to check if a seed is being used while running the REST API. If so it will exit.
    Changed App name and description.
    
    Signed-off-by: KoleBarnes <kbarnes261@gmail.com>
    KoleBarnes committed Dec 20, 2021
    Configuration menu
    Copy the full SHA
    0247fe9 View commit details
    Browse the repository at this point in the history
  14. Add node time stamp.

    Add API documentation.
    
    Signed-off-by: KoleBarnes <kbarnes261@gmail.com>
    KoleBarnes committed Dec 20, 2021
    Configuration menu
    Copy the full SHA
    924ea84 View commit details
    Browse the repository at this point in the history
  15. Fix merge issues with Dockerfile.

    Signed-off-by: KoleBarnes <kbarnes261@gmail.com>
    KoleBarnes committed Dec 20, 2021
    Configuration menu
    Copy the full SHA
    418de4f View commit details
    Browse the repository at this point in the history
  16. Added dynamically generating list that is used to select the availabl…

    …e Networks.
    
    Signed-off-by: KoleBarnes <kbarnes261@gmail.com>
    KoleBarnes committed Dec 20, 2021
    Configuration menu
    Copy the full SHA
    01acb3c View commit details
    Browse the repository at this point in the history