Translate local RC feeds (exabgp/hist) into ARTEMIS configuration primitives

[vkotronis]

Description of PR

What component(s) does this PR affect?

• Back-End (Database, Microservices, Containers, etc)
• Front-End (Flask, API, etc)
• Docs
• Build System

Configuration, Monitor, ExaBGP taps

Does the PR require changes on other components? If yes, please mark the components:

• Back-End (Database, Microservices, Containers, etc)
• Front-End (Flask, API, etc)
• Docs
• Build System

Related Issue

Resolves #253

Solution

Instruction to monitor (via configuration) to auto-update configuration based on local monitor feed.

Type

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Docs update
• None of the above

Checklist:

• I have read the contributing guide and my code conforms to the guidelines.
• This change requires a change in the documentation.
• I have updated the documentation accordingly.

Need to update https://github.com/FORTH-ICS-INSPIRE/artemis/wiki#receiving-bgp-feed-from-local-routerroute-reflectorbgp-monitor-via-exabgp and https://github.com/FORTH-ICS-INSPIRE/artemis/wiki/Configuration-file to let users know that they can use the autoconf flag for exabgp to auto-generate conf file entries. (after completion, testing and merging, ignore for now).

[slowr]

🏃 Benchmark Results 🏃

• BGP Updates (updates/sec): 703
• Hijacks Detected (hijacks/sec): 32

[vkotronis]

TODO:
[X] For already existing prefixes and asns, re-use them when learning (does not apply to original learn rule functionality)
[X] Make autoconf messaging RPC. The def handle_autoconf_updates(self, message):, should respond only after the conf is written to the file (and the next autoconf update can read it properly).
[X] Turn bgpstreamhist into a class (no global vars)
[X] Make sure that it works even when multiple autoconf streams at the same time (different sources) --> dealt with having single queue and consumer in configuration
[X] Handle withdrawals by removing the associated prefixes and rules (leave ASNs for reference)
[X] Test bgpstreamhist
[X] Test exabgp (GNS3)

[codecov]

Codecov Report

Merging #254 into master will increase coverage by 1.04%.
The diff coverage is 82.11%.

@@            Coverage Diff             @@
##           master     #254      +/-   ##
==========================================
+ Coverage   88.43%   89.48%   +1.04%     
==========================================
  Files           6        6              
  Lines        1652     1864     +212     
==========================================
+ Hits         1461     1668     +207     
- Misses        191      196       +5

Impacted Files	Coverage Δ
backend/core/mitigation.py	96% <100%> (ø)	⬆️
backend/core/database.py	91.38% <100%> (+0.59%)	⬆️
backend/core/detection.py	95.19% <100%> (ø)	⬆️
backend/core/configuration.py	79.52% <81.89%> (+10.27%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fd5babe...040aafd. Read the comment docs.

[slowr]

LGTM 🎉 we can imporve the tester/autoconftester configuration afterwards

[slowr]

🏃 Benchmark Results 🏃

• BGP Updates (updates/sec): 680
• Hijacks Detected (hijacks/sec): 32

[slowr]

🏃 Benchmark Results 🏃

• BGP Updates (updates/sec): 639
• Hijacks Detected (hijacks/sec): 32

[slowr]

🏃 Benchmark Results 🏃

• BGP Updates (updates/sec): 688
• Hijacks Detected (hijacks/sec): 29

[slowr]

🏃 Benchmark Results 🏃

• BGP Updates (updates/sec): 666
• Hijacks Detected (hijacks/sec): 33

[slowr]

🏃 Benchmark Results 🏃

• BGP Updates (updates/sec): 686
• Hijacks Detected (hijacks/sec): 34