Consolidates All NF Structs by koolzz · Pull Request #117 · sdnfv/openNetVM

koolzz · 2019-05-16T15:09:49Z

This PR merges together onvm_nf_info and onvm_nf structs, into a single onvm_nf struct with internal variable groupings for readability, also renames key structs.

Summary:

The old onvm_nf_info is re purposed as onvm_nf_init_cfg a struct used to initialize the NF with the manager. This struct is passed to onvm_mgr and is then freed when initialization is complete.
In attempt to merge these structs the usage of onvm_nf_info has been removed from all internal and external API calls. Excluding the startup sequence.
The struct onvm_nf_context was renamed to onvm_nf_local_ctx
Added a nf_function_table struct that will hold all the NF defined callback functions.
Changed all NF facing API calls to pass onvm_nf_local_ctx instead of onvm_nf as NFs might need access to the keep_running variable.

Usage:

This PR includes
Resolves issues
Breaking API changes	👍
Internal API changes	👍
Usability improvements	👍
Bug fixes
New functionality
New NF/onvm_mgr args
Changes to starting NFs
Dependency updates
Web stats updates

Merging notes:

Dependencies: None

TODO before merging :

PR is ready for review

Test Plan:

Regular testing + pktgen

Review:

Please review, release is near

This code introduces **EXPERIMENTAL** support for NFs that can efficiently run on **shared** CPUs. NFs wait on semaphores when idle and are signaled by the manager when new packets arrive. Once the NF is in wake state, no additional notifications will be sent until it goes back to sleep. All code changes are featurized using INTERRUPT_SEM macro. See the ONVM README.md file for more information and warnings. Commit log: * merges rebased to master with sdnfv#147 * reverting num_clients update. * replace tabs with spaces * Readme describing experimental shared CPU support * Add usage and known limitations for shared CPU code

…fv#185) Fixes a race condition in NF instance ID assignment. Before, if two NFs were started at the same time, they would be assigned the same instance ID. This change modifies logic in `onvm_nf_next_instance_id` to avoid assigning the same ID to two starting NFs. Commit log: * Bug Fix: Fixes instance ID assignment race condition - Fixes sdnfv#178 - Changed incrementation of next_instance_id in onvm_mgr/onvm_nf.c such that the ID can't be reused before an NF becomes active.

…ith onvm_nf struct

…nd flow_tracker). Also did some small style changes in some other documentation in onvm_nflib.h

Conflicts: README.md onvm/onvm_mgr/onvm_init.c onvm/onvm_mgr/onvm_init.h onvm/onvm_mgr/onvm_mgr.h onvm/onvm_mgr/onvm_nf.c onvm/onvm_mgr/onvm_stats.c onvm/onvm_nflib/onvm_common.h onvm/onvm_nflib/onvm_nflib.c onvm/onvm_nflib/onvm_nflib.h

…experimental/shared_cpu

- Some stats were unused and broken(?) nuked them - added stats that are possible usefull to stdout (slightly ugly)

…multithread Conflicts: examples/speed_tester/speed_tester.c

- Keeping onvm_nf_info small, moved vars to onvm_nf - Added a callback function for the scaling child thread - Cleaned up code

- Adds ability to scale NFs with advanced rings - Adds a setup function for NFs - More docs and cleanup

- Moving nf_info as passed arg to packet_handler -> allows for spawned NF to access their own nf_info - Launching new NFs now accepts arguments for packet_handler function, setup function, callback function, advanced rings function - Add a data pointer to onvm_nflib_info to maintain state - Working on a scaling example (currently testing code, not in usable state) will showcase different features of spawning NFs

- This is a major change, a lot of nflib initlization is refactored to allow spawned NFs to launch with their own service id - There is a bug when the spawn a lot of new NFs we can no longer retrieve the nf_info from nf_info_mp, this needs to be fixed as it breaks stuff

- Fixed the nf_info bug where the mempool would get messed up as children weren't cleared properly. - Updated the example scaling NF - Removed most of the debugging code

Cleaned up messy bits, replacing macro branches with if statements, encapsulating code in functions, and getting rid of unnecessary code.

- Style fixes - Removing debug code - Making code prettier - Adding better docs - Implementing requested changes

koolzz · 2019-05-29T04:21:24Z

@onvm only speed tester works will you work

onvm · 2019-05-29T04:21:31Z

@onvm only speed tester works will you work

CI Message

Your results will arrive shortly

onvm · 2019-05-29T04:27:45Z

@onvm only speed tester works will you work

CI Message

Error: ERROR: Failed to fetch results from nimbnode30

Linter Output

onvm/onvm_nflib/onvm_nflib.c:465: Lines should be <= 120 characters long [whitespace/line_length] [5]
Total errors found: 1

koolzz · 2019-05-29T04:28:34Z

I guess thats a no

koolzz · 2019-05-30T21:15:31Z

@onvm do your thing

onvm · 2019-05-30T21:15:39Z

@onvm do your thing

CI Message

Your results will arrive shortly

onvm · 2019-05-30T21:22:43Z

@onvm do your thing

CI Message

Error: ERROR: Failed to analyze results from nimbnode30

Linter Output

examples/aes_decrypt/aesdecrypt.c:173: Lines should be <= 120 characters long [whitespace/line_length] [5]
Total errors found: 1
examples/aes_encrypt/aesencrypt.c:173: Lines should be <= 120 characters long [whitespace/line_length] [5]
Total errors found: 1
examples/arp_response/arp_response.c:266: Lines should be <= 120 characters long [whitespace/line_length] [5]
Total errors found: 1
examples/basic_monitor/monitor.c:161: Lines should be <= 120 characters long [whitespace/line_length] [5]
Total errors found: 1
examples/bridge/bridge.c:142: Lines should be <= 120 characters long [whitespace/line_length] [5]
Total errors found: 1
examples/firewall/firewall.c:198: Lines should be <= 120 characters long [whitespace/line_length] [5]
Total errors found: 1
examples/flow_table/flow_table.c:208: Lines should be <= 120 characters long [whitespace/line_length] [5]
Total errors found: 1
examples/flow_tracker/flow_tracker.c:292: Lines should be <= 120 characters long [whitespace/line_length] [5]
Total errors found: 1
examples/load_balancer/load_balancer.c:488: Lines should be <= 120 characters long [whitespace/line_length] [5]
Total errors found: 1
examples/load_generator/load_generator.c:233: Lines should be <= 120 characters long [whitespace/line_length] [5]
Total errors found: 1
examples/nf_router/nf_router.c:212: Lines should be <= 120 characters long [whitespace/line_length] [5]
Total errors found: 1
examples/scaling_example/scaling.c:220: Lines should be <= 120 characters long [whitespace/line_length] [5]
Total errors found: 1
examples/simple_forward/forward.c:154: Lines should be <= 120 characters long [whitespace/line_length] [5]
Total errors found: 1
examples/speed_tester/speed_tester.c:279: Lines should be <= 120 characters long [whitespace/line_length] [5]
examples/speed_tester/speed_tester.c:517: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4]
Total errors found: 2
examples/test_flow_dir/test_flow_dir.c:155: Lines should be <= 120 characters long [whitespace/line_length] [5]
Total errors found: 1
onvm/onvm_nflib/onvm_nflib.c:453: Lines should be <= 120 characters long [whitespace/line_length] [5]
onvm/onvm_nflib/onvm_nflib.c:466: Lines should be <= 120 characters long [whitespace/line_length] [5]
Total errors found: 2

koolzz · 2019-05-30T21:40:35Z

@onvm you happy

onvm · 2019-05-30T21:40:43Z

@onvm you happy

CI Message

Your results will arrive shortly

onvm

@onvm you happy

CI Message

Run successful see results:
✔️ PR submitted to develop branch
✔️ Speed tester performance check passed
❌ Linter Failed (please fix style errors)

[Results from nimbnode30]

Median TX pps for Speed Tester: 39169623
Performance rating - 111.91% (compared to 35000000 average)

Linter Output

onvm/onvm_nflib/onvm_nflib.c:467: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4]
Total errors found: 1

koolzz · 2019-05-30T21:50:13Z

@onvm lets do 3 out of 3

onvm · 2019-05-30T21:50:21Z

@onvm lets do 3 out of 3

CI Message

Your results will arrive shortly

onvm

@onvm lets do 3 out of 3

CI Message

Run successful see results:
✔️ PR submitted to develop branch
✔️ Speed tester performance check passed
✔️ Linter passed

[Results from nimbnode30]

Median TX pps for Speed Tester: 39326084
Performance rating - 112.36% (compared to 35000000 average)

This PR merges together onvm_nf_info and onvm_nf structs, into a single onvm_nf struct with internal variable groupings for readability, also renames key structs. Summary: The old `onvm_nf_info` is re purposed as `onvm_nf_init_cfg` a struct used to initialize the NF with the manager. This struct is passed to onvm_mgr and is then freed when initialization is complete. In attempt to merge these structs the usage of `onvm_nf_info` has been removed from all internal and external API calls. Excluding the startup sequence. The struct `onvm_nf_context` was renamed to `onvm_nf_local_ctx` Added a `nf_function_table` struct that will hold all the NF defined callback functions. Changed all NF facing API calls to pass `onvm_nf_local_ctx` instead of onvm_nf as NFs might need access to the keep_running variable. Commit log: * Remove bootstrap struct from context * Nuke some extra code * Fix firewall API * Remove unused globals in example NFs * Fix style nit * Fix double define * Global rename/regrouping of onvm_nf struct * Restructure scaling logic * Rename nf_context->nf_local_ctx * Further cleanup of scaling API * Update method name * Rename callbackhandlerfunc -> callback_func * Fix linter * Remove trailing whitespace * Fix docs * Fix more docs * Free fix, code cleanup * Set pointer to NULL on cleanup * Testing function table struct * Massive rename, API fixes, remove circular refr * Don't mind me fixin whitespace * Fix speed tester double func_table init bug * Enforce 120 char limit * Rem trailing whitespace * Rem extra api call in speed_tester * Fix docs * Huuuuge linter fix, well tested

sameergk and others added 30 commits February 13, 2017 13:26

partial refactoring of api w accordance of nf_info

df6fbb7

onvm mgr compiles

e1ffe07

flow table compiles -- all examples compile

dee954c

changed onvm_get_rx and onvm_get_tx to get rx_q and tx_q associated w…

fb99a9c

…ith onvm_nf struct

resolve merge conflicts

0bff75a

Did requested changes. Also updated newer NFs (such as arp_response a…

8886eaf

…nd flow_tracker). Also did some small style changes in some other documentation in onvm_nflib.h

Corrected some incorrect references to nflib functions in NF_Dev.md

8a72816

removed typing error

a1dccf2

Update shared_cpu branch

315722a

Conflicts: README.md onvm/onvm_mgr/onvm_init.c onvm/onvm_mgr/onvm_init.h onvm/onvm_mgr/onvm_mgr.h onvm/onvm_mgr/onvm_nf.c onvm/onvm_mgr/onvm_stats.c onvm/onvm_nflib/onvm_common.h onvm/onvm_nflib/onvm_nflib.c onvm/onvm_nflib/onvm_nflib.h

Minor cleanup

71a07dc

Merge branch 'master' of https://github.com/sdnfv/openNetVM-dev into …

15c3d11

…experimental/shared_cpu

Stats cleanup

4d83219

- Some stats were unused and broken(?) nuked them - added stats that are possible usefull to stdout (slightly ugly)

Merge branch 'master' of https://github.com/sdnfv/openNetVM-dev into …

98e6b39

…multithread Conflicts: examples/speed_tester/speed_tester.c

Add functionality to scale same NF instances

ab19825

Fixed a few global vars for multithreading to work

d756709

Code clean up, comments and style fixes

262a121

Added scale return value

a51b8ef

Fixed nf_mode for running advanced rings

5d93860

Refactored the onvm_nf/onvm_nf_info passing

0e0cc5b

- Keeping onvm_nf_info small, moved vars to onvm_nf - Added a callback function for the scaling child thread - Cleaned up code

Adv rings NF scaling, setup func for spawned NFS

5485492

- Adds ability to scale NFs with advanced rings - Adds a setup function for NFs - More docs and cleanup

Fixed build errors, changed NF typedef func names

ebf7f8e

Slightly working?

bdd7a28

Fixing bugs, cleaning up code, updating example NF

7539ca9

- Fixed the nf_info bug where the mempool would get messed up as children weren't cleared properly. - Updated the example scaling NF - Removed most of the debugging code

Clean up of shared cpu branch

ce693a8

Cleaned up messy bits, replacing macro branches with if statements, encapsulating code in functions, and getting rid of unnecessary code.

The code cleaning commit has arrived

96b9c90

- Style fixes - Removing debug code - Making code prettier - Adding better docs - Implementing requested changes

Minor cleanup/whitespace fixes

af20be3

koolzz added 3 commits May 28, 2019 10:28

Free fix, code cleanup

2bf5c69

Set pointer to NULL on cleanup

de28716

Testing function table struct

55b65f6

koolzz commented May 30, 2019

View reviewed changes

Comment thread onvm/onvm_nflib/onvm_common.h

koolzz added 2 commits May 30, 2019 15:09

Massive rename, API fixes, remove circular refr

0f10977

Don't mind me fixin whitespace

a320b89

koolzz added 3 commits May 30, 2019 16:34

Fix speed tester double func_table init bug

6c41aee

Enforce 120 char limit

86aa02a

Rem trailing whitespace

3774f6a

Rem extra api call in speed_tester

ffbca4f

onvm suggested changes May 30, 2019

View reviewed changes

koolzz added 2 commits May 30, 2019 16:48

Fix docs

47753cb

Huuuuge linter fix, well tested

d52a60b

onvm approved these changes May 30, 2019

View reviewed changes

nks5295 approved these changes May 31, 2019

View reviewed changes

nks5295 changed the title ~~Restructure main onvm_nf structs~~ Consolidates All NF Structs May 31, 2019

nks5295 merged commit ac78a29 into sdnfv:develop May 31, 2019

koolzz deleted the restructure_main_structs branch June 7, 2019 23:16

Conversation

koolzz commented May 16, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary:

Merging notes:

Test Plan:

Review:

Uh oh!

koolzz commented May 29, 2019

Uh oh!

onvm commented May 29, 2019

CI Message

Uh oh!

onvm commented May 29, 2019

CI Message

Linter Output

Uh oh!

koolzz commented May 29, 2019

Uh oh!

Uh oh!

koolzz commented May 30, 2019

Uh oh!

onvm commented May 30, 2019

CI Message

Uh oh!

onvm commented May 30, 2019

CI Message

Linter Output

Uh oh!

koolzz commented May 30, 2019

Uh oh!

onvm commented May 30, 2019

CI Message

Uh oh!

onvm left a comment

Choose a reason for hiding this comment

CI Message

Linter Output

Uh oh!

koolzz commented May 30, 2019

Uh oh!

onvm commented May 30, 2019

CI Message

Uh oh!

onvm left a comment

Choose a reason for hiding this comment

CI Message

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

koolzz commented May 16, 2019 •

edited

Loading