Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Global ACL for VPCs #7150

Merged
merged 17 commits into from Nov 30, 2023
Merged

Global ACL for VPCs #7150

merged 17 commits into from Nov 30, 2023

Conversation

BryanMLima
Copy link
Contributor

Description

Currently, ACS allows the creation of ACLs for VPCs; however, it is required to create an ACL per VPC. Therefore, even if multiple VPCs had the same ACL rules, it would be required to create multiple ACLs. This PR aims to change this behavior as to be able to create global ACLs which will be available to all VPCs, similar to the default ones: default_allow and default_deny.

This PR removed the requirement of the parameter vpcId of the createNetworkACLList; therefore, if there is not a VPC ID in the parameter call, then it is a global ACL. It is important to note that only root admins can manipulate these global ACLs, but anyone can use them.

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

How Has This Been Tested?

I performed the following tests in a local lab:

T0 - createNetworkAclList

I created a global ACL with a root admin account.
(lab-bryan) 🐱 > create networkacllist name=teste-2-global description=testeglobal2 fordisplay=true 
{
  "networkacllist": {
    "description": "testeglobal2",
    "fordisplay": true,
    "id": "fb7cadc2-8209-4a5f-974f-58a7db6b1115",
    "name": "teste-2-global"
  }
}

I tried to create a global ACL with a domain admin account.

(lab-bryan-domain) 🐱 > create networkacllist name=teste-2-global description=testeglobal2 fordisplay=true 
🙈 Error: (HTTP 531, error code 4365) Only Root Admin can create global ACLs.

Expected? Yes

T1 - replaceNetworkaclList

I replaced the ACL for a network as root admin

(lab-bryan) 🐱 > replace networkacllist networkid=b539e342-e625-4cae-bc1f-9115f9f8f734 aclid=fb7cadc2-8209-4a5f-974f-58a7db6b1115
{
  "success": true
}

I replaced the ACL for a network as domain admin

(lab-bryan-domain) 🐱 > replace networkacllist networkid=b539e342-e625-4cae-bc1f-9115f9f8f734 aclid=fb7cadc2-8209-4a5f-974f-58a7db6b1115
{
  "success": true
}

Expected? Yes

T2 - createNetworkAcl

I created an ACL rule as root admin

(lab-bryan) 🐱 > create networkacl aclid=fb7cadc2-8209-4a5f-974f-58a7db6b1115 protocol=all action=allow cidrlist=0.0.0.0/0
{
  "networkacl": {
    "aclid": "fb7cadc2-8209-4a5f-974f-58a7db6b1115",
    "aclname": "teste-2-global",
    "action": "Allow",
    "cidrlist": "0.0.0.0/0",
    "fordisplay": true,
    "id": "acafd739-4dc2-472b-b571-2fed43502150",
    "number": 1,
    "protocol": "all",
    "state": "Active",
    "tags": [],
    "traffictype": "Ingress"
  }
}

I tried to create an ACL rule as a domain admin

(lab-bryan-domain) 🐱 > create networkacl aclid=fb7cadc2-8209-4a5f-974f-58a7db6b1115 protocol=all action=allow cidrlist=0.0.0.0/0
🙈 Error: (HTTP 531, error code 4365) Only Root Admins can create rules for a global ACL.

Expected? Yes

T3 - deleteNetworkAcl

I deleted an ACL rule as root admin

(lab-bryan) 🐱 > delete networkacl id=acafd739-4dc2-472b-b571-2fed43502150 
{
  "success": true
}

I tried to delete an ACL rule as domain admin

(lab-bryan-domain) 🐱 > delete networkacl id=acafd739-4dc2-472b-b571-2fed43502150 
{
  "accountid": "a522847f-936d-4347-8603-9a959df2128c",
  "cmd": "org.apache.cloudstack.api.command.user.network.DeleteNetworkACLCmd",
  "completed": "2023-01-26T14:52:53+0000",
  "created": "2023-01-26T14:52:53+0000",
  "jobid": "6785543c-d729-4725-99a0-f01eace2963a",
  "jobprocstatus": 0,
  "jobresult": {
    "errorcode": 530,
    "errortext": "Only Root Admin can delete global ACL rules."
  },
  "jobresultcode": 530,
  "jobresulttype": "object",
  "jobstatus": 2,
  "userid": "b7dcc0af-e83d-4290-bfcd-39b69abb5a0e"
}
🙈 Error: async API failed for job 6785543c-d729-4725-99a0-f01eace2963a

Expected? Yes

T4 - moveNetworkAclItem

I tried to move an ACL rule as root admin, I used a screenshot as CMK has a problem to decode the response of API `moveNetworkAclItem` (not caused by this PR, this faulty behavior was already like this).

image

I tried to move an ACL rule as domain admin
image

Expected? Yes

T5 - updateNetworkAclItem

I updated an ACL rule as root admin

(lab-bryan) 🐱 > update networkaclitem id=97c39f8e-f800-4fe5-add2-8261148ecd46 cidrlist=0.0.0.0/0,192.168.0.1/8
{
  "networkacl": {
    "aclid": "fb7cadc2-8209-4a5f-974f-58a7db6b1115",
    "aclname": "teste-2-global",
    "action": "Allow",
    "cidrlist": "0.0.0.0/0,192.168.0.1/8",
    "fordisplay": true,
    "id": "97c39f8e-f800-4fe5-add2-8261148ecd46",
    "number": 3,
    "protocol": "all",
    "state": "Add",
    "tags": [],
    "traffictype": "Ingress"
  }
}

I tried to update an ACL rule as domain Admin

(lab-bryan-domain) 🐱 > update networkaclitem id=97c39f8e-f800-4fe5-add2-8261148ecd46 cidrlist=0.0.0.0/0,192.168.0.1/8
{
  "accountid": "a522847f-936d-4347-8603-9a959df2128c",
  "cmd": "org.apache.cloudstack.api.command.user.network.UpdateNetworkACLItemCmd",
  "completed": "2023-01-26T15:09:38+0000",
  "created": "2023-01-26T15:09:38+0000",
  "jobid": "996a0316-1529-47a5-b8e9-1988575a8088",
  "jobprocstatus": 0,
  "jobresult": {
    "errorcode": 530,
    "errortext": "Only Root Admins can update global ACLs."
  },
  "jobresultcode": 530,
  "jobresulttype": "object",
  "jobstatus": 2,
  "userid": "b7dcc0af-e83d-4290-bfcd-39b69abb5a0e"
}
🙈 Error: async API failed for job 996a0316-1529-47a5-b8e9-1988575a8088

Expected? Yes

T6 - deleteNetworkAclList

I deleted an ACL rule as root admin

(lab-bryan) 🐱 > delete networkacllist id=fb7cadc2-8209-4a5f-974f-58a7db6b1115 
{
  "success": true
}

I tried to delete an ACL rule as domain admin


(lab-bryan-domain) 🐱 > delete networkacllist id=fb7cadc2-8209-4a5f-974f-58a7db6b1115 
{
  "accountid": "a522847f-936d-4347-8603-9a959df2128c",
  "cmd": "org.apache.cloudstack.api.command.user.network.DeleteNetworkACLListCmd",
  "completed": "2023-01-26T15:11:13+0000",
  "created": "2023-01-26T15:11:13+0000",
  "jobid": "40a2047d-8d23-4785-b53a-5baee1da9bf7",
  "jobprocstatus": 0,
  "jobresult": {
    "errorcode": 530,
    "errortext": "Only Root Admin can delete global ACLs."
  },
  "jobresultcode": 530,
  "jobresulttype": "object",
  "jobstatus": 2,
  "userid": "b7dcc0af-e83d-4290-bfcd-39b69abb5a0e"
}
🙈 Error: async API failed for job 40a2047d-8d23-4785-b53a-5baee1da9bf7

Expected? Yes


Furthermore, I created three VPCs and three VMs, each one attach to a tier of a VPC. After this, I created a global ACL allowing SSH connections. I tested the ssh command with all public IPs, previously created, of the VMs. All worked as expected. Furthermore, I changed the rule to deny any access to por 22 for the TCP protocol. Similarly, I tried to used ssh with all 3 public IPs and all failed.

@codecov
Copy link

codecov bot commented Jan 31, 2023

Codecov Report

Attention: 26 lines in your changes are missing coverage. Please review.

Comparison is base (60b399f) 29.12% compared to head (64abcd0) 29.16%.
Report is 5 commits behind head on main.

Files Patch % Lines
...a/com/cloud/network/vpc/NetworkACLServiceImpl.java 68.08% 4 Missing and 11 partials ⚠️
.../command/user/network/CreateNetworkACLListCmd.java 66.66% 1 Missing and 4 partials ⚠️
...ain/java/com/cloud/network/NetworkServiceImpl.java 20.00% 1 Missing and 3 partials ⚠️
...ain/java/com/cloud/network/vpc/VpcManagerImpl.java 50.00% 2 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #7150      +/-   ##
============================================
+ Coverage     29.12%   29.16%   +0.04%     
- Complexity    31030    31084      +54     
============================================
  Files          5193     5193              
  Lines        366322   366348      +26     
  Branches      53558    53557       -1     
============================================
+ Hits         106694   106855     +161     
+ Misses       245013   244832     -181     
- Partials      14615    14661      +46     
Flag Coverage Δ
simulator-marvin-tests 25.10% <48.61%> (+0.05%) ⬆️
uitests 4.49% <ø> (ø)
unit-tests 14.81% <40.35%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@DaanHoogland DaanHoogland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clg overall, will need extensive testing though

Copy link
Member

@rohityadavcloud rohityadavcloud left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the general feature - could you add more marvin/integration tests for this?

@boring-cyborg boring-cyborg bot added component:integration-test Python Warning... Python code Ahead! labels Feb 13, 2023
@BryanMLima
Copy link
Contributor Author

I like the general feature - could you add more marvin/integration tests for this?

@rohityadavcloud I added smoke tests for this feature. Let me know if you think this is enough. I used the manual tests I did as reference; however, there were methods that were not on the Marvin lib base.py file. It looks like this file is auto generated and I did not find any way to add these missing methods for the network ACL rules.

Copy link
Contributor

@DaanHoogland DaanHoogland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the test @BryanMLima , I have some doubts about it, hope you understand

test/integration/smoke/test_global_acls.py Outdated Show resolved Hide resolved
test/integration/smoke/test_global_acls.py Show resolved Hide resolved
test/integration/smoke/test_global_acls.py Show resolved Hide resolved
test/integration/smoke/test_global_acls.py Show resolved Hide resolved
test/integration/smoke/test_global_acls.py Outdated Show resolved Hide resolved
test/integration/smoke/test_global_acls.py Outdated Show resolved Hide resolved
test/integration/smoke/test_global_acls.py Show resolved Hide resolved
test/integration/smoke/test_global_acls.py Show resolved Hide resolved
test/integration/smoke/test_global_acls.py Show resolved Hide resolved
@BryanMLima
Copy link
Contributor Author

@DaanHoogland thanks for the review, I'm not used to the Marvin tests so your suggestions made it clearer to me. Let me know if there is more to change.

@DaanHoogland DaanHoogland self-requested a review February 20, 2023 08:57
@DaanHoogland
Copy link
Contributor

@blueorangutan package

@blueorangutan
Copy link

@DaanHoogland a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@apache apache deleted a comment from blueorangutan Feb 20, 2023
@apache apache deleted a comment from blueorangutan Feb 20, 2023
@apache apache deleted a comment from blueorangutan Feb 20, 2023
@apache apache deleted a comment from blueorangutan Feb 20, 2023
@apache apache deleted a comment from blueorangutan Feb 20, 2023
@blueorangutan
Copy link

Packaging result: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 5613

@DaanHoogland
Copy link
Contributor

@blueorangutan test

@blueorangutan
Copy link

@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan
Copy link

Trillian test result (tid-6209)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 40913 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7150-t6209-kvm-centos7.zip
Smoke tests completed. 109 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File

Copy link
Contributor

@DaanHoogland DaanHoogland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clgtm

@sonarcloud
Copy link

sonarcloud bot commented Mar 17, 2023

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 1 Code Smell

27.0% 27.0% Coverage
0.0% 0.0% Duplication

@BryanMLima
Copy link
Contributor Author

@DaanHoogland could you trigger the integration tests again? The error seems unrelated to this PR.

@shwstppr
Copy link
Contributor

@blueorangutan package

@blueorangutan
Copy link

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 7825

@shwstppr
Copy link
Contributor

@blueorangutan test

@blueorangutan
Copy link

@shwstppr a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan
Copy link

[SF] Trillian test result (tid-8390)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 52760 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7150-t8390-kvm-centos7.zip
Smoke tests completed. 109 look OK, 10 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_list_vms_metrics_admin Error 0.19 test_metrics_api.py
test_list_vms_metrics_history Error 0.13 test_metrics_api.py
test_list_vms_metrics_user Error 0.14 test_metrics_api.py
test_list_volumes_metrics_history Error 0.14 test_metrics_api.py
test_deploy_vm_from_iso Error 0.00 test_deploy_vm_iso.py
ContextSuite context=TestIsolatedNetworksPasswdServer>:setup Error 0.00 test_password_server.py
test_01_create_iso_with_checksum_sha1 Error 66.50 test_iso.py
test_02_create_iso_with_checksum_sha256 Error 66.49 test_iso.py
test_03_create_iso_with_checksum_md5 Error 66.50 test_iso.py
test_04_create_iso_with_no_checksum Error 66.51 test_iso.py
ContextSuite context=TestSharedNetwork>:setup Error 52.84 test_network.py
ContextSuite context=TestDeployVirtioSCSIVM>:teardown Error 309.77 test_deploy_virtio_scsi_vm.py
ContextSuite context=TestInternalLb>:setup Error 0.00 test_internal_lb.py
ContextSuite context=TestRouterDHCPHosts>:setup Error 0.00 test_router_dhcphosts.py
ContextSuite context=TestRouterDHCPOpts>:setup Error 0.00 test_router_dhcphosts.py
ContextSuite context=TestSnapshotStandaloneBackup>:setup Error 590.04 test_snapshots.py
test_01_create_template Error 11.48 test_templates.py
test_CreateTemplateWithDuplicateName Error 17.77 test_templates.py
test_02_create_template_with_checksum_sha1 Error 65.51 test_templates.py
test_03_create_template_with_checksum_sha256 Error 65.52 test_templates.py
test_05_create_template_with_no_checksum Error 135.95 test_templates.py
test_03_delete_template Error 1.09 test_templates.py
test_04_extract_template Error 1.07 test_templates.py

@shwstppr
Copy link
Contributor

@blueorangutan test

@blueorangutan
Copy link

@shwstppr a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan
Copy link

[SF] Trillian test result (tid-8391)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 58616 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7150-t8391-kvm-centos7.zip
Smoke tests completed. 117 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_08_migrate_vm Error 0.05 test_vm_life_cycle.py
test_02_redundant_VPC_default_routes Failure 380.19 test_vpc_redundant.py
test_05_rvpc_multi_tiers Failure 459.34 test_vpc_redundant.py
test_05_rvpc_multi_tiers Error 459.35 test_vpc_redundant.py

@DaanHoogland
Copy link
Contributor

[SF] Trillian test result (tid-8391) Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7 Total time taken: 58616 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7150-t8391-kvm-centos7.zip Smoke tests completed. 117 look OK, 2 have errors, 0 did not run Only failed and skipped tests results shown below:
Test Result Time (s) Test File
test_08_migrate_vm Error 0.05 test_vm_life_cycle.py
test_02_redundant_VPC_default_routes Failure 380.19 test_vpc_redundant.py
test_05_rvpc_multi_tiers Failure 459.34 test_vpc_redundant.py
test_05_rvpc_multi_tiers Error 459.35 test_vpc_redundant.py

@BryanMLima can you look to make sure the redundant vpc test failures are not a problem? Did you test redundant VPCs?

Copy link

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

@shwstppr
Copy link
Contributor

@BryanMLima any update on this? Also, merge conflicts here

@BryanMLima
Copy link
Contributor Author

[SF] Trillian test result (tid-8391) Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7 Total time taken: 58616 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7150-t8391-kvm-centos7.zip Smoke tests completed. 117 look OK, 2 have errors, 0 did not run Only failed and skipped tests results shown below:
Test Result Time (s) Test File
test_08_migrate_vm Error 0.05 test_vm_life_cycle.py
test_02_redundant_VPC_default_routes Failure 380.19 test_vpc_redundant.py
test_05_rvpc_multi_tiers Failure 459.34 test_vpc_redundant.py
test_05_rvpc_multi_tiers Error 459.35 test_vpc_redundant.py

@BryanMLima can you look to make sure the redundant vpc test failures are not a problem? Did you test redundant VPCs?

The Trillian tests are related to connection issues; this PR's changes should not impact this test, as it is not using global ACLs.

@BryanMLima
Copy link
Contributor Author

@blueorangutan package

@blueorangutan
Copy link

@BryanMLima a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@shwstppr
Copy link
Contributor

@blueorangutan package

@blueorangutan
Copy link

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 7863

@DaanHoogland
Copy link
Contributor

@blueorangutan test

@blueorangutan
Copy link

@DaanHoogland a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@rajujith rajujith removed their assignment Nov 29, 2023
@shwstppr
Copy link
Contributor

Test results from backend,

02:17:10 Smoke tests completed. 119 look OK, 0 have errors, 0 did not run
02:17:10 Only failed and skipped tests results shown below:
02:17:10 
02:17:10 
02:17:10 Test | Result | Time (s) | Test File
02:17:10 --- | --- | --- | ---

@shwstppr
Copy link
Contributor

@DaanHoogland cc @borisstoyanov do we need any further testing or it good to merge?

@DaanHoogland
Copy link
Contributor

There was a test added to the smoke test suites, so I think we are done here; merging

@DaanHoogland DaanHoogland merged commit cb62ce6 into apache:main Nov 30, 2023
26 checks passed
dhslove pushed a commit to ablecloud-team/ablestack-cloud that referenced this pull request Dec 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

9 participants