Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ovmsupport #9

Closed
wants to merge 4,348 commits into from
Closed

Ovmsupport #9

wants to merge 4,348 commits into from

Conversation

snuf
Copy link

@snuf snuf commented Jun 24, 2014

Pulling back up against master so we can keep following it a bit...

@rohityadavcloud
Copy link
Member

Hey, @snuf can you rebase this against master, let's merge it on master if it works functionally and the build passes with smoke tests?

@snuf
Copy link
Author

snuf commented Dec 6, 2014

Hey Rohit,

It’s been rebased with master two days ago, so should be a “breeze”, is there a specific point you want me to merge with ?

Cheers,

Funs

On 05 Dec 2014, at 18:31, Rohit Yadav notifications@github.com wrote:

Hey, @snuf can you rebase this against master, let's merge it on master if it works functionally and the build passes with smoke tests?


Reply to this email directly or view it on GitHub.

@rohityadavcloud
Copy link
Member

That's great Funs. I see it's already passing smoke tests on TravisCI so we can merge it on master whenever you advise? If it's functionally working with good enough coverage and tests I would say go ahead and merge it on master right away.

@snuf
Copy link
Author

snuf commented Dec 8, 2014

Hi Rohit,

Well there are a couple of things here:

1st: I work on a fork and pull in upstream master and merge that into ovmsupport (https://github.com/snuf/cloudstack/tree/ovmsupport) so in principle that makes the ovmsupport tree aligned with master including my patches.
2nd: Test coverage on part of the code is 81% (the object branch, supportive translation layer to communicate with Ovm3) and on part not where I want it to be (the hypervisor branch, where integration with Cloudstack lives).
3rd: It works within the boundaries of the feature design doc on wiki, which means all basic stuff works but some features can not be provided as Ovm3 doesn’t support them with the storage backend (running snapshots for example).
4th: The systemvm.img that’s generated from buildsystemvm.sh is a raw image so can be used for the systemvms for Ovm3.

Does this answer your question ? I genuinely think there should be some coverage for the integration layer into Cloudstack which is kind of absent?

Cheers,

Funs

On 06 Dec 2014, at 14:02, Rohit Yadav notifications@github.com wrote:

That's great Funs. I see it's already passing smoke tests on TravisCI so we can merge it on master whenever you advise? If it's functionally working with good enough coverage and tests I would say go ahead and merge it on master right away.


Reply to this email directly or view it on GitHub.

@rohityadavcloud
Copy link
Member

Hi Funs,

Thanks for replying. Alright let us know when you've good set of integration tests, I'm halfway with a new integration testing system semi-automated by ansible that uses CloudStack to test CloudStack. In that I run KVM, Xen etc. as VMs on KVM hosts all managed by CloudStack.

Regards.

Funs and others added 24 commits January 11, 2015 12:42
The User table's UUID column is restricted to 40 chars only, since we don't
know how long the nameID/userID of a SAML authenticated user will be - the fix
hashes that user ID and takes a substring of length 40 chars. For hashing,
SHA256 is used which returns a 64 char length string.

- Fix tests, add test cases
- Improve checkSAMLUser method
- Use SHA256 one way hashing to create unique UUID for SAML users

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit b2b4962)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
As per Version 1 cookies, certain characters are now allowed such as space,
colons etc but they should be url encoded using UTF8 encoding. The frontend
has a cookie value unboxing method that removes any double quotes that are added.

As per the doc http://download.oracle.com/javase/6/docs/api/java/net/URLEncoder.html
values are application/x-www-form-urlencoded and as per
http://www.w3.org/TR/html4/interact/forms.html#h-17.13.4 whitespaces are encoded
as +, therefore '+' are replaced by %20 (whitespace).

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 734bd70)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
The fix generates X509Certificate if missing from DB and uses that for eternity.
SAML SP metadata remains same since it's using the same X509 certificate and
it remains same after restarts. The certificate is serialized, base64 encoded
and stored in the keystore table under a specific name. For reading, it's
retrieved, base64 decoded and deserialized.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 4358714)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
…n.com

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 23de431)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
seasonkwok and others added 15 commits February 26, 2015 22:54
Conflicts:
	setup/db/db/schema-450to460.sql
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 9533c54)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit e000646)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

Conflicts:
	plugins/hypervisors/baremetal/src/com/cloud/baremetal/manager/BaremetalVlanManagerImpl.java
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- insecure authenticators excluded in configuration
- snapshot response should have zone
- remove vmsnapshots when removing accounts

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 5481485)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

Conflicts:
	api/src/org/apache/cloudstack/api/response/VMSnapshotResponse.java
	server/src/com/cloud/api/ApiResponseHelper.java
	server/src/com/cloud/storage/download/DownloadActiveState.java
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit dd5fb2e)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 814e557)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit f70afa1)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Adds X-XSS-Protection header
- Adds X-Content-Type-Options header
- Fixes to use json content type defined from global settings
- Uses secure cookie if enabled in global settings

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit b6b3494)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 0f819f1)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
git merge master
Merge remote-tracking branch 'cloudstack/master' into ovmsupport
@rohityadavcloud
Copy link
Member

@snuf Funs how about we merge your stuff in?

@snuf
Copy link
Author

snuf commented Mar 3, 2015

Hey Rohit,

Well all was fine until I did my last two merges from master after doing the 3.3.x integration of OVM, allocate vnet breaks and rolls back on the DB all of a sudden am trying to figure out where that is. So all bits work except for network provising for a VM (jeez luckily not important at all…), so vm rollout breaks. Am looking at where and why it breaks, but am pretty not impressed at the moment :\

Cheers,

Funs

On 02 Mar 2015, at 03:14, Rohit Yadav <notifications@github.com mailto:notifications@github.com> wrote:

@snuf https://github.com/snuf Funs how about we merge your stuff in?


Reply to this email directly or view it on GitHub #9 (comment).

@snuf
Copy link
Author

snuf commented Mar 3, 2015

Kay happy to report it was a problem in my rollout script which I changed recently cough so everything is fine, it’s just that the change left out the vlan range, so that explains a lot..

So you can go ahead and merge if you want to, I’m 3 days a way from master atm (need me to do a new pull request?).

Cheers,

Funs

On 02 Mar 2015, at 03:14, Rohit Yadav notifications@github.com wrote:

@snuf https://github.com/snuf Funs how about we merge your stuff in?


Reply to this email directly or view it on GitHub #9 (comment).

@snuf snuf closed this Mar 5, 2015
karuturi pushed a commit to karuturi/cloudstack that referenced this pull request Oct 6, 2016
ustcweizhou added a commit to ustcweizhou/cloudstack that referenced this pull request Nov 19, 2020
ustcweizhou added a commit to ustcweizhou/cloudstack that referenced this pull request Nov 23, 2020
DaanHoogland pushed a commit that referenced this pull request Nov 25, 2020
qrry pushed a commit to qrry/cloudstack that referenced this pull request Dec 4, 2020
* master: (25 commits)
  integration test: skip vlan of public ip range in get_free_vlan
  vpc vr: plugin nics by this order: public/private/guest
  vpc vr: fix Conflicting device id on private gw nic
  Adding zone name to physicalnetworkresponse (apache#4510)
  Disallowing udp for lb rules for haproxy (apache#4501)
  Make global setting non-dynamic (apache#4505)
  Adding cpuallocated percentage and value to host and hostsformigrationresponse (apache#4499)
  kvm: fix router.aggregation.command.each.timeout is reset to 600 when update other kvm configs (apache#4496)
  fix failures with test_multiple_nic_support.py (apache#4495)
  Fix hosts for migration count (apache#4500)
  sql: Fix Zones are returned in a random order (apache#3934) (apache#4494)
  integration test: update steps
  integration test: add private gateway in test
  integration test: verify public nics state
  bugfix apache#9 vpc vr: Add PREROUTING rule for vm with static nat to multiple private gateways
  bugfix apache#8 vpc: add rule for traffic between vm and private gateway
  bugfix apache#7 vpc vr: allow servers in private gateway to reach internet via the VPC VR if it is gateway
  bugfix apache#6 vpc vr: Add iptables rules for ACL of private gateway
  Revert "Fix Policy Based Routing for private gateway static routes (apache#3604)"
  Revert "Add private gateway IP to router initialization config"
  ...
nvazquez pushed a commit that referenced this pull request Feb 3, 2024
Pearl1594 added a commit that referenced this pull request Feb 6, 2024
nvazquez added a commit that referenced this pull request Mar 8, 2024
* NSX integration - skeletal code

* Fix module not loading on startup

* add upgrade path and daos
\n add nsx controller command

* add support for adding and listing nsx provider to a zone

* add license

* add default VPC offering and update upgrade path

* add global setting to enable nsx plugin

* add delete nsx controller operation

* add nsxresource

* add NSX resource , api client, create tier1 gw

* update db

* update response and add license

* Add support to create and delete nsx tier-1 gateway

* add license

* cleanup and add skeletal code for network creation

* add create/delete segment and UI integration

* add license

* address code smells - part 1

* fix test / build failure

* NSX integration - skeletal code

* Fix module not loading on startup

* add upgrade path and daos
\n add nsx controller command

* add support for adding and listing nsx provider to a zone

* add license

* add default VPC offering and update upgrade path

* add global setting to enable nsx plugin

* add delete nsx controller operation

* add nsxresource

* add NSX resource , api client, create tier1 gw

* update db

* update response and add license

* Add support to create and delete nsx tier-1 gateway

* add license

* cleanup and add skeletal code for network creation

* add create/delete segment and UI integration

* add license

* address code smells - part 1

* fix test / build failure

* add ui changes + update nsx_provider table transport zones + use NSX broadcast domain for add nics to router

* ui: fix password field, and backend changes

* add route advertisement

* update offering

* update offering

* add sleep before deletion of vpc / tier g/w for ports to be removed

* move creation of segments to design phase

* change provider to VPC router for Dhcp & dns service in an nsx offering

* Add public nic for NSX

* reserve first IP (after g/w) of subnet for router nic - NSX

* revert reserving 1st IP in vpc segments

* [NSX] Create a DHCP relay and add it to a VPC tier segment (#107)

* Create DHCP relay command and execute request

* In progress integrate with networking

* Create DHCP relay config on the network VR allocation

* Revert domain router dao changes

* Create DHCP relay con VR nic plug to NSX network

* Link DHCP relay config to segment after creation

* [NSX] Cleanup DHCP Relay config on segment deletion (#108)

* Cleanup DHCP Relay config on segment deletion

* update segment & relay name generators and call delete dhcprelay after deletion of segment

* address comment

* [NSX] Fix DHCP relay config deletion was missing zone name (#8068)

* [NSX] Refactor API wrapper operations (#8059)

* [NSX] Refactor API wrapper operations

* Big refactor

* Address review comment

* change network cidr to cidr to prevent NPE

* add domain and zone names to the various networks - vpc & tier

---------

Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>

* Nsx unit tests (#8090)

* Add tests

* add test for NsxGuestNetworkGuru

* add unit tests for NsxResource

* add unti tests for NsxElement

* cleanup

* [NSX] Refactor API wrapper operations

* update tests

* update tests - add nsxProviderServiceImpl test

* add unit test - NsxServiceImpl

* add license

* Big refactor

* Address review comment

* change network cidr to cidr to prevent NPE

* add domain and zone names to the various networks - vpc & tier

* fix tests

---------

Co-authored-by: nvazquez <nicovazquez90@gmail.com>

* modify NSX resource naming convention (#8095)

* modify NSX resource naming convention

* remove unused imports

* add a setup phase between desgin and implementation of a network for intermediary steps

* add method to all classes

* NSX: Refactor Network & VPC offering (#8110)

* [NSX] Refactor API wrapper operations

* Network offering changes for NSX

* fix services and provider combination

* address comments: rename param

* update nsx_mode parameter

---------

Co-authored-by: nvazquez <nicovazquez90@gmail.com>

* fix test

* [NSX] Allow NSX isolated networks (#8132)

* Add network offerings for NSX on isolated networks

* Fix offerings creation

* In progress NSX isolated network

* Fixes

* Fix NIC allocation to router

* NSX: Add Step for Adding Public traffic network for NSX During zone creation (#8126)

* NSX: Add Step for Adding Public traffic network for NSX

* address comments and cleanup

* address comment

* remove indent

* NSX: Create and Delete static NAT & Port forward  rules (#8131)

* NSX: Create and delete NSX Static Nat rules

* fix issues with static nat

* add static nat

* Support to add and delete Port forward rules

* add license

* fix adding multiple pf rules

* cleanup

* fix lint check

* fix smoke tests

* fix smoke tests

* Nsx add lb rule (#8161)

* NSX: Create and delete NSX Static Nat rules

* fix issues with static nat

* add static nat

* Support to add and delete Port forward rules

* add license

* fix adding multiple pf rules

* cleanup

* NSX: Add support to create and delete Load balancer rules

* fix deletion of lb rules

* add header file and update protocol detail

* build failure fix

* [NSX] Add SNAT support (#8100)

* In progress add source NAT

* Fix after merge

* Fix tests

* Fix NPE on isolated network deletion

* Reserve source NAT IP when its not passed for NSX VPC

* Create source NAT rule on VR NIC allocation

* Fix update VPC and remove VPC to update and remove SNAT rule

* Fix packaging

* Address review comment

* Fix build

* fix build - unused import

* Add defensive checks

* Add missing design to NSX public guru

---------

Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>

* NSX: Fix VR public NIC allocation (#8166)

* NSX: fix LB member addition and deletion and add defensive checks (#8167)

* Fix public NIC NPE on broadcast URI

* NSX: Router Public nic to get IP from systemVM Ip range (#8172)

* NSX: Router Public nic to get IP from systemVM Ip range

* Fix VR IP address and setSourceNatIp command

* NSX: hide systemVM reserved IP range SourceNAT

* fix test

---------

Co-authored-by: nvazquez <nicovazquez90@gmail.com>

* fix test failure

* test failure fix

* [NSX] Fix update source NAT IP (#8176)

* [NSX] Fix update source NAT IP

* Fix startup

* Fix API result

* NSX - add LB route Advertizement (#8192)

* [NSX] Add ACL types support (#8224)

* NSX: Create segment group on segment creation

* Add unit tests

* Remove group for segment before removing segment

* Create Distributed Firewall rules

* Remove distributed firewall policy on segment deletion

* Fix policy rule ID and add more unit tests

* Fix DROP action rules and transform tests

* Add new ACL rules

* Fixes

* associate security policies with groups and not to DFW and add deletion of rules

* Fix name convention

---------

Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>

* NSX: Fix creation of VPCs (#8320)

* Fix ACL rules creation (#8323)

* [NSX] Fix database views (#8325)

* NSX: Add CKS Support & Firewall rules for Isolated Networks (#8189)

* NSX: Add ALL LB IP to the list of route advertisements in tier1

* NSX: Support Source NAT on NSX Isolated networks

* NSX: Cks Support

* NSX: Create segment group on segment creation

* Add unit tests

* Remove group for segment before removing segment

* Create Distributed Firewall rules

* Remove distributed firewall policy on segment deletion

* Fix policy rule ID and add more unit tests

* Add support for routed NSX Isolated networks \n and non RFC 1918 compliant IPs

* Add support for routed NSX Isolated networks \n and non RFC 1918 compliant IPs

* Add Firewall rules

* build failure - fix unit test

* fix npes

* Add support to delete firewall rules

* update nsx cks offering

* add license

* update order of ports in PF & FW rules

* fix filter for getting transport zones

* CKS support changed - MTU updated, etc

* add LB for CKS on VPC

* address comments

* adapt upstream cks logic for vpc

* rever mtu hack

* update UI changes as per upstream fix

* change display test for CKS n/w offerings for isolated and VPC tiers

* add extra line for linter

* address comment

* revert list change

---------

Co-authored-by: nvazquez <nicovazquez90@gmail.com>

* fix ui build failure

* [NSX] Address SonarCloud Bugs (#8341)

* [NSX] Address SonarCloud Bugs

* Fix NSX API connection issues

* NSX: Add unit tests to increase coverage (#8355)

* NSX: Add unit tests

* cleanup unused imports

* add more unit tests

* add tests for publicnsxnetworkguru

* add license

* fix build failures

* address sonar comment

* fix security hotspots

* NSX: Add more unit tests (#8381)

* NSX : Unit tests

* remove unused imports

* remove unused import causing build failure

* fix build failures due to unused imports

* fix build failure

* fix test assertion

* remove unused imports

* remove unused import

* Nsx UI zone bug (#8398)

* NSX: Attempt to fix NSX Zone creation bug for public networks

* fix zone wizard public traffic issue

* add proper filtering of offerings based on VPC nsx mode

* clean up console logs

* NSX: Fix code smells and reported bugs (#8409)

* NSX: Fix code smells and reported bugs

* fox override issue

* remove unused imports

* fix test

* refactor code to reduce complexity

* add lisence

* cleanup

* fix build failure

* fix build failure

* address comments

* test - add config to ignore certain files from test coverage

* test exclusion of classes from test cov

* rever pom changes

* [NSX] Add more unit tests (#8431)

* [NSX] Add more unit tests

* More tests

* Fix build errors

* NSX: Prevent creation of L2 and Shared networks for NSX (#8463)

* NSX: Prevent creation of L2 and Shared networks for NSX

* add checks to backend to prevent creation of l2 and shared networks in nsx zones and filter only nsx offerings when creating isolated networks

* cleanup

* NSX: Fix code smells (#8436)

* NSX: Fix code smells

* Add changes to service creation logic

* CKS: Add action to during firewall rule creation (#8498)

* NSX,UI: Deduplicate network list when creating kubernetes clusters (#8513)

* NSX: Make LB service selectable in network offering (#8512)

* NSX: Make LB service selectable in network offering

* fix label

* address comments

* address comments

* NSX: Add appropriate error message when icmp type is set to -1 for NSX (#8504)

* NSX: Add appropriate error message when icmp type is set to -1 for NSX

* address comments

* update text

* fix test

* fix test - build failure

* fix test - build failure

* NSX: Cleanup NSX resources during k8s cluster cleanup (#8528)

* fix test failure

* NSX: Improve segment deletion process (#8538)

* NSX: Add passive monitor for NSX LB to test whether a server is available (#8533)

* NSX: Add passive monitor for NSX LB to test whether a server is available

* Add active monitors too

* fix build failure

* NSX: Add check for ICMP code / type for NSX zones (#8542)

* NSX: Fix Routed Mode for Isolated and VPC networks (#8534)

* NSX: Fix Routed Mode for Isolated and VPC networks

* NSX: Fix Routed mode - add checks for ports added for FW rules

* clean up code

* fix build failure

* NSX: Add retry logic with sleep to delete segments (#8554)

* NSX: Add retry logic with sleep to delete segments

* add logs

* NSX: Fix custom ACL check (#2)

* NSX: Fix custom ACL check

* NSX: Fix custom ACL check

* Nsx vpc routed mode (#5)

* NSX: Fix VPC routed mode

* NSX: VPC route mode

* remove unnecessary changes

* Nsx: Support internal LB (#4)

* NSX: Support internal LB service in NSX

* add lb removal logic

* Fix UI issue hiding internal LB tab

* Refactor method name

---------

Co-authored-by: nvazquez <nicovazquez90@gmail.com>

* NSX: Improve NSX resource cleanup process (#3)

* Fix unit test

* NSX: Add SourceNAT service to the default Routed offering for VPC (#13)

* Fix VPC restart with cleanup (#12)

* NSX: Fix ACL rule removal on replacement and fix rule order (#11)

* NSX: fix smoke test failure for ACLs (#9)

* Fix unit tests

* Fix NSX plugin pom XML

* NSX: Add support to re-order ACL rules (NSX FW rules) (#14)

* [WIP] NSX: Add support to re-order ACL rules (NSX FW rules)

* fix reordering of acl rules on all networks that it is associated to

* clean up and attempt test fix

* Fix tests

* Remove unused import

* tweak reorder logic

---------

Co-authored-by: nvazquez <nicovazquez90@gmail.com>

* Fix zone creation issue for internal load balancer

* Fix

* Fix unit test

* fix logger

* fix logger

* fix logger

* NSX: Fix VPC form to ignore source NAT IP when creating VPCs and fix label

* Move SQL changes to the newest schema file

* NSX: Last Fixes

* Fix build

---------

Co-authored-by: nvazquez <nicovazquez90@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet