Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add missing ConfigDrive entries on existing zones after upgrade #3007

Merged
merged 2 commits into from Nov 12, 2018

Conversation

@nvazquez
Copy link
Contributor

commented Nov 6, 2018

Description

After upgrade existing environments to 4.11, ConfigDrive cannot be enabled for existing zones due to missing entry on 'physical_network_service_providers' table.

The UI shows:
image

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)

How Has This Been Tested?

@nvazquez nvazquez force-pushed the shapeblue:upgradeconfigdrive branch from 6e2aa18 to 190c386 Nov 6, 2018

@nvazquez nvazquez changed the base branch from master to 4.11 Nov 6, 2018

@nvazquez nvazquez requested review from DaanHoogland and PaulAngus Nov 6, 2018

@nvazquez nvazquez added this to the 4.11.2.0 milestone Nov 6, 2018

@PaulAngus

This comment has been minimized.

Copy link
Member

commented Nov 6, 2018

@blueorangutan package

@blueorangutan

This comment has been minimized.

Copy link

commented Nov 6, 2018

@PaulAngus a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@PaulAngus

This comment has been minimized.

Copy link
Member

commented Nov 6, 2018

Jenkins says that there is a conflict

stdout: CONFLICT (modify/delete): server/src/com/cloud/network/NetworkServiceImpl.java deleted in HEAD and modified in 190c3860c59eb89ed4ade6a9444102f663576b1a. Version 190c3860c59eb89ed4ade6a9444102f663576b1a of server/src/com/cloud/network/NetworkServiceImpl.java left in tree.
Automatic merge failed; fix conflicts and then commit the result.

can you fix please @nvazquez

@blueorangutan

This comment has been minimized.

Copy link

commented Nov 6, 2018

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-2406

@nvazquez nvazquez force-pushed the shapeblue:upgradeconfigdrive branch from 190c386 to 433142a Nov 6, 2018

@nvazquez

This comment has been minimized.

Copy link
Contributor Author

commented Nov 6, 2018

Done thanks @PaulAngus
@blueorangutan package

@blueorangutan

This comment has been minimized.

Copy link

commented Nov 6, 2018

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan

This comment has been minimized.

Copy link

commented Nov 6, 2018

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-2407

@nvazquez

This comment has been minimized.

Copy link
Contributor Author

commented Nov 6, 2018

@blueorangutan

This comment has been minimized.

Copy link

commented Nov 6, 2018

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@GabrielBrascher
Copy link
Member

left a comment

Thanks for the fix, @nvazquez!
The code in general LGTM. Just a few minor comments.

Additionally, a test case might be good as well. The fact of having small and concise methods makes it a great opportunity to have a couple of test methods.

server/src/com/cloud/network/NetworkServiceImpl.java Outdated Show resolved Hide resolved
server/src/com/cloud/network/NetworkServiceImpl.java Outdated Show resolved Hide resolved
server/src/com/cloud/network/NetworkServiceImpl.java Outdated Show resolved Hide resolved
server/src/com/cloud/network/NetworkServiceImpl.java Outdated Show resolved Hide resolved
@PaulAngus
Copy link
Member

left a comment

Tried upgrading from 4.9.3 to this PR
Management server would not start

2018-11-06 17:19:18,749 WARN  [o.a.c.s.m.c.ResourceApplicationContext] (main:null) (logid:) Exception encountered during context initialization - cancelling refresh attempt: org.springframework.context.ApplicationContextException: Failed to start bean 'cloudStackLifeCycle'; nested exception is com.cloud.exception.InvalidParameterValueException: Unable to find the Network Element implementing the Service Provider 'ConfigDrive'
2018-11-06 17:19:18,750 WARN  [o.e.j.w.WebAppContext] (main:null) (logid:) Failed startup of context o.e.j.w.WebAppContext@4148db48{/client,file:///usr/share/cloudstack-management/webapp/,UNAVAILABLE}{/usr/share/cloudstack-management/webapp}
org.springframework.context.ApplicationContextException: Failed to start bean 'cloudStackLifeCycle'; nested exception is com.cloud.exception.InvalidParameterValueException: Unable to find the Network Element implementing the Service Provider 'ConfigDrive'
	at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:186)
	at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:52)
	at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:358)
	at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:159)
	at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:123)
	at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:884)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:552)
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:145)
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:122)
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:245)
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:233)
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:117)
	at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:79)
	at org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37)
	at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:71)
	at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:58)
	at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:62)
	at org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:52)
	at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:890)
	at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:532)
	at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:853)
	at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:344)
	at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1515)
	at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1477)
	at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:785)
	at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:261)
	at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:545)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:133)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:107)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
	at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:273)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:133)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:115)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:133)
	at org.eclipse.jetty.server.Server.start(Server.java:418)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:107)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
	at org.eclipse.jetty.server.Server.doStart(Server.java:385)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
	at org.apache.cloudstack.ServerDaemon.start(ServerDaemon.java:200)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.commons.daemon.support.DaemonLoader.start(DaemonLoader.java:241)
Caused by: com.cloud.exception.InvalidParameterValueException: Unable to find the Network Element implementing the Service Provider 'ConfigDrive'
	at com.cloud.network.NetworkServiceImpl.addProviderToPhysicalNetwork(NetworkServiceImpl.java:3702)
	at com.cloud.network.NetworkServiceImpl.addConfigDriveToPhysicalNetwork(NetworkServiceImpl.java:4268)
	at com.cloud.network.NetworkServiceImpl.checkConfigDriveEntriesOnZone(NetworkServiceImpl.java:645)
	... 57 more
2018-11-06 17:19:18,751 INFO  [o.e.j.s.h.ContextHandler] (main:null) (logid:) Started o.e.j.s.h.MovedContextHandler@15bfd87{/,null,AVAILABLE}
2018-11-06 17:19:18,771 INFO  [o.e.j.s.AbstractConnector] (main:null) (logid:) Started ServerConnector@4b1c1ea0{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}
2018-11-06 17:19:18,771 INFO  [o.e.j.s.Server] (main:null) (logid:) Started @36138ms
@blueorangutan

This comment has been minimized.

Copy link

commented Nov 6, 2018

Trillian test result (tid-3144)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 27591 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr3007-t3144-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Smoke tests completed. 67 look OK, 0 have error(s)
Only failed tests results shown below:

Test Result Time (s) Test File
@nvazquez

This comment has been minimized.

Copy link
Contributor Author

commented Nov 7, 2018

Thanks @rafaelweingartner @GabrielBrascher @PaulAngus for reviewing!
@PaulAngus I have pushed the fix moving the logic to the start method on NetworkModel, after all the network elements have been loaded. Can you please repeat the test?
@blueorangutan package

@blueorangutan

This comment has been minimized.

Copy link

commented Nov 7, 2018

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan

This comment has been minimized.

Copy link

commented Nov 7, 2018

Packaging result: ✔centos6 ✖centos7 ✔debian. JID-2411

@nvazquez

This comment has been minimized.

Copy link
Contributor Author

commented Nov 8, 2018

@blueorangutan package

@blueorangutan

This comment has been minimized.

Copy link

commented Nov 8, 2018

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan

This comment has been minimized.

Copy link

commented Nov 8, 2018

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-2412

@PaulAngus

This comment has been minimized.

Copy link
Member

commented Nov 8, 2018

LGTM based on manual upgrade from 4.9.3.
configdrive service appears in DB and UI as disabled.
can enable service and create an L2 network with configdrive,
creating a cloud-init enabled VM succeeds. VM starts and picks up host name from config drive.

@PaulAngus

This comment has been minimized.

Copy link
Member

commented Nov 8, 2018

@PaulAngus

This comment has been minimized.

Copy link
Member

commented Nov 8, 2018

@PaulAngus

This comment has been minimized.

Copy link
Member

commented Nov 8, 2018

@blueorangutan test matrix

@blueorangutan

This comment has been minimized.

Copy link

commented Nov 8, 2018

@PaulAngus a Trillian-Jenkins matrix job (centos6 mgmt + xs71, centos7 mgmt + vmware65, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

@nvazquez

This comment has been minimized.

Copy link
Contributor Author

commented Nov 8, 2018

@blueorangutan

This comment has been minimized.

Copy link

commented Nov 9, 2018

Trillian test result (tid-3149)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 20374 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr3007-t3149-kvm-centos7.zip
Smoke tests completed. 67 look OK, 0 have error(s)
Only failed tests results shown below:

Test Result Time (s) Test File
@blueorangutan

This comment has been minimized.

Copy link

commented Nov 9, 2018

Trillian test result (tid-3150)
Environment: vmware-65 (x2), Advanced Networking with Mgmt server 7
Total time taken: 26283 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr3007-t3150-vmware-65.zip
Smoke tests completed. 67 look OK, 0 have error(s)
Only failed tests results shown below:

Test Result Time (s) Test File
@rhtyd
rhtyd approved these changes Nov 12, 2018

@rhtyd rhtyd merged commit bb7493a into apache:4.11 Nov 12, 2018

2 checks passed

Jenkins This pull request looks good
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.