Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API job import fails on WriteListener error #4488

Closed
kristian-lesko opened this issue Feb 12, 2019 · 7 comments

Comments

@kristian-lesko
Copy link

commented Feb 12, 2019

Describe the bug
When trying to import a job to Rundeck via API, it always fail with the stacktrace listed below.
However:

  • the job is still uploaded,
  • the error does not happen when uploading a job via UI.

My Rundeck detail

  • Rundeck version: 3.13
  • install type: WAR
  • OS Name/version: CentOS 7
  • DB Type/version: Postgres

To Reproduce
Steps to reproduce the behavior:

  1. Deploy a Rundeck 3.0.13 server.
  2. Create a job definition in YAML.
  3. Try to upload the job via API (e.g., using curl, or via Python requests).
  4. Get a HTML response with the 500 error.
  5. Open Rundeck UI and see that the job was uploaded.

Expected behavior
Job should either not be uploaded, or the request should end with a 200 (not 500).

Additional context
One more strange thing is that the response from the API is a full HTML page (like one would expect to get in a browser) containing the error (listed below). One would expect the API response to be an XML, JSON or something similar (not full HTML, including Javascript code).

Job YAML definition

- description: ...
  executionEnabled: true
  group: <group>
  loglevel: INFO
  name: <name>
  nodeFilterEditable: true
  nodefilters:
    dispatch:
      excludePrecedence: true
      keepgoing: false
      rankOrder: ascending
      successOnEmptyNodeFilter: false
      threadcount: '1'
    filter: '<filter>'
  nodesSelectedByDefault: true
  options:
  - description: <param description>
    name: <param name>
    required: true
  orchestrator:
    configuration:
      count: '1'
    type: subset
  scheduleEnabled: true
  sequence:
    commands:
    - description: <command description>
      jobref:
        args: -<arg> ${option.<param name>}
        group: <group>
        name: <name>
    keepgoing: false
    strategy: step-first

Stacktrace

2019-02-14 15:42:39.698 ERROR --- [bio-8009-exec-1] c.d.r.s.plugins.RundeckPluginRegistry    : Plugin step-first for service: WorkflowStrategy was not found

Error 500: Internal Server Error

URI
    /api/17/project/project/jobs/import
Class
    java.lang.ClassNotFoundException
Message
    Handler dispatch failed; nested exception is java.lang.NoClassDefFoundError: javax/servlet/WriteListener
Caused by
    javax.servlet.WriteListener

Trace

    Line | Method
->> 2701 | privateGetDeclaredMethods in java.lang.Class
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
|   2902 | privateGetPublicMethods in     ''
|   1615 | getMethods . . . in     ''
|   1336 | getPublicDeclaredMethods in java.beans.Introspector
|   1197 | getTargetMethodInfo in     ''
|    426 | getBeanInfo      in     ''
|    173 | getBeanInfo . .  in     ''
|    290 | respondOutput    in rundeck.services.ApiService
|    581 | renderErrorXml . in     ''
|    575 | renderErrorFormat in     ''
|    684 | requireVersion . in     ''
|   3416 | apiJobsImportv14 in rundeck.controllers.ScheduledExecutionController
|    223 | invoke . . . . . in org.grails.core.DefaultGrailsControllerClass$MethodHandleInvoker
|    188 | invoke           in org.grails.core.DefaultGrailsControllerClass
|     90 | handle . . . . . in org.grails.web.mapping.mvc.UrlMappingsInfoHandlerAdapter
|    967 | doDispatch       in org.springframework.web.servlet.DispatcherServlet
|    901 | doService . . .  in     ''
|    970 | processRequest   in org.springframework.web.servlet.FrameworkServlet
|    872 | doPost . . . . . in     ''
|    846 | service          in     ''
|    549 | doAsPrivileged . in javax.security.auth.Subject
|     55 | doFilterInternal in org.springframework.boot.web.filter.ApplicationContextHeaderFilter
|    549 | doAsPrivileged . in javax.security.auth.Subject
|     67 | doFilterInternal in org.rundeck.grails.plugins.securityheaders.RundeckSecurityHeadersFilter
|    549 | doAsPrivileged . in javax.security.auth.Subject
|    317 | doFilter         in org.springframework.security.web.FilterChainProxy$VirtualFilterChain
|    127 | invoke . . . . . in org.springframework.security.web.access.intercept.FilterSecurityInterceptor
|     91 | doFilter         in     ''
|    331 | doFilter . . . . in org.springframework.security.web.FilterChainProxy$VirtualFilterChain
|    114 | doFilter         in org.springframework.security.web.access.ExceptionTranslationFilter
|     64 | doFilter . . . . in grails.plugin.springsecurity.web.UpdateRequestContextHolderExceptionTranslationFilter
|    331 | doFilter         in org.springframework.security.web.FilterChainProxy$VirtualFilterChain
|     54 | doFilterInternal in grails.plugin.springsecurity.web.filter.GrailsHttpPutFormContentFilter
|    331 | doFilter         in org.springframework.security.web.FilterChainProxy$VirtualFilterChain
|     54 | doFilter . . . . in grails.plugin.springsecurity.web.filter.GrailsAnonymousAuthenticationFilter
|    331 | doFilter         in org.springframework.security.web.FilterChainProxy$VirtualFilterChain
|    158 | doFilter . . . . in org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter
|    331 | doFilter         in org.springframework.security.web.FilterChainProxy$VirtualFilterChain
|    170 | doFilter . . . . in org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter
|    331 | doFilter         in org.springframework.security.web.FilterChainProxy$VirtualFilterChain
|    200 | doFilter . . . . in org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter
|    331 | doFilter         in org.springframework.security.web.FilterChainProxy$VirtualFilterChain
|    121 | doFilter . . . . in org.springframework.security.web.authentication.preauth.AbstractPreAuthenticatedProcessingFilter
|     44 | doFilter         in org.rundeck.security.RundeckPreauthenticationRequestHeaderFilter
|    331 | doFilter . . . . in org.springframework.security.web.FilterChainProxy$VirtualFilterChain
|     64 | doFilter         in grails.plugin.springsecurity.web.authentication.logout.MutableLogoutFilter
|    331 | doFilter . . . . in org.springframework.security.web.FilterChainProxy$VirtualFilterChain
|    105 | doFilter         in org.springframework.security.web.context.SecurityContextPersistenceFilter
|    331 | doFilter . . . . in org.springframework.security.web.FilterChainProxy$VirtualFilterChain
|     58 | doFilter         in grails.plugin.springsecurity.web.SecurityRequestHolderFilter
|    331 | doFilter . . . . in org.springframework.security.web.FilterChainProxy$VirtualFilterChain
|    214 | doFilterInternal in org.springframework.security.web.FilterChainProxy
|    177 | doFilter . . . . in     ''
|    549 | doAsPrivileged   in javax.security.auth.Subject
|     77 | doFilterInternal in org.grails.web.servlet.mvc.GrailsWebRequestFilter
|    549 | doAsPrivileged   in javax.security.auth.Subject
|     67 | doFilterInternal in org.grails.web.filters.HiddenHttpMethodFilter
|    549 | doAsPrivileged   in javax.security.auth.Subject
|    130 | doFilter . . . . in org.springframework.boot.web.support.ErrorPageFilter
|     66 | access$000       in     ''
|    105 | doFilterInternal in org.springframework.boot.web.support.ErrorPageFilter$1
|    123 | doFilter         in org.springframework.boot.web.support.ErrorPageFilter
|    549 | doAsPrivileged . in javax.security.auth.Subject
|   1149 | runWorker        in java.util.concurrent.ThreadPoolExecutor
|    624 | run . . . . . .  in java.util.concurrent.ThreadPoolExecutor$Worker
^    748 | run              in java.lang.Thread

Caused by ClassNotFoundException: javax.servlet.WriteListener
->> 2701 | privateGetDeclaredMethods in java.lang.Class
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
|   2902 | privateGetPublicMethods in     ''
|   1615 | getMethods . . . in     ''
|   1336 | getPublicDeclaredMethods in java.beans.Introspector
|   1197 | getTargetMethodInfo in     ''
|    426 | getBeanInfo      in     ''
|    173 | getBeanInfo . .  in     ''
|    290 | respondOutput    in rundeck.services.ApiService
|    581 | renderErrorXml . in     ''
|    575 | renderErrorFormat in     ''
|    684 | requireVersion . in     ''
|   3416 | apiJobsImportv14 in rundeck.controllers.ScheduledExecutionController
|    223 | invoke . . . . . in org.grails.core.DefaultGrailsControllerClass$MethodHandleInvoker
|    188 | invoke           in org.grails.core.DefaultGrailsControllerClass
|     90 | handle . . . . . in org.grails.web.mapping.mvc.UrlMappingsInfoHandlerAdapter
|    967 | doDispatch       in org.springframework.web.servlet.DispatcherServlet
|    901 | doService . . .  in     ''
|    970 | processRequest   in org.springframework.web.servlet.FrameworkServlet
|    872 | doPost . . . . . in     ''
|    846 | service          in     ''
|    549 | doAsPrivileged . in javax.security.auth.Subject
|     55 | doFilterInternal in org.springframework.boot.web.filter.ApplicationContextHeaderFilter
|    549 | doAsPrivileged . in javax.security.auth.Subject
|     67 | doFilterInternal in org.rundeck.grails.plugins.securityheaders.RundeckSecurityHeadersFilter
|    549 | doAsPrivileged . in javax.security.auth.Subject
|    317 | doFilter         in org.springframework.security.web.FilterChainProxy$VirtualFilterChain
|    127 | invoke . . . . . in org.springframework.security.web.access.intercept.FilterSecurityInterceptor
|     91 | doFilter         in     ''
|    331 | doFilter . . . . in org.springframework.security.web.FilterChainProxy$VirtualFilterChain
|    114 | doFilter         in org.springframework.security.web.access.ExceptionTranslationFilter
|     64 | doFilter . . . . in grails.plugin.springsecurity.web.UpdateRequestContextHolderExceptionTranslationFilter
|    331 | doFilter         in org.springframework.security.web.FilterChainProxy$VirtualFilterChain
|     54 | doFilterInternal in grails.plugin.springsecurity.web.filter.GrailsHttpPutFormContentFilter
|    331 | doFilter         in org.springframework.security.web.FilterChainProxy$VirtualFilterChain
|     54 | doFilter . . . . in grails.plugin.springsecurity.web.filter.GrailsAnonymousAuthenticationFilter
|    331 | doFilter         in org.springframework.security.web.FilterChainProxy$VirtualFilterChain
|    158 | doFilter . . . . in org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter
|    331 | doFilter         in org.springframework.security.web.FilterChainProxy$VirtualFilterChain
|    170 | doFilter . . . . in org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter
|    331 | doFilter         in org.springframework.security.web.FilterChainProxy$VirtualFilterChain
|    200 | doFilter . . . . in org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter
|    331 | doFilter         in org.springframework.security.web.FilterChainProxy$VirtualFilterChain
|    121 | doFilter . . . . in org.springframework.security.web.authentication.preauth.AbstractPreAuthenticatedProcessingFilter
|     44 | doFilter         in org.rundeck.security.RundeckPreauthenticationRequestHeaderFilter
|    331 | doFilter . . . . in org.springframework.security.web.FilterChainProxy$VirtualFilterChain
|     64 | doFilter         in grails.plugin.springsecurity.web.authentication.logout.MutableLogoutFilter
|    331 | doFilter . . . . in org.springframework.security.web.FilterChainProxy$VirtualFilterChain
|    105 | doFilter         in org.springframework.security.web.context.SecurityContextPersistenceFilter
|    331 | doFilter . . . . in org.springframework.security.web.FilterChainProxy$VirtualFilterChain
|     58 | doFilter         in grails.plugin.springsecurity.web.SecurityRequestHolderFilter
|    331 | doFilter . . . . in org.springframework.security.web.FilterChainProxy$VirtualFilterChain
|    214 | doFilterInternal in org.springframework.security.web.FilterChainProxy
|    177 | doFilter . . . . in     ''
|    549 | doAsPrivileged   in javax.security.auth.Subject
|     77 | doFilterInternal in org.grails.web.servlet.mvc.GrailsWebRequestFilter
|    549 | doAsPrivileged   in javax.security.auth.Subject
|     67 | doFilterInternal in org.grails.web.filters.HiddenHttpMethodFilter
|    549 | doAsPrivileged   in javax.security.auth.Subject
|    130 | doFilter . . . . in org.springframework.boot.web.support.ErrorPageFilter
|     66 | access$000       in     ''
|    105 | doFilterInternal in org.springframework.boot.web.support.ErrorPageFilter$1
|    123 | doFilter         in org.springframework.boot.web.support.ErrorPageFilter
|    549 | doAsPrivileged . in javax.security.auth.Subject
|   1149 | runWorker        in java.util.concurrent.ThreadPoolExecutor
|    624 | run . . . . . .  in java.util.concurrent.ThreadPoolExecutor$Worker
^    748 | run              in java.lang.Thread
@sjrd218

This comment has been minimized.

Copy link
Contributor

commented Feb 13, 2019

@kristian-lesko Can you tell us what container and version you are using to run your war?

@kristian-lesko

This comment has been minimized.

Copy link
Author

commented Feb 14, 2019

@sjrd218 sure, sorry for forgetting to mention that before. I was using Tomcat 7.0.76.

Update: it also happens with an older version I've had available, 7.0.69.

@kristian-lesko

This comment has been minimized.

Copy link
Author

commented Feb 14, 2019

I've been able to find an additional error in the Tomcat log:

2019-02-14 15:42:39.698 ERROR --- [bio-8009-exec-1] c.d.r.s.plugins.RundeckPluginRegistry    : Plugin step-first for service: WorkflowStrategy was not found

However, step-first sounds like a built-in plugin in Rundeck itself, so I'm not sure why it would be missing.

@sjrd218

This comment has been minimized.

Copy link
Contributor

commented Feb 14, 2019

@kristian-lesko This error occurs because you are using Tomcat 7 which does not support the Servlet 3.1 spec. The grails framework used for Rundeck 3.x expects to be run in a container that supports the 3.1 spec so it's attempting to make a call to a method that doesn't exist, which is the source of the error you are seeing. I am making a code change to work around this problem, but if you want to fix the problem before the change is rolled out you can upgrade to a more recent Tomcat container.

@kristian-lesko

This comment has been minimized.

Copy link
Author

commented Feb 14, 2019

@sjrd218 Thank you for the advice.

Is there any ETA for the code change that you mention, please? Thanks.

@gschueler gschueler closed this in d9872a9 Feb 19, 2019

gschueler added a commit that referenced this issue Feb 19, 2019

Merge pull request #4503 from rundeck/issue/4488
Fix #4488. Help Grails write output stream correctly on Tomcat 7.

@gschueler gschueler added this to the 3.0.14 milestone Feb 19, 2019

@kristian-lesko

This comment has been minimized.

Copy link
Author

commented Mar 7, 2019

Hello @gschueler,

we've found out that the same issue happens when trying to access logs from a job execution (/project//execution/renderOutput/):

Handler dispatch failed; nested exception is java.lang.NoClassDefFoundError: javax/servlet/WriteListener

Is this related? Could it be fixed in a similar way to the issue above, please? Thanks.

@sjrd218

This comment has been minimized.

Copy link
Contributor

commented Mar 7, 2019

@kristian-lesko That error should be fixed in the next release. It is this issue: #4497

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.