Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Defining datasource in groovy and using in XML routes #892

Closed
rejimathews opened this issue Jul 31, 2019 · 13 comments
Closed

Defining datasource in groovy and using in XML routes #892

rejimathews opened this issue Jul 31, 2019 · 13 comments
Labels
status/waiting-for-feedback Needs some feedback

Comments

@rejimathews
Copy link

rejimathews commented Jul 31, 2019

Hello !

Just need a quick help here. Bear with me as am not a groovy expert yet.
Since Camel k doesn't understand .. fragment, I tried adding a data-source bean via groovy script as follows

import org.apache.commons.dbcp.BasicDataSource;

context {
    registry {
		mysqlDataSource = {
		org.apache.commons.dbcp.BasicDataSource mysqlDataSource =  new org.apache.commons.dbcp.BasicDataSource()
        mysqlDataSource.url = 'jdbc:mysql://10.213.96.182/test'
        mysqlDataSource.username = 'testuser'
		mysqlDataSource.password = '******'
		mysqlDataSource.driverClassName = 'com.mysql.jdbc.Driver'
		return mysqlDataSource
		}
    }
}

My route looks like

<?xml version="1.0" encoding="UTF-8"?>
<routes xmlns="http://camel.apache.org/schema/spring" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd">
   <route streamCache="true">
      <from uri="jetty:http://0.0.0.0:9090/address" />
      <log message="Received request to query table" />
      <to uri="sql:SELECT * FROM address?dataSource=mysqlDataSource" />
      <log message="Received records ${body}" />
      <convertBodyTo type="java.lang.String" />
   </route>
</routes>

But looks like Camel-K doesn't detect this registry entry.


[1] 2019-07-31 19:29:12.984 INFO  [main] DefaultCamelContext - Apache Camel 2.23.2 (CamelContext: camel-k) is shutdown in 0.013 seconds
[1] Exception in thread "main" org.apache.camel.FailedToCreateRouteException: Failed to create route route1 at: >>> To[sql:SELECT * FROM address?dataSource=mysqlDataSource] <<< in route: Route(route1)[[From[jetty:http://0.0.0.0:9090/address]] -> [... because of Failed to resolve endpoint: sql://SELECT%20*%20FROM%20address?dataSource=mysqlDataSource due to: DataSource must be configured
[1]     at org.apache.camel.model.RouteDefinition.addRoutes(RouteDefinition.java:1352)

@lburgazzoli
Copy link
Contributor

lburgazzoli commented Jul 31, 2019

Hello !

Just need a quick help here. Bear with me as am not a groovy expert yet.
Since Camel k doesn't understand .. fragment, I tried adding a data-source bean via groovy script as follows

This is wrong, camel-k-runtime supports what Camel does but as the runtime is not based on spring-boot or blueprint, you cannot define beans in the XML.

import org.apache.commons.dbcp.BasicDataSource;

context {
    registry {
		mysqlDataSource = {
		org.apache.commons.dbcp.BasicDataSource mysqlDataSource =  new org.apache.commons.dbcp.BasicDataSource()
        mysqlDataSource.url = 'jdbc:mysql://10.213.96.182/test'
        mysqlDataSource.username = 'testuser'
		mysqlDataSource.password = '******'
		mysqlDataSource.driverClassName = 'com.mysql.jdbc.Driver'
		return mysqlDataSource
		}
    }
}

This define a bean named mysqlDataSource with type Closure, you need something like:

def ds =  new org.apache.commons.dbcp.BasicDataSource()
ds.url = 'jdbc:mysql://10.213.96.182/test'
ds.username = 'testuser'
ds.password = '******'
ds.driverClassName = 'com.mysql.jdbc.Driver'

context {
    registry {
        mysqlDataSource = ds
    }
}

Or

context {
    registry {
        mysqlDataSource = [ url: '...', username: '....' ]  as  BasicDataSource
    }
}

If you use camel-k from master, then you can do something like:

beans {
    dataSource(org.apache.commons.dbcp2.BasicDataSource) {
        driverClassName = "org.h2.Driver"
        url = "jdbc:h2:mem:camel"
        username = "sa"
        password = ""
    }
}

My route looks like

<?xml version="1.0" encoding="UTF-8"?>
<routes xmlns="http://camel.apache.org/schema/spring" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd">
   <route streamCache="true">
      <from uri="jetty:http://0.0.0.0:9090/address" />
      <log message="Received request to query table" />
      <to uri="sql:SELECT * FROM address?dataSource=mysqlDataSource" />
      <log message="Received records ${body}" />
      <convertBodyTo type="java.lang.String" />
   </route>
</routes>

But looks like Camel-K doesn't detect this registry entry.


[1] 2019-07-31 19:29:12.984 INFO  [main] DefaultCamelContext - Apache Camel 2.23.2 (CamelContext: camel-k) is shutdown in 0.013 seconds
[1] Exception in thread "main" org.apache.camel.FailedToCreateRouteException: Failed to create route route1 at: >>> To[sql:SELECT * FROM address?dataSource=mysqlDataSource] <<< in route: Route(route1)[[From[jetty:http://0.0.0.0:9090/address]] -> [... because of Failed to resolve endpoint: sql://SELECT%20*%20FROM%20address?dataSource=mysqlDataSource due to: DataSource must be configured
[1]     at org.apache.camel.model.RouteDefinition.addRoutes(RouteDefinition.java:1352)

@mathewsreji
Copy link

mathewsreji commented Sep 3, 2019

@lburgazzoli
I am trying to integrate with activemq. I have my groovy beans as follows

import org.apache.activemq.ActiveMQConnectionFactory;
import org.apache.activemq.camel.component.ActiveMQComponent;
import org.apache.activemq.pool.PooledConnectionFactory;
import org.apache.camel.component.jms.JmsConfiguration;
context {
    registry {
	
activeMQConnectionFactory = [ brokerURL: 'tcp://localhost:61616', userName: 'admin', password: 'admin' ] as ActiveMQConnectionFactory
pooledConnectionFactory = [ maxConnections: '8', connectionFactory: 'activeMQConnectionFactory' ]  as  PooledConnectionFactory 
jmsConfig = [ connectionFactory: 'pooledConnectionFactory', concurrentConsumers: '2', maxConcurrentConsumers: '10' ] as JmsConfiguration
activemq = [ configuration: 'jmsConfig'] as ActiveMQComponent
}
}

and route as follows

<?xml version="1.0" encoding="UTF-8"?>
<routes xmlns="http://camel.apache.org/schema/spring" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd">
   <route streamCache="true">
      <from uri="activemq:queue:test.queue" />
      <log message="Received request to process" />
   </route>
</routes>

Camel K build doesn't complete and I see following message

ltamehrp7v0","api-version":"camel.apache.org/v1alpha1","kind":"IntegrationContext","ns":"myproject","name":"ctx-bln9je2uoltamehrp7v0","phase":"Error","error":"failure while determining classpath: exit status 1","stacktrace":"github.com/apache/camel-k/vendor/github.com/go-logr/zapr.(*zapLogger).Error\n\t/home/nferraro/gopaths/camel-k/src/github.com/apache/camel-k/vendor/github.com/go-logr/zapr/zapr.go:128\ngithub.com/apache/camel-k/pkg/util/log.Logger.Error\n\t/home/nferraro/gopaths/camel-k/src/github.com/apache/camel-k/pkg/util/log/log.go:70\ngithub.com/apache/camel-k/pkg/controller/integrationcontext.(*buildAction).handleBuildRunning\n\t/home/nferraro/gopaths/camel-k/src/github.com/apache/camel-k/pkg/controller/integrationcontext/build.go:214\ngithub.com/apache/camel-k/pkg/controller/integrationcontext.(*buildAction).Handle\n\t/home/nferraro/gopaths/camel-k/src/github.com/apache/camel-k/pkg/controller/integrationcontext/build.go:61\ngithub.com/apache/camel-k/pkg/controller/integrationcontext.(*ReconcileIntegrationContext).Reconcile\n\t/home/nferraro/gopaths/camel-k/src/github.com/apache/camel-k/pkg/controller/integrationcontext/integrationcontext_controller.go:138\ngithub.com/apache/camel-k/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/nferraro/gopaths/camel-k/src/github.com/apache/camel-k/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:215\ngithub.com/apache/camel-k/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\t/home/nferraro/gopaths/camel-k/src/github.com/apache/camel-k/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:158\ngithub.com/apache/camel-k/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/home/nferraro/gopaths/camel-k/src/github.com/apache/camel-k/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133\ngithub.com/apache/camel-k/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/home/nferraro/gopaths/camel-k/src/github.com/apache/camel-k/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:134\ngithub.com/apache/camel-k/vendor/k8s.io/apimachinery/pkg/util/wait.Until\n\t/home/nferraro/gopaths/camel-k/src/github.com/apache/camel-k/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88"}

Any examples on connecting to activeMQ from camel k projects?

@mathewsreji
Copy link

in pooledConnectionFactory, the connectionFactory is reference to another bean defined above that named activeMQConnectionFactory. I hope its the right way to provide reference to registrybeans

@mathewsreji
Copy link

I was referring http://www.javaoutofbounds.com/apache-camel-activemq-connection-pool-example/ to configure my beans

@mathewsreji
Copy link

Tried this too

import org.apache.activemq.ActiveMQConnectionFactory;
import org.apache.activemq.camel.component.ActiveMQComponent;
import org.apache.activemq.pool.PooledConnectionFactory;
import org.apache.camel.component.jms.JmsConfiguration;
context {
    registry {

activeMQConnectionFactory = [ brokerURL: 'tcp://localhost:61616', userName: 'admin', password: 'admin' ] as ActiveMQConnectionFactory

pooledConnectionFactory = [ maxConnections: 8, setMaximumActiveSessionPerConnection: 20, blockIfSessionPoolIsFull: true, idleTimeout:50, connectionFactory: grep('activeMQConnectionFactory') ]  as  PooledConnectionFactory

jmsConfig = [ connectionFactory: grep('pooledConnectionFactory'), concurrentConsumers: 10, replyToMaxConcurrentConsumers: 10 ] as JmsConfiguration

activemq = [ configuration: grep('jmsConfig')] as ActiveMQComponent
        }
}

@WillemJiang
Copy link
Member

@contactreji
camel-k groovy provides a components DSL to help you set the activemq component properties in the closure just like this

import org.apache.activemq.camel.component.ActiveMQComponent;

context {
    components {
        activeMQ(ActiveMQComponent) {
            brokerURL = 'tcp://localhost:61616'
            userName = 'admin'
            password = 'admin'
        }
    }
}

@mathewsreji
Copy link

mathewsreji commented Sep 6, 2019

org.apache.activemq.camel.component

@WillemJiang - I get following error while starting the app.

011 seconds
[1] Exception in thread "main" org.apache.camel.FailedToCreateRouteException: Failed to create route route1: Route(route1)[[From[activemq:queue:TESTINGCONNECTION]] -> [L... because of Failed to resolve endpoint: activemq://queue:TESTINGCONNECTION due to: Username must also be provided when using username/password as credentials.
[1]     at org.apache.camel.model.RouteDefinition.addRoutes(RouteDefinition.java:217)
[1]     at org.apache.camel.impl.DefaultCamelContext.startRoute(DefaultCamelContext.java:1140)
[1]     at org.apache.camel.impl.DefaultCamelContext.startRouteDefinitions(DefaultCamelContext.java:3735)
[1]     at org.apache.camel.impl.DefaultCamelContext.doStartCamel(DefaultCamelContext.java:3440)
[1]     at org.apache.camel.impl.DefaultCamelContext$4.call(DefaultCamelContext.java:3248)

Route file

<?xml version="1.0" encoding="UTF-8"?>
<routes xmlns="http://camel.apache.org/schema/spring" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd">
   <route streamCache="true">
      <from uri="activemq:queue:TESTINGCONNECTION"/>
      <log message="Received request to process" />
   </route>
</routes>

Groovy file

import org.apache.activemq.camel.component.ActiveMQComponent;

context {
    components {
        activemq(ActiveMQComponent) {
            brokerURL = 'tcp://localhost:61616'
            userName = 'admin'
            password = 'admin'
        }
    }
}

Camel command used

kamel run --dev --name jmsintegration --trait service.port=1212 --dependency mvn:org.apache.camel/camel-jsonpath:2.24.1 --dependency mvn:org.apache.camel/camel-http4:2.24.1 --dependency mvn:org.apache.camel/camel-jetty:2.24.1 --dependency mvn:org.apache.camel/camel-jackson:2.24.1 --dependency mvn:org.apache.camel/camel-jaxb:2.24.1 --dependency  mvn:org.apache.camel/camel-restlet:2.24.1 --dependency mvn:org.apache.camel/camel-sql:2.24.1 --dependency  mvn:mysql/mysql-connector-java:5.1.34 --dependency mvn:commons-dbcp/commons-dbcp:1.4 --dependency mvn:org.apache.activemq/activemq-camel:5.15.10 --logging-level org.apache.camel.k=DEBUG routes.xml beans.groovy

@WillemJiang
Copy link
Member

@contactreji Sorry, there is a typo on the 'userName' setting, the right one should be 'username'. I think we could update the DSL by showing the wrong setting on the component properties.

@WillemJiang
Copy link
Member

Created an related issue apache/camel-k-runtime#137 in camel-k-runtime.

@mathewsreji
Copy link

Perfect. that worked! thanks @WillemJiang

@lburgazzoli
Copy link
Contributor

@contactreji can we close this one ?

@lburgazzoli lburgazzoli added the status/waiting-for-feedback Needs some feedback label Jan 1, 2020
@mathewsreji
Copy link

Closing this

@mathewsreji
Copy link

@contactreji can we close this one ?

You can close this. thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status/waiting-for-feedback Needs some feedback
Projects
None yet
Development

No branches or pull requests

4 participants