Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OGM-482 EAP JBoss Module #307

Closed
wants to merge 18 commits into from
Closed

Conversation

DavideD
Copy link
Member

@DavideD DavideD commented Mar 20, 2014

https://hibernate.atlassian.net/browse/OGM-482

This patch will create a module for Wildfly and a module fo EAP 6.2.
Setting the environment variable EAP_HOME will make the integration test run using EAP instead of WildFly. Otherwise wildfly is used. Notice that only one container is used for the integration tests.
Is this an accettable solution? Not sure how to run on both containers during the build without creating other submodules, anyone has a nice solution?

To activate the EAP profile I've opted for the activation through an environment variable because is consistent with all the other projects but I would be prone to use a system variable and run mvn -Deap.home=$EAP_HOME instead. What do you guys think?

The module will replace the default javax.persistence.api and the default javax.ws.rs.api modules on EAP because it needs later versions.

@Sanne
Copy link
Member

Sanne commented Mar 24, 2014

Hi Davide, I think we need two submodules rather than playing with environment variables: when releasing, I need you to release both modules not one.

I like the idea of a eap.home variable to test on a different version that I might want to provide, but I think by default you should take the publicly available EAP version from the Red Hat maven repositories (as we do in Hibernate Search branch 4.4)

@Sanne
Copy link
Member

Sanne commented Mar 24, 2014

About slot names; I saw you're using

<module xmlns="urn:jboss:module:1.1" name="org.hibernate" slot="ogm">

I really think we should include the version number in the slot. It could be `slot="ogm-$version"
or maybe even better to use

<module xmlns="urn:jboss:module:1.1" name="org.hibernate.ogm" slot="$version">

?

@Sanne
Copy link
Member

Sanne commented Mar 24, 2014

Why is it that you decided to have separate modules for each backend (nice) but Infinispan is fully included in the OGM module?

@Sanne
Copy link
Member

Sanne commented Mar 24, 2014

<module name="org.jboss.jts" export="true" />

Can we do something about that? and are you sure it needs to be exported too?

@Sanne
Copy link
Member

Sanne commented Mar 24, 2014

<module name="org.jboss.as.jpa.hibernate" slot="4" optional="true" />

Could you explain about this one? And why is it needed by some dialects but not all of them?

@Sanne
Copy link
Member

Sanne commented Mar 24, 2014

I see Neo4J is including the Lucene jar, you could use the module in Widlfly:
<module xmlns="urn:jboss:module:1.1" name="org.apache.lucene">
(we seem lucky it's the same version)

@Sanne
Copy link
Member

Sanne commented Mar 24, 2014

<module name="org.slf4j" />

This seems required by several modules. Are you sure ?

@Sanne
Copy link
Member

Sanne commented Mar 24, 2014

I suspect we're a bit lacking in integration tests :-/
Keep in mind that getting these dependencies "right" is quite tricky: I'd bet that any module which isn't tested is a module which doesn't work.

@Sanne
Copy link
Member

Sanne commented Mar 24, 2014

P.S. WildFly 8.0.1.Final might be released this week, if it wasn't tagged already (yes it's very late), so if you see some obvious improvement to change in the current structure of WildFly's provided modules, that could still make it but you'll need to focus on it.

@Sanne
Copy link
Member

Sanne commented Mar 24, 2014

Are you sure about needing the older ANTLR and StringTemplace libraries in the HQL module? That's unexpected, if they are really required please open a JIRA to have me fix that.

@DavideD
Copy link
Member Author

DavideD commented Mar 24, 2014

I have to check if it is possible to apply some of your remarks, in the meanwhile:

Hi Davide, I think we need two submodules rather than playing with environment variables: when releasing, I need you to release both modules not one.

I've created two submodules under the module project.
Both modules are released at once, but only one is tested. I don't like this very much as well but we can limit the damage using jobs on jenkins. I'm not sure how to test both at the same time since they require two different arquillian containers.

About slot names; I saw you're using

<module xmlns="urn:jboss:module:1.1" name="org.hibernate" slot="ogm">

I really think we should include the version number in the slot. It could be `slot="ogm-$version"
or maybe even better to use

<module xmlns="urn:jboss:module:1.1" name="org.hibernate.ogm" slot="$version">

?

This was a requirement when we have created the first OGM module, It might not be needed anymore, I will check. The ${project.version} as slot name is good for me.

Why is it that you decided to have separate modules for each backend (nice) but Infinispan is fully included in the OGM module?

I don't remember why we have decided to do this, actually I've thought is something you proposed :)
I'm fine in separating it.

Are you sure about needing the older ANTLR and StringTemplace libraries in the HQL module? That's unexpected, if they are really required please open a JIRA to have me fix that.

These versions are taken from the dependencies included in the project. I'm expecting this one is taken from the dependencies of the parser.weare using, unless something else overrides it.

I suspect we're a bit lacking in integration tests :-/
Keep in mind that getting these dependencies "right" is quite tricky: I'd bet that any module which isn't tested is a module which doesn't work.

Tests are always lacking. We don't test all the functionality in the containers but we would need a way to run all the "unit" test cases we already have on the container to increase the coverage. I would treat this as a separate issue though.

I like the idea of a eap.home variable to test on a different version that I might want to provide, but I think by default you should take the publicly available EAP version from the Red Hat maven repositories (as we do in Hibernate Search branch 4.4)

ok

I see Neo4J is including the Lucene jar, you could use the module in Widlfly:
<module xmlns="urn:jboss:module:1.1" name="org.apache.lucene">
(we seem lucky it's the same version)

Ok, I guess that when we have created the module the version was different and I didn't change the modules when I've upgraded the test to use WildFly 8

@Sanne
Copy link
Member

Sanne commented Mar 24, 2014

The ${project.version} as slot name is good for me.

Note that I changed the module name: would that be a problem interfering with org.hibernate modules?

Why is it that you decided to have separate modules for each backend (nice) but Infinispan is fully included in the OGM module?

I don't remember why we have decided to do this, actually I've thought is something you proposed :) I'm fine in separating it.

That's probably how things where before we actually had more modules.

Tests are always lacking. We don't test all the functionality in the containers but we would need a way to run all the "unit" test cases we already have on the container to increase the coverage. I would treat this as a separate issue though.

+1 For a follow up which expands on testing, but -1 for no tests at all of these modules: if you don't even have a basic operations tests I'm sure they won't work. If you think you need to simplify, maybe we should cut on the provided modules?
If a specific module can't be tested yet I'd rather not release it and focus on the other modules.

@DavideD
Copy link
Member Author

DavideD commented Mar 24, 2014

Apart the Neo4j one, which other module is not tested?
On 24 Mar 2014 13:28, "Sanne Grinovero" notifications@github.com wrote:

The ${project.version} as slot name is good for me.

Note that I changed the module name: would that be a problem interfering
with org.hibernate modules?

Why is it that you decided to have separate modules for each backend
(nice) but Infinispan is fully included in the OGM module?

I don't remember why we have decided to do this, actually I've thought is
something you proposed :) I'm fine in separating it.

That's probably how things where before we actually had more modules.

Tests are always lacking. We don't test all the functionality in the
containers but we would need a way to run all the "unit" test cases we
already have on the container to increase the coverage. I would treat this
as a separate issue though.

+1 For a follow up which expands on testing, but -1 for no tests at all of
these modules: if you don't even have a basic operations tests I'm sure
they won't work. If you think you need to simplify, maybe we should cut on
the provided modules?
If a specific module can't be tested yet I'd rather not release it and
focus on the other modules.

Reply to this email directly or view it on GitHubhttps://github.com//pull/307#issuecomment-38443577
.

@emmanuelbernard
Copy link
Member

Reading this super fast so that's just a quick comment. I don't want to impose 30s of additional test time nor impose the installation of EAP.

@Sanne
Copy link
Member

Sanne commented Mar 24, 2014

@emmanuelbernard I don't see how we can make modules and not test them.
+1 to not enforce a manual EAP download, but the extra IT tests will indeed take some extra seconds, multiplied X GridDialect and again X target platform (WildFly + EAP).

We might want to reorganize tests in proper unit tests, and defer integration testing to CI only?

Also note that I expect the work on Docker to speedup overall execution of tests, but I can't serialize this task on that one (and it's just a theory)

@DavideD
Copy link
Member Author

DavideD commented Mar 24, 2014

@Sanne Search is downloading EAP 6.1 Alpha for testing the modules but the module I've created are tested with EAP 6.2. Which one should I use?

@Sanne
Copy link
Member

Sanne commented Mar 24, 2014

EAP 6.1 Alpha is the only one available in Maven so that's what we need to use :-/
Bonus points if you make it easy to override with a different version that I could have downloaded manually (on my personal workstation or on CI).

@DavideD
Copy link
Member Author

DavideD commented Mar 25, 2014

The ${project.version} as slot name is good for me.

Note that I changed the module name: would that be a problem interfering with org.hibernate modules?

As far as I know it is, I've tried to change the group but it does not seem to work.

@DavideD
Copy link
Member Author

DavideD commented Mar 25, 2014

Can we do something about that? and are you sure it needs to be exported too?

There is no need to include it in every module but it is required in the core module with export = "true"

@DavideD
Copy link
Member Author

DavideD commented Mar 26, 2014

I've rebased the pull request and I've applied the remarks where possible.

There are two profiles added, one downloads EAP 6.1 from the repository and the other one will use the one passed through a variable. These profiles are activated if specified in the build or if the variable -Deap.home is defined when running the build.

@gunnarmorling
Copy link
Member

Jenkins, retest this please.

-->
<id>eap</id>
<properties>
<version.arquillian>1.0.3.Final</version.arquillian>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need specific Arquillian versions for the two profiles?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DavideD, could one version of Arquillian be used?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems we can use the latest version in both scenarios. I will change it

@gunnarmorling
Copy link
Member

@DavideD I just ran the build (in the default config), and this executed the integration tests on WF. What would I need to do run them on EAP? Specify that other profile by name? Is it also possible to run them on WF and EAP in one go?

@DavideD
Copy link
Member Author

DavideD commented Mar 31, 2014

The default container is wildfly, you can run the test using mvn install -Deap or mvn install -Deap.home=<jboss_home>

Arquillian does not allow two different arquillian containers on the same classpath, the test of both containers would require the creation of two additional modules for each datastore (or something like that).

Anyway there is a discussion about this on the mailing list: http://web.archiveorange.com/archive/v/SMVGgmRVpCrtD6zS7e4T

@DavideD
Copy link
Member Author

DavideD commented Mar 31, 2014

Sorry, I made a mistake, you can use this two profiles:

  • Download EAP 6.1
    mvn install -Peap
  • Use existing EAP 6.x
    mvn install -Deap.home=$EAP_HOME

@gunnarmorling
Copy link
Member

you can use this two profiles

Got it. I originally thought it would be good to be able to run both configurations as part of one Maven build (which could be done via different Maven modules as you say), but probably that's not even required and we can do this solely on the CI server via a Matrix job.

Thinking along these lines, we could have just a single integration test module which is configurable on two axes:

  • The configured datastore (requires some routine for starting/stopping the store as well as a persistence.xml with the required configuration)
  • The server to deploy to (EAP, WF; requires specific routines for preparing the right modules)

By default (i.e. when running mvn clean install) we'd just run one configuration (say ISPN on WF), while on CI we'd have a Matrix job which tests all the combinations of stores and servers (if we wish to do so).

</profile>
<profile>
<!--
This profile will download EAP 6.1.Alpha1 from the Red Hat repository but it requires
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add the URL of that repo to this comment? I don't know where that one lives.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's in the settings-example.xml, not sure if I want to duplicate it here in the commments

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, just saw it in the settings as well. That's fine then.

@janinko
Copy link

janinko commented Apr 7, 2014

Hi,
I tried this with our project on EAP 6.1.1 and got stuck with one exception. We are using @Any annotation and when I just place the modules into EAP and build&run the project it's ok. When I add new persistence unit (neo4jPU) I'm getting exception when deploying.

Persistence unit:

  <persistence-unit name="relationdbPU">
    <provider>org.hibernate.ejb.HibernatePersistence</provider>
    <jta-data-source>java:jboss/datasources/MysqlDS</jta-data-source>
    <properties>
      <property name="org.hibernate.FlushMode" value="always" />
      <property name="hibernate.dialect" value="org.hibernate.dialect.MySQL5Dialect"/>
      <property name="hibernate.hbm2ddl.auto" value="update"/>
      <property name="hibernate.show_sql" value="false"/>
      <property name="hibernate.format_sql" value="false"/>
    </properties>
  </persistence-unit>
  <persistence-unit name="neo4jPU">
    <provider>org.hibernate.ogm.jpa.HibernateOgmPersistence</provider>
    <properties>
      <property name="hibernate.transaction.jta.platform" value="org.hibernate.service.jta.platform.internal.JBossStandAloneJtaPlatform" />
      <property name="hibernate.ogm.datastore.provider" value="neo4j_embedded" />
      <property name="hibernate.ogm.neo4j.database_path" value="tmp/hbn4jqwert" />
    </properties>
  </persistence-unit>

Exception:

14:01:23,089 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 48) MSC000001: Failed to start service jboss.persistenceunit."build-tracker.ear/build-tracker-backend.jar#relationdbPU": org.jboss.msc.service.StartException in service jboss.persistenceunit."build-tracker.ear/build-tracker-backend.jar#relationdbPU": javax.persistence.PersistenceException: [PersistenceUnit: relationdbPU] Unable to build EntityManagerFactory
    at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1.run(PersistenceUnitServiceImpl.java:103)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_51]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_51]
    at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_51]
    at org.jboss.threads.JBossThread.run(JBossThread.java:122) [jboss-threads-2.1.0.Final-redhat-1.jar:2.1.0.Final-redhat-1]
Caused by: javax.persistence.PersistenceException: [PersistenceUnit: relationdbPU] Unable to build EntityManagerFactory
    at org.hibernate.ejb.Ejb3Configuration.buildEntityManagerFactory(Ejb3Configuration.java:930)
    at org.hibernate.ejb.Ejb3Configuration.buildEntityManagerFactory(Ejb3Configuration.java:904)
    at org.hibernate.ejb.HibernatePersistence.createContainerEntityManagerFactory(HibernatePersistence.java:92)
    at org.jboss.as.jpa.service.PersistenceUnitServiceImpl.createContainerEntityManagerFactory(PersistenceUnitServiceImpl.java:200)
    at org.jboss.as.jpa.service.PersistenceUnitServiceImpl.access$600(PersistenceUnitServiceImpl.java:57)
    at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1.run(PersistenceUnitServiceImpl.java:99)
    ... 4 more
Caused by: org.hibernate.AnnotationException: Property org.jboss.buildtracker.database.dao.model.Report.target has an unbound type and no explicit target entity. Resolve this Generic usage issue or set an explicit target attribute (eg @OneToMany(target=) or use an explicit @Type
    at org.hibernate.cfg.PropertyContainer.assertTypesAreResolvable(PropertyContainer.java:140)
    at org.hibernate.cfg.PropertyContainer.getProperties(PropertyContainer.java:118)
    at org.hibernate.cfg.AnnotationBinder.addElementsOfClass(AnnotationBinder.java:1356)
    at org.hibernate.cfg.InheritanceState.getElementsToProcess(InheritanceState.java:236)
    at org.hibernate.cfg.AnnotationBinder.bindClass(AnnotationBinder.java:665)
    at org.hibernate.cfg.Configuration$MetadataSourceQueue.processAnnotatedClassesQueue(Configuration.java:3468)
    at org.hibernate.cfg.Configuration$MetadataSourceQueue.processMetadata(Configuration.java:3422)
    at org.hibernate.cfg.Configuration.secondPassCompile(Configuration.java:1349)
    at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:1749)
    at org.hibernate.ejb.EntityManagerFactoryImpl.<init>(EntityManagerFactoryImpl.java:94)
    at org.hibernate.ejb.Ejb3Configuration.buildEntityManagerFactory(Ejb3Configuration.java:920)
    ... 9 more

Entity code:

@Entity
public class Report<T> implements Serializable {

    @Any(metaColumn = @Column(name = "target_type"))
    @AnyMetaDef(idType = "long", metaType = "string",
            metaValues = {
                    @MetaValue(targetEntity = Project.class, value = "P"),
                    @MetaValue(targetEntity = Release.class, value = "R"),
                    @MetaValue(targetEntity = Artifact.class, value = "A")
            })
    @JoinColumn(name = "target_id")
    private T target;

   ...

It probably doesn't really relate to this PR, but I hope you could help me.

Thanks, Honza

@DavideD
Copy link
Member Author

DavideD commented Apr 7, 2014

@janinko I'm pretty sure the problem you are having is not related to this issue but it is a problem of the way you are mapping the entity.
You should ask this question on the hibernate forum.

My guess is that you are using generics as type for target, you could try with a common interface or Object

@DavideD
Copy link
Member Author

DavideD commented Apr 7, 2014

I've rebased everything to the latest changes and unified the arquillian version.

@janinko
Copy link

janinko commented Apr 8, 2014

And do I understand it correctly that when I add the neo4jPU persistence unit, EAP uses hibernate-core-4.3.1.Final (from the modules zip I added to the EAP) instead of hibernate-core-4.2.0.SP1-redhat-1 (which is in EAP 6.1.1)? And when the neo4jPU isn't ther it uses the old hibernate-core-4.2.0.SP1-redhat-1? So I see the error because of some change in Hibernate in 4.3.1?

@DavideD
Copy link
Member Author

DavideD commented Apr 8, 2014

@janinko
Do you think you could provide a test case? Maybe you could add the classes in this project so that I can check.

The latest OGM requires hibernate core 4.3.1 so if you include the module in your application it will include that version in the classpath.

<configuration>
<artifactItems>
<artifactItem>
<groupId>${project.groupId}</groupId>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel we shouldn't mess with the modules dir of the external installation but instead assume it has been updated accordingly. This might not be what users with a separat inst. might expect.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be useful to run the integration test an a complete custom installation but when I remove the unpacking of the modules from the pom the build does not work. It seems some dependncies are missing. I would prefer to solve this as a separte issue and include what we have at the moment.

@gunnarmorling
Copy link
Member

@DavideD Nice stuff, this makes the demo work on EAP :)

In addition to the inline comments, some more remarks:

  • Could you update the ref guide (it should reflect that we have two module ZIPs now, one for AS7/EAP and one for WF) and add a short not on the different profiles to README.md?
  • We also might name the "eap" module "...jboss-as-7..." instead and mention in the docs that it also works with the corresponding product version. It's just a thought (at least to me the "eap" naming somehow suggests that it only works with EAP but not the community project).
  • The created ZIPs contain "wildfly"/"eap" twice in their name, e.g. hibernate-ogm-modules-wildfly-4.1.0-SNAPSHOT-wildfly-8-dist.zip. Shouldn't it just be something like hibernate-ogm-modules-wildfly-4.1.0-SNAPSHOT-dist.zip or better hibernate-ogm-modules-wildfly8-4.1.0-SNAPSHOT-dist.zip?

<relativePath>../pom.xml</relativePath>
</parent>

<artifactId>hibernate-ogm-modules-wildfly</artifactId>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel we should name it hibernate-ogm-modules-wildfly8 (and hibernate-ogm-modules-eap6 or hibernate-ogm-modules-jbossas7).

@DavideD
Copy link
Member Author

DavideD commented Apr 10, 2014

I've rebased and added a couple of new commits to apply the remarks.

@DavideD Nice stuff, this makes the demo work on EAP :)

Thanks, good to know :)

Could you update the ref guide (it should reflect that we have two module ZIPs now, one for AS7/EAP and one for WF) and add a short not on the different profiles to README.md?

Updated

We also might name the "eap" module "...jboss-as-7..." instead and mention in the docs that it also works with the corresponding product version. It's just a thought (at least to me the "eap" naming somehow suggests that it only works with EAP but not the community project).

I think our goal at the moment is to make it work with EAP and WildFly. The fact that it works with JBoss AS 7 is because of the way EAP is built. We did not test it with the community version.
But I don't know what other projects do in this case.

The created ZIPs contain "wildfly"/"eap" twice in their name, e.g. hibernate-ogm-modules-wildfly-4.1.0-SNAPSHOT-wildfly-8-dist.zip. Shouldn't it just be something like hibernate-ogm-modules-wildfly-4.1.0-SNAPSHOT-dist.zip or better hibernate-ogm-modules-wildfly8-4.1.0-SNAPSHOT-dist.zip?

Now the name of the modules artifact are:

  • hibernate-ogm-modules-wildfly8
  • hibernate-ogm-modules-eap6

@gunnarmorling
Copy link
Member

@DavideD Any idea what's wrong with the test build?

@DavideD
Copy link
Member Author

DavideD commented Apr 11, 2014

Jenkins, retest this please

@DavideD
Copy link
Member Author

DavideD commented Apr 11, 2014

In the console, there is this error during the creation of the distribution package:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-assembly-plugin:2.4:single (make-assembly) on project hibernate-ogm-distribution: Failed to create assembly: Error creating assembly archive dist: Problem creating TAR: request to write '0' bytes exceeds size in header of '-1619622021' bytes -> [Help 1]

It happens for the pull requests only, does it ring any bells?

@gunnarmorling
Copy link
Member

Rebased and applied. Many thanks for this, Davide! I've created OGM-499 for some of the discussed follow-ups.

Regarding the build failure I'm not sure; I don't see the issue locally. Let's see whether it occurs on master as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants