Kundera will not scan entities from a JAR file #90

Closed
dstrawson opened this Issue Jul 27, 2012 · 28 comments

Comments

Projects
None yet
4 participants

Hi -

Not sure if this is a bug or a feature request. If you take the view that this is a feature then I understand!

My entities are in a separate file (core.jar) which is included in my application as a dependency. This is because it is shared between modules. Kundera doesn't find them - neither when I use in the persistence.xml file, nor when I use kundera.annotations.scan.package. As soon as I un-jar the core.jar file into my classes/ folder, everything starts working as expected.

Setup:

Kundera-2.0.7
Cassandra-1.1.2
Maven
Sping / CXF

Collaborator

mevivs commented Jul 27, 2012

Did you try adding jar file undera
"jar-file" tag in your persistence.xml?

-Vivek

Hi Vivek -

Thanks - I admit I wasn't aware of that option - however its not ideal as the name and location of the class file in production will change, so its not the best solution I feel. My jar file will move around between development, staging and live servers, and change name between versions, although I do realise of course that I could alleviate the version issue by filtering the files at build time in maven.

Also, according to the jpa spec, my reading is that should be enough to find classes, and jar-file is used to search specific jar-files rather in the same way as kundera.annotations.scan.package, these jar files should also be specified relative to the directory or jar that contains the root of the persistence unit, so for example lib/core-1.2.3-SNAPSHOT.jar.

Daniel

Collaborator

mevivs commented Jul 28, 2012

Hi,
As per JPA:
Any entity class present within root folder of( where your persistence.xml resides) will be automatically scanned and loaded, else you need to define those as part of "mapping-class" tag OR "jar-file" tag.

Here are some excerpt about loading classes from JPA specification:


/**

  • Returns the list of the names of the mapping files that the
  • persistence provider must load to determine the mappings for
  • the entity classes. The mapping files must be in the standard
  • XML mapping format, be uniquely named and be resource-loadable
  • from the application classpath. Each mapping file name
  • corresponds to a mapping-file element in the
  • persistence.xml file.
  • @return the list of mapping file names that the persistence
  • provider must load to determine the mappings for the entity
  • classes
    */
    public List getMappingFileNames();

/**

  • Returns a list of URLs for the jar files or exploded jar
  • file directories that the persistence provider must examine
  • for managed classes of the persistence unit. Each URL
  • corresponds to a jar-file element in the
  • persistence.xml file. A URL will either be a file: URL
  • referring to a jar file or referring to a directory
  • that contains an exploded jar file, or some other URL from
  • which an InputStream in jar format can be obtained.
  • @return a list of URL objects referring to jar files or
  • directories
    */
    public List getJarFileUrls();

/**

  • Returns the URL for the jar file or directory that is the
  • root of the persistence unit. (If the persistence unit is
  • rooted in the WEB-INF/classes directory, this will be the
  • URL of that directory.)
  • The URL will either be a file: URL referring to a jar file
  • or referring to a directory that contains an exploded jar
  • file, or some other URL from which an InputStream in jar
  • format can be obtained.
  • @return a URL referring to a jar file or directory
    */
    public URL getPersistenceUnitRootUrl();

Scanning complete classpath for entities is somewhat not desirable.

I hope am not missing anything on this front. Feel free to correct, if i missed something.

-Vivek

I'm reading persistence-2_0-final-spec.pdf - specifically 8.2.1.6.3 and 8.2.1.6.4.

"A list of named managed persistence classes may be specified instead of, or in addition to, the JAR files and mapping files. Any mapping metadata annotations found on these classes will be processed ... The class element is used to list a managed persistence class."

Where was the above from?

I agree that scanning complete classpaths isn't desirable (although I've never had this problem before with say hibernate) - however I'd hope not to have to name the jar files either. Ideally I should just list the class files that contain my entities, which would be loaded as normal. My reading of the spec is that this is allowed.

Collaborator

mevivs commented Jul 28, 2012

It was from 9.5 section. i am on same spec :)

As you mentioned for 8.2.1.x section, here it is:

"The managed persistence classes may either be contained within the root of the persistence unit; or they
may be specified by reference—i.e., by naming the classes, class archives, or XML mapping files
(which in turn reference classes) that are accessible on the application classpath; or they may be speci-
fied by some combination of these means. See Section 8.2.1.6.
" (this is from "8.2.1 persistence.xml file" section).
"The managed persistence classes may either be contained within the root of the persistence unit" , i state it as for example your persistence.xml is at "classes/META-INF" folder, so root would become to " classes" folder to automatically scan for "managed persistence classes" else you need to explicitly configure them. That is what my understanding is from this point.

With Hibernate, did you work in annotation way or defined any orm.xml or hbm.xml ? As my understanding is without them or not defining them within "mapping-class" or "mapping-file" tag, it should complain about it, unless "managed instances" are not in the root folder of persistence.xml

Would love to hear your point on this.

-Vivek

Collaborator

mevivs commented Jul 28, 2012

Just noticed that you mentioned:
"Ideally I should just list the class files that contain my entities, which would be loaded as normal. My reading of the spec is that this is allowed"

That is what i am saying, should be the way. So if you have defined "mapping-class" and mapped your managed instances with them, they should automatically get loaded, provided your "jar" containing them should be in classpath!

-Vivek

Collaborator

mevivs commented Jul 28, 2012

Please share your persistence.xml. Does it contain "class" tag mapping for managed instances (available in core.jar). Then Kundera should be able to load them provided that those jars are available with "Thread.currentThread().getContextClassLoader()".

Or
Your project is based on OSGI module based class loading?

For a moment there I thought I'd done something dumb like not included the jar in the classpath, however it would never reach persist() if I had. Not knowingly using OSGI, no - I'm using spring, but spring-modules isn't involved.

Given what you are saying I'm worried that I've done something wrong, so I'll build a much smaller simpler example to replicate if that's ok, as I think I might be wasting your time otherwise,

My persistence.xml file:

<persistence xmlns="http://java.sun.com/xml/ns/persistence" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://java.sun.com/xml/ns/persistence
    http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd"
    version="2.0">
    <persistence-unit name="cassandra">
        <provider>com.impetus.kundera.KunderaPersistence</provider>
        <class>co.twickets.core.Application</class>
        <properties>            
            <property name="kundera.nodes" value="localhost"/>
            <property name="kundera.port" value="9160"/>
            <property name="kundera.keyspace" value="Twickets"/>
            <property name="kundera.dialect" value="cassandra"/>
            <property name="kundera.client.lookup.class" value="com.impetus.client.cassandra.pelops.PelopsClientFactory" />
            <property name="kundera.annotations.scan.package" value="co.twickets.core" />
            <property name="kundera.ddl.auto.prepare" value="update"/>
        </properties>       
    </persistence-unit>
</persistence>
Collaborator

mevivs commented Jul 28, 2012

Hi Daniel,
This configuration looks perfectly FINE to me.

Can you share your source code to have a look and test it at my end please?

-Vivek

Collaborator

mevivs commented Jul 28, 2012

Hi Daniel,
I think i have found out the issue. Looks like there is bug within Kundera. I will update you soon on this.

-Vivek

Collaborator

mevivs commented Jul 28, 2012

Could you please try with latest "trunk". I have added a fix for this.

Please verify.

-Vivek

Great - I've actually just built a smaller example to show the issue.

I'll try it with the trunk as you suggest.

Thanks for your help by the way :)

Collaborator

mevivs commented Jul 28, 2012

Thanks to you for pointing towards the issue :) Process was as per jpa spec, but Kundera missed out those scanned classes. It should work now.

-Vivek

OK, I've pushed an example to https://github.com/dstrawson/kunderatest. It also demonstrates issue 91, although I've not gone back to that yet, which I will do when I have this working.

I've pulled trunk / built / installed your new version and updated my dependencies / rebuilt but not managed to get it working with this example, still the same issue - when I built it for the first time it failed a UT however, so will need to go back and see why it failed the test.

Collaborator

mevivs commented Jul 28, 2012

Thanks. Will have a look and get back to you.

-Vivek

Hi Vivek -

When I build kundera-cassandra 2.0.8-SNAPSHOT I get the following :

<<< FAILURE!
Running com.impetus.client.twitter.TwissandraTest
com.impetus.kundera.loader.MetamodelLoaderException: Name conflict between classes com.impetus.client.crud.countercolumns.Counters and com.impetus.client.crud.countercolumns.Counters. Make sure no two entity classes with the same name are specified for persistence unit twissandraTest
at com.impetus.kundera.configure.MetamodelConfiguration.scanClassAndPutMetadata(MetamodelConfiguration.java:303)
at com.impetus.kundera.configure.MetamodelConfiguration.loadEntityMetadata(MetamodelConfiguration.java:212)
at com.impetus.kundera.configure.MetamodelConfiguration.configure(MetamodelConfiguration.java:97)

Collaborator

mevivs commented Jul 28, 2012

Try with mvn clean install -Dmaven.test.skip=true

Sure, that works, however I cannot get that fix to show itself I'm afraid so was assuming that the broken test might have something to do with it.

There is where I am with 2.0.8-SNAPSHOT:

  • If I only include elements in persistence.xml Kundera fails to pick up the entity classes. I think If I understood you correctly you were saying that this should work ok.?
  • If I additionally add jar-file it works ok eg ../core-package/target/core.jar - as I say above not ideal for production as will reference the version, but we could do something about that with a maven resource filter, so workable.
  • I can also unpack the jar file, where again it works ok, so there is another option there.
Collaborator

mevivs commented Jul 29, 2012

If I only include elements in persistence.xml Kundera fails to pick up the entity classes. I think If I understood you correctly you were saying that this should work ok.?

Yes. This is what i believe should work. i will have a look with your sample example.

Till then, as a workaround other 2 options should be workable for you.

-Vivek

Collaborator

mevivs commented Jul 29, 2012

Ok. Added a fix for this and verified with your github example.

Please verify at your end.

-Vivek

Thanks - that's fixed it - I'm getting a different problem now HOWEVER its more an issue with the application server than Kundera, although it should probably be considered as an improvement:

 .... 
Caused by: com.impetus.kundera.loader.MetamodelLoaderException: Name conflict between classes co.twickets.core.Account and      co.twickets.core.Account. Make sure no two entity classes with the same name  are specified for persistence unit cassandra
        at com.impetus.kundera.configure.MetamodelConfiguration.scanClassAndPutMetadata(MetamodelConfiguration.java:303)
        at com.impetus.kundera.configure.MetamodelConfiguration.loadEntityMetadata(MetamodelConfiguration.java:212)
        at com.impetus.kundera.configure.MetamodelConfiguration.configure(MetamodelConfiguration.java:97)

I saw this before, I think what's going on is that I have the same entry duplicated in my classpath for some as yet unknown reason and the code doesn't deal with it very well, I've no idea why the classpath has duplications however the fix I've made is:

(MetamodelConfiguration:204) -

        if (resources != null)
        {
***        Set<URL> resourcesSet = new HashSet<URL>(Arrays.asList(resources));
***        for (URL resource : resourcesSet)
            {
                try
                {

But like I say I will try and find the reason for the duplicates

Collaborator

mevivs commented Jul 30, 2012

Ideally there should not be 2 classes available with same complete name.I hope, same classes are not unpacked in root folder?

-Vivek

Collaborator

mevivs commented Jul 30, 2012

Hi,
I have verified it. It is not happening with build anymore. Can you please check if core.jar is being duplicated in classpath?
We need to raise this alert deliberately for duplicate check.

-Vivek

dstrawson closed this Aug 1, 2012

I have tested my code with 2.7.1 and it worked okay. But after updating to 2.8.1, Kundera failed to search my entity classes given in persistence.xml. Nothing has changed on my code.
Here is persistence.xml.

<persistence-unit name="hbaseMethodikosPu">

    <provider>com.impetus.kundera.KunderaPersistence</provider>
    <!-- <class>com.methodikos.domain.Pavement</class> -->
    <jar-file>../methodikos-shared/target/methodikos.jar</jar-file>
    <exclude-unlisted-classes>true</exclude-unlisted-classes>
    <properties>            
        <property name="kundera.nodes" value="localhost" />
        <property name="kundera.port" value="9160" />
        <property name="kundera.keyspace" value="methodikos" />
        <property name="kundera.dialect" value="hbase" />
        <property name="kundera.client.lookup.class" value="com.impetus.client.hbase.HBaseClientFactory" />
        <property name="kundera.cache.provider.class"
            value="com.impetus.kundera.cache.ehcache.EhCacheProvider" />
        <property name="kundera.cache.config.resource" value="/ehcache-test.xml" />

        <!-- <property name="index.home.dir" value="./lucene" /> -->
        <property name="kundera.ddl.auto.prepare" value="create" />
        <property name="kundera.client.property" value="hbaseMethodikos.xml" />
        <!-- <property name="kundera.annotations.scan.package" value="com.methodikos.domain" />      -->    
        <property name="" value="../methodikos-shared/target/methodikos.jar"/>
    </properties>

</persistence-unit>
Collaborator

mevivs commented Dec 12, 2013

Could you please provide a sample project for this?

Though, I have verified with https://github.com/dstrawson/kunderatest .

Modified persistence.xml for <jar-file> as:

<persistence xmlns="http://java.sun.com/xml/ns/persistence" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://java.sun.com/xml/ns/persistence
    http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd"
    version="2.0">
    <persistence-unit name="cassandra">
        <provider>com.impetus.kundera.KunderaPersistence</provider>
<!--        <class>core.Thing</class>
 -->
        <jar-file>../core-package/target/core.jar</jar-file>
        <properties>            
            <property name="kundera.nodes" value="localhost"/>
            <property name="kundera.port" value="9160"/>
            <property name="kundera.keyspace" value="example"/>
            <property name="kundera.dialect" value="cassandra"/>
            <property name="kundera.client.lookup.class" value="com.impetus.client.cassandra.pelops.PelopsClientFactory" />
            <property name="kundera.annotations.scan.package" value="core" />
            <property name="kundera.ddl.auto.prepare" value="create"/>
        </properties>       
    </persistence-unit>
</persistence>

Test1.java

package example;

import static org.junit.Assert.*;

import java.util.List;

import javax.persistence.EntityManager;
import javax.persistence.EntityManagerFactory;
import javax.persistence.Persistence;
import javax.persistence.Query;

import org.junit.Test;

import core.Thing;

public class Test1 {    


    private EntityManagerFactory emf = Persistence.createEntityManagerFactory("cassandra");

    //  @Test
    public void testNewItem() {

    //  System.out.println("class path is " + System.getProperty("java.class.path"));

        EntityManager em = emf.createEntityManager();
        try {
            Thing thing = new Thing();
            thing.setId(1000L);
            thing.setTitle("A Thing");
            thing.setNothing(null);
            em.persist(thing);
        } finally {
            em.close();
        }
    }

    @Test
    public void testQueryByPK() {
                testNewItem();
//      EntityManagerFactory emf = Persistence.createEntityManagerFactory("cassandra");
        EntityManager em = emf.createEntityManager();

        Query query = em.createQuery("SELECT t FROM Thing t WHERE t.id = :idParam");
        query.setParameter("idParam", 1000L);

        List<Thing> things;
        try {
            things = query.getResultList();
        } finally {
            em.close();
        }

        assertEquals(things.get(0).getTitle(), "A Thing");

    }



}

modify core-package and api-package for pom.xml to point to 2.8.1 release

Working fine.

-Vivek

I used Spring for JPA.

spring-data-repo.xml

<context:property-placeholder location="classpath:WEB-INF/spring/*.properties " />

    <context:component-scan base-package="com.methodikos.domain" />

    <context:annotation-config />

    <bean class="org.springframework.orm.jpa.support.PersistenceAnnotationBeanPostProcessor" />

    <!-- Entity Manager Factory -->
    <bean id="entityManagerFactory"
          class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
          <property name="persistenceUnitName" value="hbaseMethodikosPu" />
          <!-- <property name="packagesToScan" value="com.methodikos.domain" /> -->
          <!-- <property name="loadTimeWeaver">
            <bean class="org.springframework.instrument.classloading.InstrumentationLoadTimeWeaver" />
          </property> -->
    </bean>

    <jpa:repositories base-package="com.methodikos.data.repository"></jpa:repositories>

    <!-- JPA Transaction Manager -->
    <bean id="transactionManager"
          class="org.springframework.orm.jpa.JpaTransactionManager">
          <property name="entityManagerFactory" ref="entityManagerFactory" />

    </bean>

    <!-- Annotation Driven -->
    <tx:annotation-driven transaction-manager="transactionManager"/>

Test code

@ContextConfiguration(
    locations = {
        "classpath:WEB-INF/spring/spring-data-repo.xml"
})
@RunWith(SpringJUnit4ClassRunner.class)
public class HBaseTest {

    private final Logger logger = LoggerFactory.getLogger(HBaseTest.class);

    @Autowired
    EntityManagerFactory emf;

    @Test
    public void getDataTest() {

        EntityManager em = emf.createEntityManager();
                // No entity found by the name: TestPojo
        Query q = em.createQuery("Select t from TestPojo t where t.id='1'");
        List<TestPojo> results = q.getResultList();
        for (TestPojo result : results) {
            logger.debug("{}, {}, {}", result.getCol1(), result.getCol2(), result.getCol3());
        }
    }   
}

TestPojo is domain object from another maven project

@Entity
@Table(name = "test", schema = "test@hbaseMethodikosPu")
public class TestPojo {

        @Id
        String id;

        @Column(name = "col1")
        private String col1;

        @Column(name = "col2")
        private String col2;

        @Column(name = "col3")
        private String col3;

        public String getCol1() {
            return col1;
        }

        public void setCol1(String col1) {
            this.col1 = col1;
        }

        public String getCol2() {
            return col2;
        }

        public void setCol2(String col2) {
            this.col2 = col2;
        }

        public String getCol3() {
            return col3;
        }

        public void setCol3(String col3) {
            this.col3 = col3;
        }

        public String getId() {
            return id;
        }

        public void setId(String id) {
            this.id = id;
        }
}

persistence.xml

<persistence xmlns="http://java.sun.com/xml/ns/persistence"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://java.sun.com/xml/ns/persistence
    http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd"
    version="2.0">

    <persistence-unit name="hbaseMethodikosPu">

        <provider>com.impetus.kundera.KunderaPersistence</provider>
        <!-- <class>com.methodikos.domain.Pavement</class> -->
        <jar-file>../methodikos-shared/target/methodikos.jar</jar-file>
        <!-- <exclude-unlisted-classes>true</exclude-unlisted-classes> -->
        <properties>            
            <property name="kundera.nodes" value="localhost" />
            <property name="kundera.port" value="9160" />
            <property name="kundera.keyspace" value="methodikos" />
            <property name="kundera.dialect" value="hbase" />
            <property name="kundera.client.lookup.class" value="com.impetus.client.hbase.HBaseClientFactory" />
            <property name="kundera.cache.provider.class"
                value="com.impetus.kundera.cache.ehcache.EhCacheProvider" />
            <property name="kundera.cache.config.resource" value="/ehcache-test.xml" />

            <!-- <property name="index.home.dir" value="./lucene" /> -->
            <property name="kundera.ddl.auto.prepare" value="create" />
            <property name="kundera.client.property" value="hbaseMethodikos.xml" />
            <!-- <property name="kundera.annotations.scan.package" value="com.methodikos.domain" />      -->    
            <!-- <property name="" value="../methodikos-shared/target/methodikos.jar"/> -->
        </properties>

    </persistence-unit>

</persistence>

Log when I use 2.8.1

...
DEBUG | main | o.s.beans.factory.support.DefaultListableBeanFactory    | Invoking afterPropertiesSet() on bean with name 'entityManagerFactory' 
INFO | main | com.impetus.kundera.KunderaPersistence                  | Loading Core 
INFO | main | com.impetus.kundera.loader.CoreLoader                   | Loading Kundera Core Metdata ...  
INFO | main | o.s.orm.jpa.LocalContainerEntityManagerFactoryBean      | Building JPA container EntityManagerFactory for persistence unit 'hbaseMethodikosPu' 
INFO | main | com.impetus.kundera.KunderaPersistence                  | Loading Application MetaData and Initializing Client(s) For Persistence Unit(s) hbaseMethodikosPu 
INFO | main | c.i.kundera.configure.PersistenceUnitConfiguration      | Loading Metadata from persistence.xml ... 
INFO | main | c.i.kundera.configure.PersistenceUnitConfiguration      | Finishing persistence unit metadata configuration ... 
INFO | main | com.impetus.kundera.configure.MetamodelConfiguration    | No class to scan for persistence unit hbaseMethodikosPu. Entities will be loaded from classpath/ context-path 
INFO | main | com.impetus.kundera.configure.ClientMetadataBuilder     | Loading client factory for persistence unit hbaseMethodikosPu 
INFO | main | com.impetus.kundera.client.ClientResolver               | Initializing client factory for: hbaseMethodikosPu 
INFO | main | com.impetus.kundera.client.ClientResolver               | Finishing factory initialization 
INFO | main | com.impetus.kundera.configure.SchemaConfiguration       | Configuring schema export for : hbaseMethodikosPu 
WARN | main | com.impetus.kundera.configure.AbstractPropertyReader    | File hbaseMethodikos.xml not found, Caused by  
WARN | main | org.apache.hadoop.hbase.HBaseConfiguration              | instantiating HBaseConfiguration() is deprecated. Please use HBaseConfiguration#create() to construct a plain Configuration 
DEBUG | main | org.apache.hadoop.security.Groups                       |  Creating new Groups object 
DEBUG | main | org.apache.hadoop.security.Groups                       | Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping; cacheTimeout=300000 
....

Log when I use 2.7.1

...
2013-12-12 17:12:22,141  INFO | main | com.impetus.kundera.KunderaPersistence                  | Loading Core 
2013-12-12 17:12:22,143  INFO | main | com.impetus.kundera.loader.CoreLoader                   | Loading Kundera Core Metdata ...  
2013-12-12 17:12:22,148  INFO | main | o.s.orm.jpa.LocalContainerEntityManagerFactoryBean      | Building JPA container EntityManagerFactory for persistence unit 'hbaseMethodikosPu' 
2013-12-12 17:12:22,148  INFO | main | com.impetus.kundera.KunderaPersistence                  | Loading Application MetaData and Initializing Client(s) For Persistence Unit(s) hbaseMethodikosPu 
2013-12-12 17:12:22,163  INFO | main | c.i.kundera.configure.PersistenceUnitConfiguration      | Loading Metadata from persistence.xml ... 
2013-12-12 17:12:22,227  INFO | main | c.i.kundera.configure.PersistenceUnitConfiguration      | Finishing persistence unit metadata configuration ... 
2013-12-12 17:12:22,229  INFO | main | com.impetus.kundera.configure.MetamodelConfiguration    | No class to scan for persistence unit hbaseMethodikosPu. Entities will be loaded from classpath/ context-path 
2013-12-12 17:12:22,499  INFO | main | com.impetus.kundera.configure.ClientMetadataBuilder     | Loading client factory for persistence unit hbaseMethodikosPu 
2013-12-12 17:12:22,502  INFO | main | com.impetus.kundera.client.ClientResolver               | Initializing client factory for: hbaseMethodikosPu 
2013-12-12 17:12:22,523  INFO | main | com.impetus.kundera.client.ClientResolver               | Finishing factory initialization 
2013-12-12 17:12:22,523  INFO | main | com.impetus.kundera.configure.SchemaConfiguration       | Configuring schema export for : hbaseMethodikosPu 
2013-12-12 17:12:22,549  WARN | main | com.impetus.kundera.configure.AbstractPropertyReader    | File hbaseMethodikos.xml not found, Caused by  
...

Log seems to be same for both but 2.7.1 is able to find my entity and 2.8.1 not.

I am using Java 1.7. I checked dependency change between 2.7.1 and 2.8.1 but found nothing special, although I am thinking there are some dependency issues.

Thank you

Collaborator

mevivs commented Dec 13, 2013

Please email sample project to me at vivek.mishra@impetus.co.in

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment