Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmarking more cases #17

Closed
roookeee opened this issue Jul 12, 2019 · 6 comments
Closed

Benchmarking more cases #17

roookeee opened this issue Jul 12, 2019 · 6 comments

Comments

@roookeee
Copy link

roookeee commented Jul 12, 2019

The OrderFactory currently only returns one case which could be seen as the most "happiest" case: nothing is null which introduces several issues:

  • all the Mappers don't actually have to be implemented / defined in a null-safe manner. It's uncertain right now whether every implemented mapper is actually null safe (the tests don't cover that either). Each mappers null-safe declaration could also be way more verbose than it is right know which would skew the first impression how succinct / readable each mapping framework actually is in a nullable scenario
  • the performance characteristics of each mapper could be drastically different for "unhappy" cases e.g. every field that should be mapped is null

I would propose the OrderManager to return a List<Order> which contains a happy, unhappy and middle case which is iterated for each mapper. Each Mapper will suffer in its final score if it handles one of those cases poorly. Maybe use an Order[] instead of a List<Order> to avoid skewing the results by having to instantiate an Iterator<Order> in each benchmark call. An extended for-loop on an array desugars into an indexed based loop that won't impact the results as much

@roookeee
Copy link
Author

I would volunteer for a PR if this issue is accepted

@arey
Copy link
Owner

arey commented Jul 14, 2019

The 2 source and target data structures are similar. There is no upper case conversion or lastname + firstname to name conversion. Thus nullable fields should not be a problem. Nullable relationship or empty collections could be.
Before going further, I would like some precisions. Just a few samples could be fine.

@roookeee
Copy link
Author

Looking at the manual implementation everything is indeed null checked but the input values for the benchmark are never null. One could conclude that 1) the manual implementation does unnecessary null checks and may lose some performance compared to the mapping frameworks or 2) that the input of the benchmark does not reflect the nullability of some fields.

What I am proposing: use Order values in the benchmark that actually have null values. As I see it most if not all mappers included in this repository do a null check implicitly (without further configuration) BUT some mappers might need additional configuration to add a null check (see my just added datus implementation). I could have omitted the null checks as I know that the current benchmark values are never null which would improve datus performance - cheating yes, but there is no unit test that shows that all mappers can handle null fields of the input value and the benchmark doesn't reflect null values in the fields too. I would propose the following unit-test and benchmark cases:

//case 1: everything is present
Order order1 = new Order();
Customer customer = new Customer();
order1.setCustomer(customer);
customer.setName("Joe Smith");
Address billingAddress = new Address();
customer.setBillingAddress(billingAddress);
billingAddress.setStreet("1234 Market Street");
billingAddress.setCity("San Fran");
Address shippingAddress = new Address();
customer.setShippingAddress(shippingAddress);
shippingAddress.setStreet("1234 West Townsend");
shippingAddress.setCity("Boston");
List<Product> products = new ArrayList<Product>(2);
order1.setProducts(products);
products.add(new Product("socks"));
products.add(new Product("shoes"));

//case 2: everything is null
Order order2 = new Order();

//case 3: some fields are null (both addresses)
Order order3 = new Order();
Customer customer3 = new Customer();
order3.setCustomer(customer3);
customer3.setName("Joe Smith");
customer3.setShippingAddress(shippingAddress);
shippingAddress.setStreet("1234 West Townsend");
shippingAddress.setCity("Boston");
List<Product> products3 = new ArrayList<Product>(2);
order3.setProducts(products3);
products3.add(new Product("socks"));
products3.add(new Product("shoes"));

@arey
Copy link
Owner

arey commented Jul 16, 2019

I've done a test with the partial Order (case 3).
Only the Jmapper test fails and requires some additional configuration :

com.googlecode.jmapper.exceptions.JMapperException: com.googlecode.jmapper.exceptions.NestedBeanNullException: the field street is null, error occured in mapping phase between the fields: OrderDTO.shippingStreetAddress and Order.customer

	at com.googlecode.jmapper.config.JmapperLog.error(JmapperLog.java:46)
	at com.googlecode.jmapper.JMapper.getDestination(JMapper.java:100)
	at com.javaetmoi.benchmark.mapping.mapper.jmapper.JMapperMapper.map(JMapperMapper.java:35)
	at com.javaetmoi.benchmark.mapping.mapper.dozer.AbstractMapperTest.map(AbstractMapperTest.java:16)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runners.Suite.runChild(Suite.java:128)
	at org.junit.runners.Suite.runChild(Suite.java:27)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: com.googlecode.jmapper.exceptions.NestedBeanNullException: the field street is null, error occured in mapping phase between the fields: OrderDTO.shippingStreetAddress and Order.customer
	at com.googlecode.jmapper.config.Error.nestedBeanNull(Error.java:127)
	at comjavaetmoibenchmarkmappingmodeldtoOrderDTOcomjavaetmoibenchmarkmappingmodelentityOrder741697001.nullVSouAllAll(comjavaetmoibenchmarkmappingmodeldtoOrderDTOcomjavaetmoibenchmarkmappingmodelentityOrder741697001.java)
	at com.googlecode.jmapper.JMapper.getDestination(JMapper.java:98)
	... 32 more

I'm not sure to add those kind of test. I'll prefer to enhance the model with more advanced features: enum mapping, transcoding, String to numeric conversion ...

@roookeee
Copy link
Author

roookeee commented Jul 16, 2019

Your call :) Maybe just make a comment in the readme that you expect or don't expect new mappers to handle null, a unit test would do too. Thanks for your time. If you change your mind give me a heads up, will do a PR then

@arey
Copy link
Owner

arey commented Jul 19, 2019

Thank you. I've added some unit tests and complete the readme.md.

@arey arey closed this as completed Jul 19, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants