New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow MapStruct to be used together with Project Lombok #510

Closed
panov-andy opened this Issue Mar 20, 2015 · 66 comments

Comments

Projects
None yet
@panov-andy
Copy link

panov-andy commented Mar 20, 2015

Hi.
I'm using Project Lombok and unable generate mappers with maven.
I use @DaTa annotation (lombok) if it make sense.

@gunnarmorling

This comment has been minimized.

Copy link
Member

gunnarmorling commented Mar 20, 2015

There has been a discussion about the combination of MapStruct + Lombok on the Google group recently: https://groups.google.com/forum/#!topic/mapstruct-users/91fBwD4hNZE. An example for using both together is here: https://github.com/mapstruct/mapstruct-examples

If this does not help, it'd be good to know what the exact issue is and how your set-up does look like.

@panov-andy

This comment has been minimized.

Copy link
Author

panov-andy commented Mar 20, 2015

Suddenly it's not an option for me to split project to modules.
Anyway I'll thing about it.
Thanks.

@dave-lo

This comment has been minimized.

Copy link

dave-lo commented May 27, 2015

I was able to get the mapstruct working with Lombok on versions < 1.16.0 because the annotationprocessor was still exposed. Starting 1.16.0+ the annotation processor is hidden so the maven compiler can't use it and thus this solution won't work

@agudian

This comment has been minimized.

Copy link
Member

agudian commented May 28, 2015

That's too bad - it's likely that there is not much that we can do on our side. Perhaps the devs at lombok can take care of supporting annotation-processors again...

@rzwitserloot

This comment has been minimized.

Copy link

rzwitserloot commented Nov 22, 2015

We can't do much without more info on how @dave-lo fixed this issue. I don't even know what the issue is :/

@gunnarmorling

This comment has been minimized.

Copy link
Member

gunnarmorling commented Nov 23, 2015

Hey @rzwitserloot, thanks for dropping by. Yes, it would help to know what @dave-lo did do, e.g. it's not clear to me what "Starting 1.16.0+ the annotation processor is hidden" means. I suppose somehow he managed to establish an ordering of Lombok and then MapStruct.

Generally, the issue is that MapStruct generates code based on accessors of JavaBeans properties which don't exist yet in the source code when using Lombok's @DaTa annotation. I'm not sure whether an annotation processor such as MapStruct can reliably "see" bytecode generated by another participant of the same compilation (as that case generally is not foreseen by the compiler environment).

Another idea for making the integration better would be to let MapStruct take fields into account as well (we want that anyways) but let it generate accessor-based code instead of field-accessing code by means of some configuration (or e.g. when spotting the @Data annotation). That'd by my preferred approach as it should be more robust and e.g. also work in IDEs.

@rzwitserloot

This comment has been minimized.

Copy link

rzwitserloot commented Nov 23, 2015

@gunnarmorling Yes, MapStruct should be able to reliably see the stuff that lombok generates, if lombok runs first. Lombok tries to run first but it can't always make this happen, so apparently here MapStruct runs first and thus can't see the stuff lombok adds.

The 'hiding' that is being referred to, is that lombok's annotation processors are squirreled away in public inner classes of private classes, which make them visible for the various command line and jar launcher mechanisms of java (such as MANIFEST.MF's Main-Class key), but invisible to java code. This to avoid having a bunch of lombok internals 'taint' a project's namespace. I haven't yet figured out what part of this change is making it harder to make lombok run first.

For what it's worth, lombok forces (or, it should, perhaps that is the bug?) a new annotation processing round after it changes things, so that other APs can always pick up on the changes even if MS ran first.

The way the annotation processor API design solves this problem is that the flow should be either:

  1. javac parses
  2. javac fires a round
  3. lombok gets to go first and adds some accessor methods
  4. MapStruct gets to go second and sees them, and generates some stuff.
  5. javac is done with the round, but because lombok said stuff changed, another round starts.
  6. lombok fires again but doesn't change anything as everything available has already been inspected.
  7. MapStruct should draw the same conclusion.
  8. javac is done with the round, notices nothing changed, and fires a final no-changes round.
  9. lombok picks this up, does nothing.
  10. MapStruct picks this up, dos nothing.
  11. the final round is done without error and thus javac moves on to bytecode generation.

Or, alternatively, if the order is reversed:

  1. javac parses
  2. javac fires a round
  3. MapStruct gets to go first and doesn't see the accessor methods yet.
  4. lombok goes second and adds them. Also marks to javac that more rounds are needed.
  5. javac is done with the round, but because lombok said stuff changed, another round starts.
  6. MapStruct gets to go again and should pick up on the changes here, modifying what it has done so far / doing more.
  7. lombok fires again but doesn't change anything as everything available has already been inspected.
  8. javac is done with the round, notices nothing changed, and fires a final no-changes round.
  9. MapStruct picks this up, dos nothing.
  10. lombok picks this up, does nothing.
  11. the final round is done without error and thus javac moves on to bytecode generation.

An alternate take is that MS doesn't do anything but gather statistics in all the rounds until the final round and then generates it all, this is the most flexible design for a lot of AP processing (basically, if you don't ever generate any new sources, it's probably the right choice: Just make a list of what you need to do in all the rounds, until the final round, then do everything the list tells you to).

Perhaps MapStruct doesn't do this. If you only ever use 1 Annotation Processor, just doing all the work in the first round and ignoring all further rounds works fine. It only breaks when multiple processors are involved.

So far, either:

  1. Lombok is buggy and doesn't trigger another round.
  2. MapStruct has a flaw in the design where it doesn't stick to the appropriate rules about how to deal with rounds.
  3. MapStruct DOES deal with rounds properly, but that part is bugged. They often can be; it rarely comes up!
  4. Something else is going on that I don't yet understand.

Any guesses on your side of idea 2 or idea 3 might be relevant here?

@rspilker

This comment has been minimized.

Copy link

rspilker commented Nov 23, 2015

  1. Something goes wrong with identifying lomboks annotation processor. The correct processor is lombok.launch.AnnotationProcessorHider$AnnotationProcessor. Possibly the $ is a . or the old processor is used, the one shipped in pre 1.16.0 versions.
@rmannibucau

This comment has been minimized.

Copy link

rmannibucau commented Nov 26, 2015

A workaround based on maven - but making IDEs a bit lost - is to delombokize the sources (build config):

<sourceDirectory>${project.build.directory}/generated-sources/delombok</sourceDirectory>

<plugins>
  <plugin> <!-- normally optional but here we use mapstruct so we need this step -->
    <groupId>org.projectlombok</groupId>
    <artifactId>lombok-maven-plugin</artifactId>
    <version>1.16.6.1</version>
    <executions>
      <execution>
        <phase>generate-sources</phase>
        <goals>
          <goal>delombok</goal>
        </goals>
        <configuration>
          <sourceDirectory>src/main/java/</sourceDirectory>
          <addOutputDirectory>true</addOutputDirectory>
        </configuration>
      </execution>
    </executions>
  </plugin>
  <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-compiler-plugin</artifactId>
    <version>3.3</version>
    <executions>
      <execution>
        <id>default-compile</id>
        <goals>
          <goal>compile</goal>
        </goals>
        <configuration>
          <annotationProcessors>
            <annotationProcessor>org.mapstruct.ap.MappingProcessor</annotationProcessor>
          </annotationProcessors>
        </configuration>
      </execution>
    </executions>
  </plugin>
@gunnarmorling

This comment has been minimized.

Copy link
Member

gunnarmorling commented Nov 26, 2015

@rzwitserloot, @rspilker Thanks for that thorough explanation; I am aware of the round-based environment for annotation processors and I don't know of any apparent issues in MapStruct due to this.

So far I assumed the processing environment could not handle altered types (as that's not foreseen traditionally) and thus might not call for another round if Lombok has added getters/setters, but your description indicates otherwise. If I find a minute, I'll set up a test project for debugging what's going on.

@rmannibucau Thanks for chiming in and providing that workaround!

@sjaakd

This comment has been minimized.

Copy link
Contributor

sjaakd commented Nov 26, 2015

but your description indicates otherwise

Hmm. I wander what happens when we have a mixed scenario. Partially Lombok partially not. Will MapStruct be able to cope with that scenario. Generating mappers, waiting for Lombok and then?

@rspilker

This comment has been minimized.

Copy link

rspilker commented Dec 6, 2015

If think the problem is the $ in the class name. On the command line, that might be interpreted as a variable. I was able to run the following command successfully:

javac -processor 'lombok.launch.AnnotationProcessorHider$AnnotationProcessor' -cp lombok.jar Test.java

However, without the single quotes it would give me an error.

@rmannibucau

This comment has been minimized.

Copy link

rmannibucau commented Dec 7, 2015

@rspilker

This comment has been minimized.

Copy link

rspilker commented Dec 7, 2015

The code also suggests that somehow switching the compiler to in-process would be a workaround.Is that possible?

@rmannibucau

This comment has been minimized.

Copy link

rmannibucau commented Dec 7, 2015

It is but not a reliable option for existing builds.

@rspilker

This comment has been minimized.

Copy link

rspilker commented Dec 7, 2015

I just wanted to verify our assumptions regarding the cause. If I understand the code, it generates a file that's passed to javac using `@arguments-file-name'.

I find it a bit strange that you would still need to escape entries in that file. Why would they be interpreted, unless javac does it. And if javac does it, possibly we can always add the single quotes. Having them on the command line does not work in Windows. Possible we can use double quotes and an escape symbol.

But I do agree that either the Plexus compiler or javac does not handle the $ sign in program arguments correctly.

@rspilker

This comment has been minimized.

Copy link

rspilker commented Dec 8, 2015

I think the problem is that Maven itself interprets it as a property. Possibly the following works:

lombok.launch.AnnotationProcessorHider&#36;AnnotationProcessor
@gunnarmorling

This comment has been minimized.

Copy link
Member

gunnarmorling commented Dec 12, 2015

An alternate take is that MS doesn't do anything but gather statistics in all the rounds until the final round and then generates it all, this is the most flexible design for a lot of AP processing

Hey @rzwitserloot, @rspilker, I tried that, and indeed this lets MapStruct see the Lombok-generated accessors. The problem though is that the compiler issues a warning saying "File for type XYZ created in the last round will not be subject to annotation processing". I take that as that it's a bad practice to generate new files in the last round. And really this makes sense as it isn't friendly towards other processors.

So I'd have to somehow postpone file creation to that additional round triggered by Lombok but I don't see a reliable way for doing so. Specifically, in the first round - where MapStruct is called for its @Mapper annotation - I don't know whether there will be another non-final round (as triggered by Lombok) or not. I also tried to generate the files in the first round and the second one - if there is one - but then javac complains about the same file being generated twice.

Atm. I am left behind a bit clueless, the only reliable approach I can see is to anticipate that there will be accessors when Lombok's annotations are present and generate my code based on that anticipation. But I'd dislike the fact to rely on such knowledge and assumptions about another processor.

@rmannibucau

This comment has been minimized.

Copy link

rmannibucau commented Dec 12, 2015

What about allowing mapstruct to know other processors by conf and let mapstruct handles their lifecycle - kind of delegate/decorator pattern?

@agudian

This comment has been minimized.

Copy link
Member

agudian commented Dec 12, 2015

What about allowing mapstruct to know other processors by conf and let mapstruct handles their lifecycle - kind of delegate/decorator pattern?

That shouldn't be the responsibility of MapStruct. Plus, each processor is only called for those types that carry the annotation(s) that the processor feels responsible for. So adding an uber-processor isn't something that the APT API considers.

Making sure that lombok is executed before mapstruct by passing the list of processors accordingly doesn't do it?

          <annotationProcessors>
            <annotationProcessor>lombok.launch.AnnotationProcessorHider$AnnotationProcessor</annotationProcessor>
            <annotationProcessor>org.mapstruct.ap.MappingProcessor</annotationProcessor>
          </annotationProcessors>
@rmannibucau

This comment has been minimized.

Copy link

rmannibucau commented Dec 12, 2015

@agudian agree but the point is making mapstruct usable. Doesnt work as mentionned cause of a mvn issue. Waiting or hacking the mvn fix is surely the best option.

@agudian

This comment has been minimized.

Copy link
Member

agudian commented Dec 12, 2015

Waiting or hacking the mvn fix is surely the best option.

I'm touching those parts of the code right now anyway and can take a closer look into https://issues.apache.org/jira/browse/MCOMPILER-256 while I'm at it.

@gunnarmorling

This comment has been minimized.

Copy link
Member

gunnarmorling commented Dec 13, 2015

Making sure that lombok is executed before mapstruct by passing the list of processors accordingly doesn't do it?

The issue is that both processors are called in the same round - first Lombok, then MapStruct - but within that same round the TypeElement s examined by MapStruct do not yet expose the accessors added by Lombok from what I can see. There will be another non-final round triggered by Lombok in which I can see the Lombok generated stuff, so if I wait for that round to write out the stuff from MapStruct it works. It's only that I don't know whether there will be such round or not (in case Lombok is not present).

Essentially, one would have to wait until all source and target types of MapStruct-generated mappers are stable - i.e. not modified by Lombok or amended by generating super-types - and only then generate the mapper implementations. I am not sure though I can identify that point of time.

@gunnarmorling

This comment has been minimized.

Copy link
Member

gunnarmorling commented Dec 15, 2015

I've thought about a conceptually similar case that actually is supported officially by JSR 269: The generation of super-classes.

We can make this case work as we can identify types with super-types not yet generated (because they are to be generated in a subsequent round or by another processor in the same round); the super-type elements will have kind ERROR in this case. This allows to defer handling of the affected sub-types to another round. I think that's the way to do it correctly as envisioned by the JSR 269 creators.

The issue is that I don't see how it could be done with Lombok-modified entities. I can't find out whether a type has been amended by Lombok in the same round, not allowing for the decision whether to defer handling of the types or not.

I think this is one of the cases where Lombok's modification of types really causes trouble, because it makes collobaration with other processors as intended by the JSR 269 impossible from what I can say. I'd be more than happy to be convinced otherwise, though.

@rmannibucau

This comment has been minimized.

Copy link

rmannibucau commented Dec 16, 2015

Well you can also state the jsr 269 was badly designed ;) - just trying to be fair from both point of view.

More seriously do you care bringing these explanations there rzwitserloot/lombok#973 ? Think working between both projects will be more efficient and a solution seems possible with this last comment.

@gunnarmorling

This comment has been minimized.

Copy link
Member

gunnarmorling commented Dec 16, 2015

Added a comment over there; Not sure how it could look like, but maybe @rzwitserloot et al. have a good idea.

@rmannibucau

This comment has been minimized.

Copy link

rmannibucau commented Dec 16, 2015

Thanks Gunnar!

@gunnarmorling

This comment has been minimized.

Copy link
Member

gunnarmorling commented Jan 18, 2017

Cool, looking forward to the outcome. Let us know in case there's anything we can help with.

gunnarmorling added a commit to gunnarmorling/mapstruct that referenced this issue Jan 21, 2017

mapstruct#510 Adding experimental SPI for letting AST-modifying annot…
…ation processors such as Lombok tell us about future modifications

gunnarmorling added a commit to gunnarmorling/mapstruct that referenced this issue Jan 31, 2017

mapstruct#510 Adding experimental SPI for letting AST-modifying annot…
…ation processors such as Lombok tell us about future modifications

gunnarmorling added a commit to gunnarmorling/mapstruct that referenced this issue Jan 31, 2017

gunnarmorling added a commit that referenced this issue Jan 31, 2017

#510 Adding experimental SPI for letting AST-modifying annotation pro…
…cessors such as Lombok tell us about future modifications

@gunnarmorling gunnarmorling added this to the 1.2.0.Beta1 milestone Jan 31, 2017

@gunnarmorling

This comment has been minimized.

Copy link
Member

gunnarmorling commented Jan 31, 2017

The SPI has been merged to master. Thanks @rzwitserloot, @rspilker and everyone else for making it happen!

@gunnarmorling gunnarmorling changed the title fail if Project Lombok is used (projectlombok.org) Allow MapStruct to be used together with Project Lombok Jan 31, 2017

@msymonov

This comment has been minimized.

Copy link

msymonov commented Feb 6, 2017

Hi everyone, when do we expect this feature to be released? What is actually required? Will we need both artifacts to be released? And thanks for everyone's efforts

@gunnarmorling

This comment has been minimized.

Copy link
Member

gunnarmorling commented Feb 6, 2017

You'll need MapStruct 1.2.0.Beta1 (should be out very soon, we hope to do it next week) and the latest "Edge" release of Lombok. @rzwitserloot, @rspilker, any indication when there will be a regular Lombok release with your change?

@rspilker

This comment has been minimized.

Copy link

rspilker commented Feb 7, 2017

I expect a new release within a week.

@rzwitserloot

This comment has been minimized.

Copy link

rzwitserloot commented Feb 10, 2017

Lombok release 1.16.14 is out with this feature.

@fabiohbarbosa

This comment has been minimized.

Copy link

fabiohbarbosa commented Feb 13, 2017

Problem resolved with new lombok version!
Thanks

@Maaartinus

This comment has been minimized.

Copy link

Maaartinus commented Feb 16, 2017

Cool! Could someone update the http://mapstruct.org/faq/?

@gunnarmorling

This comment has been minimized.

Copy link
Member

gunnarmorling commented Feb 16, 2017

@Maaartinus Maybe you could do it? Just click the "Edit on GitHub" link on the upper right of the FAQ page and you could prepare a PR. We've planned the Beta1 release for Monday next week.

@avk2

This comment has been minimized.

Copy link

avk2 commented Apr 11, 2017

If I add lombok to entities too, then it's not working. Lombok is not meant to be used with entities?

@filiphr

This comment has been minimized.

Copy link
Member

filiphr commented Apr 11, 2017

It doesn't matter whether you use lombok with entities and not. If your project is correctly setup and uses the versions defined in this faq then it should work. If it doesn't please create a new issue, displaying the problem

@avk2

This comment has been minimized.

Copy link

avk2 commented Apr 12, 2017

@filiphr I originally had

	<plugin>
		<groupId>org.apache.maven.plugins</groupId>
		<artifactId>maven-compiler-plugin</artifactId>
		<version>3.6.1</version><!--$NO-MVN-MAN-VER$ -->
		<configuration>
			<annotationProcessorPaths>
				<path>
					<groupId>org.mapstruct</groupId>
					<artifactId>mapstruct-processor</artifactId>
					<version>${org.mapstruct.version}</version>
				</path>
			</annotationProcessorPaths>
		</configuration>
	</plugin>

I fixed it commenting out the <configuration> part and adding mapstruct-processor as dependency

Maybe the installation instructions should be updated? http://mapstruct.org/documentation/installation/

@filiphr

This comment has been minimized.

Copy link
Member

filiphr commented Apr 13, 2017

@avk2 if you add lombok as path in the annotationprocessorPaths it will also work. The thing with the annotationProcessorPaths is that if it is specified there, then the maven compiler will only use what is there, if that is empty, then it will use the classpath for the APT.

Maybe we need to write in the installation instructions that one should add other APT via path there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment