Plugin with Leiningen 2 doesn't pull in dependencies correctly. #11
Comments
|
Hi Dave, In general, two projects shouldn't share the same source folder. The standard layout used by Leiningen (and Maven) is 1 folder per project with a src folder inside.Also IntelliJ itself works on the assumption that you have 1 project with n modules - so you can't have two projects open in the same window. I'd suggest the following folder layout: your-project (the main project, actually just an aggregator):
Have a look at the ring project: https://github.com/mmcgrana/ring they also have a project with several sub-projects. If for whatever reason you cannot adhere to the default structure or if you still have the issue when using the default structure, it would help me a lot if you could provide a small sample project showing your issue. Also, can you tell me, which version of Leiningen you are using? Hope this helps. Kind regards, |
|
The first issue (with multiple modules being created) was some sort of problem with the IntelliJ project files. I deleted the .iml file and the .idea/ directory and recreated the project and that problem is gone now. The real problem is with the dependencies. I'm using Leiningen 2.0preview6. This version no longer copies the jar files into the lib/ directory, so IntelliJ can't find them. As a workaround I used Leiningen 1 (lein1 deps) to copy the dependencies into lib/ and set that up as a library in my project. |
|
Ok, thanks for keeping me updated. As far as I can see Leiningen 2 uses the .m2 repo. I'll see if I can modify the plugin so it looks for dependencies there and links them into the libraries section. That will probably introduce a dependency to the maven plugin but since that comes bundled with IntelliJ it shouldn't be too much of an issue. |
|
Leiningen 2 uses Pomegranate at the core of its dependency management. Would it be possible to call the get-classpath function from pomegranate.clj and add those to the module? I took a look at the IntelliJ plugin documentation, and it's not very clear when it comes to this use case. A maven project seems to update the module file (and write it out) when the pom changes. It would be nice to emulate this feature with the project.clj. Also, we'll have to keep in mind the new profiles when we're looking for :dependencies as profiles can change the classpath. All of this functionality exists for Maven, but I can't find much of the source in order to attempt to emulate it and I don't know the IntelliJ APIs well enough to take a good guess as to where I should start. |
|
I'm not sure if this "read the classpath and add it to the project" would actually work, I assume leiningen itself pulls some dependencies that are not necessarily parts of the project dependencies and one would have to filter these out with a more or less hacky heuristic. The cleanest way IMHO would be to call lein deps, which will download all the necessary deps into ~/.m2 and then collect the jars from there using the information from the project.clj file and add them to IDEAs module dependencies. Profiles are a different topic I think as they would probably require some larger refactoring on the plugin. In fact you need a way to enable/disable plugins on the UI and then the plugin needs to take all the rules into account to build the resulting dependency tree. I'm actually not too happy that leiningen introduced profiles as it turns more and more into maven with all it's complexity. I have a few hours to spare this week, so let's see if I can come up with a solution for the dependencies first and tackle the profile thing afterwards. |
|
Sounds great, thanks Jan!! |
|
This all just got a whole level more complicated. The magic word is "transitive dependencies". With Leiningen 1 that is not a problem, just import the whole lib folder and that's it. With Leiningen 2, I have to do the dependency resolution in the plugin. And it should better yield the same results as Leiningen itself. So I guess I have to use pomegranate for this. Sadly, my Clojure is not that good, so that is going to take a while... |
|
What functions do you need? If they're pure functions, I could help write the interactions with pomegranate. If you know the inputs/outputs, I can toy with pomegrantate to make an interface that you can leverage. |
|
Well let me write it in Java-notation, before I make a fool of myself :): /**
Fetches all direct and transitive dependencies of the given artifact
*/
List<Artifact> getDependencies(Artifact a)
where Artifact is something like:
class Artifact {
public String groupId;
public String artifactId;
public String version;
public String type;
public String classifier;
public String scope;
}
So maybe you could add a new class to the leiningen-interop.jar where I can access such a function from Java and which does the pomegranate magic. Once I got the artifacts, I can easily grab them from the local m2 repo and add them to the module dependencies with correct scope. Meanwhile I get to brush up on my clojure :D |
|
I edited the title of this issue to better reflect what the issue is |
|
For Leiningen 2 we've spun out the functionality you need for this into the separate https://clojars.org/leiningen-core http://leiningen.org/reference.html That's what the Leiningen support in CounterClockwise uses. You should be able to pull in that library and set the classpath using |
|
Sounds cool I'll check it out tonight :) Thank you! |
|
In the meantime you can just run lein pom, then setup the maven facet. It'll setup the classpath just fine. If you just want a full string of the classpath, you can also run the lein classpath target which will output it as a string. It includes everything leiningen would use. |
|
@technomancy no luck. Everytime I add something like (use 'leiningen.core.project) to my clojure file I get a: java.lang.ClassNotFoundException: clojure.lang.ILookupHost Same happens when I just do lein repl and do the (use .. ) there. I have not the slightest idea what I might be doing wrong. |
|
It seems a Clojure version mismatch problem. |
|
I just tried to run the plugin configuration, and it failed with the same exception upon opening a project. Updating the clojure version in the plugin EDIT: just FYI, ILookupHost is looked up on |
|
Hmm, still no joy. I have updated the dependency to 1.4.0: (defproject de.janthomae/leiningenplugin "1.1.0-SNAPSHOT"
:description "IntelliJ plugin for controlling Leiningen"
:dependencies [[org.clojure/clojure "1.4.0"]
[leiningen-core "2.0.0-SNAPSHOT"]]
:plugins [[lein-pprint "1.1.1"]]
:dev-dependencies [[junit/junit "4.8.2"]]
:source-paths ["src"]
:resource-paths ["lein-resources"]
:compile-path "classes"
:aot :all
; :disable-implicit-clean true
:omit-source true
:target-dir "out"
:jar-name "leiningen-interop.jar")Still getting an exception: |
|
If you replace the clojure.jar (or clojure-1.2.0.jar) in the plugin directory with the clojure 1.4 jar this should take care of this problem. |
|
@64BitChris I'm running this from a shell not using the Intellij plugin (just to make sure it's not a bug in the IntelliJ-plugin). I have deleted the lib folder which had an old clojure.jar inside but still the same exception. |
|
Adding a :eval-in :leiningen to the project file seemed to do the trick. I have no idea why this is necessary but I can live with that as long as it works. |
|
I'm trying to update this plugin to work against Lein 2. However, there are some very fundamental differences in the architecture from Lein 1 and Lein 2 - at least in the approach to how the plugin interacts with leiningen. In the current version of the plugin, the interaction is made through calls to Leiningen and thus needs to know the lein home, lein bat file, and lein path. With Lein 2, we can programmatically configure a project using the leiningen-core library. I've tested this and validated that I can create a project this way. I've created a :gen-class to front the behavior to the leiningen-core library. The problem I'm having is that when I try to call my generated class from inside the plugin I get a FileNotFound exception saying that it can't load clojure/core or clojure/core_init from the classpath. The odd part about this is that the clojure.lang.RT.load function is on the stack as well as other clojure.lang.RT. RT and core are in the same jar, it's finding RT but not core and I can't figure out why. When I access the clojure.core_init.class.getClassLoader() from the debugger, I get an exception (the FileNotFound exception) the first time, but if I evaluate it again, it returns the PluginClassLoader, which I think it should do at the beginning. Does anyone have any ideas as to why the IDEA plugin system is having problems with finding the clojure core classes? I have clojure.jar in my plugin lib directory and it gets deployed to the sandbox-plugins directory. If I can get past this problem, I won't be far from being able to upgrade this plugin to support leiningen 2. Thanks! |
|
Well this is where I hit the wall as well. I have no clue why the classes cannot be found especially they are clearly within the classpath. I also tried some manual invocations using the debugger (e.g. stopping a line before the clojure interop part and doing a class.forname myself which yielded a result). Seems like clojure is pulling off some weird magic inside but I'm by no means a clojure expert, so maybe someone with experience in clojure classloading could help here. |
|
Okay, I finally figured it out. If I switch the current thread's classloader to the Leiningen Plugin Classloader and THEN call the Clojure libraries, everything is okay. Thread.currentThread().setContextClassLoader(LeiningenConstants.class.getClassLoader()); Where LeiningenConstants can be anything that is specific to the Leiningen Plugin Classpath. I made this call before using the clojure libraries and everything works. The current thread in this context is the AWT-EventQueue thread, and the plugin's classloader doesn't have a parent set, so something in IntelliJ must be switching it back as everything else seems to work. Anyway, this resolves the issue, but it makes me wonder if there's something in the plugin I have to do to tell the thread to switch to the plugin classloader and then switch back. Jan, are you doing any active work that this helps you fix? I'm basically trying to add IDEA Module support - so I can go to the 'Add Module' dialog and then add a leiningen module. Since I'd have to introspect the dependencies, I decided to look at upgrading so that the leiningen plugin would use lein 2.0. I discussed this in #clojure and they suggested only supporting 2.0 instead of 1.7 and 2.0 (people could always use earlier versions for 1.7 support). I just want to make sure we don't do duplicate work, so if you don't have plans to do anything with this for a while, then I think we're all good :) |
|
I am trying to get the dependencies when working with Leiningen 2, so that you can just load up the leiningen project and have all dependencies satisfied and no red code in your project. From what I read you are trying the same. I'd suggest that you send me a pull request once you got the dependency resolution working, so I can build on this when importing the project into IntelliJ. You could then continue to work on the module support. Another priority is IDEA 12 support, which will probably require a branch as several APIs have been changed. |
|
Sounds good, I'll try to get the dependencies pulled in via leiningen-core with the current plugin and then submit the pull request to you. Probably won't get much focused time until this weekend. Do we want to follow the La Clojure plugin's approach by having a branch for 11 and 12? We can create two branches now that point at the current head and then let them diverge from there. Do you have a good link to a place that documents the IDEA Plugin APIs and the changes from 11 to 12? I'm using the Maven Plugin's source code to figure out what needs to be done and have to figure out what each component does. Ultimately, I'd like to have the Leiningen Plugin be on par with the Maven plugin, in that they both do similar things, just with different underlying technologies. I think that level of support will make IDEA one of the top platforms for building Clojure apps. I can help with a lot of this as I have some extra time before the end of the year. |
|
No link except this one http://confluence.jetbrains.net/display/IDEADEV/PluginDevelopment . But it's rather dated. I got most of my knowledge from reverse engineering other stuff, mainly the Maven plugin. Regarding the branch, we should definitely have one for IDEA 11 and one for IDEA 12. I'm not so fond of the idea of dropping support for Leiningen 1, but I have no data about what's in use right now. If we drop support for Leiningen 1 with IDEA 12 that also means that if you want to do a Leiningen 1 project, you have to go back to IDEA 11, because the older plugin versions that still support Leiningen 1 won't work on IDEA 12, as they have changed the API. Maybe we should go the way of the maven plugin here and support both versions (such as the Maven plugin supports Maven 2 & 3). That is unless you got a source that Leiningen 1 is no longer in use. If we decide to support both versions of Leiningen I'd rather wait with the branch until we got basic Leiningen 2 support up and running, so we don't have to do a ton of merges back and forth to backport Leiningen 2 functionality to IDEA 11. I like your vision of making the plugin on par with the Maven plugin :) |
|
I just noticed that IDEA 12 has been released and wanted to post that I've got the integration with leiningen.core almost complete. It will pull in the transitive dependencies as well and list them just like the maven plugin (except displayed more leiningen-ish). I will be submitting a pull request in the next couple days, with the latest being before the end of the weekend. |
|
Okay, completed this and submitted pull request: #14 |
|
Can we close this one now? |
|
Yup. Just uploaded a new version to the plugin repo which contains your fix. Again thank you :) |
I'm having several issues using this plugin. The first is that for some reason it is adding modules into a project that shouldn't be there. For example, I have two separate projects in /src/project1 and /src/project2. Each has its own project.clj and .iml files. When I open project1 Leiningen adds project2 as a module inside of project1. Really weird. Both projects are listed in the Leiningen window.
The second problem is that IntelliJ can't find any of the dependencies. It seems that "lein deps" doesn't copy the dependency jars into /lib in version 2 any longer.
Very, very frustrated with this, so any advice would be much appreciated. I'd much rather use Leiningen than go back to Maven.
The text was updated successfully, but these errors were encountered: