Finding usages across multiple source directories

[bobpaige]

* If you are a Codenvy customer or have a trial license please file your issues with Support through email. GitHub issues do not have a response SLA.*

I have a Java project with 16 submodules and as many Java source directories. I open one of the business objects, select the class name, and select "Find usages" from the Assistant menu. It only shows usages within the same source directory as the business object.

How do I make it search across my entire project?

Reproduction Steps:

1. create a project with multiple source directories, business objects in one source directory, logic in another
2. open a business object and select "Find usages"

Expected behavior:

I expect it will search across my entire project. Better yet (maybe a configurable option), search across my entire workspace.

Observed behavior:

It only shows usages within the same source directory.

Codenvy version: 5.0.0-M5
OS and version: Server: Docker (some version); client: Windows, Chrome

Additional information:

Problem only started happening recently, didn't happen in an older version of Codenvy: This is a new in-house trial of Codenvy.

Problem can be reliably reproduced: Yes

Diagnostic bundle
Execute the script on this page and post public link to the bundle here.

[TylerJewell]

When you say repositories - do you mean different git repos? If so, our workspaces can have multiple projects, with each project representing a single repo. And then each of those projects having their own project type with intellisense associated with it. I am unsure if our intellisense spans across multiple projects in a single workspace, but would need to check. But you usually manage this by having a maven multi-module project - single project with many modules.

[bobpaige]

Thanks for the quick response!

I have two projects in my workspace that are from two different repos, but even in one of those projects (which itself has multiple source directories) it doesn't find all usages across source directories.

You mention maven; we happen to be using gradle, but I had to manually configure the source directories (using the "Configure classpath" dialog) because I could find no other way to have it find my sources.

Our Gradle projects have multiple submodules (same idea as a maven multi-module project)

[bobpaige]

Another data point: I opened a unit test that uses a particular business object (that did not show up in the list of usages). When I select the business object and select "find usages", I see the same list of 111 usages, excluding the source file I'm in at that moment.

So it seems to understand that this business object is the one I'm referencing, but Codenvy doesn't show sources in my unit test source directory.

Both the business objects and the unit tests are on the classpath as source directories.

[TylerJewell]

I am fairly confident that we would not traverse across multiple projects, but we should absolutely parse against a single one. We cannot address both problems at the same time so let's make this about the single project. Engineers would need to reproduce the issue to try and diagnose it so can you provide those instructions.

[tolusha]

@bobpaige
I've just checked on http://github.com/eclipse/che project and Finding usages works fine inside single project. Could you pls point out which project do you use?

[bobpaige]

I'm using Codenvy 5.0.0-M5, editing an in-house project of 2,400+ source files in 12+ source directories. Plus some source directories that are generated at build time. Smaller than eclipse/che :)

I manually added my source directories (through the Configure Classpath dialog) to my project. When I search for all occurrences of a given business object, it finds 100+ uses but only in files from that source directory, not other source directories in the same project. This is a "Java" project (not maven or blank).

I tried importing the che sources as an example. Definitely hit by this issue (#1037) constantly, as well as Chrome claiming pages are unresponsive while it re-reads the project structure.

Pulled up the DTO.java class (in some source path; hard to tell in Codenvy but I'll start a different issue for that) and looked for usages. It definitely found 710 occurrences across several source directories, but that is a maven project, not generic java.

Maybe that is the issue? Maven, not Java?

[tolusha]

If it is opensource can you give me a link to download sources?

[tolusha]

I would like to try and to reproduce the issue.

[bobpaige]

Sorry; in-house, not open source.

[bobpaige]

Curiously, I increased the RAM for my workspace from 2G to 4G and restarted it. Then opened DTO.java and did a "Find occurrences" again; this time I get only 353 occurrences, whereas last time I got 710 (see message above).

Which is it?

[bobpaige]

Left Codenvy up in the background for an hour and, when I thought about it, tried "Find occurrences" on the DTO.java class from eclipse/che. This time I got 710 occurrences but is complaining it can't find the sources:

Double-clicking the references does open the file, but the "Usage" panel seems to be confused.

Related question: I increased RAM for my workspace to 4G, but what about the RAM configuration for the Codenvy process itself; can I update that as a user, or should that option be set to a certain value as part of the install?

[TylerJewell]

Right now we do not give you control over the size of the "ws-agent" that runs inside of your workspace. You can set the overall workspace RAM, and then the agent consumes as much of that as it needs. It's usually not much, but if you are running the Java languaage server based upon JDT, it can be a memory hog.

In a future version of Che / Codenvy, we are going to move all non-root agents to be running as containers that are separate from the workspace itself. This would be all language server agents and the ws-agent. They get access to projects through volume mounts. SSH and terminal would still reside inside the workspace because they need root access to the entire container.

But by doing this, we can give users more control over how the system is managing a workspace + agents.

[bobpaige]

So you are confident the behaviors I'm seeing now (inconsistent results from "Find usages" and the "Can't find sources" error above) are not memory related? That is good.

Do you have guidelines for memory allocation for workspaces, or the size of projects that Codenvy will support? eclipse/che appears to have about 3x the number of source files as our core code, but we have a dozen (and growing) projects built on this core, so I'm trying to understand the best way to layout our workspaces.

Should I have one workspace that loads all projects, including the core? Seems like overkill.

Should have one workspace for each project, plus the core? Makes it easy to debug/update the core along with the other project, but then any local branches can't be shared with other workspaces (leading to lots of temporary branches on our main Git server, and no, I don't see us using forks internally).

Or maybe I should continue to press for separation between our core code and other project code, meaning each workspace should have precisely one of these projects in it, not both. If you need to update the core, make it an official "feature-version" and push the build artifacts to Artifactory before you try to integrate. But I can imagine lots of pushback on this option.

Sorry, the logistics/politics are my issue to resolve, not yours, but I would like to know the technical limits/expectations of Codenvy. And I'm always open to process feedback :)

[TylerJewell]

Well, I am personally not confident that the memory issues are what caused the inconsistency with Find Usages. But I suspect that it has something to do with processes running inside the container, not the IDE. We are launching JDT processes in the background, and if they are not completed, then you can get erroneous errors. In many cases we do not block the user from seeing intermediate results, and I think more advanced systems hide all information until the background analysis has completed. We have opted to try and provide more incremental updates as the processes unfolds. But it's possible that even the intermediate steps of presentation are just wrong -and it was why Anatoliy wanted to try and reproduce the issue.

On the guidance for how to structure your workspace, that is unique to every company. What we try to say is that each workspace at a minimum needs about 256MB of overhead for our agents. If you add intellisense, it can be anywhere from 50-1.5GB of RAM. The JDT is a memory hog and the JDT will eat up as much memory as it can get. So you need to size your worksapces around this concept that is there. The more workspaces that you have running at the same time the more memory you need. Also, we are not doing anything to do intellisense across workspaces, so you will need to consider that unit of work as well.

Generally, we want to work towards a world where each workspace = 1 set of compose services, which are identical to what you would put into production. And so the workspace management reflects production as close as possible.

[TylerJewell]

May I close this issue, or is there still a core open issue related to the cross-repo scanning? We need to make sure that we have each issue uniquely named and tagged, otherwise it will get lose over time.

[bobpaige]

Yes, thanks for checking.

Cross-repo scanning would be great, but I see how it doesn't really fit into your plan at the moment.