[FEATURE] handle WAR file and EAR file

[chingchichuang]

Hi,

I would like to ask if DepClean has plan to implement how to handle WAR file and EAR file?
An enterprise archive (EAR) is a wrapper for a Java EE application, which consists of web archive (WAR) and Java archive (JAR) files.
Thank you!

Best regards,
Ching-Chi

[cesarsotovalero]

Hi @chingchichuang, thanks for reaching out.

I've implemented a way for DepClean to decompress and analyze the bytecode in EAR and WAR files, see here. So it should be able to report dependency usages in those files.

Please clone and install DepClean from source, then try it in your codebase and let me know if it works for you.

Question: Do you know about any open-source Maven project that uses a EAR or WAR dependency to test it?

[chingchichuang]

Hi @cesarsotovalero, thank you for the help^______^

I am testing this new feature and find out that this function needs to update too.
It seems that project.getArtifacts() does not return dependencies of WAR file.
This article may be a solution.

For the question about EAR and WAR file, I can't find other open-source Maven project that uses EAR or WAR. The Maven project I use is a proprietary project in my internship company. I have checked with my supervisors and they also don't know about this. If I have more information about this, I will let you know! Thank you!!

[cesarsotovalero]

Hi @chingchichuang,

Thanks for pointing at this.

Would you like to contribute by making a pull request with the working changes as in theShareableJarsInWarsExtractor?

Otherwise, I will try to implement this solution.

[chingchichuang]

Hi @cesarsotovalero,

Sure, I am glad to implement this solution.
I will send a pull request in this week.
Thank you for giving me this opportunity^^

[chingchichuang]

Hi @cesarsotovalero,

I follow the probable solution mentioned previously, but it does not detect dependencies inside the war file.
So, I try another implementation like below. I add another if condition in this method.

Since WAR files's dependencies are all included under the folder WEB-INF/lib/, I try to decompress WAR file and use classAnalyzer to analyze individual jar file (WEB-INF/lib/*.jar). This step works fine.

However, after getting classes by classAnalyzer, I need to put classes and artifact into artifactClassMap.
My current problem is that I don't know how to create an artifact data structure for jar file (WEB-INF/lib/*.jar).

I try to create an artifact by using DefaultArtifact.
DefaultArtifact needs 7 arguements: groupId, artifactId, version, scope, type, classifier, artifactHandler.
I can parse artifactid and version for each jar file, but I don't know how to give classifier and artifactHanlder parameters. Also, I don't know where to get groupId for jar file (WEB-INF/lib/*.jar).
In order to compile successfully, I use the groupId of war file instead and give arbitrary string as classifier parameter.

I am still testing now, and I will update the result later.

P.S. Currently, I use a workaround. I go to the repository of each WAR file and manually copy their dependencies (in pom.xml) to the pom.xml analyzed by DepClean.

      } else if (file != null && file.getName().endsWith(".war")) {
        URL url = file.toURI().toURL();
        acceptWar(url, artifactClassMap, artifact.getGroupId());
      }

  private void acceptWar(URL url, Map<Artifact, Set<String>> artifactClassMap, String wid) throws URISyntaxException {
    File file = new File(new URI(url.toString()));
    decompressDependencyFiles(file.getAbsolutePath());
    try (ZipFile zip = new ZipFile(file)) {
      Enumeration<? extends ZipEntry> zipFileEntries = zip.entries();
      while (zipFileEntries.hasMoreElements()) {
        ZipEntry entry = zipFileEntries.nextElement();
        String currentEntry = entry.getName();
        if (currentEntry.matches("^WEB-INF/lib/.*-.*\\.jar$")) {
          String name = currentEntry.substring(currentEntry.lastIndexOf("/") + 1);
          String artifactId = name.substring(0, name.lastIndexOf("-"));
          String version = name.substring(name.lastIndexOf("-") + 1, name.lastIndexOf("."));
          String jarPath = url.toString().substring(0, url.toString().length() - 4) + "/WEB-INF/lib/" + name;
          Set<String> classes = classAnalyzer.analyze(new URL(jarPath));
          Artifact artifact = new DefaultArtifact(wid, artifactId, version, "runtime", "jar", "jdk8", null);
          artifactClassMap.put(artifact, classes);
        }
      }
    } catch (IOException e) {
      log.info(e.toString());
    }
  }

P.S. for the method decompressDependencyFiles, I reuse your method as below.

  private void decompressDependencyFiles(String zipFile) {
    File file = new File(zipFile);
    try (ZipFile zip = new ZipFile(file)) {
      String newPath = zipFile.substring(0, zipFile.length() - 4);
      new File(newPath).mkdir();
      Enumeration<? extends ZipEntry> zipFileEntries = zip.entries();
      // Process each entry
      while (zipFileEntries.hasMoreElements()) {
        // grab a zip file entry
        ZipEntry entry = zipFileEntries.nextElement();
        String currentEntry = entry.getName();
        File destFile = new File(newPath, currentEntry);
        File destinationParent = destFile.getParentFile();
        // create the parent directory structure if needed
        destinationParent.mkdirs();
        if (!entry.isDirectory()) {
          BufferedInputStream is = new BufferedInputStream(zip.getInputStream(entry));
          int currentByte;
          // establish buffer for writing file
          byte[] data = new byte[BUFFER_SIZE];
          // write the current file to disk
          FileOutputStream fos = new FileOutputStream(destFile);
          try (BufferedOutputStream dest = new BufferedOutputStream(fos, BUFFER_SIZE)) {
            // read and write until last byte is encountered
            while ((currentByte = is.read(data, 0, BUFFER_SIZE)) != -1) {
              dest.write(data, 0, currentByte);
            }
            dest.flush();
            is.close();
          }
        }
      }
    } catch (IOException e) {
      log.info(e.toString());
    }
  }

[cesarsotovalero]

Hi @chingchichuang,
Please use the latest version of DepClean and confirm if this issue has been fixed or not.
Thanks!

[chingchichuang]

Hi @cesarsotovalero,
Thank you! Just see the message today. I will update it this week.
Sorry for not updating for a long time^^

[chingchichuang]

Hi @cesarsotovalero,

I finally have time to test the new version.
The way I test is to turn on the log.
After download and install, I run "mvn se.kth.castor:depclean-maven-plugin:2.0.3-SNAPSHOT:depclean -Dorg.slf4j.simpleLogger.defaultLogLevel=debug" in order to print out the log as shown in the following link.
https://github.com/castor-software/depclean/blob/master/depclean-maven-plugin/src/main/java/se/kth/depclean/graph/MavenDependencyGraph.java#L60
I want to see if new version can find the dependency of WAR file.
The result is that I only see the maven coordinate of WAR file but not the child dependency of WAR file.
I guess if we only declare WAR in POM, it is not enough for the following method to get WAR file's child.
https://github.com/castor-software/depclean/blob/master/depclean-maven-plugin/src/main/java/se/kth/depclean/graph/MavenDependencyGraph.java#L123
So, I believe the problem is how we declare WAR in POM in order for maven API to get WAR's child.
Therefore, I think this issue can be closed.

Currently, I have one month left before my contract with my internship company ends.
I probably won't spend time on figuring out how to add WAR dependency for maven API getting their children dependencies.
I even do not know if this assumption is correct.
However, if you have idea, I am welcome to test it:)
But that would not be relevant to this issue.
I think your new implementation has already covered many cases to add dependencies from POM file, which is awesome!
Thanks for your help!!

[cesarsotovalero]

Hi @chingchichuang,
Thanks for the feedback.
Please do not hesitate in opening another issue if you want to keep contributing!