Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Symbolic links are added as symlinks to the sources JAR instead of being followed #160

Open
mirabilos opened this issue Jan 8, 2021 · 13 comments
Open

Symbolic links are added as symlinks to the sources JAR instead of being followed #160

mirabilos opened this issue Jan 8, 2021 · 13 comments

Comments

@mirabilos
Copy link

I’ve got symbolic links in my resources (files that get reused in multiple places in a larger repository). They are correctly followed when creating the main binary JAR but the sources JAR contains the symbolic links themselves¹, which then point to nirvana (i.e. to outside the JAR).

In #47 (comment) it was suggested to make the “follow symlinks?” mode configurable. Implementing this would completely solve my issue if it is configurable in the necessary places; I believe that implementing it in the manner shown will fix it.

Related: https://issues.apache.org/jira/browse/MRESOURCES-237 where 3.x versions of the maven-resources-plugin cause similar problems by not following symlinks any more (which is why I’m pinning it to a 2.x version in my projects).

Fixing this in plexus-archiver might fix the maven-resources-plugin problem as well… unsure.

① Funnily enough, this even causes other software to crash when exploring the resulting archives…
@michael-o
Copy link
Member

The reported crash isn't reproducible for me with:

$ mc -V
GNU Midnight Commander 4.8.24
Kompiliert mit GLib 2.66.2
Die ncurses-Bibliothek benutzen
Mit eingebautem Editor
Mit Unterstützung für Hintergrundtätigkeiten
Mit Maus-Unterstützung für xterm
Mit Unterstützung für X11-Ereignisse
Mit Internationalisierungs-Unterstützung
Mit Unterstützung mehrerer Codepages
Virtuelles Dateisystem: cpiofs, tarfs, sfs, extfs, ftpfs, fish
Datentyp: char: 8; int: 32; long: 64; void *: 64; size_t: 64; off_t: 64;

This needs to be verified with Plexus Archiver.

@mirabilos
Copy link
Author

It’s not about the crash, this is just a funny side effect.

I need to be able to have the maven-source-plugin follow symlinks when creating the sources JAR instead of adding the symlinks themselves into the JAR. This is what this is about.

@michael-o
Copy link
Member

Likely both modes would be required. Retention and resolution.

@mirabilos
Copy link
Author

Yes, of course, hence the request to make it configurable.

Per resource set, as @petr-ujezdsky suggested in his comment, would probably match best.

@mirabilos
Copy link
Author

Setting collection.setFollowingSymLinks(true); does not seem to be enough; the resulting JAR file still contains the symbolic link instead of the pointed-to file :(

@mirabilos
Copy link
Author

Some debugging later…

Archiver archiver;
archiver.addDirectory( sourceDirectory, pIncludes, pExcludes );

        getLog().info( "resource {{{" );
        ResourceIterator ri = archiver.getResources();
        while (ri.hasNext()) {
            getLog().info( "resource: " + fmt(ri.next()) );
        }
        getLog().info( "resource }}}" );
    }

    private static String fmt(final ArchiveEntry e) {
    	return String.format("%d(%o)<%s>", e.getType(),e.getMode(),e.getName());
    }

… it clearly shows that, despite…

--- a/src/main/java/org/codehaus/plexus/archiver/AbstractArchiver.java
+++ b/src/main/java/org/codehaus/plexus/archiver/AbstractArchiver.java
@@ -367,7 +367,7 @@ public void addFileSet( @Nonnull final FileSet fileSet )
         // The PlexusIoFileResourceCollection contains platform-specific File.separatorChar which
         // is an interesting cause of grief, see PLXCOMP-192
         final PlexusIoFileResourceCollection collection = new PlexusIoFileResourceCollection();
-        collection.setFollowingSymLinks( false );
+        collection.setFollowingSymLinks( true );
 
         collection.setIncludes( fileSet.getIncludes() );
         collection.setExcludes( fileSet.getExcludes() );

…, the collection entry is still a symlink:

[INFO] indeed running the patched plugin
[INFO] resource {{{
[INFO] resource: 2(40755)<>
[INFO] resource: 2(40755)<org>
[INFO] resource: 2(40755)<org/evolvis>
[INFO] resource: 2(40755)<org/evolvis/tartools>
[INFO] resource: 2(40755)<org/evolvis/tartools/mvnparent>
[INFO] resource: 2(40755)<org/evolvis/tartools/mvnparent/examples>
[INFO] resource: 1(100644)<org/evolvis/tartools/mvnparent/InitialiseLogging.java>
[INFO] resource: 1(100644)<org/evolvis/tartools/mvnparent/examples/Main.java>
[INFO] resource: 2(40755)<META-INF>
[INFO] resource: 2(40755)<META-INF/legal>
[INFO] resource: 2(40755)<META-INF/legal/org.evolvis.tartools>
[INFO] resource: 2(40755)<META-INF/legal/org.evolvis.tartools/maven-parent-lib>
[INFO] resource: 3(120777)<META-INF/legal/org.evolvis.tartools/maven-parent-lib/LICENCE>
[INFO] resource }}}

This is about the last file, a symbolic link:

lrwxrwxrwx 1 me me 54 Jan  8 15:35 lib/src/main/resources/META-INF/legal/org.evolvis.tartools/maven-parent-lib/LICENCE -> ../../../../../../../../src/main/ancillary/LICENCE.hdr

I hope the fact that the amount of ../s brings it out of the lib/ subdirectory is not the problem here, because this is how files are shared in multi-module projects…

@mirabilos
Copy link
Author

This might be due to plexus-io ResourceFactory…

public static PlexusIoResource createResource( File f, String name, final ContentSupplier contentSupplier,
                                               InputStreamTransformer inputStreamTransformer,
                                               PlexusIoResourceAttributes attributes )
    throws IOException
{
    boolean symbolicLink = attributes.isSymbolicLink();
    return symbolicLink ? new PlexusIoSymlinkResource( f, name, attributes )
        :  new PlexusIoFileResource(f, name, attributes, contentSupplier, inputStreamTransformer);
}

… in PlexusIoFileResourceCollection…

        File f = new File( dir, sourceDir );

        PlexusIoResourceAttributes attrs = new FileAttributes( f, cache1, cache2 );
        attrs = mergeAttributes( attrs, f.isDirectory() );

        String remappedName = getName( name );

        PlexusIoResource resource =
            ResourceFactory.createResource( f, remappedName, null, getStreamTransformer(), attrs );

… and FileAttributes…

        Map<String, Object> attrs = Files.readAttributes( path, "unix:permissions,gid,uid,isSymbolicLink,mode", LinkOption.NOFOLLOW_LINKS );

… so the collection.setFollowingSymLinks(true); seems to be only about following symbolic links to directories when encountered; additional remapping of symbolic links in plexus-io is needed.

@mirabilos mirabilos mentioned this issue Jan 8, 2021
Open
mirabilos added a commit to qvest-digital/plexus-archiver that referenced this issue Jan 8, 2021
@mirabilos
forked plexus-io
df727bf 
maybe advances debugging related to
codehaus-plexus#160
(even if this fork is only concerned about following symlinks,
not (yet) about making it configurable, since that is beyond
my knowledge about this project)
mirabilos added a commit to qvest-digital/maven-parent that referenced this issue Jan 8, 2021
@mirabilos
experiment with a forked plexus archiver to follow symlinks
096688f 
cf. codehaus-plexus/plexus-archiver#160
mirabilos added a commit to qvest-digital/plexus-archiver that referenced this issue Jan 8, 2021
@mirabilos
make following symlinks configurable in plexus-archiver
11bee26 
default is still true here, will be false when submitting upstream

REQUIRES a version of plexus-io with the following PR merged:
codehaus-plexus/plexus-io#38

advances codehaus-plexus#160
even if I don’t know how to make this configurable from Maven
@mirabilos
Copy link
Author

I’m currently trying to implement this inside plexus-archiver in a configurable way (support both retention and resolution of symbolic links, as you said), but I have absolutely no idea how to then tie this in with the maven-source-plugin because the Maven Archiver configuration does not expose any plexus-archiver configuration, if there is any.

@michael-o can you help me with that?

@mirabilos
Copy link
Author

mirabilos commented Jan 8, 2021 •
edited
Loading

My WIP branch on top of the 4.2.1 release (since that’s what the maven-source-plugin uses) now has this implemented, even extending the ZipArchiverTest.testSymlinkFileSet() case so it tests both retaining and resolving symlinks at creation time.

Even if this necessarily must wait for a new plexus-io release with that PR applied, review (also whether I’m on the correct path here) welcome…

@michael-o
Copy link
Member

You need to pass this through all layers. All plugins using Maven Archiver expose XML configuration for it.

@mirabilos
Copy link
Author

mirabilos commented Jan 8, 2021 via email

@plamentotev
Copy link
Member

For me the topic of symlinks is at times quite confusing and hard to follow (no pun intended). It would help if we write down the requirements what is the desired behavior when the follow symbolic links. Plexus Archiver does have a complex system of including/excluding files, filenames mappings, overriding file attributes etc. Lets say we have the following directory structure (-> denotes symlink)

/regularDir
    /nestedDir
        regularFileC
    regularFileA
    regurlarFileB
regularFileD
symLinkDir -> regularDir
symLinkA -> regularFileD
symLinkB -> regularDir/regularFileA

So if follow links is true then what should be the result. For files I think it is relatively straightforward. We should apply all exclusion/inclusion, file mappings, etc on the symLinkA and symLinkB. Just when compressing we should include the content of regularFileD and regularDir/regularFileA instead of having symlinks in the archive. But what about the file attributes? And what about dirs.

p.s. @mirabilos please keep in mind that the follow sylinks change you mentioned was applied for Windows - platform on which Plexus Archiver didn't support symlinks at the time. That is why reverting the change on all system does not work as expected. I was intended for systems where symlinks are not supported not for ones where it is supported but should be followed.

@mirabilos
Copy link
Author

mirabilos commented Jan 9, 2021 via email

@mirabilos mirabilos mentioned this issue Apr 18, 2021
Open
mirabilos added a commit to qvest-digital/maven-parent that referenced this issue Jun 25, 2024
@mirabilos
experiment to see whether we can revert to upstream plexus-io
5bfddce 
and with it latest maven-source-plugin

if codehaus-plexus/plexus-archiver#160
is, indeed, fixed now…
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@michael-o @mirabilos @plamentotev