Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Potential Feature] How to identify an entry as symlink? #39

Open
ayushmanchhabra opened this issue Jan 31, 2024 · 6 comments · May be fixed by #47
Open

[Potential Feature] How to identify an entry as symlink? #39

ayushmanchhabra opened this issue Jan 31, 2024 · 6 comments · May be fixed by #47

Comments

@ayushmanchhabra
Copy link

export async function unzip(zippedFile, cacheDir) {
  const zip = await yauzl.open(zippedFile);

  let entry = await zip.readEntry();

  while (entry !== null) {
    let entryPathAbs = path.join(cacheDir, entry.filename);

    // Create the directory beforehand to prevent `ENOENT: no such file or directory` errors.
    await fs.promises.mkdir(path.dirname(entryPathAbs), { recursive: true });

    const readStream = await entry.openReadStream();
    const writeStream = fs.createWriteStream(entryPathAbs);
    await stream.promises.pipeline(readStream, writeStream);

    entry = await zip.readEntry();
  }

  await zip.close();
}

This is currently how I use this package to unzip files. Is there a way to know if an entry is a symlink? Currently if an entry is a symlink, it is decompressed as a file with the name of directory as file name and the file path as data in that file.

For example, if Resources is a symlink, it is decompressed as a file with file contents:

Versions/Current/Resources
@overlookmotel
Copy link
Owner

I can't actually remember! Or perhaps the ZIP spec doesn't actually cover this, and it's been tacked-on as custom extensions by different ZIP implementations.

I'd suggest looking at the ZIP spec and also using a bit of trial and error looking at entry.generalPurposeBitFlag and entry.extraFields - comparing the 2 for normal files and symlinks.

If you find the answer, please let me know and we can maybe add an API for it.

Sorry but I don't have time to figure it out myself right now (and symlinks are probably a niche case), but I hope the above is useful.

@ayushmanchhabra ayushmanchhabra changed the title [Question] How to identify an entry as symlink? [Potential Feature] How to identify an entry as symlink? Jan 31, 2024
@ayushmanchhabra
Copy link
Author

I can't actually remember! Or perhaps the ZIP spec doesn't actually cover this, and it's been tacked-on as custom extensions by different ZIP implementations.

I'd suggest looking at the ZIP spec and also using a bit of trial and error looking at entry.generalPurposeBitFlag and entry.extraFields - comparing the 2 for normal files and symlinks.

If you find the answer, please let me know and we can maybe add an API for it.

Sorry but I don't have time to figure it out myself right now (and symlinks are probably a niche case), but I hope the above is useful.

Sounds good, I'll look into it!

@ayushmanchhabra
Copy link
Author

ayushmanchhabra commented Feb 8, 2024

According to this Stack Overflow post and how this library has implemented symlinks, we have to get the externalFileAttributes from Zip._readEntryAt to calculate the file mode which would allow us to identify if an entry is a symlink or not.

I think with this information, I can identify symlinks without making any changes to yauzl-promise. In the future, an entry.isSymlink() function would be great!

Update:

I'm not sure it'd be the same on Windows

https://stackoverflow.com/questions/4939802/what-are-the-possible-mode-values-returned-by-powershells-get-childitem-cmdle
This might come in handy. Linking here for future reference.

@overlookmotel
Copy link
Owner

overlookmotel commented Feb 8, 2024

Hi @ayushmanchhabra. Thanks very much for looking into it. From reading those, I'm not sure it'd be the same on Windows, as it has a very different file permissions model, from what I understand.

I would be happy to accept a PR, as long as there's a test for unzipping a ZIP file containing symlinks, and that test passes on a range of OSes. If you're willing, that would be great.

But otherwise, I'm afraid I'm not going to be able to find the time to do it myself.

Happy to provide any help/guidance you need in that process. Please let me know if you want to.

@ayushmanchhabra
Copy link
Author

I'm not sure it'd be the same on Windows, as it has a very different file permissions model

How typical of me to ignore Windows! :P

I would be happy to accept a PR, as long as there's a test for unzipping a ZIP file containing symlinks, and that test passes on a range of OSes. If you're willing, that would be great.

I'm planning to test out the symlinks behaviour in nw-builder. Once it works there, I'd be happy to submit a PR!

ayushmanchhabra added a commit to nwutils/nw-builder that referenced this issue Feb 15, 2024
* Do not mix Promise and Callback APIs.
* Remove `createSymlinks` function workaround

Refs:
overlookmotel/yauzl-promise#39 (comment)
@ayushmanchhabra
Copy link
Author

ayushmanchhabra commented Feb 15, 2024

I was successfully able to identify symlinks in nw-builder!

Before implementing entry.isSymlink()

function modeFromEntry(entry) {
  const attr = entry.externalFileAttributes >> 16 || 33188;

  return [448 /* S_IRWXU */, 56 /* S_IRWXG */, 7 /* S_IRWXO */]
    .map(mask => attr & mask)
    .reduce((a, b) => a + b, attr & 61440 /* S_IFMT */);
}

async function unzip(zippedFile, cacheDir) {
  const zip = await yauzl.open(zippedFile);
  let entry = await zip.readEntry();

  while (entry !== null) {
    let entryPathAbs = path.join(cacheDir, entry.filename);
    /* Create the directory beforehand to prevent `ENOENT: no such file or directory` errors. */
    await fs.promises.mkdir(path.dirname(entryPathAbs), { recursive: true });
    /* Check if entry is a symbolic link */
    const isSymlink = ((modeFromEntry(entry) & 0o170000) === 0o120000);
    const readStream = await entry.openReadStream();
    
    if (isSymlink) {
      const chunks = [];
      readStream.on("data", (chunk) => chunks.push(chunk));
      await stream.promises.finished(readStream);
      const link = Buffer.concat(chunks).toString('utf8').trim();
      await fs.promises.symlink(link, entryPathAbs)
    } else {
      const writeStream = fs.createWriteStream(entryPathAbs);
      await stream.promises.pipeline(readStream, writeStream);
    }

    // Read next entry
    entry = await zip.readEntry();
  }
}

After implementing entry.isSymlink():

async function unzip(zippedFile, cacheDir) {
  const zip = await yauzl.open(zippedFile);
  let entry = await zip.readEntry();

  while (entry !== null) {
    let entryPathAbs = path.join(cacheDir, entry.filename);
    /* Create the directory beforehand to prevent `ENOENT: no such file or directory` errors. */
    await fs.promises.mkdir(path.dirname(entryPathAbs), { recursive: true });
    const readStream = await entry.openReadStream();
    
    if (entry.isSymlink()) {
      const chunks = [];
      readStream.on("data", (chunk) => chunks.push(chunk));
      await stream.promises.finished(readStream);
      const link = Buffer.concat(chunks).toString('utf8').trim();
      await fs.promises.symlink(link, entryPathAbs)
    } else {
      const writeStream = fs.createWriteStream(entryPathAbs);
      await stream.promises.pipeline(readStream, writeStream);
    }

    // Read next entry
    entry = await zip.readEntry();
  }
}

The user will still have to create symlinks on their own and this should not be implemented by yauzl-promise. Thoughts?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants