New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
archive.on("progress") returns wrongs values v4 -> v5 #475
Comments
do you have example code? It could be the glob change since its more or less reading fs as needed vs huge swaths at a time. |
@ctalkington sure , const archiver = require('archiver'); // You can change this by something bigger!
|
I noticed this today on a mac. |
I noticed this as well. It seems that Example Codeconst archiver = require("archiver");
const fs = require("fs");
const Path = require("path");
const TEST_FILES_DIR = "./random_test_files"; //A directory with a few random files to test zipping
function setup(fileName) {
const archive = archiver('zip', {
zlib: {level: 9} // Sets the compression level.
});
const writeStream = fs.createWriteStream(fileName);
archive.pipe(writeStream);
archive.on("progress", (progress) => {
console.log("TOTAL", progress.entries.total, "PROCESSED", progress.entries.processed);
});
return archive;
}
(async () => {
const folderName = Path.basename(TEST_FILES_DIR)
console.log("========================= ZIPPING WITH archive.directory =========================")
const archiveDir = setup("directory.zip");
archiveDir.directory(TEST_FILES_DIR, folderName);
await archiveDir.finalize();
console.log("========================= ZIPPING WITH MANUAL TRAVERSAL =========================");
const archiveManual = setup("manual.zip");
const files = fs.readdirSync(TEST_FILES_DIR)
for (const file of files) {
archiveManual.file(Path.join(TEST_FILES_DIR, file), {name: Path.join(folderName, file)});
}
await archiveManual.finalize();
})() Output:
Looking through the source of To get around this behavior, I'd imagine the best solution is to count the number of files in the directory manually using something like recursive-readdir before calling const recursive = require("recursive-readdir");
const archiver = require("archiver");
const archive = archiver('zip', {
zlib: {level: 9} // Sets the compression level.
});
const writeStream = fs.createWriteStream("test.zip");
archive.pipe(writeStream);
let total = 0
archive.on("progress", (progress) => {
console.log("TOTAL", total, "PROCESSED", progress.entries.processed);
});
total = (await recursive(TEST_FILES_DIR)).length;
archive.directory(TEST_FILES_DIR, folderName);
await archive.finalize(); |
I have the same problem. const fs = require('fs');
const path = require('path');
const archiver = require('archiver');
const log = console.log.bind(console);
const error = console.error.bind(console);
const platform = process.platform;
const projectRoot = path.resolve(__dirname, '..');
const nodeModulesDir = path.resolve(projectRoot, 'node_modules');
const toolkitVersion = require('toolkit/package.json').version;
const zipName = `${toolkitVersion}-${platform}.zip`;
const outputPath = path.resolve(projectRoot, 'zips', zipName);
if (fs.existsSync(zipName)) {
log(`Removing existing zip file: ${zipName}`);
fs.unlinkSync(zipName);
}
if (!fs.existsSync(nodeModulesDir)) {
error('node_modules not found. Please run `yarn` first.');
process.exit(1);
}
const output = fs.createWriteStream(outputPath);
const archive = archiver('zip', {
zlib: { level: 9 }
});
output.on('close', function() {
console.log(archive.pointer() + ' total bytes');
});
archive.on('warning', function(err) {
if (err.code === 'ENOENT') {
error(err);
} else {
throw err;
}
});
archive.on('error', function(err) {
throw err;
});
archive.on("progress", progress => {
const percent = Math.round(progress.entries.processed * 100.0 / progress.entries.total);
log(`\rZipping: ${percent.toFixed(2)}%`);
})
archive.pipe(output)
archive.directory(nodeModulesDir, 'node_modules')
.finalize(); |
Have somebody solution? |
In my situation this.zip.on('progress', (progress) => {
const newProgress = progress.fs.processedBytes / this.totalSize;
// ... do something with progress ...
}); |
Any updates from the devs? I mean hopefully it gets fixed soon, workarounds are cool but not necessary. With Code: archive.on("progress", (progress) => {
const percentage = (progress.entries.processed / progress.entries.total) * 100;
console.log("Processed:", progress.entries.processed, "FilesTotal:", progress.entries.total, "Fortschritt: ", percentage.toFixed(2), "%");
}); Workaround for Version >=5.0.0 by calculating the total entries by myself (with help of fs) By a massive amount of files this is not a good solution: function countFiles(directory) {
let count = 0;
function countFilesRecursive(currentPath) {
const contents = fs.readdirSync(currentPath);
contents.forEach(file => {
count++; // Zähle jede Datei und jedes Verzeichnis
const fullPath = path.join(currentPath, file);
if (fs.lstatSync(fullPath).isDirectory()) {
countFilesRecursive(fullPath);
}
});
}
countFilesRecursive(directory);
return count;
} const filesTotal = countFiles(workspacePath); // Zählt die Dateien im Verzeichnis
// Fügen Sie hier den progress Event-Handler hinzu: Überwachung der gesamten Daten umgewandelt in % 0-100 aber ist nicht wirklich 0-100% da alle Daten angeschaut werden und mal 100 gerechnet werden aber nicht korrekt wenn manche einzelne Daten sehr viel größer sind als kleinere, dennoch ausreichend. Anstelle filesTotal sollte eigentlich: progress.entries.total aber das ist seit 5.0 archiver verbuggt siehe: https://github.com/archiverjs/node-archiver/issues/475
archive.on("progress", (progress) => {
const percentage = (progress.entries.processed / filesTotal) * 100;
console.log("Processed:", progress.entries.processed, "FilesTotal:", filesTotal, "Fortschritt: ", percentage.toFixed(2), "%");
});
archive.pipe(res); // Stecke Archiv ins eine Ende der Pipe und schicke es durch bis res -- wird dann in kleineren Junks zur Echtzeit immer in Stückchen gesendet an Browser
archive.directory(workspacePath, false); // Pfad nimmt alle Ordner und Unterdateien und packt sie an die root der .zip
archive.finalize(); // Stecke einen Korken ans Ende der Pipe - definiert die Blöcke Best solution for this Problem: So just change the archiver version in the package.json: "archiver": "^4.0.2", and update the version with npm install. archive.on("progress", (progress) => {
const percentage = (progress.entries.processed / progress.entries.total) * 100;
console.log("Processed:", progress.entries.processed, "FilesTotal:", progress.entries.total, "Fortschritt: ", percentage.toFixed(2), "%"); // toFixed (2) also Zweistellen nach dem Komma (z. B. 85.51 %) wenn man keine oder weniger haben möchten Zahl bis 0 veringern.
});
archive.pipe(res);
archive.directory((workspacePath), false);
archive.finalize(); @ctalkington Please work on this please. I think its not a big bug just something which came up from 4.0.2 --> 5.0.0 as I tested. |
in "archiver": "4.0.2", when i used archive.on("progress", progress => {}) the values of
progress.entries.processed and progress.entries.total were corrected - different values.
for example, Math.round(p.entries.processed * 100.0 / p.entries.total) were giving me the current percentage.
but in "archiver": "5.1.0" im getting same values in processed and total.
The text was updated successfully, but these errors were encountered: