Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unpigz -l failed with "can't destroy locked resource" #96

Closed
wrjiafang opened this issue Jan 13, 2022 · 3 comments
Closed

unpigz -l failed with "can't destroy locked resource" #96

wrjiafang opened this issue Jan 13, 2022 · 3 comments

Comments

@wrjiafang
Copy link

wrjiafang commented Jan 13, 2022

Create a file about 200M size named test.txt and pigz this file to get test.txt.gz and then "unpigz -l" will failed, error as:
$ ./unpigz -l test.txt.gz
compressed original reduced name
228799 209715200 99.9% test.txt
unpigz: can't destroy locked resource (pigz.c:2622:mutex_destroy)
unpigz: abort: internal threads error

or

$ ./unpigz -l test.txt.gz
unpigz: skipping: test.txt.gz unrecognized format
unpigz: can't destroy locked resource (pigz.c:2622:mutex_destroy)
unpigz: abort: internal threads error

It seems like there are data race when use valgrind tool to check.
We found that, "unpigz" will g.decode = 1; and then in process(), use in_init(); method = get_header(1);
However, the following g.list, list_info(), will also in_init();method = get_header(1);
This will lead to get_header()-->GET()-->load()-->
if (g.procs > 1) {
// if first time, fire up the read thread, ask for a read
if (g.in_which == -1) {
g.in_which = 1;
g.load_state = new_lock(1);
g.load_thread = launch(load_read, NULL);
}
Double init g.load_state and create load_read thread.
Then there are many data race and the thread will be failed to destroy with error (EBUSY: Device or resource busy)

@wrjiafang
Copy link
Author

wrjiafang commented Jan 13, 2022

To avoid the double init and create thread, but get the correct method value. We can fix it by:

+    
+       // if requested, just list information about input file
+    if (g.list) {
+        list_info();
+        load_end();
+        return;
+    }

     // if decoding or testing, try to read gzip header
     if (g.decode) {
@@ -4048,13 +4055,6 @@
         }
     }

-    // if requested, just list information about input file
-    if (g.list) {
-        list_info();
-        load_end();
-        return;
-    }
-

Put the list_info before any decode, since it does not make sense to do any decode job if just list info.
Please give some suggestions. Thanks.
.

@madler
Copy link
Owner

madler commented Jan 14, 2022

Thank you for the report. Actually, it does make sense to decode for listing, for pigz -lt. However I did not consider the possibility of what you're requesting, which is equivalent to pigz -ld. The fix is similar to your proposal, but adding a check on g.decode:

@@ -4021,6 +4021,13 @@ local void process(char *path) {
     }
     SET_BINARY_MODE(g.ind);
 
+    // if requested, just list information about the input file
+    if (g.list && g.decode != 2) {
+        list_info();
+        load_end();
+        return;
+    }
+
     // if decoding or testing, try to read gzip header
     if (g.decode) {
         in_init();
@@ -4062,13 +4069,6 @@ local void process(char *path) {
         }
     }
 
-    // if requested, just list information about input file
-    if (g.list) {
-        list_info();
-        load_end();
-        return;
-    }
-
     // create output file out, descriptor outd
     if (path == NULL || g.pipeout) {
         // write to stdout

This will be updated in a future version.

@madler madler closed this as completed Jan 14, 2022
@wrjiafang
Copy link
Author

@madler Thanks very much for your quick reply. Understood, let's look forward to your new version.

halstead pushed a commit to openembedded/openembedded-core that referenced this issue Jan 17, 2022
Refer [1], "unpigz -l" failed with error:
$ ./unpigz -l test.txt.gz
compressed original reduced name
228799 209715200 99.9% test.txt
unpigz: can't destroy locked resource (pigz.c:2622:mutex_destroy)
unpigz: abort: internal threads error

or

$ ./unpigz -l test.txt.gz
unpigz: skipping: test.txt.gz unrecognized format
unpigz: can't destroy locked resource (pigz.c:2622:mutex_destroy)
unpigz: abort: internal threads error

[1] madler/pigz#96

Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
offa pushed a commit to jhnc-oss/poky that referenced this issue Jan 17, 2022
Refer [1], "unpigz -l" failed with error:
$ ./unpigz -l test.txt.gz
compressed original reduced name
228799 209715200 99.9% test.txt
unpigz: can't destroy locked resource (pigz.c:2622:mutex_destroy)
unpigz: abort: internal threads error

or

$ ./unpigz -l test.txt.gz
unpigz: skipping: test.txt.gz unrecognized format
unpigz: can't destroy locked resource (pigz.c:2622:mutex_destroy)
unpigz: abort: internal threads error

[1] madler/pigz#96

(From OE-Core rev: f1d4ce46f07f4d2a0213da2efeb0f025083f0999)

Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
halstead pushed a commit to openembedded/openembedded-core that referenced this issue Jan 17, 2022
Refer [1], "unpigz -l" failed with error:
$ ./unpigz -l test.txt.gz
compressed original reduced name
228799 209715200 99.9% test.txt
unpigz: can't destroy locked resource (pigz.c:2622:mutex_destroy)
unpigz: abort: internal threads error

or

$ ./unpigz -l test.txt.gz
unpigz: skipping: test.txt.gz unrecognized format
unpigz: can't destroy locked resource (pigz.c:2622:mutex_destroy)
unpigz: abort: internal threads error

[1] madler/pigz#96

Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
seambot pushed a commit to seamapi/poky that referenced this issue Jan 17, 2022
Refer [1], "unpigz -l" failed with error:
$ ./unpigz -l test.txt.gz
compressed original reduced name
228799 209715200 99.9% test.txt
unpigz: can't destroy locked resource (pigz.c:2622:mutex_destroy)
unpigz: abort: internal threads error

or

$ ./unpigz -l test.txt.gz
unpigz: skipping: test.txt.gz unrecognized format
unpigz: can't destroy locked resource (pigz.c:2622:mutex_destroy)
unpigz: abort: internal threads error

[1] madler/pigz#96

(From OE-Core rev: 856d9e07e60b6f85681782107934a1bdc12c6429)

Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
halstead pushed a commit to openembedded/openembedded-core that referenced this issue Feb 1, 2022
Refer [1], "unpigz -l" failed with error:
$ ./unpigz -l test.txt.gz
compressed original reduced name
228799 209715200 99.9% test.txt
unpigz: can't destroy locked resource (pigz.c:2622:mutex_destroy)
unpigz: abort: internal threads error

or

$ ./unpigz -l test.txt.gz
unpigz: skipping: test.txt.gz unrecognized format
unpigz: can't destroy locked resource (pigz.c:2622:mutex_destroy)
unpigz: abort: internal threads error

[1] madler/pigz#96

Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
halstead pushed a commit to openembedded/openembedded-core that referenced this issue Feb 1, 2022
Refer [1], "unpigz -l" failed with error:
$ ./unpigz -l test.txt.gz
compressed original reduced name
228799 209715200 99.9% test.txt
unpigz: can't destroy locked resource (pigz.c:2622:mutex_destroy)
unpigz: abort: internal threads error

or

$ ./unpigz -l test.txt.gz
unpigz: skipping: test.txt.gz unrecognized format
unpigz: can't destroy locked resource (pigz.c:2622:mutex_destroy)
unpigz: abort: internal threads error

[1] madler/pigz#96

Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
seambot pushed a commit to seamapi/poky that referenced this issue Feb 1, 2022
Refer [1], "unpigz -l" failed with error:
$ ./unpigz -l test.txt.gz
compressed original reduced name
228799 209715200 99.9% test.txt
unpigz: can't destroy locked resource (pigz.c:2622:mutex_destroy)
unpigz: abort: internal threads error

or

$ ./unpigz -l test.txt.gz
unpigz: skipping: test.txt.gz unrecognized format
unpigz: can't destroy locked resource (pigz.c:2622:mutex_destroy)
unpigz: abort: internal threads error

[1] madler/pigz#96

(From OE-Core rev: 6aff6c269015385b20806a26beed8e69003599c4)

Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
seambot pushed a commit to seamapi/poky that referenced this issue Feb 1, 2022
Refer [1], "unpigz -l" failed with error:
$ ./unpigz -l test.txt.gz
compressed original reduced name
228799 209715200 99.9% test.txt
unpigz: can't destroy locked resource (pigz.c:2622:mutex_destroy)
unpigz: abort: internal threads error

or

$ ./unpigz -l test.txt.gz
unpigz: skipping: test.txt.gz unrecognized format
unpigz: can't destroy locked resource (pigz.c:2622:mutex_destroy)
unpigz: abort: internal threads error

[1] madler/pigz#96

(From OE-Core rev: b8e0bcecce31af29d04d4b694ebddb33b1e5710a)

Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants