savannah: VFS optimization request

[mc-butler]

Important

This issue was migrated from Trac:

Origin	https://midnight-commander.org/ticket/3
Reporter	slavazanko (@slavaz)

Original: http://savannah.gnu.org/bugs/?8136

Submitted by:	None	Submitted on:	Fri 12 Mar 2004 11:00:45 PM UTC
Category:	VFS	Severity:	3 - Normal
Status:	None	Privacy:	Public
Assigned to:	None	Open/Closed:	Open
Release:	4.6.0	Operating System:	GNU/Linux

Original submission:

Not a bug, but feature request.

I am using VFS (extfs). I have written program (backup extractor) 
which handle "copyout" and "list" commands from mc. But if I want to
 copyout a whole directory (or a lot of files from archive), it is 
called FOR EACH extracted file. In my case, extractor (duplicity 
backupper, http://www.nongnu.org/duplicity/ ) starts very long time.
 But it can extract whole directory quickly.

I think if mc would pass to extfs script environment variable 
containing names of ALL the selected files (AND DIRECTORIES) on the 
panel before extracting, it would get ability to optimize extract 
script. For example, I can fully extract all needed files in 
temporary directory by ONE call to duplicity, and then just copy 
these files from temp directory to pathes needed by mc.

Comment 1 by Anonymous at Sat 13 Mar 2004 12:19:29 AM UTC:

Here is a simple patch (also attached). Hope you will apply it in
next release.

--- mc-4.6.1-pre1/src/file.c 2003-10-27 20:36:19.000000000 +0300
+++ mc-4.6.1-pre1.patched/src/file.c 2004-03-12 03:13:05.000000000 +0300
@@ -1865,4 +1865,27 @@
}
}
+
+ /*
+ * Dmitry Koteroff <d[at]koteroff.ru>
+ * Patch to save selected items in environment variable.
+ * This variable would be passed to extfs scripts and may be used
+ * to optimize depacking performance.
+ */
+ {{{{
+ char* env = g_strconcat("", NULL);
+ for (i = 0; i < panel->count; i++) {
+ char prevEnv, p;
+ if (!panel->dir.list[i].f.marked) continue;
+ p = concat_dir_and_file (panel->cwd, panel->dir.list[i].fname);
+ prevEnv = env;
+ env = g_strconcat(prevEnv, p, "\n", NULL);
+ g_free(p);
+ g_free(prevEnv);
+ }
+ setenv("MC_SELECTION", env, 1);
+ g_free(env);
+ }}}}
+
+
#ifdef WITH_BACKGROUND
/* Did the user select to do a background operation? */

Comment 2 by Anonymous at Sat 13 Mar 2004 06:40:31 PM UTC:

Here is the better patch, it also consider current cursor position.

P.S.
"return 0" in background operation instead of "goto glean_up" isn't a bug?..

--- mc-4.6.0/src/file.c 2002-12-26 22:04:10.000000000 +0300
+++ mc-4.6.0.patched/src/file.c 2004-03-12 21:22:17.000000000 +0300
@@ -1895,4 +1895,30 @@
}
}
+
+ /*
+ * Dmitry Koteroff <d[at]koteroff.ru>
+ * Patch to save selected items in environment variable.
+ * This variable would be passed to extfs scripts and may be used
+ * to optimize depacking performance.
+ */
+ {{{{
+ char* env = g_strconcat("", NULL);
+ for (i = 0; i < panel->count; i++) {
+ char prevEnv, p;
+ if (!panel->dir.list[i].f.marked) continue;
+ p = concat_dir_and_file (panel->cwd, panel->dir.list[i].fname);
+ prevEnv = env;
+ env = g_strconcat(prevEnv, p, "\n", NULL);
+ g_free(p);
+ g_free(prevEnv);
+ }
+ setenv("MC_SELECTION", env, 1);
+ g_free(env);
+ char *cur = concat_dir_and_file (panel->cwd, source);
+ setenv("MC_CURRENT", cur, 1);
+ g_free(cur);
+ }}}}
+
+
#ifdef WITH_BACKGROUND
/* Did the user select to do a background operation? */
@@ -1913,4 +1939,5 @@
mc_setctl (dest, MCCTL_FORGET_ABOUT, NULL);
/* file_op_context_destroy (ctx); */
+ setenv("MC_SELECTION", "", 1); setenv("MC_CURRENT", "", 1);
return 0;
}
@@ -2123,4 +2150,5 @@
} /* Many files */
clean_up:
+ setenv("MC_SELECTION", "", 1); setenv("MC_CURRENT", "", 1);
/* Clean up */

Note

Original attachments:

• mc-4.6.0.diff (raw) by slavazanko (@slavaz) on Dec 24, 2008 at 9:50 UTC
• mc-4.6.1-pre1.diff.zip (raw) by slavazanko (@slavaz) on Dec 24, 2008 at 9:51 UTC

[mc-butler]

Changed by slavazanko (@slavaz) on Dec 24, 2008 at 9:50 UTC

• Attachment mc-4.6.0.diff (raw) added

[mc-butler]

Changed by slavazanko (@slavaz) on Dec 24, 2008 at 9:51 UTC

• Attachment mc-4.6.1-pre1.diff.zip (raw) added

[mc-butler]

Changed by metux (@metux) on Dec 26, 2008 at 21:28 UTC (comment 1)

Wouldnt it be better to introduce some new vfs operation
(bulk_copy?) for that ?

[mc-butler]

Changed by styx (@styx) on Feb 23, 2009 at 19:08 UTC (comment 2)

• Milestone set to 4.7

[mc-butler]

Changed by styx (@styx) on May 27, 2009 at 10:59 UTC (comment 3)

• Milestone changed from 4.7 to VFS Standardisation

[mc-butler]

Changed by birdie (aros@….com) on Aug 1, 2010 at 18:27 UTC (comment 4)

• Severity set to no branch

Also I have a firm belief that VFS plugins must be rewritten using a faster (than shell) language. It's already done for CPIO and TAR but other archivers need this treatment too. However I have no idea which language can suit this task because C/C++ language is difficult to code and patch quickly, while script languages are not really as fast as they need to be.

Try to enter an archive containing over 50 thousand files and you'll instantly notice how much time MC spends parsing a VFS plugin output.

[mc-butler]

Changed by slavazanko (@slavaz) on Oct 29, 2010 at 11:23 UTC

• Blocking set to #1952

(In #1952) Ticket frozen until #3 unfixed.

[mc-butler]

Changed by TiGR (@TiGR) on Aug 30, 2011 at 19:26 UTC (comment 6)

• Branch state set to no branch

Are there any news on this ticket? Currently, unpacking archives is a real pain, while it is a very common use case.

[mc-butler]

Changed by andrew_b (@aborodin) on Aug 31, 2011 at 7:11 UTC (comment 7)

A lot of changes in VFS layer are required for this issue. That will be possible only after 4.8.0 release.

[mc-butler]

Changed by ossi (@ossilator) on Jan 11, 2014 at 15:19 UTC (comment 8)

• Description edited

[mc-butler]

Changed by andrew_b (@aborodin) on Jun 18, 2015 at 18:27 UTC (comment 9)

• Version set to master
• Milestone changed from 4.8 to Future Releases

[mc-butler]

Changed by andrew_b (@aborodin) on May 7, 2023 at 9:47 UTC

• Blocking #1952 deleted

[mc-butler]

Changed by zaytsev (@zyv) on Feb 21, 2025 at 7:29 UTC (comment 11)

Ticket #2051 has been marked as a duplicate of this ticket.

[mc-butler]

Changed by zaytsev (@zyv) on Feb 21, 2025 at 7:37 UTC (comment 12)

Ticket #2317 has been marked as a duplicate of this ticket.