Importing a file list — a virtual pane from outside XeFM, starting with the clipboard

[crftwr]

Split out of #446. That thread argues the plugin-API side: a registry so a search engine can be driven from XeFM's own dialog. Working through Everything made it clear that the cheaper and more general answer is not a plugin API at all — it is letting a list of paths from anywhere become a pane.

Everything already has a search UI that is better at searching Everything's index than anything XeFM would build: as-you-type over the index, its whole query language (size:, dm:, ext:, !, |, <>), its own history and sorting. The division of labour writes itself — search there, work here — and it needs no contract between the two, only a list of paths.

The same door serves find, fd, rg -l, git ls-files, a build log, a spreadsheet column, or a list someone pasted in chat.

---

1. This is the missing inverse of an action that already ships

Three of the four pieces exist:

piece	where	state
The wire format	copy_paths_to_clipboard (app.py:5074) — one full path per line	ships (Cmd-Shift-P)
Reading the clipboard	Panel.get_clipboard() (panel.py:2578 → backend.py:794)	ships, every backend, process-local fallback on curses
Turning paths into a pane	_feed_search_results (app.py:4307)	ships
An action that joins them	—	missing

copy_paths_to_clipboard already decided what a file list looks like when it leaves XeFM. Reading one back in is its inverse, and file_list_manager.py:44-48 already says the destination holds:

For a virtual pane (a search-results feed, pane_data['virtual']) … sort, filter, and post-op reconciliation Just Work on a virtual pane.

2. Why this is worth doing on its own merits

Deliberately echoing #342's framing, because the same test applies.

1. It is testable on the machine I actually use. This is the reason Everything is not going into XeFM (Pluggable file search — an engine registry, a matcher registry, and why they don't compose #446): the only test path would be a Windows machine with Everything installed and indexed, and there is no CI. An import action is exercised by find . -name '*.py' | pbcopy on macOS, today, in the normal loop.
2. There is no contract to keep. No threading promise, no cancellation, no option translation, no query-language ownership, no capability set. A list of paths cannot break across versions, and nothing about pane["virtual"] gets published.
3. It is not tool-specific. "Import a file list" is a better feature than "Everything support", and it costs less.
4. It inherits a UI rather than reimplementing one. Nothing to maintain on the search side at all.

What it does not give is Migemo over an external index — #432's other half. The search happens in the other tool's box, where XeFM's matcher is not. The honest mitigation is a small action that puts the Migemo regex for a typed query on the clipboard, to be pasted into Everything's regex:. Clunky, but it is one action and it does not pretend.

3. The boundary with #342

#342 sets its own test:

If we ever find ourselves explaining both as "copy", the design has failed.

Both read the clipboard, both are about paths, and they must not blur:

	#342's paste	Import
Means	"put these files here"	"show me these files"
Touches the filesystem	yes	no
Destination	the current directory	the pane's view
Undo story	a file operation	⌫ (_exit_virtual, app.py:2800)

Different verbs, and they should not share a key. #342 names "Explorer / Finder interop" as one of its three reasons, which is the same transport this needs — so the two want the same PuiKit work (§5) for opposite purposes.

There are two clipboard flavours, and the distinction matters here more than anywhere:

• Plain text, one path per line — what copy_paths_to_clipboard writes, and what Everything's "Copy Full Name to Clipboard" writes.
• A native file list (CF_HDROP on Windows, NSFilenamesPboardType on macOS) — what Explorer, Finder, and Everything's plain Ctrl+C write.

Import should accept both. Only the first is reachable today.

4. Transports, in order of what they cost

	transport	cost
1	Clipboard, plain text	One action. No new API anywhere. The format is already defined by its inverse
2	A file — Everything's File→Export (TXT / CSV / EFU), or any list on disk	Nearly the same. Handles sets too large for a clipboard and leaves a re-openable artifact
3	An external command's stdout — es.exe <query>, rg -l, git ls-files	external_programs.py already exists. This is where the headless half of #446 actually lands
4	Pushed from outside — Everything's context menu can run a program with the selection	Needs a CLI entry point and a way to reach a running instance. Biggest, least certain
5	Drag and drop into the pane	The most natural gesture and the most expensive: PuiKit has no drop target at all (§5), GUI-only, and not verifiable on my machine for the Windows half

1 and 2 are the whole feature for most purposes. 3 overlaps #446 and is the better home for engines that have no GUI.

5. PuiKit: the read side of a file-list clipboard does not exist

Worth stating precisely, because the asymmetry is total.

• get_clipboard() returns plain text only (backend.py:794). There is no get_clipboard_files().
• clipboard_rich is a write-side capability: set_clipboard_rich(text, html=…, rtf=…) (backend.py:808). macOS advertises it, Windows does not, web is "security-limited" (capability.py:101).
• Windows already knows how to build a CF_HDROP — _win32_dragdrop.py exists precisely for that, "exposing exactly one format (CF_HDROP over a real DROPFILES global memory)" — but only as a drag source for begin_file_drag (backend.py:982). It has never had to read one.
• os_drag_drop is likewise a source capability. There is no drop target anywhere.

So: plain-text import needs nothing from PuiKit. Importing an Explorer/Finder file copy and accepting a drag into the pane are the same underlying work — "read a file list from the OS" — and it is a PuiKit change, per backend, with the Windows half needing hands-on verification as usual. It is also the work #342 wants for its own interop.

That suggests the same staging #378 used: ship the part that needs no PuiKit change first, and let the richer transport arrive behind a capability.

6. What the destination already handles, and what it does not

Handles. Sorting, filtering and post-operation reconciliation on a virtual pane (file_list_manager.py:44-48). Pruning entries that no longer exist, on every refresh (file_list_manager.py:62). Backing out with ⌫ (_exit_virtual, app.py:2800). Relative-name display against a root (file_pane.py:358, #383).

Does not.

• kind finally gets a second value. _feed_search_results writes "kind": "search" (app.py:4338) and nothing has ever read it — the same shape as the ten capability methods with no callers in User-defined virtual folders — a scheme registry, and the base class that has to ship with it #426 §5. It was written for exactly this.

• The header would lie. app.py:789-791 reads virtual["mode"] directly and renders ⌕ "query" — N results (filename). An imported list is neither a filename nor a content search, and has no query.

• root has no answer, and root=None is not the one. Imported paths span drives. root feeds the relative-name column, the sort's rel_root (Option to use directory name + filename when sorting virtual file list (search result), not only filename #383), the cwd handed to external programs (app.py:4627), and app.py:6607. My first thought was root=None → full paths in the name column, and that is simply wrong about the code: name_key.rel_name (name_key.py:52-68) "falls back to the basename when there is no root". With root=None an imported list renders as bare a.txt rows — precisely the "location-less name that says nothing about which of several hits it was" that "Copy Name(s)" command shoud copy relative path in virtual file list #433 removed.

  Better, and derived from the data rather than from where the user happens to be standing: use the imported set's own common ancestor as root. rg -l inside one repository gives the repository, so the pane shows src/main.py. A list all under one folder gives that folder. Everything results across drives give nothing, and that is the honest answer. It degrades gracefully — one outlier drags the ancestor up to a drive root — and it gives app.py:4627's cwd something better than None.

  Where there is no common ancestor the pane still has to show whole paths, and rel_name has only two states (relative to a root, or basename). So this needs a third: the branch belongs in file_pane.py:358 / :605, choosing str(f), not in name_key — the basename fallback is load-bearing for ordinary directory panes.

• There is no cap. _RESULT_CAP = 1000 belongs to the search dialog, not to the pane. Everything can select a million rows.

7. The format, and parsing what arrives

What XeFM writes: absolute, verbatim. copy_paths_to_clipboard already chose absolute, and it is the only defensible choice — the clipboard is a cross-application bus that carries no context, so a relative path in it is unresolvable, and paths leaving XeFM get pasted into shells and editors that resolve against their cwd. copy_names_to_clipboard also already settled the encoding question, and the reasoning transfers directly to import:

Verbatim, though, not the NFC form the name column renders: a copied name gets pasted into a shell or an editor, where it has to address the file that is actually on disk

So: normalize for display, address the filesystem with exactly what arrived. NFC-normalizing before the existence check loses decomposed paths on macOS.

What XeFM accepts: either. Asymmetric on purpose, and the anchor for a relative path differs by transport:

transport	relative paths resolve against
A file	the list file's own directory — the M3U / .gitignore / response-file convention
Clipboard	the active pane, because there is no other anchor

This is the one place the feature can be silently wrong: src/main.py resolving against the wrong repository finds a real file that is not the one meant. It cannot be prevented, only reported — so the status line should say how many entries were resolved relatively, not just how many landed.

The clipboard holds whatever was in it. The rest of the rule should be dull on purpose:

• split on newlines; trim whitespace and a surrounding pair of quotes
• skip empty lines
• keep what exists, count what did not, and say so in the status line
• if nothing survives, leave the pane alone and say that instead

Deliberately not doing: guessing a delimiter, parsing CSV columns, accepting globs, or reading URLs. A list of paths is a list of paths; anything cleverer belongs in the tool that produced it. (Everything's .efu export is CSV and would be a separate, explicit reader under transport 2 — it carries size and dates, which is the §8 note below.)

8. One thing worth getting right early: attributes, not bare paths

Feeding bare paths means the pane stats every entry. For a list of a hundred thousand files spanning several drives that is the whole cost of the feature.

filters.py already promises the opposite for a pane's own listing:

size and mtime are free, seeded from the listing's own attribute record

Several transports can supply them: Everything's .efu export carries size and dates, Everything3_AddSearchPropertyRequest returns them, find -printf can print them. So the import path should accept (path, attrs) where a transport has them and fall back to statting lazily where it does not — the same shape listdir_attrs was added for. This also decides whether a config-defined FILTERS predicate over an imported list is free or ruinous.

9. Stages

		needs
①	Import from the clipboard as plain text; kind honoured in the header; the root=None decision	nothing new
②	Import from a file, including a .efu reader that keeps size and dates	①
③	Import from a command's stdout	①, external_programs.py
④	PuiKit: read a native file list (CF_HDROP / NSFilenamesPboardType), then accept a drop	a capability, per backend, Windows by hand

① is a small, self-contained change and settles the two questions (kind, root) that #446 would otherwise have to settle blind.

---

Open questions

1. The verb, and the key. "Import", "Paste as list", "Feed"? It must not read as Clipboard model for file operations #342's paste, and it should not take a key Clipboard model for file operations #342 will want.
2. root when the set has no common ancestor. Whole paths in the name column is clear (§6), but the external-program cwd is not — fall back to the directory the pane was showing before the import? And is a computed common ancestor ever surprising: a list of two files from opposite ends of a drive gets the drive root, which is correct and useless.
3. Should the import remember where it came from? The header has room to say "list from clipboard · 4,812 items" the way it says ⌕ "query" — N results. A file or command import could name its source; a clipboard one cannot.
4. A cap, and what it means. Truncating an imported list silently is worse than truncating a search — the user chose this exact set. Ask above some threshold, or accept the cost?
5. Does an imported list belong in history? ⌫ backs out to the directory; there is no way to get the list back without re-importing. A file-based import is re-openable, a clipboard one is not.

[crftwr]

More sources for the same destination

Everything in #453 is about the destination — a list of paths becomes a virtual pane — with the clipboard as the first source. A few more sources fit the same door, and I think they are worth naming now so the action is designed as "import from a source" rather than "paste from clipboard."

Command output

Midnight Commander's External panelize is the precedent: run a command, read stdout, one path per line, show it as a panel. Every example in #453 — find, fd, rg -l, git ls-files — goes from "pipe to clipboard, then paste" to one step. Everything's own CLI (es.exe) makes #432 work with no plugin API at all.

Details this adds on top of the clipboard version:

• Relative paths resolve against the directory the command ran in (the active pane).
• Re-run on refresh. The pane remembers its command, so refresh re-runs it. A clipboard list has no equivalent — refresh behavior depends on the source.
• Async and cancellable. A slow command must not freeze the window; this wants the job machinery, and argues for a built-in action rather than something done through the user API.
• Output encoding on Windows (cp932 and friends) needs an explicit decision.
• Named commands in config, like PROGRAMS, for the ones used repeatedly.
• Remote panes, later. On an ssh:// pane the command could run on the server and its output be read as ssh:// paths — a genuinely new capability. On s3:// there is nothing to run, so the capability simply isn't offered.

Saved lists

A virtual pane that remembers its source can be saved and reopened: Finder smart folders, Directory Opus collections. With command output as a source, "saved search" is just "saved command."

Flat view

Total Commander's Branch View — every file under the current directory, flattened — is a virtual list whose source is a recursive walk. Worth considering as a built-in source rather than a separate feature.

From outside XeFM

A control CLI over the local socket planned for MCP (… | xefm-ctl show-list) and results from an AI assistant are both sources too. They depend on the socket design and are tracked there; noting them here so the source abstraction doesn't preclude them.

[crftwr]

Stages ①–③ shipped (#492–#499)

A list of paths from three sources now opens as the pane, under Go → Open List. All three actions ship unbound: open_list_from_clipboard, open_list_from_command, open_list_from_file. A config's own search gets in the same way through PaneApi.open_list(paths, title=...).

How the open questions were answered

1. The verb, and the key. The verb is open, not import or paste. Opening a list is like opening an archive: the pane shows something that isn't a directory, and go_parent leaves it. That's also why it sits in Go and not Edit, and why it can't be read as Clipboard model for file operations #342's paste. No default keys.
2. root with no common ancestor. The root is the common ancestor of the paths that survive, so a missing line can't drag it up. When the survivors share no ancestor, a sentinel, name_key.WHOLE_PATH, names each row by its whole path. None keeps its meaning of basename for ordinary panes. External programs then run from the directory the pane showed before, the one ⌫ returns to. The header gives the root priority: [Clipboard] ~/src/xefm. The title is cut before the root, and the row count moved to the footer.
3. Remembering the source. The header names it: Clipboard, the command line, or the list file.
4. A cap. There is none. Existence is checked on the listing worker with one attribute read per path, and that read is the existence check. Each location's root is tried once first, so an unreachable ssh host costs one timeout, not one per row. Missing and unreachable rows are counted separately in the log line, and if none survive the pane is left alone.
5. History. The command source keeps one: the last 100 commands, in the state DB.

§6's "the header would lie" is fixed: virtual["kind"] is read now, after never having been read. Virtual panes, search results included, are also re-listed on a worker now. The old refresh called exists() and then stat() per row on the UI thread, which a capped local search hid and an uncapped ssh:// list would not.

The command source (③)

It runs through the shell in the pane's directory, with the XEFM_* environment, on a Task: a progress dialog counts lines, and Esc kills the whole process tree. Paths are shown whatever the exit code, since rg and grep exit 1 for no match.

• Everything: es.exe -path . ext:py, with no plugin API involved.
• Encoding on Windows: A GUI's child gets an OEM-code-page console (cp437 on an English system), so es.exe and dir /b printed Japanese names as ????. chcp 65001 fixes external programs, but cmd caches the code page at startup, so the command runs in a second cmd started after chcp. The command reaches that cmd through delayed expansion of an environment variable, so only the inner cmd ever parses it.
• Refresh does not re-run the command, unlike the proposal in the follow-up comment. Refresh here is the post-operation reconciliation, which fires after every delete and move. Re-running find there is slow and reorders rows under the cursor. Re-running is the history's job.
• The history turned out to need its own dialog, not the picker. With a picker, typing rg -l while rg -l TODO was remembered ran the old command. HistoryInputDialog is a field over its history, and Enter always uses the field. ↓ copies a row in without re-filtering, as an address bar does, and Enter in the list chooses the row rather than using it. The ; Filter prompt uses the same dialog.

The file source (②)

It opens the text file under the cursor (or the selected ones, merged), also available as Open as List on the context menu. Relative lines resolve against the list file's own folder. Byte-order marks decide the encoding first, UTF-16 included, since that is what Windows PowerShell 5.1's > writes.

Deliberately left out

• The .efu reader. Everything's File List is CSV with sizes and dates; reading it would be Everything support by another name. Everything's TXT export and es.exe cover the need.
• Attributes from the source (EntryInfo in open_list, find -printf). This can be added later without breaking anything.
• Saved lists and named commands. They need a "run it again" verb that isn't the post-operation refresh.
• Running a command on an ssh:// pane. It is refused, not run locally.

Still open

④: reading a native file list (CF_HDROP / NSFilenamesPboardType) and accepting a drop. This is per-backend PuiKit work, and the Windows half needs checking by hand.