Filetags

Meta-organize your files with only a rename, and without touching the original files.

Most image-gallery apps typically have meta-organization features, such as “albums”, which allow for a new level of convenient and clean organization of files, without the duplication of the original.

But in a basic UNIX filesystem environment, that convenience doesn’t exist. The closest you can get is to create a “favorites” directory and either copy, move, or symlink your desired file. The first two are very messy, and the latter is very tedious, manual, finicky, prone to failure, and requires the command-line. But if those issues didn’t exist, then using symlinks, it could be possible to achieve the same kind of convenience and clean meta-organization of files that you can get in fancy image-gallery applications, but in the UNIX filesystem environment. File-type agnostic, purpose agnostic, location agnostic, and GUI agnostic.

This project tries to achieve just that. By simply “tagging”, or renaming a file with a filename cookie (substring), such as prefixing it with an underscore, this program automatically creates a symlink to it in a directory of your choice.

In a config file, you specify the directories to “watch”, directories to create symlinks in, and the regexes to match your filename cookies.

And broken or inappropriate symlinks are deleted, so the state of your “favorites” directory will always remain accurate and up-to-date.

Purpose

TODO: update: change this to how i came up with this idea. make it a bit shorter.

Imagine that you’re using a music DAW (Digital Audio Workstation) and want to mark several files as “favorite” so you can quickly come back to them later. Most DAWs don’t provide such functionality, so the only realistic options are to either copy or move your favorite samples to a dedicated “favorites” directory. This is not very ideal. It can be very time-consuming and meticulous to move or copy files from one location to another, especially from within your DAW, if it even supports it.

So what if there was a way to automatically organize any arbitrary file to a dedicated directory, without any file duplication or moving? And without ever having to leave your DAW (or any application for that matter) for any of it?

This project seeks to solve exactly that, by automatically symlinking files with a matching filename substring/cookie to a specified directory.

So for example, if I find a sample that I really like, all I would need to do to tag it as a “favorite”, would be to rename the file to include a filename cookie in regex, such as "^_.*" (a filename prefixed with an underscore) to automatically symlink that file to my “favorites” directory. Thus, streamlining this style of file organization by tenfold, and making this possible no matter the front-end, as long as it can rename files.

And because it’s file-agnostic, it can be used to organize your photos, movies, documents, and anything else. As long as it’s a file.

How does it work?

It runs a daemon in the background that recursively monitors all files from a directory for changes in filename. When it observes such a change, it matches the filename against a specified filename cookie regex, and if it matches, it creates a symlink as appropriate.

TODO: more info.

Features

• specify multiple “rules”, where each rule specifies directories to monitor and directories to symlink to.
• broken symlinks are automatically deleted periodically.
• low memory footprint even with many files being watched.
• currently only supports Linux, but planning on adding support for other operating systems.

To Do

• [X] during post-serialization of config, create a copy of config with the rule-specific settings replaced with the defaults if none (or similar).
• [X] go through all source code and rename all dest_dirs/dest_dir/dest to link_dirs/link_dir/link.
• [X] refactor everything very freely without fear of painful compilation issues (it’ll literally only take 2mins, always)
• [X] extend the clone_vars! macro so that it supports cloning with Arc, creating a new var like arc_{}.
  • ideas:

    clone_vars!((Arc: config)) // => let arc_config = Arc::new(config);
            

• [X] implement Serialization for config, for more reliable integration tests
• [X] make a macro to generate config file in fs to then deserialize (also ensures two-way ‘se & ‘de).

Notes

• instead of with_barrier, return a collected vector of closures, vec.len() + 1 will be the barrier sum, and then evaluate each closure in a new thread.

Main process

1. args, config_gen
2. init checks (ensure all source_dirs and link_dirs exist, creating them if needed)
3. start responder (the main place where events are processed)
4. run clean all inappropriate links in all link_dirs.
5. start periodic_link_cleaner
6. run test hook
7. wait until responder thread completes

Implementing design patterns!!

• builder pattern
  • crates: bon
  • for building Config?
• typestate pattern
  • different stages of initialization? maybe not necessary with my channels method?
• newtype pattern
  • a newtype for Paths for link and watch dirs?
    • implement methods for these newtypes?
• factory pattern
  • a factory for creating threads (watchers, cleaners, etc)
• a WorkerBuilder that creates either a responder/queue, watcher, or cleaner.
  • re-organize the file structure so that src/channels/worker/{watcher, periodic_cleaner}.
  • struct Channels { responder_handle: (), }

File tree

• lib.rs
• dispatcher/
  • symlinks/
    • filesystem.rs
    • symlinking.rs
    • cleaning.rs
  • workers/
    • responder.rs
    • watcher.rs
    • periodic_cleaner.rs

Assurances

• Init:
  • mandatory clean
• Startup:
  • start cleaners
  • start watchers
• Test_hook.
• hang…

• Clean operation:
  • [ ] link_dirs: exists? (create dirs or crash)
  • for each link_file:
    • match file type:
      • cannot get metadata => log error, next.
      • not a symlink => next.
      • is a symlink =>
        • Delete symlink if any:
          • if doesn’t match any regex
          • if broken symlink
          • if symlink’s target doesn’t exist
          • if symlink’s target is not a subdir of any watch_dir

  implementation

  • link_dirs_all .for_each(|f| { if !f.exists() (create dir or terminate) });
  • watch_dirs_all .for_each(|f| { if !f.exists() (log but don’t intervene) });
  • link_dirs_all .map(|d| { WalkDir::new(d).map(|entry| { let entry = entry?; let path = entry.path();

    let file_info = (get file metadata)?; if file_info.file_type().is_symlink() { … } }) });