option to "clean up" zotfile folder

[cpence]

I'm using ZotFile to store files in a custom location as linked attachments, in a single flat directory. When I delete linked attachments or items from the Zotero library, though, the files continue to clutter that directory. It'd be great if ZotFile had an option to go through the specified folder, check to see if each file is present and linked to some entry in the library, and move any files that aren't to a subdirectory for cleanup (say, the "orphans" directory).

(I'd imagine that people who were using subdirectory storage would want ZotFile to automatically remove those directories if they were completely empty, as well, though I don't use subdirectories.)

This would be a really helpful feature for me!

[jlegewie]

Makes sense. Another useful feature would be to have a click-able popup that asks users whether they want to delete the file when they remove a linked attachment.

I won't get to this anytime soon though and contributions are always welcome!

[zoomphys]

I second this feature. For a while, I have been using Zotero+Zotfile to store potentially interesting papers for me to skim through. I later found out it's so cumbersome to get rid of them, so now I have to carefully read the papers first and decide later if I really want to save them to Zotero.

[ThSGM]

I third this feature. Really important suggestion!

[roey-angel]

Same here, seems to me like a crucial option that zotfile will be able to monitor the custom storage dir and delete unlinked files.

[tychen742]

Same here. Much needed feature! ^^

[jogrue]

Hi there! This would be great. I am not a coder, but I think an easy solution would be an option to clean up, that runs only once (instead of ZotFile taking care of cleaning up all the time). One would have to get a list of all files in the ZotFile folder and then iterate through this list checking if there is a link to it in the Zotero database. If there is not, it can be deleted (or a window could pop up showing the files that are not linked and allow users to select which ones to delete).

[eklim530]

Same here. Thanks.

[andersjohansson]

I guess it would be possible to adapt @mronkko's perl script for that purpose: https://github.com/mronkko/ZoteroCleanOrphans/.

Kind of the only thing that would need to be changed is the paths and the SQL question:
'SELECT key FROM items' to 'SELECT path FROM itemAttachments'

Then we would have a comparison between files in db and files in directory, and the files that are not referenced in Zotero can be deleted by the script.

Perhaps I missed thinking about something. I Haven't tested it at all.

[DmitriyC]

I'd really love to have this feature as well...please, and thank you!

[chrpinedo]

I would also like to have this feature!

[yangyangoceanographer]

+1

[mguzmann]

+1

[jlegewie]

This seems to be the most requested feature so I gave it a shot. Unfortunately, I wasn't able to finish it and I am not sure when I will be able to return to it. Documenting my progress here just in case someone wants to pick up.

Description
clean_folder takes a path and looks for any files in that path are not zotero attachments. If it finds any, the function asks the user whether he/she wants to delete them. For testing purposes, the actual deleting is commented our right now. Instead, the function returns an array with the file objects that should be deleted. The easiest way to test and work on this is in Firefox's Scratchpad when you enable the Browser environment.

Remaining issues

• There is a bug with some file paths. It might be related to umlaute but I am not sure. Here is the files I am having problems with: ~/Documents/Literatur/temp/Blüthgen et al_2006_Effects of sequestration on signal transduction cascades.pdf
• The function does not delete empty folders right now
• It needs serious testing both on windows in unix systems considering that it deletes files

var files_unattached = function(path, atts) {
    var dir = Zotero.ZotFile.createFile(path),
        files = dir.directoryEntries,
        files_unlinked = [];
    while (files.hasMoreElements()) {
        // get next file
        var file = files.getNext();
        file.QueryInterface(Components.interfaces.nsIFile);
        // look in sub-directories
        if (file.isDirectory())
               files_unlinked = files_unlinked.concat(files_unattached(file.path, atts));
        // continue if file is folder or hidden
        if (file.isDirectory() || file.isHidden())
          continue;
        // continue if filetype not included
        // if (!this.checkFileType(file)) 
        //   continue;
        // continue if file is zotero attachment
        if (atts.map(att => att.equals(file)).some(x => x == true))
        // if (atts.map(att => att.path == file.path).some(x => x == true))
        // if (Zotero.ZotFile.normalize_path(file.path) in att_paths)
            continue;
       // add file to list
       files_unlinked.push(file);
    }
    return files_unlinked;
};

var clean_folder = function(path) {
    // get attachment files
    // var items_dead = Zotero.Items.getAll().filter(item => item.isAttachment()).filter(item => !item.fileExists());
    var atts = Zotero.Items.getAll()
        .filter(att => att.isAttachment())
        .filter(att => att.fileExists())
        .map(att => att.getFile());
        // .map(Zotero.ZotFile.normalize_path);
    // get tablet files
    var atts_tablet = Zotero.Items.getAll()
        .filter(att => Zotero.ZotFile.getTabletStatus(att))
        .map(att => Zotero.ZotFile.getTabletFile(att, false))
        .filter(file => file && file.exists());
    atts = atts.concat(atts_tablet);
    // get files in folder that are not Zotero attachments
    var files_unlinked = files_unattached(path, atts);
    // confirm deletion of files
    if (files_unlinked.length == 0)
        return;
    if (!confirm("Do you want to delete " + files_unlinked.length + " files in '" + path + "' that are not Zotero attachments?"))
        return;
    // delete files
    // files.forEach(f => f.remove(false));
    // delete empty folders
    // ...
    return(files_unlinked)
}

// get destination folder
var path = Zotero.ZotFile.prefs.getComplexValue("tablet.dest_dir", Components.interfaces.nsISupportsString).data;
var path = Zotero.ZotFile.prefs.getComplexValue("dest_dir", Components.interfaces.nsISupportsString).data;
files = clean_folder(path)
// files is an array of `nsIFile` objects (https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Reference/Interface/nsIFile)
// you can get their paths with `files.map(file => file.path)`

[jacknlliu]

It's a very important function for managing the amounts of documents.

[DancingQuanta]

I vote for this feature!

[ghost]

This would really be a great feature.

[laclaro]

+1

There is also an very old discussions in the zotero forums
https://forums.zotero.org/discussion/comment/265284/

I would suggest to make it optional like on deleting items from trash: "Do you also want to remove linked files?"

[lampuiho]

Is there an alternative solution for now (to clean up the custom folder)?

[alanlivio]

+1
This would really be a great feature.

[lampuiho]

• It needs serious testing both on windows in unix systems considering that it deletes files
  I would say adding an option to just move those files to a certain folder would make it a lot safer.

[ammoniac1984]

This feature would be super useful!

[harshita-gupta]

+1

[moffat]

+1 on this.

[redleafnew]

I would like to have this feature

[jotelha]

+1

[toftul]

+1

[toftul]

At least there is one workaround:

1. Stop any syncing serves. Just in case.
2. Change zotfile folder to any temp directory.
3. In Zotero select all files -> right click -> Manage Attachments -> Rename Attachments. That will move files which are only in your Zotero library.
4. Remove everything from the original folder. At this point it is better to sync intermediate state to avoid any conflicts in the future.
5. Change zotfile folder to the original directory.
6. In Zotero select all files -> right click -> Manage Attachments -> Rename Attachments.
7. Profit.

[shunzh]

The workaround above is nice for occasional cleanup. Still hope to have an option to delete the removed file.

[Yunuuuu]

This would be fine

[tonigi]

Made this tiny script https://github.com/giorginolab/zotfile_doctor , if useful for others.

[weiyongxu]

+1

[ChrisTrivedi]

Made this tiny script https://github.com/giorginolab/zotfile_doctor , if useful for others.

Huge thanks for this! Just used and it was super simple and easy. While most of my files were synced up I still had a few duplicates. It turns out the items were linked under the correct reference but trying to open the linked pdf resulted in an "item not found" error. I simply deleted them, emptied the Zotero trash and ran the script again. Zero duplicates!

Thanks again for a nice little script to help us keep things cleaned up. Much appreciated for your efforts!

[RangerChu]

+1

[dmi3kno]

+1, please

[kongdd]

+1

[gwurm2023]

+1

[rspreafico]

+1

[ThachNgocTran]

+1