Add 'filepath' source attribute to tiddlywiki.files

[michsa]

Adds a new source attribute to facilitate importing subdirectories with the searchSubdirectories attribute.

The filepath atttribute is like filename except it includes the full pathname relative to the directory path. Consequently it only works with directories sections (not tiddlers) in tiddlywiki.files.

Here's an example tiddlywiki.files that uses this attribute to import all image files within the files directory and its subfolders as _canonical_uri tiddlers:

/* tiddlers/images/tiddlywiki.files */

{
    "directories": [
        {
            "path": "../../files/",
            "filesRegExp": "^.*\\.(?:jpg|jpeg|png|gif)$",
            "isTiddlerFile": false,
            "searchSubdirectories": true,
            "fields": {
                "title": {"source": "basename-uri-decoded"},
                "created": {"source": "created"},
                "modified": {"source": "modified"},
                "type": "image/jpeg",
                "tags": ["image"],
                "text": "",
                "_canonical_uri": { "source": "filepath", "prefix": "files/" }
            }
        }
    ]
}

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated
tiddlywiki5	✅ Ready (Inspect)	Visit Preview	Mar 9, 2023 at 7:31PM (UTC)

[michsa]

Side note: for my use case, I also wanted to customize tags on imported files based on the pathname (eg, files in files/memes/cats would be tagged with memes and cats). To do this, I added a second source attribute, subdirectories, that splits the subdirectories on the relative path into an array (which I used in my tags field). I'm not sure how generally useful that would be, but I can include it in this PR (or a separate one) if there's interest.

[Jermolene]

Thanks for your patience @michsa, and welcome to the project.

This looks good to me – just one small thing, please could you prepare a separate PR to the 'tiddlywiki-com' branch with your signature for the Contributor License Agreement (see contributing.md). Many thanks

[pmario]

Side note: for my use case, I also wanted to customize tags on imported files based on the pathname (eg, files in files/memes/cats would be tagged with memes and cats). To do this, I added a second source attribute, subdirectories, that splits the subdirectories on the relative path into an array (which I used in my tags field). I'm not sure how generally useful that would be, but I can include it in this PR (or a separate one) if there's interest.

How would the JSON config look like with these changes added?

[Jermolene]

[deleted my previous comment for being incorrect]

How would the JSON config look like with these changes added?

Hi @pmario there's an example of using the new filepath attribute in the OP:

"_canonical_uri": { "source": "filepath", "prefix": "files/" }

[michsa]

Sorry for the delay! @Jermolene I've signed the CLA here: #7342

How would the JSON config look like with these changes added?

@pmario you would use the source attribute subdirectories:

"tags": { "source": "subdirectories" },

this sets the field to an array of subdirectories between the root (files in the example in the PR description) and the file which is being loaded to a tiddler. Eg, for files in files/photos this would be ['photos'], and for files in files/photos/family it would be ['photos', 'family'].

The implementation is really simple - it's just another case in the fieldInfo.source switch statement. I'm gonna go ahead and add it to this PR.

[Jermolene]

Thank you @michsa – and apologies for not recognising your name in your CLA signature.

[pmario]

this sets the field to an array of subdirectories between the root (files in the example in the PR description) and the file which is being loaded to a tiddler. Eg, for files in files/photos this would be ['photos'], and for files in files/photos/family it would be ['photos', 'family'].

Sorry, but that's not enough context. Please insert the info from your reply, into your JSON below. .. The JSON is several levels deep. So the info on which level the config has to be is essential. .. Also for the user docs

/* tiddlers/images/tiddlywiki.files */

{
    "directories": [
        {
            "path": "../../files/",
            "filesRegExp": "^.*\\.(?:jpg|jpeg|png|gif)$",
            "isTiddlerFile": false,
            "searchSubdirectories": true,
            "fields": {
                "title": {"source": "basename-uri-decoded"},
                "created": {"source": "created"},
                "modified": {"source": "modified"},
                "type": "image/jpeg",
                "tags": ["image"],
                "text": "",
                "_canonical_uri": { "source": "filepath", "prefix": "files/" }
            }
        }
    ]
}

[Jermolene]

@michsa we're now in the freeze for the release of v5.2.6 (see #7279 for details of the roadmap), and so we won't be able to merge this until afterwards. I've added this PR to the list at #7345. Don't worry about the reference to v5.2.6 in this PR, I'll correct that when we merge it.

@pmario I think that the docs that @michsa has produced are inline with the existing docs. I don't disagree that more examples would be helpful, but that isn't really in the scope of this PR that is just adding one more property.

[michsa]

@pmario is right, it'd be silly not to have an example in the docs. I added one along with the subdirectories attribute (and updated the 5.2.6 references to 5.3.0 while I was at it - might as well save some effort later)

[Jermolene]

Great thanks @michsa much appreciated.

[Jermolene]

Thanks for your patience @michsa

[joshuafontany]

Oooooh, great addition. Thanks @michsa !!!