Ability to structure input files in a directory hierarchy

[peturv]

Hi!

I may have done things in the wrong order, but heres for getting it right.

I'm creating an IG with a moderate amount of files and found my fish tank to be getting quite crowded hand hard to get an overview of. My natural instincts said to do what I always do and structure things with subdirectories (either based on resource areas, types or whatever is suitable in the given context).

Unfortunately SUSHI expects all .fsh files to be in the root directory.

To get around this I added an option to SUSHI to recurse all directories under root when looking for .fsh files. While I'm not sure if you want this option, I've created a PR with my changes #236 . This does not affect the output hierarchy.

If this is of interest I'll ask for a review, if not I'll close my PR again.

[cmoesel]

Hi @peturv! Thanks for filing the issue and submitting the PR!

I think this is of interest and can be a helpful feature. There are a few things we need to determine before we can merge it, however.

We've heard other requests to allow folder organization in the FSH Tank, but those requests also suggested that perhaps it should affect the output. Looking at the IG Template documentation, however, the set of valid "resource" folders supported by the IG Publisher is limited -- so allowing arbitrary folder names on SUSHI output probably wouldn't work. So I'm inclined to go with the approach of your PR; which is that input folders do not affect output folders. But I need to discuss that with others as well.

The other question I have is if the -r flag is necessary. Are there use cases where you would have subfolders that you don't want traversed? It seems to me that perhaps recursive traversal should just be the default approach. What do you think?

[peturv]

We've heard other requests to allow folder organization in the FSH Tank, but those requests also suggested that perhaps it should affect the output. Looking at the IG Template documentation, however, the set of valid "resource" folders supported by the IG Publisher is limited -- so allowing arbitrary folder names on SUSHI output probably wouldn't work. So I'm inclined to go with the approach of your PR; which is that input folders do not affect output folders. But I need to discuss that with others as well.

For my part I see affecting input and output folder hierarchies as two widely different features. If one is to be allowed to modify the output hierarchy, I'm thinking that some sort of configuration would be more logical, depending on how the output hierarchy is to be based.
I.e. should it be split into subfolders based on input folder path, based on resource type, both (input_folder_path/resource_type) or on something completely different? In any case the features probably can and should be added independent of each other. If needed the relative paths to each file can be extracted from the input and utilized in whatever output solution is or is not chosen.

The other question I have is if the -r flag is necessary. Are there use cases where you would have subfolders that you don't want traversed? It seems to me that perhaps recursive traversal should just be the default approach. What do you think?

I agree, but didn't want to be too aggressive on default behaviour. For my part I usually either delete or change file extension of any file I don't want included. Should I remove the flag and change the default behaviour?

[cmoesel]

For reference (for anyone else reading this), further conversation here:

• https://chat.fhir.org/#narrow/stream/215610-shorthand/topic/Announcing.20SUSHI.200.2E9.2E0/near/188410428
• https://chat.fhir.org/#narrow/stream/215610-shorthand/topic/SUSHI.20IG.20folder.20structure

@peturv -- we agree that configuration of output should be treated as a separate issue.

If you could update the PR so that recursive processing happens by default, that would be great. I don't think we need a flag at all -- if others want flat processing, we can add that back later -- but I suspect no one will want/need it.