Support custom extractors

[LunarWhisper]

The current interface doesn't allow registering custom extractors, which limits the package's functionality.
For example, I need to unpack .MSI and .MSP packages.

The hacky approach with UNKNOWN isn't suitable, as it's explicitly filtered.

I'd like to either replace the enum with a string representation or add an alternative exctractor resolution mechanism.

[LunarWhisper]

@gfs , @danfiedler-msft

May I suggest this solution design to make it possible to extend the library without significantly changing/duplicating existing interfaces?

[TestClass]
public class CustomExtractorTests
{
    [DataTestMethod]
    [DataRow("Lorem.txt", true)]
    [DataRow("Lorem.txt", false)]
    public void ExtractFlatFile(string fileName, bool requireTopLevelToBeArchive)
    {
        var extractor = new Extractor();
        extractor.SetExtractor(ArchiveFileType.UNKNOWN, new TestTxtExtractor());
        var path = Path.Combine(Directory.GetCurrentDirectory(), "TestData", "TestData", fileName);
        var results = extractor.Extract(path, new ExtractorOptions() { RequireTopLevelToBeArchive = requireTopLevelToBeArchive }).ToList();
        // var results = await extractor.ExtractAsync(path, new ExtractorOptions(){ RequireTopLevelToBeArchive = requireTopLevelToBeArchive }).ToListAsync();
            
        Assert.AreEqual(9, results.Count);
    }

    private sealed class TestTxtExtractor : AsyncExtractorInterface
    {
        public IEnumerable<FileEntry> Extract(FileEntry fileEntry, ExtractorOptions options, ResourceGovernor governor, bool topLevel = true)
        {
            if (Path.GetExtension(fileEntry.Name) != ".txt")
            {
                yield return fileEntry;
                yield break;
            }
            
            var index = 0;
            var input = new StreamReader(fileEntry.Content);
            while (!input.EndOfStream)
            {
                var line =  input.ReadLine();
                if (line is null)
                    continue;
                
                var bytes = Encoding.UTF8.GetBytes(line);
                var stream = StreamFactory.GenerateAppropriateBackingStream(options, bytes.Length);
                stream.Write(bytes, 0, bytes.Length);
                stream.Flush();
                stream.Position = 0;
                yield return new FileEntry($"{++index:0000}.txt", stream, parent: fileEntry, modifyTime: fileEntry.ModifyTime, memoryStreamCutoff: options.MemoryStreamCutoff);
            }
        }

        public IAsyncEnumerable<FileEntry> ExtractAsync(FileEntry fileEntry, ExtractorOptions options, ResourceGovernor governor, bool topLevel = true)
        {
            throw new System.NotImplementedException(); // todo
        }
    }
}

[gfs]

@LunarWhisper Thanks for the suggestion. I appreciate the use case for wanting to programatically extend the library. I wonder though, this seems like it gets a little sticky if you were to implement multiple new extractor types - you can't set multiple extractors for the same type and you wouldn't be able to route them either, and would still require some custom logic to allow for checking the unknown type rather than the extant logic that treats those as literal files.

The current type of archive detection logic operates on the bytes of the archive using the minimagic utility, rather than file extension. My thoughts on how this might work best, with supporting arbitrary number of extensions, is to add a new property to Extractor.cs for an arbitrary number of custom extractors. For these custom extractors, maybe there could be a CustomIAsyncExtractor interface that also implements a CanExtract method that checks for binary signatures - like mini magic does - and then the generic extractor can when encountering something that isn't default supported 1. check if there are any custom extractors defined 2. if there are iterate over them executing the CanExtract on the Stream 3. when one of them indicates it can extract, use that one.

Thoughts on something like that? would that serve your use case?

[LunarWhisper]

@gfs, yes, definitely, although these are significant API changes that I wanted to avoid. :)
But of course it will be much better this way.