Custom Drivers

Writing a custom driver

Any backend — S3, Azure Blob, SFTP, HTTP, a database — plugs into FileHub by deriving from three base types:

• DirectoryEntry
• FileEntry
• IFileHub (factory that produces a root DirectoryEntry)

Optionally implement IUrlAccessible on the file class for URL-reachable items.

Skeleton

using FileHub;

public interface IMyFileHub : IFileHub { }

public class MyFileHub : IMyFileHub
{
    public DirectoryEntry Root { get; }

    public MyFileHub() => Root = new MyDirectory("root", parent: null);
}

public class MyDirectory : DirectoryEntry
{
    public override string Path { get; }
    public override DirectoryEntry Parent { get; }
    public override DateTime CreationTimeUtc { get; } = DateTime.UtcNow;
    public override DateTime LastWriteTimeUtc => CreationTimeUtc;

    public MyDirectory(string name, MyDirectory parent) : base(name, rootPath: null)
    {
        Parent = parent;
        Path = parent == null ? name : System.IO.Path.Combine(parent.Path, name);
    }

    public override bool Exists() => true;

    public override Task<bool> FileExistsAsync(string name, CancellationToken ct = default)      => /* ... */;
    public override Task<bool> DirectoryExistsAsync(string name, CancellationToken ct = default) => /* ... */;

    public override async Task<FileEntry> CreateFileAsync(string name, CancellationToken ct = default)
    {
        ThrowIfReadOnly();
        ValidateName(name);
        // Single-arg CreateFile refuses an existing target: probe and throw
        // FileAlreadyExistsException. The overwrite overload is what clobbers.
        if (await FileExistsAsync(name, ct).ConfigureAwait(false))
            throw new FileAlreadyExistsException(CombineChildPath(name));
        await /* backend call */;
        return new MyFile(this, name);
    }

    // + TryOpenFileAsync, CreateDirectoryAsync, TryOpenDirectoryAsync,
    //   DeleteAsync(recursive), DeleteAsync(name, recursive), GetFiles, GetDirectories
    //
    // Async is the source of truth: those are the abstract members. The base
    // class provides every sync sibling (CreateFile, FileExists, Delete, ...)
    // by bridging to your async implementation — override a sync method only
    // when your backend is natively synchronous (see below). Enumeration
    // (GetFiles/GetDirectories) is the exception: the abstract surface is the
    // sync pull model, and the *Async enumerators wrap it unless you override
    // them for native pagination.
}

public class MyFile : FileEntry
{
    public override string Path => System.IO.Path.Combine(Parent.Path, Name);
    public override DirectoryEntry Parent { get; }
    public override long Length => /* ... */;
    public override DateTime CreationTimeUtc { get; }
    public override DateTime LastWriteTimeUtc => /* ... */;

    public MyFile(DirectoryEntry parent, string name) : base(name) => Parent = parent;

    public override bool Exists() => /* ... */;

    public override Stream GetReadStream()  => /* ... */;
    public override Stream GetWriteStream(FileWriteOptions options = null) { ThrowIfReadOnly(); /* ... */ }

    // + Delete, Rename, MoveTo
}

Rules

1. ThrowIfReadOnly() is the first line of every write. That's how AsReadOnly() works — see Security.
2. ValidateName(name) on every caller-supplied leaf name. Portable rule (PathUtil.ValidateName, identical on every OS): blocks null, empty, ., .., path separators and control characters. Drivers backed by a real file system layer PathUtil.ValidateLocalName on top for the OS-specific characters.
3. Enforce the sandbox if your backend has an addressable namespace. Use ResolveSafePath / EnsureWithinRoot on DirectoryEntry, or an equivalent prefix check (see PathUtil.EnsureWithinRootPrefix in the core package).
4. Don't leak SDK types. Wrap external SDK exceptions into FileHubException when they represent I/O failure; keep SDK types internal.
5. Ship a marker interface — public interface IMyFileHub : IFileHub { } — so DI consumers can bind to your driver without widening to IFileHub.
6. Implement FileExistsAsync and DirectoryExistsAsync separately. Each abstract method should use the cheapest probe your backend offers (one call apiece is the target). Callers that want either combine the two themselves.
7. Honour overwrite if you override the copy/move fast path. FileEntry.CopyTo/MoveTo take bool overwrite = false; the base guards it via DirectoryEntry.Exists/ExistsAsync and throws FileAlreadyExistsException on an existing destination unless overwrite: true. If you override for a native same-backend path (server-side copy/rename), re-check the destination with an Exists probe before the server op when overwrite is false — don't let the backend clobber silently. Likewise Rename must never overwrite: probe first and throw FileAlreadyExistsException on a taken name.
8. Optionally override ExistsAsync(string name) / CombineChildPath. The base ExistsAsync(name) does a file-then-directory probe; if one backend request can answer "is anything at this name?" (e.g. a single LIST on object storage), override it. Override CombineChildPath when child paths don't use the driver-neutral / separator (the Local driver joins with the OS-native separator).

Async first

On DirectoryEntry the abstract surface is async — you implement DeleteAsync, CreateFileAsync, etc., and the base class supplies every sync sibling by bridging to them (thread-pool offload + block, deadlock-safe under a SynchronizationContext). Don't implement both rails independently:

public override async Task DeleteAsync(CancellationToken ct = default)
{
    ThrowIfReadOnly();
    await client.DeleteAsync(Key, ct).ConfigureAwait(false);
    // base Delete() bridges here — nothing else to write
}

Matches the pattern used by the S3/OCI/FTP drivers.

For a natively synchronous backend (in-process state, local file system), invert locally: put the logic in a sync override and implement the async abstract as a thin wrapper, skipping the bridge hop:

public override void Delete() { ThrowIfReadOnly(); /* sync backend call */ }

public override Task DeleteAsync(CancellationToken ct = default)
{
    ct.ThrowIfCancellationRequested();
    Delete();
    return Task.CompletedTask;
}

Matches the Memory/Local drivers.

Nested paths

CreateDirectoryAsync / TryOpenDirectoryAsync receive the whole nested path ("a/b/c"). Split and validate it with PathUtil.SplitAndValidateSegments, then resolve it in the fewest operations your backend supports (one recursive mkdir, one PUT/LIST on object storage) — see the S3/OCI/FTP directories for reference implementations.

Cached metadata

If your backend requires a network round-trip to stat a file (like OCI or FTP), don't do that I/O inside a property getter — it risks deadlock under a SynchronizationContext when a sync caller is nested in async code. Instead:

1. Populate _length / _creationTimeUtc / _lastWriteTimeUtc at construction time from data your TryOpenFile / GetFiles already fetched.
2. Update _length from the write pipeline as bytes stream (see FtpStream / OciObjectStream).
3. Implement IRefreshable so callers who need authoritative state can ask explicitly.

public class MyFile : FileEntry, IRefreshable
{
    private long _length;

    public override long Length => _length;                 // pure getter

    public void Refresh() => RefreshAsync().GetAwaiter().GetResult();

    public async Task RefreshAsync(CancellationToken ct = default)
    {
        var stat = await client.StatAsync(Key, ct);
        _length = stat.Size;
        /* ... */
    }
}

Drivers whose state is already authoritative in process (Memory) or tracked by the OS (Local) don't need this — their getters read the live state directly.

Per-object metadata

If your backend has a native per-object metadata surface (content type, cache-control, user tags), wire it up by overriding the write stream (it receives the FileWriteOptions) and GetMetadataAsync on FileEntry. Drivers without such a surface silently ignore the options — never throw. Document the supported fields in your driver's README so callers know what round-trips.

1. Apply options on write — override GetWriteStreamAsync (or the sync GetWriteStream for non-async backends); the other write methods — SetBytesAsync, SetTextAsync, CopyFromStreamAsync — funnel through it. Keep the options on the stream, not on the file, so an abandoned write can't leak forward:

public override Task<Stream> GetWriteStreamAsync(FileWriteOptions options = null, CancellationToken ct = default)
{
    ThrowIfReadOnly();
    ct.ThrowIfCancellationRequested();
    return Task.FromResult<Stream>(new MyWriteStream(this, options));  // stream applies options on commit
}

2. Return an immutable snapshot on read — override GetMetadataAsync to return a FileMetadata. Replace the cached snapshot wholesale on each refresh/write rather than mutating in place:

public override async Task<FileMetadata> GetMetadataAsync(CancellationToken ct = default)
{
    if (!IsLoaded) await RefreshAsync(ct);
    return _metadata;   // immutable; safe to hand out directly
}

3. (Optional) Expose backend-specific fields with subclasses — derive from FileWriteOptions for write knobs and from FileMetadata for read-only typed fields, mirroring S3WriteOptions / AmazonS3FileMetadata. Downcast the incoming FileWriteOptions in your write path. Drivers that don't support a field must ignore it silently — never throw.

Lazy open-or-create

If your backend lets you produce a directory handle without a server round-trip, override OpenOrCreateChildDirectoryAsync (the hook called by OpenDirectory(name, createIfNotExists: true) and the descent step of nested CreateFile/OpenFile) to skip the probe entirely. For files, override CreateFileAsync and TryOpenFileAsync to return a stub directly:

• The file stub starts with Length = -1, default timestamps, empty metadata.
• Writes materialise it; Exists() triggers the first stat; reads from a never-materialised stub return 0 bytes.
• Reading bytes from a missing stub silently returns 0 bytes — Exists() is the contract for "is there really something here".

Pair this with ILazyLoad so consumers can detect unloaded handles:

public class MyFile : FileEntry, ILazyLoad
{
    public bool IsLoaded { get; private set; }

    public void Refresh()                                          => RefreshAsync().GetAwaiter().GetResult();
    public async Task RefreshAsync(CancellationToken ct = default) { /* stat + populate fields */; IsLoaded = true; }
}

Flip IsLoaded to true after any of: a successful Refresh, an Exists() that hit and populated state, a successful write commit, or an entry resolved by TryOpenFile / OpenFile. The S3 and OCI drivers follow this contract.

IUrlAccessible

public class MyFile : FileEntry, IUrlAccessible
{
    public bool IsPublic => /* ... */;

    public Uri GetPublicUrl() =>
        IsPublic
            ? new Uri($"https://cdn.example.com/{Path}")
            : throw new InvalidOperationException("Not public.");

    public Uri GetSignedUrl(TimeSpan expiresIn)
        => GetSignedUrlAsync(expiresIn).GetAwaiter().GetResult();

    public async Task<Uri> GetSignedUrlAsync(TimeSpan expiresIn, CancellationToken ct = default)
    {
        if (expiresIn <= TimeSpan.Zero)
            throw new ArgumentOutOfRangeException(nameof(expiresIn));
        return await signer.SignAsync(Path, expiresIn, ct);
    }
}

Abstract the SDK for tests

Define an internal interface for the operations your driver needs; depend on it from MyDirectory / MyFile; implement it twice (real SDK + in-memory fake). The OCI driver does this with IOciClient.

internal interface IMyBackendClient
{
    Task<Stream> GetAsync(string key, CancellationToken ct);
    Task        PutAsync(string key, Stream body, CancellationToken ct);
    Task        DeleteAsync(string key, CancellationToken ct);
}

DI

Consumers register your driver with the standard helpers — no work on your side:

services.AddFileHub<IMyFileHub>(sp => new MyFileHub(config));

See Dependency Injection for named hubs and lifetime control.

Checklist

• MyFileHub : IMyFileHub : IFileHub with a Root
• MyDirectory : DirectoryEntry — every abstract overridden
• MyFile : FileEntry — every abstract overridden
• ThrowIfReadOnly() on every mutation
• ValidateName(name) on every caller-supplied leaf
• Sandbox enforcement where the backend has a path/key namespace
• SDK types/exceptions kept internal; wrapped into FileHubException when I/O-shaped
• Async abstracts implemented against the backend's native rail (async for network-bound, sync-wrapped for in-process/local)
• Optional: IRefreshable on file/directory classes that cache metadata
• Optional: per-object metadata — apply FileWriteOptions on the write path + override GetMetadataAsync
• Optional: ILazyLoad on file classes that can be returned in an unloaded state
• Optional: override OpenOrCreateChildDirectoryAsync (and return file stubs from CreateFileAsync/TryOpenFileAsync) for zero-call open-or-create
• Optional: IUrlAccessible on the file class
• Optional: Dispose() overridden for unmanaged resources