File storage package

[Pavel910]

Create a standalone file storage package with support for drivers.
Searching the npm gave me nothing I would want to use, google did not help either.

Goals:

• ability to provide CDN URL
• ability to create date folder structure
• abstract interface to support 3rd party drivers
• local filesystem driver (as a separate package) Create a local filesystem driver for webiny-file-storage #29
• AWS S3 driver (as a separate package) Create an AWS S3 driver for webiny-file-storage #30

[Pavel910]

@Adrian1358 @SvenAlHamad

FileStorageDriver interface proposal (taken from current PHP implementation).
Some things may change due to async nature of node. I am also considering support for file streams as a parameter to setContents.

declare interface IFileStorageDriver {
    supportsTouching: boolean;
    supportsDirectories: boolean;
    supportsFileSizeRetrieval: boolean;
    supportsAbsolutePathRetrieval: boolean;

    /**
     * Reads the contents of the file
     */
    getContents(key: string): Promise<string | boolean>;

    /**
     * Writes the given File
     */
    setContents(key: string, contents: string, append: boolean): Promise<number | boolean>;

    /**
     * Checks whether the file exists
     */
    keyExists(key: string): Promise<boolean>;

    /**
     * Returns an array of all keys (files and directories)
     *
     * For storage that doesn't support directories, both parameters are irrelevant.
     *
     * @param key       (Optional) Key of a directory to get keys from. If not set - keys will be read from the storage root.
     * @param recursive (Optional) Read all items recursively. Pass integer value to specify recursion depth.
     *
     * @return array
     */
    getKeys(key: string, recursive: number | boolean): Promise<Array<string>>;

    /**
     * Returns the last modified time
     */
    getTimeModified(key: string): Promise<number | boolean>;

    /**
     * Deletes the file
     */
    deleteKey(key: string): Promise<boolean>;

    /**
     * Renames a file
     */
    renameKey(sourceKey: string, targetKey: string) : Promise<boolean>;

    /**
     * Returns most recent file key that was used by a storage
     */
    getRecentKey(): string | null;

    /**
     * Returns public file URL
     */
    getURL(key: string): string;

    /**
     * Does this storage create a date folder structure?
     */
    createDateFolderStructure(): boolean;
}

[adrians5j]

Looks good @Pavel910.

Just wondering, is word 'Key' necessary in methods like renameKey, deleteKey ?

[Pavel910]

@Adrian1358 delete is a keyword, I'm afraid we will run into troubles if we use it as a method name. And if it is present in deleteKey then it is better to keep it consistent.

[adrians5j]

Good point @Pavel910

[Pavel910]

@Adrian1358 @SvenAlHamad

Here is the second revision, cleaned up and improved comparing to the PHP version:

• almost everything is converted to async functions returning Promises
• Directory class is removed entirely, not really needed for our current plans
• added set/get meta methods - it will be up to the driver to implement these methods if possible
• renamed to delete and rename @Adrian1358 after reading about compatibility - should be fine

declare interface IFileData {
    data: string | Buffer | ReadableStream,
    meta?: Object
}

declare interface IFileStorageDriver {
    /**
     * Reads the contents of the file
     */
    getFile(key: string, options?: { encoding: string }): Promise<IFileData | boolean>;

    /**
     * Writes the given File
     */
    setFile(key: string, file: IFileData): Promise<boolean>;

    /**
     * Get meta data
     */
    getMeta(key: string): Promise<Object>;

    /**
     * Set meta data
     */
    setMeta(key: string, meta: Object): Promise<boolean>;

    /**
     * Checks whether the file exists
     */
    exists(key: string): Promise<boolean>;

    /**
     * Returns an array of all keys (files and directories)
     *
     * For storage that doesn't support directories, both parameters are irrelevant.
     *
     * @param key       (Optional) Key of a directory to get keys from. If not set - keys will be read from the storage root.
     * @param filter    (Optional) Glob pattern to filter returned files
     *
     * @return array
     */
    getKeys(key?: string, filter?: string | null): Promise<Array<string>>;

    /**
     * Returns the last modified time
     */
    getTimeModified(key: string): Promise<number | boolean>;

    /**
     * Deletes the file
     */
    delete(key: string): Promise<boolean>;

    /**
     * Renames a file
     */
    rename(sourceKey: string, targetKey: string): Promise<boolean>;

    /**
     * Returns public file URL
     */
    getURL(key: string): string;

    /**
     * Does this storage create a date prefix?
     */
    createDatePrefix(): boolean;

    /**
     * Get file size (if supported)
     */
    getSize(key: string): Promise<number | null>;

    /**
     * Get absolute file path (if supported)
     */
    getAbsolutePath(key: string): Promise<string | null>;
}

declare interface IFile {
    /**
     * Constructor
     */
    constructor(key: string, storage: IFileStorageDriver): IFile;

    /**
     * Get file storage
     */
    getStorage(): IFileStorageDriver;

    /**
     * Get file key
     */
    getKey(): string;

    /**
     * Get public file URL
     */
    getUrl(): string;

    /**
     * Get file contents
     */
    getContents(options?: { encoding: string }): Promise<string | Buffer | ReadableStream>;

    /**
     * Get file meta
     */
    getMeta(): Promise<Object>;

    /**
     * Get time modified
     */
    getTimeModified(): Promise<number>;

    /**
     * Set file contents (writes contents to storage)
     */
    setContents(data: IFileData): Promise<boolean>;

    /**
     * Set file meta
     */
    setMeta(meta: Object): Promise<boolean>;

    /**
     * Rename a file
     */
    rename(newKey: string): Promise<boolean>;

    /**
     * Delete a file
     */
    delete(): Promise<boolean>;

    /**
     * Get absolute file path.
     * If storage driver does not support absolute paths (cloud storage), returns file key
     *
     * @return string
     */
    getAbsolutePath(): Promise<string | null>;

    /**
     * Get file size in bytes
     *
     * @return int|null Number of bytes or null
     */
    getSize(): Promise<number | null>;
}

[SvenAlHamad]

Why do we duplicate some methods, like getMeta between IFileStorageDriver and IFile? Shouldn't the storage just return an IFile instance and then those methods are available via that instance only.

[SvenAlHamad]

Never mind ... I got my answer :) The interfaces look good! 👍

[adrians5j]

Awesome!