fs_dlk

pyfilesystem2 interface to Azure Datalake

Installing

pip install fs-dlk

Opening FS Datalake

Authentication with username and password:

from fs_dlk import DLKFS
dlkfs = DLKFS("/path/to/the/remote/dir", username="username", password="password")

Authentication with tenant secret:

from fs_dlk import DLKFS
dlkfs = DLKFS("/path/to/the/remote/dir", tenant_id="tenant id", client_id="client id", client_secret="client_secret")

Specifying custom store name:

from fs_dlk import DLKFS
dlkfs = DLKFS("/path/to/the/remote/dir", store="store name", **auth_args_here)

Authentication with connection strings:

Username and password auth:

import fs
dlkfs = fs.open_fs("dlk://username:password@/store_name/path/to/remote")

Tenant secret auth:

import fs
dlkfs = fs.open_fs("dlk://username:password@tenant-id/store_name/path/to/remote")

Downloading files

with open("local_file", "wb") as local_file:
    dlkfs.download("path/to/remote/file", local_file)

Uploading files

with open("local_file", "wb") as local_file:
    dlkfs.upload("path/to/remote/file", local_file)