v1.0.0-beta3

JuiceFS v1.0.0-beta3 is the third beta release for v1.0. It has 247 commits from 22 contributors, thanks to @SandyXSD @zhoucheng361 @davies @zhijian-pro @yuhr123 @sanwan @AIXjing @rayw000 @xiaogaozi @Suave @showjason @tangyoupeng @201341 @solracsf @guo-sj @chnliyong @DeanThompson @zwwhdls @wph95 @lidaohang @sjp00556 @DEvmIb !

Highlights

• Supports etcd as a new metadata engine. It can be a handy choice when you only need a small volume but cares more about the data availability and persistence.
• Supports Redis Cluster and other compatible services (Amazon MemoryDB for Redis) as metadata engines.
• When using SQL metadata engines, file names not encoded by UTF-8 can now be properly handled after manual modification to the table schema, see details.
• A new session management format is introduced. Old clients are unable to detect sessions with version 1.0.0-beta3 or higher via juicefs status or juicefs destroy command, see details.
• If trash is enabled, compacted slices are kept as well in case they are needed to recover file data. These slices will be cleaned up automatically after trash-days, and can be deleted manually via juicefs gc command.
• A lot of improvements have been made to juicefs sync command.
• A lot of protection checks against unintentional misuse have been added.

New

• Supports etcd as metadata engine (#1638)
• Supports Redis in cluster mode while using one slot for each file system (#1696)
• Supports handling file names not encoded by UTF-8 for SQL metadata engines (#1762)
• Supports TLS when using TiKV as metadata engine or object storage (#1653, #1778)
• Supports Oracle Object Storage as data storage (#1516)
• Supports setting umask for S3 Gateway (#1537)
• Java SDK now supports pushing metrics to Graphite (#1586)
• Added a new option --heartbeat for the mount command to adjust heartbeat interval (#1591, #1865)
• Added many improvements for sync command to make it more handy (#1554, #1619, #1651, #1836, #1897, #1901)
• Added a new option --hash-prefix for the format command to add a hashed prefix for objects (#1657)
• Added a new client option --storage to allow customized storage type (#1912)

Changed

• compacted slices will be kept for trash-days if trash is enabled (#1790)
• cmd: support using integer for duration flags (#1796)
• cmd: use homedir as default working directory for non-root users (#1869)
• cmd/format: create a uuid object in the target bucket (#1548)
• cmd/dump&load: dump load behavior supports non-ascii characters (#1691)
• cmd/dump: omit empty value in dumped JSON (#1676)
• cmd/dump: remove secret key (#1569)
• meta: encrypt the secret-key and encrypt-key in setting (#1562)
• meta: create subdir automatically (#1712)
• meta: specify the format field preventing update (#1776)
• meta: escape meta password from env (#1879)
• meta/redis: check redis version (#1584)
• meta/redis: use smaller retry backoff in sentinel mode (#1620)
• meta/redis: retry transaction for connection error or EXECABORT (#1637)
• meta/sql: retry transaction after too many connections (#1876)
• meta/sql: add primary key for all tables (#1913, #1919)
• meta&chunk: Set max retries of meta & chunk according to the config io-retries (#1713, #1800)
• chunk: limit number of upload goroutines (#1625)
• chunk/store: limit max retry for async upload as well (#1673)
• object/obs: Verify Etag from OBS (#1715)
• object/redis: implement listAll api for redis (#1777)
• fuse: automatically add ro option if mount with --read-only (#1661)
• vfs/backup: reduce the limit for skipping backup (#1659)
• sync: reduce memory allocation when write into files (#1644)
• SDK: use uint32 for uid,gid (#1648)
• SDK: handle removeXAttr return code (#1775)
• Dependency: switch to Go 1.17 (#1594)
• Dependency: fix jwt replace (#1534)
• Dependency: upgrade golang-cross version to v1.17.8 (#1539)
• Dependency: upgrade tikv to v2.0.0 (#1643)
• Dependency: reduce dep from minio (#1645)
• Dependency: upgrade gjson to 1.9.3 (#1647)
• Dependency: upgrade sdk for object storage (#1665)
• Dependency: upgrade qiniu sdk (#1697)

Bugfix

• cmd/format: fix setting quota (#1658)
• cmd/mount: fix parsing of cache dir (#1758)
• cmd/warmup: fix handling of relative paths (#1735)
• cmd/sync: fix sync command not working when destination is webdav (#1636)
• cmd/gateway: fix s3 gateway DeleteObjects panic (#1527)
• meta: forbid empty name for dentry (#1687)
• meta: lock counters when loading entries (#1703)
• meta: fix snap not released if error occurs when dumping meta (#1669)
• meta: don't update parent attribute if it's a trash directory (#1580)
• meta/redis: fix loading large directory into Redis (#1858)
• meta/redis: update used space/inodes in memory whenever needed (#1573)
• meta/sql: use upsert to update xattr for PG (#1825)
• meta/sql: split insert batch (#1831)
• meta/sql: fix wrong result from scanning SQL database (#1854)
• chunk/cache: Fix read disk_cache not existed remove cache key (#1677)
• object: fallback to List only if ListAll is not supported (#1623)
• object/b2: check returned bucket from B2 (#1745)
• object/encrypt: fix parse rsa from pem (#1724)
• object/encrypt: Add JFS_RSA_PASSPHRASE environment variable prompt information (#1706)
• object/sharding: fix ListAll returning invalid objects (#1616)
• object/ceph: fix listAll hangs if there are many objects (#1891)
• vfs: write control file asynchronously (#1747)
• vfs: fix getlk in access log (#1788)
• sync: Fix copied and copiedBytes (#1801)
• utils: fix the problem that the progress bar loses the log (#1756)
• SDK: rename libjfs atomic (#1939)