Skip to content

Commit

Permalink
Merge pull request #55 from trapexit/policyperfunc
Browse files Browse the repository at this point in the history
per FUSE function policies. closes #52, #53
  • Loading branch information
trapexit committed Mar 6, 2015
2 parents e0f0bb5 + 12f393a commit a359c88
Show file tree
Hide file tree
Showing 32 changed files with 570 additions and 123 deletions.
84 changes: 67 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ mergerfs - another FUSE union filesystem

# SYNOPSIS

mergerfs -ocreate=epmfs,search=ff <srcpoints> <mountpoint>
mergerfs -o<options> <srcpoints> <mountpoint>

# DESCRIPTION

Expand All @@ -20,10 +20,11 @@ Why create mergerfs when those exist? mhddfs isn't really maintained or flexible

###options###

| Option | Default |
|--------|--------|
| search | ff |
| create | epmfs |
All [FUSE](http://fuse.sourceforge.net) functions which have a category (see below) are option keys. The syntax being `<func>=<policy>`.

To set all function policies in a category use `category.<category>=<policy>` such as `category.create=mfs`.

They are evaluated in the order listed so if the options are `rmdir=rand,category.action=ff` the `action` category setting will override the `rmdir` setting.

###srcpoints###

Expand All @@ -46,14 +47,15 @@ In /etc/fstab it'd look like the following:

# POLICIES

Filesystem calls are broken up into 2 categories: search and create. There are also some calls which have no policy attached due to state being kept between calls. These categories can be assigned a policy which dictates how [mergerfs](http://github.com/trapexit/mergerfs) behaves. Any policy can be assigned to a category though some aren't terribly practical. For instance: rand (Random) may be useful for **create** but could lead to very odd behavior if used for **search**.
Filesystem calls are broken up into 3 categories: action, create, search. There are also some calls which have no policy attached due to state being kept between calls. These categories can be assigned a policy which dictates how [mergerfs](http://github.com/trapexit/mergerfs) behaves. Any policy can be assigned to a category though some aren't terribly practical. For instance: rand (Random) may be useful for **create** but could lead to very odd behavior if used for **search**.

#### Functional classifications ####
| Class | FUSE calls |
|-------|------------|
| search | access, getattr, getxattr, listxattr, open, readlink, chmod, link, removexattr, rmdir, setxattr, truncate, unlink, utimens |
| action | chmod, chown, link, removexattr, rename, rmdir, setxattr, truncate, unlink, utimens |
| search | access, getattr, getxattr, listxattr, open, readlink, symlink |
| create | create, mkdir, mknod |
| none | fallocate, fgetattr, fsync, ftruncate, ioctl, read, readdir, rename, statfs, symlink, write, release |
| N/A | fallocate, fgetattr, fsync, ftruncate, ioctl, read, readdir, statfs, symlink, write, release |

#### Policy descriptions ####
| Policy | Description |
Expand Down Expand Up @@ -100,22 +102,66 @@ There is a pseudo file available at the mountpoint which allows for the runtime

Even if xattrs are disabled the [{list,get,set}xattrs](http://linux.die.net/man/2/listxattr) calls will still work.

The keys are:
##### Keys #####
* user.mergerfs.srcmounts
* user.mergerfs.create
* user.mergerfs.search
* user.mergerfs.category.action
* user.mergerfs.category.create
* user.mergerfs.category.search
* user.mergerfs.func.access
* user.mergerfs.func.chmod
* user.mergerfs.func.chown
* user.mergerfs.func.create
* user.mergerfs.func.getattr
* user.mergerfs.func.getxattr
* user.mergerfs.func.link
* user.mergerfs.func.listxattr
* user.mergerfs.func.mkdir
* user.mergerfs.func.mknod
* user.mergerfs.func.open
* user.mergerfs.func.readlink
* user.mergerfs.func.removexattr
* user.mergerfs.func.rename
* user.mergerfs.func.rmdir
* user.mergerfs.func.setxattr
* user.mergerfs.func.symlink
* user.mergerfs.func.truncate
* user.mergerfs.func.unlink
* user.mergerfs.func.utimens

##### Example #####

```
[trapexit:/tmp/mount] $ xattr -l .mergerfs
user.mergerfs.srcmounts: /tmp/a:/tmp/b
user.mergerfs.create: epmfs
user.mergerfs.search: ff
[trapexit:/tmp/mount] $ xattr -p user.mergerfs.search .mergerfs
user.mergerfs.category.action: ff
user.mergerfs.category.create: epmfs
user.mergerfs.category.search: ff
user.mergerfs.func.access: ff
user.mergerfs.func.chmod: ff
user.mergerfs.func.chown: ff
user.mergerfs.func.create: epmfs
user.mergerfs.func.getattr: ff
user.mergerfs.func.getxattr: ff
user.mergerfs.func.link: ff
user.mergerfs.func.listxattr: ff
user.mergerfs.func.mkdir: epmfs
user.mergerfs.func.mknod: epmfs
user.mergerfs.func.open: ff
user.mergerfs.func.readlink: ff
user.mergerfs.func.removexattr: ff
user.mergerfs.func.rename: ff
user.mergerfs.func.rmdir: ff
user.mergerfs.func.setxattr: ff
user.mergerfs.func.symlink: ff
user.mergerfs.func.truncate: ff
user.mergerfs.func.unlink: ff
user.mergerfs.func.utimens: ff
[trapexit:/tmp/mount] $ xattr -p user.mergerfs.category.search .mergerfs
ff
[trapexit:/tmp/mount] $ xattr -w user.mergerfs.search ffwp .mergerfs
[trapexit:/tmp/mount] $ xattr -p user.mergerfs.search .mergerfs
[trapexit:/tmp/mount] $ xattr -w user.mergerfs.category.search ffwp .mergerfs
[trapexit:/tmp/mount] $ xattr -p user.mergerfs.category.search .mergerfs
ffwp
[trapexit:/tmp/mount] $ xattr -w user.mergerfs.srcmounts +/tmp/c .mergerfs
Expand All @@ -131,6 +177,8 @@ ffwp
/tmp/a:/tmp/b:/tmp/c
```

##### Extra Details #####

For **user.mergerfs.srcmounts** there are several instructions available for manipulating the list. The value provided is just as the value used at mount time. A colon (':') delimited list of full path globs.

| Instruction | Description |
Expand All @@ -144,6 +192,8 @@ For **user.mergerfs.srcmounts** there are several instructions available for man
| =[list] | set |
| [list] | set |

Categories and funcs take a policy as described in the previous section. When reading funcs you'll get the policy string. However, with categories you'll get a coma separated list of policies for each type found. For example: if all search functions are `ff` except for `access` which is `ffwp` the value for `user.mergerfs.category.search` will be `ff,ffwp`.

#### mergerfs file xattrs ####

While they won't show up when using [listxattr](http://linux.die.net/man/2/listxattr) mergerfs offers a number of special xattrs to query information about the files served. To access the values you will need to issue a [getxattr](http://linux.die.net/man/2/getxattr) for one of the following:
Expand Down
2 changes: 1 addition & 1 deletion src/access.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ namespace mergerfs
const ugid::SetResetGuard ugid(fc->uid,fc->gid);
const rwlock::ReadGuard readlock(&config.srcmountslock);

return _access(*config.search,
return _access(*config.access,
config.srcmounts,
fusepath,
mask);
Expand Down
2 changes: 2 additions & 0 deletions src/category.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -35,12 +35,14 @@ namespace mergerfs
const std::vector<Category> Category::_categories_ =
buildvector<Category,true>
(CATEGORY(invalid))
(CATEGORY(action))
(CATEGORY(create))
(CATEGORY(search));

const Category * const Category::categories = &_categories_[1];

const Category &Category::invalid = Category::categories[Category::Enum::invalid];
const Category &Category::action = Category::categories[Category::Enum::action];
const Category &Category::create = Category::categories[Category::Enum::create];
const Category &Category::search = Category::categories[Category::Enum::search];

Expand Down
4 changes: 3 additions & 1 deletion src/category.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,8 @@ namespace mergerfs
{
invalid = -1,
BEGIN = 0,
create = BEGIN,
action = BEGIN,
create,
search,
END
};
Expand Down Expand Up @@ -88,6 +89,7 @@ namespace mergerfs
static const std::vector<Category> _categories_;
static const Category * const categories;
static const Category &invalid;
static const Category &action;
static const Category &create;
static const Category &search;
};
Expand Down
2 changes: 1 addition & 1 deletion src/chmod.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ namespace mergerfs
const config::Config &config = config::get();
const rwlock::ReadGuard readlock(&config.srcmountslock);

return _chmod(*config.search,
return _chmod(*config.chmod,
config.srcmounts,
fusepath,
mode);
Expand Down
2 changes: 1 addition & 1 deletion src/chown.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ namespace mergerfs
const ugid::SetResetGuard ugid(fc->uid,fc->gid);
const rwlock::ReadGuard readlock(&config.srcmountslock);

return _chown(*config.search,
return _chown(*config.chown,
config.srcmounts,
fusepath,
uid,
Expand Down
42 changes: 38 additions & 4 deletions src/config.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@
#include "rwlock.hpp"
#include "fs.hpp"

#define POLICYINIT(X) X(policies[FuseFunc::Enum::X])

using std::string;
using std::vector;

Expand All @@ -43,14 +45,46 @@ namespace mergerfs
: destmount(),
srcmounts(),
srcmountslock(),
create(policies[Category::Enum::create]),
search(policies[Category::Enum::search]),
POLICYINIT(access),
POLICYINIT(chmod),
POLICYINIT(chown),
POLICYINIT(create),
POLICYINIT(getattr),
POLICYINIT(getxattr),
POLICYINIT(link),
POLICYINIT(listxattr),
POLICYINIT(mkdir),
POLICYINIT(mknod),
POLICYINIT(open),
POLICYINIT(readlink),
POLICYINIT(removexattr),
POLICYINIT(rename),
POLICYINIT(rmdir),
POLICYINIT(setxattr),
POLICYINIT(symlink),
POLICYINIT(truncate),
POLICYINIT(unlink),
POLICYINIT(utimens),
controlfile("/.mergerfs")
{
pthread_rwlock_init(&srcmountslock,NULL);

create = &Policy::epmfs;
search = &Policy::ff;
setpolicy(Category::Enum::action,Policy::Enum::ff);
setpolicy(Category::Enum::create,Policy::Enum::epmfs);
setpolicy(Category::Enum::search,Policy::Enum::ff);
}

void
Config::setpolicy(const Category::Enum::Type category,
const Policy::Enum::Type policy_)
{
const Policy *policy = Policy::find(policy_);

for(int i = 0; i < FuseFunc::Enum::END; i++)
{
if(FuseFunc::fusefuncs[i] == category)
policies[(FuseFunc::Enum::Type)FuseFunc::fusefuncs[i]] = policy;
}
}

const Config&
Expand Down
29 changes: 26 additions & 3 deletions src/config.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@
#include <vector>

#include "policy.hpp"
#include "category.hpp"
#include "fusefunc.hpp"

namespace mergerfs
{
Expand All @@ -44,14 +44,37 @@ namespace mergerfs
public:
Config();

public:
void setpolicy(const Category::Enum::Type category,
const Policy::Enum::Type policy);

public:
std::string destmount;
std::vector<std::string> srcmounts;
mutable pthread_rwlock_t srcmountslock;

const Policy *policies[Category::Enum::END];
public:
const Policy *policies[FuseFunc::Enum::END];
const Policy *&access;
const Policy *&chmod;
const Policy *&chown;
const Policy *&create;
const Policy *&search;
const Policy *&getattr;
const Policy *&getxattr;
const Policy *&link;
const Policy *&listxattr;
const Policy *&mkdir;
const Policy *&mknod;
const Policy *&open;
const Policy *&readlink;
const Policy *&removexattr;
const Policy *&rename;
const Policy *&rmdir;
const Policy *&setxattr;
const Policy *&symlink;
const Policy *&truncate;
const Policy *&unlink;
const Policy *&utimens;

public:
const std::string controlfile;
Expand Down
2 changes: 1 addition & 1 deletion src/create.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ namespace mergerfs
const ugid::SetResetGuard ugid(fc->uid,fc->gid);
const rwlock::ReadGuard readlock(&config.srcmountslock);

return _create(*config.search,
return _create(*config.create,
*config.create,
config.srcmounts,
fusepath,
Expand Down
29 changes: 1 addition & 28 deletions src/fs.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,6 @@
#include <sys/types.h>
#include <unistd.h>
#include <glob.h>
#include <fnmatch.h>

#include "fs.hpp"
#include "xattr.hpp"
Expand Down Expand Up @@ -492,7 +491,7 @@ namespace fs
if(rv == -1 && errno != ENOTTY)
return -1;

return (errno = 0);
return 0;
}

void
Expand All @@ -515,32 +514,6 @@ namespace fs
globfree(&gbuf);
}

void
erase_fnmatches(const vector<string> &patterns,
vector<string> &strs)
{
vector<string>::iterator si;
vector<string>::const_iterator pi;

si = strs.begin();
while(si != strs.end())
{
int match = FNM_NOMATCH;

for(pi = patterns.begin();
pi != patterns.end() && match != 0;
++pi)
{
match = fnmatch(pi->c_str(),si->c_str(),0);
}

if(match == 0)
si = strs.erase(si);
else
++si;
}
}

namespace find
{
int
Expand Down
3 changes: 0 additions & 3 deletions src/fs.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -110,9 +110,6 @@ namespace fs
void glob(const vector<string> &patterns,
vector<string> &strs);

void erase_fnmatches(const vector<string> &patterns,
vector<string> &strs);

namespace find
{
int invalid(const vector<string> &basepaths,
Expand Down

0 comments on commit a359c88

Please sign in to comment.