Provide an "exclude" option for the b2 sync command

[segphault]

It'd be really useful to be able to exclude paths that match a certain pattern from synchronization. For example, if I'm synchronizing a bunch of node.js projects, I might want to have b2 sync ignore all the node_modules folders.

[bwbeach]

Good idea.

Do you want the excluded things to be wildcards? Regexp or glob? Match the entire path, or just the file name?

I assume that if a folder is excluded, everything under it should be excluded, too.

[segphault]

For the case that I care about personally, I want to be able to match just the file name rather than the whole path, but I could see the latter also being useful in some situations. I'd like to be able to do something like --exclude "node_modules" and have it exclude directories that match that name everywhere they appear in the hierarchy.

Wildcards would also be great, I think either glob or regex would work well. And yep, it'd also skip the contents of excluded folders. Thanks!

[segphault]

I decided to take a crack at implementing this myself. Given the way that the sync feature builds its file list, matching against the full path ended up being the most practical and least intrusive approach. I also decided on using a regex for the match.

[sund]

I would also like to see rsync-style exclusion and inclusion, i.e. from the rsync help:

--exclude=PATTERN     exclude files matching PATTERN

--exclude-from=FILE     read exclude patterns from FILE

--include=PATTERN     don't exclude files matching PATTERN

--include-from=FILE     read include patterns from FILE

Just my thoughts.

[ppolewicz]

@sund this seems like a very small change after #153 is merged. Would you like to implement the options you mentioned?

[TJC]

Just wanted to say -- I think you should try and implement the same exclude patterns as rsync; they've become reasonably well-understood by many people, and supported by third-party apps.

[ppolewicz]

@TJC @sund @segphault do you still need this?

[simongreen-net]

I would like to see this option.

[ppolewicz]

@simongreen-net okay. Would you like to implement it?

We encourage outside contributors to perform changes on our codebase. Many such changes have been merged already. In order to make it easier to contribute, core developers of this project:

• provide guidance (through the issue reporting system)
• provide tool assisted code review (through the Pull Request system)
• maintain a set of integration tests (run with a production cloud)
• maintain a set of (well over a hundred) unit tests
• automatically run unit tests on 14 versions of python (including osx, Jython and pypy)
• format the code automatically using yapf
• use static code analysis to find subtle/potential issues with maintainability
• maintain other Continous Integration tools (coverage tracker)

this particular change is rather easy to implement.

[simongreen-net]

Sorry, my Python skills aren't at a level where I would be able to write a patch.

[sund]

I would also like this. But, like, @simongreen-net my python-fu depends on google more than it should.

[ppolewicz]

ok, I have somebody working on this. Assigning myself as a placeholder.

[TJC]

@ppolewicz -- Yes, I'd still like to see this feature.. but my Python skills aren't really up there, sorry.

[therecluse26]

Just my two cents... I think that instead of (or inaddition to) having a regex-based ignore function it'd be way more helpful to enable a sort of "ignore" file with a list of all the excluded sync directories, much like the way that the .gitignore file works.

[dougwollison]

@therecluse26 agreed. I'm personally very fond of the --filter=[file] option in rsync; I can have a nice include/exclude ruleset so that I can sync just what I need.

[therecluse26]

Any updates on this?

[ppolewicz]

#244 has been merged

as for --exclude-from=FILE and --include-from=FILE, maybe one day someone will contribute it, when doing so please reference this issue. For now, lets close it.