Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.Sign up
Filtering non-cookbook files leave empty directories that are still scp-ed #429
We use test-kitchen & kitchen-ec2 as part of our CI testing on travis. By default travis will bundle in to
We were getting builds that would appear to hang randomly while transferring files for the cookbook being tested. I SSH-ed in to the box while it was transferring files, ran strace against the scp process, and there I saw that it was copying bunch of "files" in the installed bundle directory. Investigating further, I found that they were actually all empty directories.
Looking at the code it looks like all of the resolved cookbooks are copied to a temp directory and then ignorable files are removed. However, directories that these files were in are left behind, due to filtering non-files from the removal list.
Uploading files to the instance happens in
The issue is caused by the ~2000 directories that get created by a pretty reasonable set of installed gems, namely berkshelf, foodcritic, chefspec, & test kitchen. For now I've worked around this issue by having bundle install to another location, outside of the work directory.
I'm not sure what the best way to "fix" this would be, so this is just a bug report for now. I think it would make sense to figure out a way to avoid having these directories end up in the upload at all, whether that means a second round of deleting or switching to a white-list copy.
It appears some discussion happened a while back about a better way to upload cookbooks, which might also solve the slowness issue even if these ignorable directories were still uploaded