Out of memory error when processing large folder

[iamfinky]

So I have 1.7m files to upload to RackSpace Cloud Files and someone recommended your tool.
However, running the tool on a VM with 2GB of RAM caused it to run out of memory.

Is your tool loading ALL file information into memory before uploading, or is it processing each folder/subfolder independently?

[cloudnull]

hi @ilithium turbolift will index the content within the source directory and then upload. The practice is similar to that of rsync. In order to accomplish the upload of all of your files you might need to upload targeted directories in separate containers. While that'll be more work upfront it will make managing content within swift (cloudfiles) a whole lot nicer, IMO.

[cloudnull]

Ping me on IRC if you have any questions or if I can help out in some way. /me @cloudnull.