Feature request: S3 crawler

[jamesinc]

I'd really like to be able to crawl an S3 bucket. I feel sketchy serving web traffic out of my Dropbox!

[sochix]

Hi! FTP or SMB crawler will not work for you?

[jamesinc]

It would be possible to work around it using s3fs. I was just hoping for a native solution that didn't require mounting a bucket into a directory. It would work, but it is not ideal, as it would want to cache the contents of the bucket to the local disk. In this case, I'm trying to avoid needing lots of storage on the instance running Ambar.

[jmgilman]

+1

[sochix]

In the latest release you can mount S3 folder to local crawler

[AyKarsi]

how can I mount a s3 folder to the local crawler?

[sochix]

@AyKarsi just google on how to map s3 folder to local folder, and then map this local folder to the crawler.