Save images on remote server

[bezkos]

I want to save images on a remote server via (FTP?). Is this possible with scrapy?
(I know about S3 or Google but i mean in a custom FTP server).

[Gallaecio]

https://docs.scrapy.org/en/latest/search.html?q=ftp&check_keywords=yes&area=default

[bezkos]

I dont think scrapy support saving files, images to FTP servers except S3 or Google.
To achieve that, i patched files.py, images.py and pipelines.py and created a new class FTPStore.

[Gallaecio]

Sorry, I missed the ‘images’ bit 🤦‍♂️

[webtekindo]

Hi @bezkos, we are also looking for the same kind of solution, do you mind sharing your class FTPStore ? That will be very helpfull 👍

[bezkos]

class FTPStore(object):

    FTP_USERNAME = None
    FTP_PASSWORD = None    

    def __init__(self, uri):
        assert uri.startswith('ftp://')
        serverip = uri.split('://', 1)[1]
        self.ftp = FTP()
        # self.ftp.set_debuglevel(2)
        self.ftp.connect(serverip, 21)
        self.ftp.login(self.FTP_USERNAME, self.FTP_PASSWORD)        

    def persist_file(self, path, buf, info, meta=None, headers=None):        
        buf.seek(0)
        self.ftp.storbinary('STOR %s' %path, buf)	
    
    def stat_file(self, path, info):
        Custom code here cause i needed to download images only 1 time without expiration.

        return {'last_modified': last_modified, 'checksum': path}

Hope I helped...

[OmarFarrag]

Working on that

[faizan2700]

Kindly let me know if someone is still working on this if not please assign this to me. If someone is working on this can you tell me which "good first issues" (or easy ones ) are unclaimed yet I have tried to get one other before and that was already claimed by someone. Thank you in advance.

[Gallaecio]

Kindly let me know if someone is still working on this

See #3961

[faizan2700]

I will study codebase according to that @Gallaecio

[hk1997]

Is someone still working on this?
I would love to contribute

[faizan2700]

I looked at the data base and tried to understand but for now I am not working on this issue and @Gallaecio will tell you if someone else is workkng on this issue

[Gallaecio]

@hk1997 You can see “May be fixed by #3961” at the top of this issue, right under the issue title.