Feature Request: Provide different ways to upload and store attachments, and drag attachment to card to upload it

[fnkr]

Large databases are not easy to handle and to backup.
It would be useful to store attachments as file in the local filesystem (or maybe on a Cloud Storage provider like Google Cloud Storage or Amazon S3?).

[shekar73]

i agree 👍

[ocdtrekkie]

Local filesystem would definitely be nice. Not sure someone who wants a self-hosted board is going to really want to store data on Google or Amazon clouds though, as that largely defeats the point.

[dcworldwide]

Users may or may not find value in self hosted attachments. I'd prefer if i could like to my google drive account. Any ideas of how we might achieve non-gridfs storage?

[xet7]

I will add S3, Google Drive support etc. I'll add this to my todo list.

[ahashibon]

supporting something like owncloud or next cloud would be great, also supporting access to davros on sandstorm would also be pretty cool! I doubt that using google, or AS would help much, this defeats the point point of own hosting !

[xet7]

Planning to use this package https://atmospherejs.com/malibun23/files

[GavinLilly]

Storing attachments in a different manner would certainly help organisations using Wekan self-hosted. The last thing they want is to have data stored in many places. In my mind AWS and GDrive would be the most applicable places to store and they're both supported by the "files" package

[azban]

I will add S3, Google Drive support etc. I'll add this to my todo list.

@xet7 this hasn't been completed yet, has it? is there any reason, other than a lack of time/priority?

[rossmansfield]

What would be a fair bounty to prioritize this for 1Q 2020? Ross Mansfield, PE O: (281) 888-0808 M: (713) 510-5000 E: Ross@KingwoodSmartHomes.com KWSH From: azban [mailto:notifications@github.com] Sent: Thursday, November 14, 2019 7:41 PM To: wekan/wekan Cc: rossmansfield; Mention Subject: Re: [wekan/wekan] Feature Request: Provide different ways to upload and store attachments, and drag attachment to card to upload it (#142) I will add S3, Google Drive support etc. I'll add this to my todo list. @xet7 <https://github.com/xet7> this hasn't been completed yet, has it? is there any reason, other than a lack of time/priority? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#142?email_source=notifications&email_token=AHMVLCNRVGHFWE6QHCTQKXDQTX43JA5CNFSM4A4UYZW2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEED7QCI#issuecomment-554170377> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHMVLCI5LSP5MMR6OHT6P23QTX43JANCNFSM4A4UYZWQ> . <https://github.com/notifications/beacon/AHMVLCNIE2L543JHJI6ZMOTQTX43JA5CNFSM4A4UYZW2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEED7QCI.gif>

[xet7]

My 1Q 2020 is full.

Does some other Wekan contributor have time to do this at 1Q ?

[razum2um]

@ocdtrekkie @ahashibon please, note, s3 does not "defeat the point" because it's a protocol and there's https://github.com/minio/minio self-hosted alternative with the same api (@rossmansfield mentioned it as well)

[razum2um]

@xet7 @blaggacao did you have time to get back on this btw? could you maybe add hacktoberfest topic to the repo?

p.s oh, it seems like tech discussion continues here #3272

[razum2um]

@rossmansfield feel free to plegde anything https://www.bountysource.com/ or in crypto https://gitcoin.co/ do you still need this?

[blaggacao]

The current implementation on #3273 is just a special implementation of a storage backend towards GridFS MongoDB Buckets.

With the same strategy, those can be stored anywhere given a connector is provided. See for example here. I'd assume, storing to file system is as simple as not deleting the uploaded files and not storing them into a specific backend.

[razum2um]

@blaggacao thanks for such a quick reply, btw, what do you think about another idea: given there's already a self-hosted minio and some images inside - can we just render direct img src= to the direct location, i.e. not uploading twice, but specifying the link?

[blaggacao]

Yes the download location of ostrio:files is configurable. Upload goes through meteor, though. With upload comes the proper initialization of the image reference in the mongo database. So

given there's already a self-hosted minio and some images inside

would be rather awkward to support. Though nothing speaks against using a self hosted minion and populate it through wekan uploads.

[razum2um]

already a self-hosted minio and some images inside

would be rather awkward to support

oh, sorry, didn't realised input can contain just html #2635 (comment)

[blaggacao]

Here is where a hook intercepts the upload and stores it into the back-end (and deletes the uploaded files):

https://github.com/wekan/wekan/pull/3273/files#diff-4a858d7364de26a72d82ca64882d740c4ecf2f5efe5eab5bd481a3a909d49518R26

[xet7]

From @razum2um

feel free to plegde anything https://www.bountysource.com/ or in crypto https://gitcoin.co/ do you still need this?

For Wekan, current payment options are at https://wekan.team/commercial-support/ . Those BountySource and Gitcoin or anything else are not in use at Wekan, it was already a lot of work to move away from BountySource #3177 .

[xet7]

@mfilser @NotTheEvilOne @blaggacao

Any ideas how to fix my first try of AWS S3 code 21e2eab so that after upload and optionally checking file with ClamAV etc, it would upload file to S3 ? And show attachments at cards, be able to move files to S3, etc? Beginnings of those changes are at feature-s3 branch https://github.com/wekan/wekan/tree/feature-s3

Related https://github.com/veliovgroup/Meteor-Files/blob/master/docs/aws-s3-integration.md

[NotTheEvilOne]

Hi,

Fortunately from the validation part no changes are required at all. Validation and storage movement are separated steps by now and can easily work with your changes. I'll try to find some time to look more detailed into the S3 branch over the holidays.

[xet7]

@NotTheEvilOne

I fixed some typos and added some more code so that WeKan starts properly with those S3 related changes, but I have not yet tested does it work. Have you made any changes?

028633b

[xet7]

@mfilser @NotTheEvilOne

I listed related S3 commits at https://github.com/wekan/wekan/blob/master/CHANGELOG.md#upcoming-wekan--release

I merged feature-s3 branch to master, and deleted feature-s3 branch, because it seems WeKan starts anyway with these changes.

Currently I'm getting this error, when trying to click button to move attachment to S3. Trying to figure it out.

I20221226-06:37:48.337(2)? Exception while invoking method 'moveAttachmentToStorage' TypeError: this.classFileStoreStrategyS3 is not a constructor
I20221226-06:37:48.338(2)?     at FileStoreStrategyFactory.getFileStrategy (models/lib/fileStoreStrategy.js:56:15)
I20221226-06:37:48.338(2)?     at models/lib/fileStoreStrategy.js:458:52
I20221226-06:37:48.338(2)?     at Array.forEach (<anonymous>)
I20221226-06:37:48.338(2)?     at moveToStorage (models/lib/fileStoreStrategy.js:456:33)
I20221226-06:37:48.338(2)?     at MethodInvocation.moveAttachmentToStorage (models/attachments.js:147:7)
I20221226-06:37:48.339(2)?     at packages/check/match.js:118:15
I20221226-06:37:48.339(2)?     at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1329:12)
I20221226-06:37:48.339(2)?     at Object._failIfArgumentsAreNotAllChecked (packages/check/match.js:116:43)
I20221226-06:37:48.339(2)?     at maybeAuditArgumentChecks (packages/ddp-server/livedata_server.js:1899:18)
I20221226-06:37:48.339(2)?     at getCurrentMethodInvocationResult (packages/ddp-server/livedata_server.js:772:38)
I20221226-06:37:48.339(2)?     at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1329:12)
I20221226-06:37:48.339(2)?     at packages/ddp-server/livedata_server.js:791:46
I20221226-06:37:48.339(2)?     at new Promise (<anonymous>)
I20221226-06:37:48.340(2)?     at Session.method (packages/ddp-server/livedata_server.js:739:23)
I20221226-06:37:48.340(2)?     at packages/ddp-server/livedata_server.js:603:43

[xet7]

I got some S3 example working at https://github.com/wekan/meteor-files-website where I made these changes wekan/meteor-files-website@c56185e so that it would start properly with:

1. settings.json copied to parent directory
2. S3 settings added
3. Started with start.sh

So with that example, I'm able to upload to S3. I'll look what is different with that example, and current WeKan code, what would be required to get WeKan code working.

[xet7]

I asked from Meteor-Files is there possibility for Custom Endpoint for MinIO veliovgroup/Meteor-Files#862

[xet7]

It's possible to use MinIO and other cloud storage as filesystem mounted with Rclone, docs here:
https://github.com/wekan/wekan/wiki/Rclone

It works with move to filesystem button (not move to S3 button, that does not work yet).