Zipline v4 Announcement & Ideas

[diced]

Please use this thread for any ideas or features you would like to see in Zipline's 4th major release.

Over this summer (and beyond probably), I will be completely rewriting Zipline, to make it more robust and better for everyone.

Due to this, development will focus only on fixing bugs present in 3.7.0, there will be no more minor releases (bugs fixed will be released under a patch release). You will need to use the upstream versions for any fixes.

Roadmap & FAQ: https://diced.notion.site/Zipline-v4-Roadmap-058aceb8a35140e7af4c726560aa3db1?pvs=4

[PhoenixSheppy]

LDAP authentication would be cool. Perhaps proper multi-domain configuration options? I own domains with (.com) and (.net) suffixes, and having the ability to chose, or randomly select them would be nice.

UI looks nice, but there's infinite possibilities for that, so not much to comment on there.

Maybe a gallery option for non-registered users, something they could look through (from a list of approved images set by upload author, etc.)

[diced]

• idrk anything about ldap
• Multi-domain can be done but it's weird I can elaborate on this if you'd like
• 👍
• I think this would be better of as a folder but maybe

[kashalls]

idrk anything about ldap

Instead of LDAP, try implementing support for authelia or authentik. Both of those have support for it and they allow you to use the OAUTH specification.

[diced]

Authentik is supported in v4, but it was also brought into light that authentik is just a OIDC implementation, you can see in the PR #372.

[Ollie1101]

Ability to whitelist/blacklist file extensions.
Small text spot on the login page for abuse email
Better analytics view (all-time/year/month/week/daily) instead of just 10 days or whatever it is
Better invite system thats integrated into the webui
Per user storage usage/limiter/ratelimter
discord upload bypass pretty please (in discussions already)
discord webhook function that posts the link in a channel so that discord can embed it faster when the user uses the link
video preview in gallery, plays when hovered over
no need for a /u/ (or whatever) in upload links but make sure there's backwards compatibility :)

[PhoenixSheppy]

I really like the idea for the report abuse text, perhaps expand upon that with a fill-able form vs. email since some people prefer not to send emails.

[febryanvaldo]

no need for a /u/ (or whatever) in upload links but make sure there's backwards compatibility :)

So you don't know? you can just use / without /u/ for upload links. So it would be DOMAIN.com/NameofImages.jpg instead of DOMAIN.com/u/NameofImages.jpg. Just edit the UPLOADER_ROUTE=/ in your configuration or docker yaml if you are using Docker.

[Ollie1101]

huh.. i wish i knew this before uploading 30,000 files. cause you cant change it after the fact without getting dead links

[febryanvaldo]

huh.. i wish i knew this before uploading 30,000 files. cause you cant change it after the fact without getting dead links

Yeah, maybe because of the database thing.

[diced]

• this exists already, I think I might add some sort of extension/(white/black)list setting instead of it just being a blacklist
• meh idk
• this is planned, so yeah hopefully
• idk what this is supposed to mean but I did revamp it a bit in the current v4
• planned
• is this referring to the 50mb limit bypass? If so then, yes it is planned
• you can already do this
• like youtube I guess, I don't really know if it's a good idea though
• unfortunately I can't do anything about this - maybe some redirect for those who have already posted links that go to one route

[LypticZ]

ability to checkmark uploads to bulk move into a folder
an video/ audio player
maybe designing options for viewing sites
edit the name of a file
see the file options directly on the dashboard without having to click on the file

a good example imo for design and shit is https://upload.systems

[diced]

• good idea
• not sure what this means? I won't be adding a custom video player since there isn't really anything like different qualities and such that you can do on Zipline's videos
• not sure what this means
• maybe
• this can be done through the table view

[Jeyso215]

Cloudflare Captcha to Prevent Spam Bots
#429

Better Image & Text View
#430

Server-Side Encryption Option In Settings
#431

[ItzDerock]

Serverless support would be very nice, that way no matter what region someone is in, they get to see the images at blazingly fast speeds. Tasks like stat calculation could make use of vercel cron tasks (maybe generate the json in as part of a vercel build script) but then that leaves other platforms in the dust.

[diced]

This is something I've wanted to do in v4, though I'm not sure it will be possible to support multiple types of serverless platforms (which would be the goal). The way Zipline v3 works, as a lot of functionality within the custom server, and I'm not sure how else it would be done.

[ItzDerock]

you can probably still use a custom server with serverless if you use something like hono instead of fastify, not sure though.

[diced]

I forgot that zipline has a lot of stuff with worker_threads 😔, so I don't think it will be possible to support serverless platforms

[diced]

I was thinking about this earlier, and was thinking about some sort of proxy that would serve images from Zipline, but I'm not sure. It would be a pretty limited Zipline if serverless.

[ItzDerock]

I was thinking about this earlier, and was thinking about some sort of proxy that would serve images from Zipline, but I'm not sure. It would be a pretty limited Zipline if serverless.

what I was thinking is maybe handle just the image serving using serverless. maybe like with cf workers that only respond to the /u/ path or whatever it's set to and everything else just goes to the main Zipline instance.

idrk, it sounds complex but this is definitely something I'd like to see implemented in some form. maybe this could be an extension that advanced users could setup, but Zipline can still run standalone.

[Jeyso215]

Support Matrix Meta Tags for Embed

[TacticalTechJay]

Are there any docs for this?

[diced]

Matrix is a decentralized chat network, and relies on matrix clients. These clients must parse meta tags and opengraph data for link previews.

[BLACK4585]

Add API integration for Shlink for shortened URLs. I already have a URL shortener, and it would be nice if I could use it. They already have an API. For this, a credentials tab in the settings would be cool.
Here are their API docs: https://api-spec.shlink.io/
Then when uploading new files, you could be able to choose whether to use the Zipline domain or one by Shlink. If you choose Shlink, you are able to set all the settings you want. https://api-spec.shlink.io/#/Short%20URLs/createShortUrl The same goes to shorten URLs via Zipline. Overall, you could go to Shlink directly to create a short URL, but so you have everything in one place.

[Jeyso215]

It has URL Shortener built-in

[BLACK4585]

Yes I know. But imo Shlink is better for this because it is just built for shorten and has way more options. That's why I wrote in the last line that this integration could be also implemented in creating short URLs in Zipline, so everything is in one place.
Also, I want to use my short URL domain for Shlink because, as already mentioned, it has more options to customize the short link. So I don't want to redirect the Domain to Zipline.

[diced]

Adding integration for another url shortening service when Zipline has it's own seems counter-productive. If you would like more options consider suggesting them?

[katanacrimson]

Content warning capability. Blur overlay for images/videos with a textual overlay, potentially custom message indicating why the content is flagged with a warning.

[diced]

not sure what a content warning means? is it like a spoiler and u have to click on it to see the image?

[katanacrimson]

Either a clickthrough or a CSS blur that unblurs on hover, yeah; a content warning could function as a spoiler warning, nsfw content warning, or somesuch.

[vinceycodes]

What are you doing to do that for every image or do you want some sort of ai to try to identify it

[katanacrimson]

Let the user specify it for image uploads themselves. Trying to involve AI for that is overcomplicating things.

[Jeyso215]

Add keybase.io Meta Tags for Embed

[TacticalTechJay]

Sounds a bit vague... What sort of meta tags? What sort of functionality would be gained from adding it?

[diced]

Not sure what this is supposed to mean, but I don't think they have any custom stuff and usually opengraph is the standard for link previews as well as other meta tags, so this won't be added(?)

[Jeyso215]

Add https://haveibeenpwned.com to check if passwords has been in databreach.

Add to see if your images has been trained by AI. https://haveibeentrained.com/

[diced]

This seems out of the scope of Zipline and imo, unnecessary for zipline.

[spiritLHLS]

I think the API key needs to be given different levels of permissions, can only be used to upload files, can only be used to view files, can files be deleted, etc., to give the operations all a hierarchy.

[diced]

Maybe, I think this is a bunch of stuff that could be added later but not really that important for the major release.

[spiritLHLS]

I don't know exactly how it was developed, I guessed before that it involved the underlying design, so I made this suggestion.
If it's not something that needs to be considered at the outset subsequently it can always be attached and I'll wait patiently.

[ItzDerock]

Prometheus support.
basically add a /stats endpoint with authorization (API key maybe?) and just use node-prom then I can setup Prometheus to scrape that endpoint and get cool grafana dashboard because grafana > the Zipline stats page.

[kashalls]

Should actually be /metrics, and if possible add support for it to be pulled only by an Authorization header.

[kashalls]

How about multi-domain support with https://zipline.diced.sh/docs/guides/upload-options#override-domain except you can specify a URL path for that domain. I know you can do this with nginx by rewriting the target path /(.*) with /path/$1 but when uploading using the ShareX uploader it still adds that http(s)://files.example.com/path/AuLOs1.png. The recommended fix for changing UPLOADER_ROUTE to / wouldnt exactly work because it enforces it on all domains.

So TLDR, add an option to override the path too?

[diced]

maybe

[tigattack]

Searching files by name (and, as an awesome stretch, tags) as mentioned in #282 would be amazing and make this so much more usable for me.

I can't speak for others, but since the OAuth PR was merged, lack of search is the only thing holding me back from migrating to Zipline at this point.

Also, is there a technical reason for removing the ability to view EXIF metadata? This is really useful and I'd love to see it stick around in v4.

[diced]

• I think all of this except tags is introduced in v4 as of now (i will work on tags later)
• good to hear! what are you using currently?
• Kind of(?), viewing exif required a library that called a executable and was pretty clunky and I didn't like how it worked. The new way just reads the files bytes and removes GPS metadata. So now we can't view exif metadata - and I thought it was a fair tradeoff since the amount of people wanting to view exif metadata would be pretty small. Unfortunately you are one of the people in that small group, I would like to hear what metadata you would even look at (imo I think its just useless unless you upload pictures from your phone, digital cameras, etc)

[tigattack]

• That's great to hear, thanks!
• I currently use XBackBone, it's nice but it's beginning to feel a little clunky compared to the more recent solutions such as Zipline, and overall has fewer features.
• That makes sense, now I understand why you felt that way about EXIF support! I will fairly often be uploading images from various cameras so yes, I fall into the subset of users who would still find that useful, but it is something I can live without. The ability to strip GPS coordinates is great; I would appreciate the option to strip all EXIF data from images, but I'm guessing if you were to implement this then you'd essentially be back where you were before in terms of calling binaries and so on.

[iiTranquility]

Discussed in Discord but some way to log/send data to a datahandler, then get data from the data handler to something like grafana?

It would query the current stats so we can have that in a dashboard :)

[diced]

as discussed in the discord I think in the near future I will make a zipline-prometheus repository for users to put stats in grafana and such.

[mrpops2ko]

automated dedupe

often times i upload the same file(s) when sharing, either the same reaction gifs, same file or same image

i'd imagine it'd be something like hash each upload and store the hashes, once the same hash is seen then rather than storing a duplicate of the file, it would simply generate an additional url for it and reflink to the original file

this would be especially useful in multi-tenant setups

[diced]

I think this is a good idea, but the only time this would be used is when the files have the same name right? For example, using the name upload format instead of the others like random, uuid, etc. anyway, I think a simple implementation of this would just be when it detects a file of the same name, it can change the files name to hello_2.abc if a file called hello.abc already existed.

[mrpops2ko]

no it'd be used whenever the files turn up the same hash. i'd suggest a hash on the file itself as best practice, especially if you like the idea of easy multi-tenant dedupe.

so if you had say 15 friends all using your zipline instance and they all frequently used the same reaction gif, rather than having 45+ instances of it, the hash of the file itself would be computed and just reflinked if it matches back

then you could have file1.gif file2.gif all the way up to file file45.gif all linking back to the same storage backend file

the unfortunate implication of this would be that if one user selected 'delete file' then it wouldn't actually delete it from the storage, because its referenced in 44 other file link addresses. deleting it would invalidate that one specific link, but the 44 other ones would still be accessible should the user know the different unique string to locate it

edit: the goal being that in these scenarios it would significantly reduce the space utilisation, because lets assume this gif is 1mb, rather than storing 45mb of the same file you'd only store 1mb and reference it 45 times

[diced]

ah I see, i've added this to the roadmap though I'm not sure if itll be in the stable release. I think I'll add it later in.

[maxinetakeba]

Folder structure. Allow folders within folders.

Edit: On the topic of folders.

When viewing folders = view by tree/list and allow multi select

[teslazap]

A way to save Upload Options. The user changes some Upload Options and has the possibility to decide if the new options should be persistent or not. Or at least the admin decides default Upload Options for everyone and be able to adjust them over time.

A way to enforce/disable some Upload Options to users, for example the "Override Domain". If the server will only allow downloads from a specific domain, perhaps different from the one the GUI is being served from, users shouldn't be able to change it

[diced]

• already in v4
• I think this is a good idea, from what I understand the best way to implement this is a setting where you set disallowed headers/options and those will or will not show up in the upload options & if the user tries to use them when uploading the API will not let them

[teslazap]

thank you diced, i'm in a situation where the web gui is served via https (enforced), but the port 443 is restricted to specific address spaces. the files can't be downloaded, not even if the "s" of https is manually dropped by the user, because of the redirect.
the enforced custom domain would allow all the links to use a specific hostname with port 80 access allowed.

in this scenario a way for the user to have access to the aforementioned custom link after the file has been uploaded from the files section would be appreciated. otherwise one must write the custom url by hand to share it again, because I understand the copy-url button uses the web gui address by design

[dotcomUNDERGROUND]

Feature Request: URL Upload

Uploading a file with is URL would be great.

[Permanently]

You can already do this on ShareX.

I also think this could open up a can of worms full of vulnerabilities.

[diced]

Yeah, gonna stick with just file uploads for now.

[barsa2000]

feature request: make zipline a pwa so it's faster to use with a phone

it's not a complex feature to add, just needs a <link> tag in the _document.tsx and a manifest.json to be implemented

docs: https://developer.mozilla.org/en-US/docs/Web/Progressive_web_apps/Guides/Making_PWAs_installable

[diced]

I think this would be cool to add in, I'll take a look and see how things can be implemented soon.

[barsa2000]

Yeah, it's really easy to implement and nice to have, I implemented it locally if you want I can just create a PR

[diced]

I did a bit of research, it even looks like you can make share targets (like right clicking on a file in explorer > open with > zipline, or on phone just selecting the zipline pwa). I'll also add this in if it seems to work well :)

https://developer.mozilla.org/en-US/docs/Web/Manifest/share_target