1.0 Release #57

Merged
merged 4 commits into from Nov 16, 2015

Projects

None yet

6 participants

@Acconut
Member
Acconut commented Feb 3, 2015

More than a year ago the last release, 0.2.2 was published. Now the final 1.0 release is just around the corner introducing breaking changes and a lot of new features.

The major changes towards the core include the addition of the TUS-Resumable, TUS-Max-Size, TUS-Extension and TUS-Version headers while making the first one mandatory. All these headers must be returned in the new OPTIONS request in order to enable protocol discovery. In addition the Offset header must not be greater or lower than the current offset for the file.

The biggest changes were made by introducing the Upload-Expiration, Checksum, Stream, Retries, Termination, Concatenation (formerly Merge) and Metadata extensions.

After all of this work the protocol is now considered stable and ready for use in production environments. Speaking of implementations the official tus-jquery-client and tusd repositories are currently being updated to support the 1.0 release.

This pull request will be merged once these changes are done. Furthermore, final feedback may be submitted to adjust minor things.

@Acconut Acconut added this to the 1.0 milestone Feb 3, 2015
@qsorix
qsorix commented Feb 3, 2015

Awesome!
I've noticed one typo: "The biggest changes were made by introducing the Upload-Expiration, Checksum, Stream, Retires, Termination, Merge and Metadata extensions."

@Acconut
Member
Acconut commented Feb 3, 2015

@qsorix Fixed, thanks. A blog post will follow soon. :)

@kvz
Member
kvz commented Feb 3, 2015

💯

@kvz
Member
kvz commented Feb 3, 2015

If I had to nitpick, was there a reason to go for Merge over Concatenate? I personally thought the latter was more precise.

@vayam vayam and 1 other commented on an outdated diff Feb 5, 2015
+```
+POST /files HTTP/1.1
+Merge: partial
+Entity-Length: 6
+
+HTTP/1.1 204 No Content
+Location: http://tus.example.org/files/b
+```
+
+The next step is to create the final upload consisting of the two earlier
+generated partial uploads. In following request no `Entity-Length` header is
+presented.
+
+```
+POST /files HTTP/1.1
+Merge: final; /files/a http://tus.example.org/files/b
@vayam
vayam Feb 5, 2015 Member

change http://tus.example.org/files/b to /files/b

@Acconut
Acconut Feb 5, 2015 Member

I want to keep the hostname and scheme to demonstrate that both URLs are valid.

@vayam
vayam Feb 5, 2015 Member

Ah K, makes sense.

@vayam vayam and 1 other commented on an outdated diff Feb 5, 2015
@@ -1,10 +1,11 @@
# tus resumable upload protocol
-**Version:** 0.2.2 ([SemVer](http://semver.org))<br>
+**Version:** 1.0.0 ([SemVer](http://semver.org))<br>
**Date:** 2014-01-26<br>
@vayam
vayam Feb 5, 2015 Member

just a reminder to update the date when you release!

@Acconut
Acconut Feb 5, 2015 Member

Thanks, I had a note by myself. It will be updated before the PR is merged into master.

@vayam vayam and 1 other commented on an outdated diff Feb 5, 2015
+In order to indicate that this extension is supported by the server it MUST
+include the `streams` element in the `TUS-Extension` header.
+
+#### Example
+
+After creating a new upload using the file creation extension, 100 bytes are
+uploaded. The next request transfers additional 100 bytes and the total entity
+length. In the end of this example the server knows that the resource will have
+a size of 300 bytes but only the first 200 are transferred.
+
+**Request:**
+
+```
+POST /files HTTP/1.1
+Host: tus.example.org
+TUS-Resumable: 1.0.0
@vayam
vayam Feb 5, 2015 Member

shouldn't the example include the header
Entity-Length: streaming

@Acconut
Acconut Feb 5, 2015 Member

Thanks, I forgot that one!

@Acconut
Member
Acconut commented Feb 5, 2015

@kvz: If I had to nitpick, was there a reason to go for Merge over Concatenate? I personally thought the latter was more precise.

Sorry, I missed this. I'm ok with renaming the extension to Concatenation since merging may seem as if the chunks are mixed randomly and not ordered. Anyway, I would like to rename the header to Concat instead of Concatenate for simplicity but I am not religious about that. Opinions?

/cc @vayam

@kvz
Member
kvz commented Feb 5, 2015

Concat is fine by me!

On Thu, Feb 5, 2015 at 5:27 PM, Marius notifications@github.com wrote:

@kvz https://github.com/kvz: If I had to nitpick, was there a reason to
go for Merge over Concatenate? I personally thought the latter was more
precise.

Sorry, I missed this. I'm ok with renaming the extension to Concatenation
since merging may seem as if the chunks are mixed randomly and not ordered.
Anyway, I would like to rename the header to Concat instead of Concatenate
for simplicity but I am not religious about that. Opinions?

/cc @vayam https://github.com/vayam


Reply to this email directly or view it on GitHub
#57 (comment)
.

Kevin van Zonneveld (kvz.io)
Co-Founder, Transloadit (transloadit.com)

@vayam
Member
vayam commented Feb 5, 2015

@Acconut sorry I missed that. Question: If I am implementing the server and I don't want to support Merge:Final before individual chunks are completely upload. Can I just return 412 or 400? What do you suggest?

@Acconut
Member
Acconut commented Feb 6, 2015

@vayam Good thought. What about updating the spec to do following:

By default it is not allowed to concatenate uploads when not all of them are finished. The response is not specified by the spec but 412 is already used to mark unsupported versions (see TUS-Resumable). Of course, you can use 400 or if it is too general 422 Unprocessable Entity.
In the case that none of the above work just make your own status code, it is allowed by the HTTP spec. :)

If servers decide to support this feature they should add concatenate-unfinished to TUS-Extension.
You are correct, this eases implementations a lot. Thanks for this mention.

@vayam
Member
vayam commented Feb 6, 2015

422 Unprocessable Entity works!
concatenate-unfinished sounds good.

@Acconut
Member
Acconut commented Feb 6, 2015

@vayam See aa20f72. Does this work for you?

@vayam
Member
vayam commented Feb 8, 2015

@Acconut looks good!

@Acconut
Member
Acconut commented Feb 12, 2015

While implementing the Concatenation extension for tusd I stumbled across following thought:
Should the maximum defined in TUS-Max-Size also apply for concatenated files? For example, the server's max. size is 2GB. A client now uploads three partial uploads each 1GB. Until now everything is ok since the sizes are in all cases lower. Now the client wants to concatenate these three files resulting in a 3GB file.

The question is whether the specification should prohibit/allow if this action (a concatenated upload is bigger than the maximum size) or leave this point up to the implementation?

@kvz
Member
kvz commented Feb 15, 2015

@Acconut Tough one. I expect that the main reason for using TUS-Max-Size is the worry about limitations of storage or further processing steps. With that use-case in mind, I think it makes sense if the tus server rejects both on PATCH and on concat, should it calculate that the resulting file will become bigger than the max allowed size. There should probably be a remark about this in the concat extension. Open to hearing other views on this.

@Acconut
Member
Acconut commented Feb 15, 2015

@kvz The reasons for limiting upload sizes are not always possibilities but also safety to not flood your server (e.g., you don't need to allow uploads with GBs for a simple 100x100 avatar).

Anyway, I think it makes sense to add that if a limit is presented it must be applied for concatenation as well. The spec defines the behaviour already if no static value is able to be determinated:

If no hard-limit is presented or the server is not able to calculate it this header MUST be omitted.

In the meantime I will have a look at whether we can use the Vary header to indicate that TUS-Max-Size may change for different requests.

@Acconut
Member
Acconut commented Feb 15, 2015

The Vary header contains a list of request headers depending on which the response body will change (e.g. user agent string). This is very important for proxies, caches and crawlers but not usable for our case.

@vayam
Member
vayam commented Feb 19, 2015

@kvz @Acconut w.r.t TUS-Max-Size I realized that one of my requirements is to return the max upload size allowed based on user storage quota. What do you guys think of returning TUS-Max-Size in response to every POST creation request instead of being generic response to OPTIONS /files request? Again if you think it is not generic requirement, I am ok to handle it outside TUS protocol.

@Acconut
Member
Acconut commented Feb 19, 2015

@vayam I would like to keep the TUS-Max-Size header in the OPTIONS response for separation.
As you mentioned, feel free to extend the protocol if you feel like it 👍

@Acconut
Member
Acconut commented Mar 2, 2015

I just want to throw in a quick update:
The update for tusd is nearly finished, most of the basic extensions are implemented (expect Streams, I am not sure if I'll do it before the release or not) and the core, too, of course.
Our jquery-plugin does support the core but no additional extensions yet, although I plan to only implement Metadata for now.
Sadly, I am not able to work on the iOS project since I am not able to use Objective-C. Any help is appreciated.

@kvz
Member
kvz commented Sep 12, 2015

Hi Justin,

We're waiting on a final round of feedback, but I don't expect that will result in many changes from the current 1.0 proposal so we're very close to completing this.

All contributions are welcome, anything particular you had in mind?

Kind regards,

Kevin van Zonneveld (kvz.io)
Co-Founder, Transloadit (transloadit.com)

On 11 Sep 2015 at 15:51:40, Justin Henck (notifications@github.com) wrote:

Hi all, may I ask what the status is on this? We're interested in using tus in a project (and
are happy to be contributors), but want to know if there are other efforts I'm not aware
of.


Reply to this email directly or view it on GitHub:
#57 (comment)

Edit: Hm did Justin remove his question or am I doing something silly?

@Acconut
Member
Acconut commented Sep 13, 2015

I just wanted to give a quick status update for myself:
The new X-HTTP-Method-Override header needs to be updated in tusd and tus-java-client. In addition I would like to update the texts on http://tus.io and there is still the Developer Guide left to be written. But all of these tasks do not require work on the protocol spec, which you can call finished.

Hm did Justin remove his question or am I doing something silly?

No, you're not becoming silly :)

@cjhenck
Contributor
cjhenck commented Sep 15, 2015

I decided to try to find a mailing list vs. using a pull request for my own gain :).

I may have a chance to take a look at the java client.

@kvz
Member
kvz commented Sep 15, 2015

I decided to try to find a mailing list vs. using a pull request for my own gain :).

Makes sense! We don't have a mailing list - we do have a Slack channel tho and be happy to shoot you an invite to chat. We're considering switching to gitter.im to lower this bar of entry.

I may have a chance to take a look at the java client.

Excellent!

@kvz
Member
kvz commented Sep 19, 2015

Working on a blog post with Project Status that we'll publish soon.

@kvz kvz referenced this pull request Sep 19, 2015
Closed

Project status? #33

@Acconut
Member
Acconut commented Sep 23, 2015

The blog post, @kvz mentioned is online now: http://tus.io/blog/2015/09/19/project-status/

@kvz kvz merged commit 046960a into master Nov 16, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment