New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1.0 Release #57

Merged
merged 4 commits into from Nov 16, 2015

Conversation

Projects
None yet
6 participants
@Acconut
Member

Acconut commented Feb 3, 2015

More than a year ago the last release, 0.2.2 was published. Now the final 1.0 release is just around the corner introducing breaking changes and a lot of new features.

The major changes towards the core include the addition of the TUS-Resumable, TUS-Max-Size, TUS-Extension and TUS-Version headers while making the first one mandatory. All these headers must be returned in the new OPTIONS request in order to enable protocol discovery. In addition the Offset header must not be greater or lower than the current offset for the file.

The biggest changes were made by introducing the Upload-Expiration, Checksum, Stream, Retries, Termination, Concatenation (formerly Merge) and Metadata extensions.

After all of this work the protocol is now considered stable and ready for use in production environments. Speaking of implementations the official tus-jquery-client and tusd repositories are currently being updated to support the 1.0 release.

This pull request will be merged once these changes are done. Furthermore, final feedback may be submitted to adjust minor things.

@Acconut Acconut added this to the 1.0 milestone Feb 3, 2015

@qsorix

This comment has been minimized.

Show comment
Hide comment
@qsorix

qsorix Feb 3, 2015

Awesome!
I've noticed one typo: "The biggest changes were made by introducing the Upload-Expiration, Checksum, Stream, Retires, Termination, Merge and Metadata extensions."

qsorix commented Feb 3, 2015

Awesome!
I've noticed one typo: "The biggest changes were made by introducing the Upload-Expiration, Checksum, Stream, Retires, Termination, Merge and Metadata extensions."

@Acconut

This comment has been minimized.

Show comment
Hide comment
@Acconut

Acconut Feb 3, 2015

Member

@qsorix Fixed, thanks. A blog post will follow soon. :)

Member

Acconut commented Feb 3, 2015

@qsorix Fixed, thanks. A blog post will follow soon. :)

@kvz

This comment has been minimized.

Show comment
Hide comment
@kvz

kvz Feb 3, 2015

Member

💯

Member

kvz commented Feb 3, 2015

💯

@kvz

This comment has been minimized.

Show comment
Hide comment
@kvz

kvz Feb 3, 2015

Member

If I had to nitpick, was there a reason to go for Merge over Concatenate? I personally thought the latter was more precise.

Member

kvz commented Feb 3, 2015

If I had to nitpick, was there a reason to go for Merge over Concatenate? I personally thought the latter was more precise.

Show outdated Hide outdated protocol.md
```
POST /files HTTP/1.1
Merge: final; /files/a http://tus.example.org/files/b

This comment has been minimized.

@vayam

vayam Feb 5, 2015

Member

change http://tus.example.org/files/b to /files/b

@vayam

vayam Feb 5, 2015

Member

change http://tus.example.org/files/b to /files/b

This comment has been minimized.

@Acconut

Acconut Feb 5, 2015

Member

I want to keep the hostname and scheme to demonstrate that both URLs are valid.

@Acconut

Acconut Feb 5, 2015

Member

I want to keep the hostname and scheme to demonstrate that both URLs are valid.

This comment has been minimized.

@vayam

vayam Feb 5, 2015

Member

Ah K, makes sense.

@vayam

vayam Feb 5, 2015

Member

Ah K, makes sense.

Show outdated Hide outdated protocol.md
@@ -1,10 +1,11 @@
# tus resumable upload protocol
**Version:** 0.2.2 ([SemVer](http://semver.org))<br>
**Version:** 1.0.0 ([SemVer](http://semver.org))<br>
**Date:** 2014-01-26<br>

This comment has been minimized.

@vayam

vayam Feb 5, 2015

Member

just a reminder to update the date when you release!

@vayam

vayam Feb 5, 2015

Member

just a reminder to update the date when you release!

This comment has been minimized.

@Acconut

Acconut Feb 5, 2015

Member

Thanks, I had a note by myself. It will be updated before the PR is merged into master.

@Acconut

Acconut Feb 5, 2015

Member

Thanks, I had a note by myself. It will be updated before the PR is merged into master.

Show outdated Hide outdated protocol.md
```
POST /files HTTP/1.1
Host: tus.example.org
TUS-Resumable: 1.0.0

This comment has been minimized.

@vayam

vayam Feb 5, 2015

Member

shouldn't the example include the header
Entity-Length: streaming

@vayam

vayam Feb 5, 2015

Member

shouldn't the example include the header
Entity-Length: streaming

This comment has been minimized.

@Acconut

Acconut Feb 5, 2015

Member

Thanks, I forgot that one!

@Acconut

Acconut Feb 5, 2015

Member

Thanks, I forgot that one!

@Acconut

This comment has been minimized.

Show comment
Hide comment
@Acconut

Acconut Feb 5, 2015

Member

@kvz: If I had to nitpick, was there a reason to go for Merge over Concatenate? I personally thought the latter was more precise.

Sorry, I missed this. I'm ok with renaming the extension to Concatenation since merging may seem as if the chunks are mixed randomly and not ordered. Anyway, I would like to rename the header to Concat instead of Concatenate for simplicity but I am not religious about that. Opinions?

/cc @vayam

Member

Acconut commented Feb 5, 2015

@kvz: If I had to nitpick, was there a reason to go for Merge over Concatenate? I personally thought the latter was more precise.

Sorry, I missed this. I'm ok with renaming the extension to Concatenation since merging may seem as if the chunks are mixed randomly and not ordered. Anyway, I would like to rename the header to Concat instead of Concatenate for simplicity but I am not religious about that. Opinions?

/cc @vayam

@kvz

This comment has been minimized.

Show comment
Hide comment
@kvz

kvz Feb 5, 2015

Member

Concat is fine by me!

On Thu, Feb 5, 2015 at 5:27 PM, Marius notifications@github.com wrote:

@kvz https://github.com/kvz: If I had to nitpick, was there a reason to
go for Merge over Concatenate? I personally thought the latter was more
precise.

Sorry, I missed this. I'm ok with renaming the extension to Concatenation
since merging may seem as if the chunks are mixed randomly and not ordered.
Anyway, I would like to rename the header to Concat instead of Concatenate
for simplicity but I am not religious about that. Opinions?

/cc @vayam https://github.com/vayam


Reply to this email directly or view it on GitHub
#57 (comment)
.

Kevin van Zonneveld (kvz.io)
Co-Founder, Transloadit (transloadit.com)

Member

kvz commented Feb 5, 2015

Concat is fine by me!

On Thu, Feb 5, 2015 at 5:27 PM, Marius notifications@github.com wrote:

@kvz https://github.com/kvz: If I had to nitpick, was there a reason to
go for Merge over Concatenate? I personally thought the latter was more
precise.

Sorry, I missed this. I'm ok with renaming the extension to Concatenation
since merging may seem as if the chunks are mixed randomly and not ordered.
Anyway, I would like to rename the header to Concat instead of Concatenate
for simplicity but I am not religious about that. Opinions?

/cc @vayam https://github.com/vayam


Reply to this email directly or view it on GitHub
#57 (comment)
.

Kevin van Zonneveld (kvz.io)
Co-Founder, Transloadit (transloadit.com)

@vayam

This comment has been minimized.

Show comment
Hide comment
@vayam

vayam Feb 5, 2015

Member

@Acconut sorry I missed that. Question: If I am implementing the server and I don't want to support Merge:Final before individual chunks are completely upload. Can I just return 412 or 400? What do you suggest?

Member

vayam commented Feb 5, 2015

@Acconut sorry I missed that. Question: If I am implementing the server and I don't want to support Merge:Final before individual chunks are completely upload. Can I just return 412 or 400? What do you suggest?

@Acconut

This comment has been minimized.

Show comment
Hide comment
@Acconut

Acconut Feb 6, 2015

Member

@vayam Good thought. What about updating the spec to do following:

By default it is not allowed to concatenate uploads when not all of them are finished. The response is not specified by the spec but 412 is already used to mark unsupported versions (see TUS-Resumable). Of course, you can use 400 or if it is too general 422 Unprocessable Entity.
In the case that none of the above work just make your own status code, it is allowed by the HTTP spec. :)

If servers decide to support this feature they should add concatenate-unfinished to TUS-Extension.
You are correct, this eases implementations a lot. Thanks for this mention.

Member

Acconut commented Feb 6, 2015

@vayam Good thought. What about updating the spec to do following:

By default it is not allowed to concatenate uploads when not all of them are finished. The response is not specified by the spec but 412 is already used to mark unsupported versions (see TUS-Resumable). Of course, you can use 400 or if it is too general 422 Unprocessable Entity.
In the case that none of the above work just make your own status code, it is allowed by the HTTP spec. :)

If servers decide to support this feature they should add concatenate-unfinished to TUS-Extension.
You are correct, this eases implementations a lot. Thanks for this mention.

@vayam

This comment has been minimized.

Show comment
Hide comment
@vayam

vayam Feb 6, 2015

Member

422 Unprocessable Entity works!
concatenate-unfinished sounds good.

Member

vayam commented Feb 6, 2015

422 Unprocessable Entity works!
concatenate-unfinished sounds good.

@Acconut

This comment has been minimized.

Show comment
Hide comment
@Acconut

Acconut Feb 6, 2015

Member

@vayam See aa20f72. Does this work for you?

Member

Acconut commented Feb 6, 2015

@vayam See aa20f72. Does this work for you?

@vayam

This comment has been minimized.

Show comment
Hide comment
@vayam

vayam Feb 8, 2015

Member

@Acconut looks good!

Member

vayam commented Feb 8, 2015

@Acconut looks good!

@Acconut

This comment has been minimized.

Show comment
Hide comment
@Acconut

Acconut Feb 12, 2015

Member

While implementing the Concatenation extension for tusd I stumbled across following thought:
Should the maximum defined in TUS-Max-Size also apply for concatenated files? For example, the server's max. size is 2GB. A client now uploads three partial uploads each 1GB. Until now everything is ok since the sizes are in all cases lower. Now the client wants to concatenate these three files resulting in a 3GB file.

The question is whether the specification should prohibit/allow if this action (a concatenated upload is bigger than the maximum size) or leave this point up to the implementation?

Member

Acconut commented Feb 12, 2015

While implementing the Concatenation extension for tusd I stumbled across following thought:
Should the maximum defined in TUS-Max-Size also apply for concatenated files? For example, the server's max. size is 2GB. A client now uploads three partial uploads each 1GB. Until now everything is ok since the sizes are in all cases lower. Now the client wants to concatenate these three files resulting in a 3GB file.

The question is whether the specification should prohibit/allow if this action (a concatenated upload is bigger than the maximum size) or leave this point up to the implementation?

@kvz

This comment has been minimized.

Show comment
Hide comment
@kvz

kvz Feb 15, 2015

Member

@Acconut Tough one. I expect that the main reason for using TUS-Max-Size is the worry about limitations of storage or further processing steps. With that use-case in mind, I think it makes sense if the tus server rejects both on PATCH and on concat, should it calculate that the resulting file will become bigger than the max allowed size. There should probably be a remark about this in the concat extension. Open to hearing other views on this.

Member

kvz commented Feb 15, 2015

@Acconut Tough one. I expect that the main reason for using TUS-Max-Size is the worry about limitations of storage or further processing steps. With that use-case in mind, I think it makes sense if the tus server rejects both on PATCH and on concat, should it calculate that the resulting file will become bigger than the max allowed size. There should probably be a remark about this in the concat extension. Open to hearing other views on this.

@Acconut

This comment has been minimized.

Show comment
Hide comment
@Acconut

Acconut Feb 15, 2015

Member

@kvz The reasons for limiting upload sizes are not always possibilities but also safety to not flood your server (e.g., you don't need to allow uploads with GBs for a simple 100x100 avatar).

Anyway, I think it makes sense to add that if a limit is presented it must be applied for concatenation as well. The spec defines the behaviour already if no static value is able to be determinated:

If no hard-limit is presented or the server is not able to calculate it this header MUST be omitted.

In the meantime I will have a look at whether we can use the Vary header to indicate that TUS-Max-Size may change for different requests.

Member

Acconut commented Feb 15, 2015

@kvz The reasons for limiting upload sizes are not always possibilities but also safety to not flood your server (e.g., you don't need to allow uploads with GBs for a simple 100x100 avatar).

Anyway, I think it makes sense to add that if a limit is presented it must be applied for concatenation as well. The spec defines the behaviour already if no static value is able to be determinated:

If no hard-limit is presented or the server is not able to calculate it this header MUST be omitted.

In the meantime I will have a look at whether we can use the Vary header to indicate that TUS-Max-Size may change for different requests.

@Acconut

This comment has been minimized.

Show comment
Hide comment
@Acconut

Acconut Feb 15, 2015

Member

The Vary header contains a list of request headers depending on which the response body will change (e.g. user agent string). This is very important for proxies, caches and crawlers but not usable for our case.

Member

Acconut commented Feb 15, 2015

The Vary header contains a list of request headers depending on which the response body will change (e.g. user agent string). This is very important for proxies, caches and crawlers but not usable for our case.

@vayam

This comment has been minimized.

Show comment
Hide comment
@vayam

vayam Feb 19, 2015

Member

@kvz @Acconut w.r.t TUS-Max-Size I realized that one of my requirements is to return the max upload size allowed based on user storage quota. What do you guys think of returning TUS-Max-Size in response to every POST creation request instead of being generic response to OPTIONS /files request? Again if you think it is not generic requirement, I am ok to handle it outside TUS protocol.

Member

vayam commented Feb 19, 2015

@kvz @Acconut w.r.t TUS-Max-Size I realized that one of my requirements is to return the max upload size allowed based on user storage quota. What do you guys think of returning TUS-Max-Size in response to every POST creation request instead of being generic response to OPTIONS /files request? Again if you think it is not generic requirement, I am ok to handle it outside TUS protocol.

@Acconut

This comment has been minimized.

Show comment
Hide comment
@Acconut

Acconut Feb 19, 2015

Member

@vayam I would like to keep the TUS-Max-Size header in the OPTIONS response for separation.
As you mentioned, feel free to extend the protocol if you feel like it 👍

Member

Acconut commented Feb 19, 2015

@vayam I would like to keep the TUS-Max-Size header in the OPTIONS response for separation.
As you mentioned, feel free to extend the protocol if you feel like it 👍

@Acconut

This comment has been minimized.

Show comment
Hide comment
@Acconut

Acconut Mar 2, 2015

Member

I just want to throw in a quick update:
The update for tusd is nearly finished, most of the basic extensions are implemented (expect Streams, I am not sure if I'll do it before the release or not) and the core, too, of course.
Our jquery-plugin does support the core but no additional extensions yet, although I plan to only implement Metadata for now.
Sadly, I am not able to work on the iOS project since I am not able to use Objective-C. Any help is appreciated.

Member

Acconut commented Mar 2, 2015

I just want to throw in a quick update:
The update for tusd is nearly finished, most of the basic extensions are implemented (expect Streams, I am not sure if I'll do it before the release or not) and the core, too, of course.
Our jquery-plugin does support the core but no additional extensions yet, although I plan to only implement Metadata for now.
Sadly, I am not able to work on the iOS project since I am not able to use Objective-C. Any help is appreciated.

@Acconut Acconut referenced this pull request Mar 3, 2015

Merged

Add collaborators #58

@kvz

This comment has been minimized.

Show comment
Hide comment
@kvz

kvz Sep 12, 2015

Member

Hi Justin,

We're waiting on a final round of feedback, but I don't expect that will result in many changes from the current 1.0 proposal so we're very close to completing this.

All contributions are welcome, anything particular you had in mind?

Kind regards,

Kevin van Zonneveld (kvz.io)
Co-Founder, Transloadit (transloadit.com)

On 11 Sep 2015 at 15:51:40, Justin Henck (notifications@github.com) wrote:

Hi all, may I ask what the status is on this? We're interested in using tus in a project (and
are happy to be contributors), but want to know if there are other efforts I'm not aware
of.


Reply to this email directly or view it on GitHub:
#57 (comment)

Edit: Hm did Justin remove his question or am I doing something silly?

Member

kvz commented Sep 12, 2015

Hi Justin,

We're waiting on a final round of feedback, but I don't expect that will result in many changes from the current 1.0 proposal so we're very close to completing this.

All contributions are welcome, anything particular you had in mind?

Kind regards,

Kevin van Zonneveld (kvz.io)
Co-Founder, Transloadit (transloadit.com)

On 11 Sep 2015 at 15:51:40, Justin Henck (notifications@github.com) wrote:

Hi all, may I ask what the status is on this? We're interested in using tus in a project (and
are happy to be contributors), but want to know if there are other efforts I'm not aware
of.


Reply to this email directly or view it on GitHub:
#57 (comment)

Edit: Hm did Justin remove his question or am I doing something silly?

@Acconut

This comment has been minimized.

Show comment
Hide comment
@Acconut

Acconut Sep 13, 2015

Member

I just wanted to give a quick status update for myself:
The new X-HTTP-Method-Override header needs to be updated in tusd and tus-java-client. In addition I would like to update the texts on http://tus.io and there is still the Developer Guide left to be written. But all of these tasks do not require work on the protocol spec, which you can call finished.

Hm did Justin remove his question or am I doing something silly?

No, you're not becoming silly :)

Member

Acconut commented Sep 13, 2015

I just wanted to give a quick status update for myself:
The new X-HTTP-Method-Override header needs to be updated in tusd and tus-java-client. In addition I would like to update the texts on http://tus.io and there is still the Developer Guide left to be written. But all of these tasks do not require work on the protocol spec, which you can call finished.

Hm did Justin remove his question or am I doing something silly?

No, you're not becoming silly :)

@cjhenck

This comment has been minimized.

Show comment
Hide comment
@cjhenck

cjhenck Sep 15, 2015

Contributor

I decided to try to find a mailing list vs. using a pull request for my own gain :).

I may have a chance to take a look at the java client.

Contributor

cjhenck commented Sep 15, 2015

I decided to try to find a mailing list vs. using a pull request for my own gain :).

I may have a chance to take a look at the java client.

@kvz

This comment has been minimized.

Show comment
Hide comment
@kvz

kvz Sep 15, 2015

Member

I decided to try to find a mailing list vs. using a pull request for my own gain :).

Makes sense! We don't have a mailing list - we do have a Slack channel tho and be happy to shoot you an invite to chat. We're considering switching to gitter.im to lower this bar of entry.

I may have a chance to take a look at the java client.

Excellent!

Member

kvz commented Sep 15, 2015

I decided to try to find a mailing list vs. using a pull request for my own gain :).

Makes sense! We don't have a mailing list - we do have a Slack channel tho and be happy to shoot you an invite to chat. We're considering switching to gitter.im to lower this bar of entry.

I may have a chance to take a look at the java client.

Excellent!

@kvz

This comment has been minimized.

Show comment
Hide comment
@kvz

kvz Sep 19, 2015

Member

Working on a blog post with Project Status that we'll publish soon.

Member

kvz commented Sep 19, 2015

Working on a blog post with Project Status that we'll publish soon.

@kvz kvz referenced this pull request Sep 19, 2015

Closed

Project status? #33

@Acconut

This comment has been minimized.

Show comment
Hide comment
@Acconut

Acconut Sep 23, 2015

Member

The blog post, @kvz mentioned is online now: http://tus.io/blog/2015/09/19/project-status/

Member

Acconut commented Sep 23, 2015

The blog post, @kvz mentioned is online now: http://tus.io/blog/2015/09/19/project-status/

@kvz kvz merged commit 046960a into master Nov 16, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment