Empty endpoint doesn't work during domain registration

[tweist]

Hi,

First off, thanks a lot for putting up this free service. I tried to register a new domain following the instruction.

Hostname: sa-sanofi.tsibiocomputing.com

I left "Optional S3 endpoint" blank since I use AWS US standard.

However, I got the following error message:

invalid AWS S3 region 's3.amazonaws.com'; invalid domain

Do you know what is wrong?

[yegor256]

Try to use s3 as an endpoint. Empty text doesn't work at the moment.

[yegor256]

I'll try to fix this UI problem in the nearest future, thanks for reporting.

[tweist]

That worked. Would be nice if you update the webpage instruction.

Thanks a lot!

-Wei

On Thu, Feb 14, 2013 at 11:46 AM, Yegor Bugayenko
notifications@github.comwrote:

Try to use s3 as an endpoint. Empty text doesn't work at the moment.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/43#issuecomment-13564147.

Wei Tao, Ph.D.
TSI Biocomputing LLC
617-564-0934

This e-mail and any files transmitted with it may contain privileged and
confidential information and are intended solely for the use of the
individual or entity to which they are addressed. If you are not the
intended recipient or the person responsible for delivering the e-mail to
the intended recipient, you are hereby notified that any dissemination or
copying of this e-mail or any of its attachment(s) is strictly prohibited.
If you have received this e-mail in error, please immediately notify the
sending individual or entity by e-mail and permanently delete the original
e-mail and attachment(s) from your computer system. Thank you for your
cooperation.

[tweist]

Now new problem. I was repeated asked for user/password even though I'm
sure I typed in correct. I also tried
http://sa-sanofi.tsibiocomputing.com/.htpasswd and got 28 byte back. What
else should I try?

Thanks!

-Wei

On Thu, Feb 14, 2013 at 11:52 AM, Wei Tao wei.tao@tsibiocomputing.comwrote:

That worked. Would be nice if you update the webpage instruction.

Thanks a lot!

-Wei

On Thu, Feb 14, 2013 at 11:46 AM, Yegor Bugayenko <
notifications@github.com> wrote:

Try to use s3 as an endpoint. Empty text doesn't work at the moment.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/43#issuecomment-13564147.

Wei Tao, Ph.D.
TSI Biocomputing LLC
617-564-0934

This e-mail and any files transmitted with it may contain privileged and
confidential information and are intended solely for the use of the
individual or entity to which they are addressed. If you are not the
intended recipient or the person responsible for delivering the e-mail to
the intended recipient, you are hereby notified that any dissemination or
copying of this e-mail or any of its attachment(s) is strictly prohibited.
If you have received this e-mail in error, please immediately notify the
sending individual or entity by e-mail and permanently delete the original
e-mail and attachment(s) from your computer system. Thank you for your
cooperation.

Wei Tao, Ph.D.
TSI Biocomputing LLC
617-564-0934

This e-mail and any files transmitted with it may contain privileged and
confidential information and are intended solely for the use of the
individual or entity to which they are addressed. If you are not the
intended recipient or the person responsible for delivering the e-mail to
the intended recipient, you are hereby notified that any dissemination or
copying of this e-mail or any of its attachment(s) is strictly prohibited.
If you have received this e-mail in error, please immediately notify the
sending individual or entity by e-mail and permanently delete the original
e-mail and attachment(s) from your computer system. Thank you for your
cooperation.

[tweist]

I also got this message:

sa-sanofi.tsibiocomputing.com with .htpasswd(0 user(s) updated 1ms ago)

Thanks!

On Thu, Feb 14, 2013 at 12:07 PM, Wei Tao wei.tao@tsibiocomputing.comwrote:

Now new problem. I was repeated asked for user/password even though I'm
sure I typed in correct. I also tried
http://sa-sanofi.tsibiocomputing.com/.htpasswd and got 28 byte back. What
else should I try?

Thanks!

-Wei

On Thu, Feb 14, 2013 at 11:52 AM, Wei Tao wei.tao@tsibiocomputing.comwrote:

That worked. Would be nice if you update the webpage instruction.

Thanks a lot!

-Wei

On Thu, Feb 14, 2013 at 11:46 AM, Yegor Bugayenko <
notifications@github.com> wrote:

Try to use s3 as an endpoint. Empty text doesn't work at the moment.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/43#issuecomment-13564147.

Wei Tao, Ph.D.
TSI Biocomputing LLC
617-564-0934

This e-mail and any files transmitted with it may contain privileged and
confidential information and are intended solely for the use of the
individual or entity to which they are addressed. If you are not the
intended recipient or the person responsible for delivering the e-mail to
the intended recipient, you are hereby notified that any dissemination or
copying of this e-mail or any of its attachment(s) is strictly prohibited.
If you have received this e-mail in error, please immediately notify the
sending individual or entity by e-mail and permanently delete the original
e-mail and attachment(s) from your computer system. Thank you for your
cooperation.

Wei Tao, Ph.D.
TSI Biocomputing LLC
617-564-0934

This e-mail and any files transmitted with it may contain privileged and
confidential information and are intended solely for the use of the
individual or entity to which they are addressed. If you are not the
intended recipient or the person responsible for delivering the e-mail to
the intended recipient, you are hereby notified that any dissemination or
copying of this e-mail or any of its attachment(s) is strictly prohibited.
If you have received this e-mail in error, please immediately notify the
sending individual or entity by e-mail and permanently delete the original
e-mail and attachment(s) from your computer system. Thank you for your
cooperation.

Wei Tao, Ph.D.
TSI Biocomputing LLC
617-564-0934

This e-mail and any files transmitted with it may contain privileged and
confidential information and are intended solely for the use of the
individual or entity to which they are addressed. If you are not the
intended recipient or the person responsible for delivering the e-mail to
the intended recipient, you are hereby notified that any dissemination or
copying of this e-mail or any of its attachment(s) is strictly prohibited.
If you have received this e-mail in error, please immediately notify the
sending individual or entity by e-mail and permanently delete the original
e-mail and attachment(s) from your computer system. Thank you for your
cooperation.

[yegor256]

What encryption mechanism are you using for your password? Looks like the server managed to load your .htpasswd, but couldn't understand it.

[tweist]

SHA1. Here is the sample entry in the .htpasswd file:

htpasswd -nbs s3auth s3auth

-Wei

On Thu, Feb 14, 2013 at 12:25 PM, Yegor Bugayenko
notifications@github.comwrote:

What encryption mechanism are you using for your password? Looks like the
server managed to load your .htpasswd, but couldn't understand it.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/43#issuecomment-13567353.

Wei Tao, Ph.D.
TSI Biocomputing LLC
617-564-0934

This e-mail and any files transmitted with it may contain privileged and
confidential information and are intended solely for the use of the
individual or entity to which they are addressed. If you are not the
intended recipient or the person responsible for delivering the e-mail to
the intended recipient, you are hereby notified that any dissemination or
copying of this e-mail or any of its attachment(s) is strictly prohibited.
If you have received this e-mail in error, please immediately notify the
sending individual or entity by e-mail and permanently delete the original
e-mail and attachment(s) from your computer system. Thank you for your
cooperation.

[yegor256]

Try this tool: http://aspirine.org/htpasswd_en.html

[yegor256]

Your .htpasswd file should look like (for example):

yegor:{SHA}witfkXg0JglCjW9RssWvTAveakI=

[tweist]

I see. The instruction could be a little clearer on what content to put in
the .htpasswd file.

That worked. Thanks so much!

-Wei

On Thu, Feb 14, 2013 at 12:31 PM, Yegor Bugayenko
notifications@github.comwrote:

Your .htpasswd file should look like (for example):

yegor:{SHA}witfkXg0JglCjW9RssWvTAveakI=

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/43#issuecomment-13567691.

Wei Tao, Ph.D.
TSI Biocomputing LLC
617-564-0934

This e-mail and any files transmitted with it may contain privileged and
confidential information and are intended solely for the use of the
individual or entity to which they are addressed. If you are not the
intended recipient or the person responsible for delivering the e-mail to
the intended recipient, you are hereby notified that any dissemination or
copying of this e-mail or any of its attachment(s) is strictly prohibited.
If you have received this e-mail in error, please immediately notify the
sending individual or entity by e-mail and permanently delete the original
e-mail and attachment(s) from your computer system. Thank you for your
cooperation.

[yegor256]

Make sense, I'll update instructions! Thanks for using the service!

[tweist]

Thanks again for the wonderful service that you provide.

A side point: is the main reason that you don't support a non-domain type
bucket name is that it can't easily be redirected to relay.s3auth.com?

On Thu, Feb 14, 2013 at 12:43 PM, Yegor Bugayenko
notifications@github.comwrote:

Make sense, I'll update instructions! Thanks for using the service!

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/43#issuecomment-13568283.

Wei Tao, Ph.D.
TSI Biocomputing LLC
617-564-0934

This e-mail and any files transmitted with it may contain privileged and
confidential information and are intended solely for the use of the
individual or entity to which they are addressed. If you are not the
intended recipient or the person responsible for delivering the e-mail to
the intended recipient, you are hereby notified that any dissemination or
copying of this e-mail or any of its attachment(s) is strictly prohibited.
If you have received this e-mail in error, please immediately notify the
sending individual or entity by e-mail and permanently delete the original
e-mail and attachment(s) from your computer system. Thank you for your
cooperation.

[yegor256]

I just can't find a few hours of free time to do it. It's already requested: #41

Hopefully will do it in a few weeks.

[yegor256]

You can add your comment in issue #41, so that we will know that more people are waiting for it

[tweist]

I added comments in issue 41. It will be a huge plus if we can use existing
buckets. Moving files around in S3 is no fun.

Thanks again!

-Wei

On Thu, Feb 14, 2013 at 1:04 PM, Yegor Bugayenko
notifications@github.comwrote:

You can add your comment in issue #41#41,
so that we will know that more people are waiting for it

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/43#issuecomment-13569290.

Wei Tao, Ph.D.
TSI Biocomputing LLC
617-564-0934

This e-mail and any files transmitted with it may contain privileged and
confidential information and are intended solely for the use of the
individual or entity to which they are addressed. If you are not the
intended recipient or the person responsible for delivering the e-mail to
the intended recipient, you are hereby notified that any dissemination or
copying of this e-mail or any of its attachment(s) is strictly prohibited.
If you have received this e-mail in error, please immediately notify the
sending individual or entity by e-mail and permanently delete the original
e-mail and attachment(s) from your computer system. Thank you for your
cooperation.

[tweist]

Hi Yegor,

I was trying to use S3auth service to access private S3 files using a tool
called IGV (http://www.broadinstitute.org/igv/). However, I have difficulty
to load a type of file called BAM. I contacted the IGV developer and got
the following response:

The BAM requests are missing the range-byte headers...
If it works by forwarding the request be sure that it forwards the entire
request, including all headers (crucially the Range header).

This is a little bit cryptic to me. Does this response ring a bell to you?
Also, does the response content from http request go through your relay
server? Or it just handles the authentication?

Thanks for your help!

-Wei

On Thu, Feb 14, 2013 at 1:22 PM, Wei Tao wei.tao@tsibiocomputing.comwrote:

I added comments in issue 41. It will be a huge plus if we can use
existing buckets. Moving files around in S3 is no fun.

Thanks again!

-Wei

On Thu, Feb 14, 2013 at 1:04 PM, Yegor Bugayenko <notifications@github.com

wrote:

You can add your comment in issue #41#41,
so that we will know that more people are waiting for it

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/43#issuecomment-13569290.

Wei Tao, Ph.D.
TSI Biocomputing LLC
617-564-0934

This e-mail and any files transmitted with it may contain privileged and
confidential information and are intended solely for the use of the
individual or entity to which they are addressed. If you are not the
intended recipient or the person responsible for delivering the e-mail to
the intended recipient, you are hereby notified that any dissemination or
copying of this e-mail or any of its attachment(s) is strictly prohibited.
If you have received this e-mail in error, please immediately notify the
sending individual or entity by e-mail and permanently delete the original
e-mail and attachment(s) from your computer system. Thank you for your
cooperation.

Wei Tao, Ph.D.
TSI Biocomputing LLC
617-564-0934

This e-mail and any files transmitted with it may contain privileged and
confidential information and are intended solely for the use of the
individual or entity to which they are addressed. If you are not the
intended recipient or the person responsible for delivering the e-mail to
the intended recipient, you are hereby notified that any dissemination or
copying of this e-mail or any of its attachment(s) is strictly prohibited.
If you have received this e-mail in error, please immediately notify the
sending individual or entity by e-mail and permanently delete the original
e-mail and attachment(s) from your computer system. Thank you for your
cooperation.

[yegor256]

Well, this is how our server works:

1. It receives a request from an HTTP client (a web browser, for example). This request contains two important pieces of information: Host HTTP header and request URI.
2. According to the information in the Host header the server finds your S3 "bucket" in one of the accounts registered with us.
3. Using the request URI the server finds the "key" in your bucket.
4. The server makes a request to Amazon S3 using their RESTful API, providing them the bucket and key names.
5. Amazon responds with some binary data (content of your files stored in Amazon S3) and a few HTTP response headers. The server ignores all HTTP response headers received from Amazon, except two: Content-Length and Content-Type.
6. The server adds its own HTTP headers to those two received from Amazon and sends them back, together with data, to the HTTP client (the browser).

Thus, for me the text "The BAM requests are missing the range-byte headers" is a bit cryptic too. I would understand if they would say "The BAM responds are missing...", since we're not returning back any "range related" headers.

Can you explain what network components are you trying to connect and how? Maybe I'd able to help..

[tweist]

Hi Yegor,

Thanks for your thorough response. I think I got it. And the "The BAM
requests are missing the range-byte headers" should read BAM response. So
could you add range-byte header in the heads returned (item 5)? That in
theory should solve the problem.

Best,

-Wei

On Sunday, March 3, 2013, Yegor Bugayenko wrote:

Well, this is how our server works:

It receives a requesthttp://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol#Request_messagefrom an HTTP client (a web browser, for example). This request contains two
important pieces of information: Host HTTP header and request URI.
2.

According to the information in the Host header the server finds your
S3 "bucket" in one of the accounts registered with us.
3.

Using the request URI the server finds the "key" in your bucket.
4.

The server makes a request to Amazon S3 using their RESTful API,
providing them the bucket and key names.
5.

Amazon responds with some binary data (content of your files stored in
Amazon S3) and a few HTTP response headers. The server ignores all HTTP
response headers received from Amazon, except two: Content-Length and
Content-Type.
6.

The server adds its own HTTP headers to those two received from Amazon
and sends them back, together with data, to the HTTP client (the browser).

Thus, for me the text "The BAM requests are missing the range-byte
headers" is a bit cryptic too. I would understand if they would say "The
BAM responds are missing...", since we're not returning back any "range
related" http://en.wikipedia.org/wiki/Byte_serving headers.

Can you explain what network components are you trying to connect and how?
Maybe I'd able to help..

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/43#issuecomment-14342906
.

Wei Tao, Ph.D.
TSI Biocomputing LLC
617-564-0934

This e-mail and any files transmitted with it may contain privileged and
confidential information and are intended solely for the use of the
individual or entity to which they are addressed. If you are not the
intended recipient or the person responsible for delivering the e-mail to
the intended recipient, you are hereby notified that any dissemination or
copying of this e-mail or any of its attachment(s) is strictly prohibited.
If you have received this e-mail in error, please immediately notify the
sending individual or entity by e-mail and permanently delete the original
e-mail and attachment(s) from your computer system. Thank you for your
cooperation.

[yegor256]

Yes, we'll do it in #47

[tweist]

Hi Yegor,

Thank you! Do you have an ETA?

-Wei

On Sun, Mar 3, 2013 at 7:50 AM, Yegor Bugayenko notifications@github.comwrote:

Yes, we'll do it in #47 #47

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/43#issuecomment-14346438
.

Wei Tao, Ph.D.
TSI Biocomputing LLC
617-564-0934

This e-mail and any files transmitted with it may contain privileged and
confidential information and are intended solely for the use of the
individual or entity to which they are addressed. If you are not the
intended recipient or the person responsible for delivering the e-mail to
the intended recipient, you are hereby notified that any dissemination or
copying of this e-mail or any of its attachment(s) is strictly prohibited.
If you have received this e-mail in error, please immediately notify the
sending individual or entity by e-mail and permanently delete the original
e-mail and attachment(s) from your computer system. Thank you for your
cooperation.

[yegor256]

yes, I'll fix #47 today

[tweist]

Sounds good. Thanks so much for your prompt response. When you fix #47, let
me know and I can test it again to see if it resolves the issues we
encountered.

-Wei

On Monday, March 4, 2013, Yegor Bugayenko wrote:

yes, I'll fix #47 #47 today

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/43#issuecomment-14365713
.

Wei Tao, Ph.D.
TSI Biocomputing LLC
617-564-0934

This e-mail and any files transmitted with it may contain privileged and
confidential information and are intended solely for the use of the
individual or entity to which they are addressed. If you are not the
intended recipient or the person responsible for delivering the e-mail to
the intended recipient, you are hereby notified that any dissemination or
copying of this e-mail or any of its attachment(s) is strictly prohibited.
If you have received this e-mail in error, please immediately notify the
sending individual or entity by e-mail and permanently delete the original
e-mail and attachment(s) from your computer system. Thank you for your
cooperation.

[yegor256]

I think #47 will take a bit longer than I expected (see comments there). Will take a closer look tomorrow.

[tweist]

Hi Yegor,

I wonder if you have to couple issues 45 and 47 together and if it will
take long to make change and just allow http headers to pass through.
Thanks!

-Wei

On Mon, Mar 4, 2013 at 2:35 PM, Yegor Bugayenko notifications@github.comwrote:

I think #47 #47 will take a
bit longer than I expected (see comments there). Will take a closer look
tomorrow.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/43#issuecomment-14400271
.

Wei Tao, Ph.D.
TSI Biocomputing LLC
617-564-0934

This e-mail and any files transmitted with it may contain privileged and
confidential information and are intended solely for the use of the
individual or entity to which they are addressed. If you are not the
intended recipient or the person responsible for delivering the e-mail to
the intended recipient, you are hereby notified that any dissemination or
copying of this e-mail or any of its attachment(s) is strictly prohibited.
If you have received this e-mail in error, please immediately notify the
sending individual or entity by e-mail and permanently delete the original
e-mail and attachment(s) from your computer system. Thank you for your
cooperation.

[yegor256]

Yes, looks like #47 and this issue will be solved as soon as #45 is solved. I'm working on #45 now... will update you soon.

[tweist]

Hi Yegor,

Not trying to rush you... any good news on resolving issue 45?

Thanks!

-Wei

On Wed, Mar 6, 2013 at 9:35 AM, Yegor Bugayenko notifications@github.comwrote:

Yes, looks like #47 #47 and
this issue will be solved as soon as #45https://github.com/yegor256/s3auth/issues/45is solved. I'm working on
#45 #45 now... will update you
soon.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/43#issuecomment-14502055
.

Wei Tao, Ph.D.
TSI Biocomputing LLC
617-564-0934

This e-mail and any files transmitted with it may contain privileged and
confidential information and are intended solely for the use of the
individual or entity to which they are addressed. If you are not the
intended recipient or the person responsible for delivering the e-mail to
the intended recipient, you are hereby notified that any dissemination or
copying of this e-mail or any of its attachment(s) is strictly prohibited.
If you have received this e-mail in error, please immediately notify the
sending individual or entity by e-mail and permanently delete the original
e-mail and attachment(s) from your computer system. Thank you for your
cooperation.

[yegor256]

we're almost there. a massive refactoring happened in #48 and is going to be released in a few hours. right after it #45 will be solved. ETA is end of this week, the latest.

[tweist]

Look forward to it coming on line. Thanks!

-Wei

On Friday, March 15, 2013, Yegor Bugayenko wrote:

we're almost there. a massive refactoring happened in #48https://github.com/yegor256/s3auth/issues/48and is going to be released in a few hours. right after it
#45 #45 will be solved. ETA is
end of this week, the latest.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/43#issuecomment-14957728
.

Wei Tao, Ph.D.
TSI Biocomputing LLC
617-564-0934

This e-mail and any files transmitted with it may contain privileged and
confidential information and are intended solely for the use of the
individual or entity to which they are addressed. If you are not the
intended recipient or the person responsible for delivering the e-mail to
the intended recipient, you are hereby notified that any dissemination or
copying of this e-mail or any of its attachment(s) is strictly prohibited.
If you have received this e-mail in error, please immediately notify the
sending individual or entity by e-mail and permanently delete the original
e-mail and attachment(s) from your computer system. Thank you for your
cooperation.

[yegor256]

#45 is fixed and tested. Seems to be working fine. Please let me know whether it works for you now.

[tweist]

Hi Yegor,

It worked. Thanks so much!

-Wei

On Mon, Mar 18, 2013 at 2:14 PM, Yegor Bugayenko
notifications@github.comwrote:

#45 #45 is fixed and tested.
Seems to be working fine. Please let me know whether it works for you now.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/43#issuecomment-15071628
.

Wei Tao, Ph.D.
TSI Biocomputing LLC
617-564-0934

This e-mail and any files transmitted with it may contain privileged and
confidential information and are intended solely for the use of the
individual or entity to which they are addressed. If you are not the
intended recipient or the person responsible for delivering the e-mail to
the intended recipient, you are hereby notified that any dissemination or
copying of this e-mail or any of its attachment(s) is strictly prohibited.
If you have received this e-mail in error, please immediately notify the
sending individual or entity by e-mail and permanently delete the original
e-mail and attachment(s) from your computer system. Thank you for your
cooperation.

[yegor256]

You're welcome. Thanks for using our service! If you have any other problems - don't hesitate to create new issues. I'm not going to close this one since some of the comments you made above are not yet fixed (mostly related to the user manual and our UI).

[goojs]

Hi. I am an engineer who is reasonably familiar with classical Apache based web. I am trying to migrate to Amazon S3 based web with some protection. This looks like an excellent choice but my trial wasn't successful.

I would appreciate if you can answer a few questions about instructions in s3auth.com:

(1) Where in AWS could I do "Point a CNAME DNS record of your domain to relay.s3auth.com"?

(2) The example policy doesn't work (I replaced "bucket-1.example.com" --> "my_own_bucket_name"). Is the following policy fine (basically I allowed access to everybody)?
{
"Version":"2008-10-17",
"Statement":[{
"Sid":"AllowPublicRead",
"Effect":"Allow",
"Principal": {
"AWS": ""
},
"Action":["s3:GetObject"],
"Resource":["arn:aws:s3:::bucket_name/"
]
}
]
}

(3) "Register your domain, IAM key, and IAM secret" --> "AWS" key & secret key, not "IAM", right?

[yegor256]

@goojs thanks for your request, but would be better if you open a new issue with this text. I will answer your questions there. We're trying to separate problems and discussions by issues.

[yegor256]

@tweist could you please close this issue if there are no more open questions?

[yegor256]

@davvd please close the issue