Unable to download dom

[scherepn]

Attempting to download dom docset via thor docs:download results in:

(1/1) DOM FAILED (Errno::ECONNRESET: Connection reset by peer)
Done

Every other docset worked without issue, so the problem does not appear to be on my end.

[ojeytonwilliams]

thor docs:download --default
(1/6) DOM Events OK
(2/6) HTTP OK
(3/6) HTML OK
(4/6) JavaScript OK
(5/6) CSS OK
(6/6) DOM OK

That's what I see locally. I did notice that downloading docs isn't 100% reliable, though. Hopefully things should improve after I finish the migration to Cloudflare.

Could you give it another go and see if it's fixed itself?

[scherepn]

user@host /d/d/devdocs> thor docs:download --default
(1/6) DOM Events OK
(2/6) HTML OK
(3/6) HTTP OK
(4/6) CSS OK
(5/6) JavaScript OK
(6/6) DOM FAILED (Errno::ECONNRESET: Connection reset by peer)
Done

Run 5 times, one minute apart. It's always dom

[ojeytonwilliams]

I'm guessing it must be the CDN, because it's behaving differently for us. Just out of curiosity, if you run curl http://dl.devdocs.io/dom.tar.gz > dom.tar.gz directly, do you get the files?

[scherepn]

Looks like a CDN problem.

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
 59 12.4M   59 7562k    0     0  12.8M      0 --:--:-- --:--:-- --:--:-- 12.8M
curl: (56) Recv failure: Connection reset by peer

Gave it a few tries. Failed at 59% every time.

[ojeytonwilliams]

Right. Well, I'm figuring a few things out, but I'm making progress on the migration and I'll let you know when it's in a state that you can test this.

For now, is this a blocking problem? If it is, I'll just share that file another way.

[scherepn]

No, I keep local backups of these files.

This isn't the first time I've had this problem, but this is the first time it's happened to one of the default docsets and I figured it was only a matter of time before someone else hit the same problem.

[MasterEnoc]

@scherepn Devdocs has changed the infraestructure a little bit and this shouldn't be an issue anymore, try again to see if the issue has been solved

[scherepn]

I can confirm that dom works, but I think the move to the new CDN introduced some other issues.

5 docsets return a 403 HTTP error when attempting to download them

$ thor docs:download openjdk
(1/1) OpenJDK 15 FAILED (OpenURI::HTTPError: 403 Forbidden)
Done

I can open a new issue if this is too different of a problem.

[scherepn]

For reference, I downloaded every docset and these were the ones with this issue:

• OpenJDK 15
• Vue Router 3
• Vue Router 4
• VueJS 3
• Vuex 3

[simon04]

@ojeytonwilliams, can you please check the permissions of those files on s3://devdocs-downloads – thank you!

[richardkmichael]

Not sure if I should open another issue, but since this now seems to be about doc installs failing with 403, I'd like to mention the Erlang docs are also affected.

$ be thor docs:download erlang
(1/1) Erlang 23 FAILED (OpenURI::HTTPError: 403 Forbidden)
Done

Curl returns the full Access Denied S3 error:

$ curl 'https://downloads.devdocs.io/erlang~23.tar.gz'
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>ZD72CH25RCRHEKPZ</RequestId><HostId>cZHIgN8mI4OWE7zx0iqG3t0CzPDkkl/d0fPROfPEV034XkSN7drMwsuJg7MTbvmu1A3/ffj1GVs=</HostId></Error>

In the meantime, I downloaded Erlang docs and used thor docs:generate erlang.

Please let me know if I should provide more information or open a new issue.

[ojeytonwilliams]

Hey everyone, I've recreated them from the docs and uploaded them, so the problem should be resolved. @richardkmichael @scherepn could you confirm that everything's working for you?

If this happens again, please @ me and I'll sort it out.

[richardkmichael]

@ojeytonwilliams Thank you, the Erlang docs now install! Unfortunately there is another problem.

The installed Erlang docs do not show up in the devdocs UI, and it appears to be because the docs are nested:

public/docs/erlang~23/erlang~23/<FILES>.

I moved the child directory content up so that the docs are directly inside (as is the case with Ruby and others):

public/docs/erlang~23/<FILES>

.. and it works.

Perhaps the Erlang archive has been mistakenly been built with the parent directory, or the top-level directory should be removed during extraction? I'm afraid I don't know about devdocs packaging.

[ojeytonwilliams]

Thanks @richardkmichael, yep, you're exactly right. I messed up the packaging - I'll sort it out.

[ojeytonwilliams]

Right, that should be working @richardkmichael. Thanks for patience with this, it should all be fine now.

[richardkmichael]

@ojeytonwilliams Works now, thank you!

[ojeytonwilliams]

@richardkmichael glad to hear it! If the problem arises again, let me know, but I think this can be closed.