Proxy error, getting Boards and Libraries updates

[iggapres]

Hi,

Using Aduino IDE 1.8.1 Linux 64 bits
OS is Fedora 25:

# uname -a
Linux xxxx.xxx.xxx 4.9.3-200.fc25.x86_64 #1 SMP Fri Jan 13 01:01:13 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

Java is OpenJDK:

$ java -version
openjdk version "1.8.0_111"
OpenJDK Runtime Environment (build 1.8.0_111-b16)
OpenJDK 64-Bit Server VM (build 25.111-b16, mixed mode)

When trying to "Tools / Board / Boards Manager...", or "Sketch / Include Libraries / Manage Libraries...", I get the following error:

Using proxy HTTP @ /192.168.0.254:3128
 Error downloading http://downloads.arduino.cc/packages/package_index.json
java.lang.RuntimeException: java.lang.Exception: Error downloading http://downloads.arduino.cc/packages/package_index.json
	at cc.arduino.contributions.packages.ui.ContributionManagerUI.lambda$onUpdatePressed$0(ContributionManagerUI.java:151)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.Exception: Error downloading http://downloads.arduino.cc/packages/package_index.json
	at cc.arduino.contributions.DownloadableContributionsDownloader.download(DownloadableContributionsDownloader.java:131)
	at cc.arduino.contributions.packages.ContributionInstaller.download(ContributionInstaller.java:334)
	at cc.arduino.contributions.packages.ContributionInstaller.downloadIndexAndSignature(ContributionInstaller.java:309)
	at cc.arduino.contributions.packages.ContributionInstaller.updateIndex(ContributionInstaller.java:287)
	at cc.arduino.contributions.packages.ui.ContributionManagerUI.lambda$onUpdatePressed$0(ContributionManagerUI.java:147)
	... 1 more
Caused by: java.io.IOException: Received invalid http status code from server: 416
	at cc.arduino.utils.network.FileDownloader.downloadFile(FileDownloader.java:194)
	at cc.arduino.utils.network.FileDownloader.download(FileDownloader.java:128)
	at cc.arduino.contributions.DownloadableContributionsDownloader.download(DownloadableContributionsDownloader.java:129)
	... 5 more

No matter how I set my proxy (Arduino IDE, environment variables, gnome settings, ...) I have the same issue.
Of course, all other programs on my system work fine with this proxy.

I wrote a little Java program to test Java & Proxy together: it's working great.

[facchinm]

Hi @iggapres ,
if you are able to access http://downloads.arduino.cc/packages/package_index.json from a browser with the same proxy you are using in the IDE it's worth investigating. Otherwise, the 416 error (from previous reports) looks like a server-side problem, so it could have been a temporary downtime.

[iggapres]

Hi @facchinm ,

At the time of the bug report, I could download the file via a regular browser or even using curl and the same proxy.

I tried again today with no success on the Arduino IDE side, but still successfully with my Firefox.

I have access to my proxy (squid) logs:
This one is the request made by my Firefox (the file is already cached by the proxy)

==> access.log <==
1485349469.766     38 192.168.0.202 TCP_MISS/304 438 GET http://downloads.arduino.cc/packages/package_index.json - DIRECT/149.202.160.25 -

==> store.log <==
1485349469.766 RELEASE -1 FFFFFFFF 6ACC5934932DBA9C08AF1D8C51E0D694  304 1485350654 1483983451 1485350654 unknown -1/0 GET http://downloads.arduino.cc/packages/package_index.json

This one is the request made by the Arduino IDE

==> access.log <==
1485349500.144    113 192.168.0.202 TCP_MISS/416 498 GET http://downloads.arduino.cc/packages/package_index.json - DIRECT/149.202.160.25 text/html

==> store.log <==
1485349500.144 RELEASE -1 FFFFFFFF A874B12AC61A9D03DCBED6D8DE860CD9  416        -1        -1        -1 text/html -1/242 GET http://downloads.arduino.cc/packages/package_index.json

[oliver-gilmour-twi]

Hi @facchinm ,

I can confirm that I have the same issue as @iggapres . I'm using Windows 7, 64 bit, and I've installed everything. I've edited the preference file to have my proxy settings embedded -- and you can see that loaded when you restart the Arduino IDE. But I cannot connect to the repository to download the necessary packages (required for MKR 1000). However I can directly type in the link to Firefox and download the package contents - so I know its not an issue with our proxy nor server side issue.

I think its an implementation issue with the Arduino IDE, and it sounds like this issue existed from version 1.6.x, from what I saw on google. I've restarted, run as administrator (had to convince IT department!), included domain with backslash before username... still access error.

Coupled with the fact that I cannot seem to get the online editor to work either (plugin installed but somehow not recognised), I currently have no way to program the MKR1000; and I need to have some code written ASAP for a commercial feasibility I am working on.

I would appreciate it if you can address this with urgency.

[facchinm]

I just installed squid in my local environment (so I have no fancy enterprise networking/NAT setup) and it just works (once configured via the IDE Preference/Network tab). access.log reads

1486032696.970     67 127.0.0.1 TCP_MISS/206 30604 GET http://downloads.arduino.cc/packages/package_arduino.cc_linux_index.json - HIER_DIRECT/149.202.160.25 application/json

Using a Windows 10 host configured to connect through this same proxy and I get

1486033013.356    125 192.168.0.188 TCP_MISS/206 141295 GET http://downloads.arduino.cc/packages/package_index.json - HIER_DIRECT/149.202.160.25 application/json

I really can't help with your problem unless I'm able to replicate it, so I need some more debug information or a complete description of your proxy setup

[iggapres]

Hi,

The only thing I can add is that the proxy, and the computer it's installed on, is configured in a way that only requests through the proxy can reach the web.
It's kind of an applicative firewall. If you do not go through proxy, you cannot go outside and the proxy only allows requests on its 3128 and 443 ports.

I can't make tests anymore right now because of unusable computer.

Thank you anyways for trying.

[oliver-gilmour-twi]

Hi,

we still think it's an implementation issue with the IDE.

For starters the IDE doesn't seem to want to remember the HTTP proxy crendentials in preferences (resets on reloading of preferences - I think this may be related to the root cause of this issue), but you can type in the details in plain text (which may be a security issue but that's another story) in the configuration script (....\AppData\Local\Arduino15\Preferences.txt) and the IDE has the right settings on start up.

If I load up board manager, without trying to use the proxy, we get a bunch of errors pointing to connection time out. If we set the Arduino IDE to use HTTP proxy, with deliberately the wrong credentials the console outputs:

http://downloads.arduino.cc/packages/package_index.json file signature verification failed. File ignored.

and I get the same output even after using the correct settings.
This makes me believe that durring the access phase the IDE isn't passing the correct proxy credentials to the proxy server entred.

Again, using Firefox and the same proxy settings I can access both:
http://downloads.arduino.cc/packages/package_index.json and
https://downloads.arduino.cc/packages/package_index.json

and read the package index in plain text.

[per1234]

I notice #6628 also involved a 416 error.

[per1234]

I see from #7573 that some issues with proxy handling were fixed (I guess by #9023), so I'll go ahead and close this as fixed. If I'm mistaken and the issue is still occurring with the latest hourly build, please comment here and I'll reopen.