The URL is unreachable #20

ghost · 2016-11-22T16:19:48Z

Doesn't work to me. Got always "The URL is unreachable" no matter what URL I use - even with google.com. Any ideas?

mtakaki · 2016-11-23T01:57:11Z

I'm not sure why it's not working for you. How are you running it?

If you're running through docker, could you please try this and send me the output?

$ docker exec -it container_id bash
# curl -v google.com

Otherwise, could you please send me the command you're using to run it?

ghost · 2016-11-23T08:19:04Z

I'm not running docker but debian.
I start cachet-url-monitor with:

$ source bin/activate
$ pip install -r requirements.txt
$ python cachet_url_monitor/scheduler.py config.yml

The script is running but got no positive result.

mtakaki · 2016-11-24T03:40:47Z

I know this is a stupid question, but what do you get when you run this?

$ curl -v google.com

Also could you please send me the output of the following:

$ source bin/activate
$ pip install -r requirements.txt
$ python -c 'import requests; print requests.get("http://www.google.com").status_code'

ghost · 2016-11-24T09:01:48Z

Here we go:

$ curl -v google.com

<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>302 Moved</TITLE></HEAD><BODY>
<H1>302 Moved</H1>
The document has moved
<A HREF="http://www.google.de/?gfe_rd=cr&amp;ei=qak2WPmbLKGh8wfJ_rrACg">here</A>.
</BODY></HTML>

$ source bin/activate:

No output

$ pip install -r requirements.txt:

Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.11 in ./lib/python2.7/site-packages (from -r requirements.txt (line 1))
Requirement already satisfied (use --upgrade to upgrade): requests==2.9.1 in ./lib/python2.7/site-packages (from -r requirements.txt (line 2))
Requirement already satisfied (use --upgrade to upgrade): schedule==0.3.2 in ./lib/python2.7/site-packages (from -r requirements.txt (line 3))
Cleaning up...

$ python -c 'import requests; print requests.get("http://www.google.com").status_code':

200

mtakaki · 2016-11-25T03:35:50Z

What do you have in your configuration file? It should be working as you can clearly hit google.
This library actually uses sorta the same code you just used.

kirtiranjan · 2016-11-25T09:40:37Z

@mtakaki I am also getting the same issue"The URL is unreachable".Any solution .Also how can we pass parameters along with POST request ?

mtakaki · 2016-11-27T02:19:10Z

@kirtiranjan not sure why it's failing for both of you. I just tested it and it's working fine. Could you please share your configuration file?

kirtiranjan · 2016-11-28T04:48:28Z

@mtakaki here is my config file

endpoint:
url: http://www.google.com
method: GET
timeout: 0.05
expectation:
- type: HTTP_STATUS
status: 200
- type: LATENCY
threshold: 1
- type: REGEX
regex: '.(<body).'
cachet:
api_url: http://status.example.com/api/v1
token: #################
component_id: 1
#metric_id: 1
create_incident: true
frequency: 30

mtakaki · 2016-11-28T05:26:03Z

@kirtiranjan I would start with a simple configuration and you add more criteria as you clear them off. For example, your timeout is 50ms. Is it enough? The HTTP status that google returns is 302 Found and forwards it to another URL. So, I would remove the HTTP_STATUS criteria for now.

I'll update the sample configuration to something that works right away. It seems that it's confusing and that may be leading you guys to this problem.

kirtiranjan · 2016-11-28T06:54:59Z

Irrespective of what I do I am always getting "The URL is unreachable"

mtakaki · 2016-11-28T07:28:47Z

If you look at the source code, an ConnectionError is what causes the log message "The URL is unreachable." For whatever reason you're being able to reach google's page. Did you follow the same steps I've asked to @Hfroese and you got the same results?

waltrinehart · 2016-11-29T22:42:49Z

Howdy,

I am also getting this issue. I am running the container pulled from dockerhub. I also tried doing a docker compose from the github and am getting the same thing.

Config.yaml

  url: http://www.google.com
  method: GET
  timeout: 0.01
  expectation:
    - type: HTTP_STATUS
      status: 200
    - type: LATENCY
      threshold: 1
    - type: REGEX
      regex: '.*(<body).*'
cachet:
  api_url: http://status/api/v1
  token: #######################
  component_id: 1
  #metric_id: 1
  create_incident: true
frequency: 10

If I docker exec into the container and curl -v www.google.com:
curl -v www.google.com

Rebuilt URL to: www.google.com/
Hostname was NOT found in DNS cache
Trying 216.58.217.36...
Connected to www.google.com (216.58.217.36) port 80 (#0)

GET / HTTP/1.1
User-Agent: curl/7.38.0
Host: www.google.com
Accept: /

< HTTP/1.1 200 OK (there is some more junk after this that is just the web page)

On the plus side, the container is correctly talking to the Cachet API and changing the status when I get the error: WARNING [2016-11-29 22:43:34,997] cachet_url_monitor.configuration.Configuration - The URL is unreachable: GET http://www.google.com

mtakaki · 2016-11-30T06:14:50Z

I'm really not sure what's going on for you guys. I just pulled the docker image and it's working fine for me. Lemme do more tests here.

waltrinehart · 2016-11-30T15:57:59Z

Further information that might help, I have tested this both inside and outside of my companies VPN. Same results both ways.

Lemme know if you need anything else, this seems like a pretty rad tool that'll save me a whole lot of time.

mtakaki · 2016-12-01T16:54:54Z

Thanks, @waltrinehart! Could you please try removing the REGEX expectation? Also, could you please try going with a super simple configuration and then adding more things? For example, just HTTP status, then you add latency, etc.

mtakaki · 2016-12-01T16:55:24Z

This should help isolating what exactly is failing for you.

waltrinehart · 2016-12-01T17:38:55Z

Here is the output when I just do HTTP_Status:
sh-3.2# docker run --rm -it -v /Users/wrinehart/cachet-url-monitor/my_config:/usr/src/app/config/ mtakaki/cachet-url-monitor
INFO [2016-12-01 17:30:22,319] cachet_url_monitor.configuration.Configuration - Monitoring URL: GET http://www.google.com
INFO [2016-12-01 17:30:22,320] cachet_url_monitor.configuration.Configuration - Registered expectation: 'HTTP status: 200'
INFO [2016-12-01 17:30:22,321] cachet_url_monitor.scheduler.Scheduler - Starting monitor agent...
WARNING [2016-12-01 17:30:37,378] cachet_url_monitor.configuration.Configuration - The URL is unreachable: GET http://www.google.com
INFO [2016-12-01 17:30:37,460] cachet_url_monitor.configuration.Configuration - Component update: status [3]

Here is the when I add latency:
sh-3.2# docker run --rm -it -v /Users/wrinehart/cachet-url-monitor/my_config:/usr/src/app/config/ mtakaki/cachet-url-monitor
INFO [2016-12-01 17:28:22,745] cachet_url_monitor.configuration.Configuration - Monitoring URL: GET http://www.google.com
INFO [2016-12-01 17:28:22,746] cachet_url_monitor.configuration.Configuration - Registered expectation: 'HTTP status: 200'
INFO [2016-12-01 17:28:22,747] cachet_url_monitor.configuration.Configuration - Registered expectation: 'Latency threshold: 1.0000'
INFO [2016-12-01 17:28:22,747] cachet_url_monitor.scheduler.Scheduler - Starting monitor agent...
WARNING [2016-12-01 17:28:37,832] cachet_url_monitor.configuration.Configuration - The URL is unreachable: GET http://www.google.com
INFO [2016-12-01 17:28:37,922] cachet_url_monitor.configuration.Configuration - Component update: status [3]

waltrinehart · 2016-12-01T17:40:40Z

Maybe it might be a good idea to copy/pasta your config you are using? Maybe something is awry with the format of the config file I have.

mtakaki · 2016-12-02T04:13:52Z

Ok, my config looks like this:

endpoint:
  url: http://www.google.com
  method: GET
  timeout: 10
  expectation:
    - type: HTTP_STATUS
      status: 200
    - type: LATENCY
      threshold: 10
cachet:
  api_url: http://localhost/api/v1
  token: my_token
  component_id: 1
  create_incident: true
frequency: 15

And this is my output:

$ docker run --rm -it -v "$PWD":/usr/src/app/config/ mtakaki/cachet-url-monitor
     INFO [2016-12-01 17:39:18,435] cachet_url_monitor.configuration.Configuration - Monitoring URL: GET http://www.google.com
     INFO [2016-12-01 17:39:18,437] cachet_url_monitor.configuration.Configuration - Registered expectation: 'HTTP status: 200'
     INFO [2016-12-01 17:39:18,437] cachet_url_monitor.configuration.Configuration - Registered expectation: 'Latency threshold: 10.0000'
     INFO [2016-12-01 17:39:18,439] cachet_url_monitor.scheduler.Scheduler - Starting monitor agent...
     INFO [2016-12-01 17:39:34,071] cachet_url_monitor.configuration.Configuration - Component update: status [1]

waltrinehart · 2016-12-02T16:50:24Z

That worked! There must be a discrepancy between the config included and the config you posted here. Maybe a formatting issue or a variable that it just doesn't like.

Is there anyway to get a list of the possible Expectations the config will accept?

mtakaki · 2016-12-02T16:55:47Z

How are you creating your config file? Are you using the config.yml included here or are you copying/pasting it from the README?

The possible expectations are:

HTTP_STATUS
LATENCY
REGEX

waltrinehart · 2016-12-02T16:56:32Z

The one I got working was the copy you posted above.

mtakaki · 2016-12-02T16:57:02Z

Sorry, I meant the original one, the one that didn't work.

waltrinehart · 2016-12-02T16:58:07Z

Ah, I also posted that one in my first post. I -believe- I got that config from the README, but I also could have gotten it from the sample included in the git repo.

mtakaki · 2016-12-02T17:00:56Z

I noticed the indentation was incorrect, but I thought it was just a pasting issue. Would you mind editing it and putting it into a code block? You will need to wrap it with ```, on the top and on the bottom.

mtakaki · 2016-12-02T17:02:38Z

I just created #21 to have better validation and to list the available expectations.

waltrinehart · 2016-12-02T17:43:27Z

endpoint:
  url: http://www.google.com
  method: GET
  timeout: 0.01
  expectation:
    - type: HTTP_STATUS
      status: 200
#    - type: LATENCY
#      threshold: 1
#    - type: REGEX
#      regex: '.*(<body).*'
cachet:
  api_url: http://status.net/api/v1
  token: token
  component_id: 1
  #metric_id: 1
  create_incident: true
frequency: 10

mtakaki · 2016-12-02T17:44:44Z

I can't see anything wrong with it. Could you please try increasing the timeout? Maybe you're not getting a response in 10ms.

waltrinehart · 2016-12-02T17:49:26Z

I took the above config and adjusted the timeout to 10 (like in your config). It is working correctly.

0.01 (which is in the sample) must be far too low.

I swear at one point when I was dicking with this I increased the timeout and got "the url is unreachable", I must not have adjusted it enough.

mtakaki · 2016-12-03T02:36:03Z

Well, the interesting thing is that you should get this message, instead:

  WARNING [2016-12-02 17:32:55,637] cachet_url_monitor.configuration.Configuration - Request timed out
     INFO [2016-12-02 17:32:55,815] cachet_url_monitor.configuration.Configuration - Component update: status [3]

So it still doesn't explain what's happening...

mtakaki · 2016-12-05T06:37:16Z

If this is not happening anymore for you guys @Hfroese @kirtiranjan @waltrinehart, I'll close this issue. But I'd like some action items to prevent it from happening to other people. Still not sure what's going on, to be honest.

shaun-ba · 2016-12-27T16:08:02Z

@mtakaki I also had this issue, then had a "Regex did not match anything in the body".

I've since removed the regex, upp'ed the timeout and latency to 10 and now i don't receive any updates to the monitor, it's just stuck at "starting monitor agent" with no other logs for 5minutes. I'll assume this code isn't stable enough to use :(

mtakaki · 2016-12-27T18:04:16Z

What's in your frequency, @shaun-ba ? This code is stable enough to me, but nonetheless you should use it at your own risk.

shaun-ba · 2016-12-27T18:05:01Z

30seconds, i've not managed to get it to work at all unfortunately, very strange and different results on each run as other above reported

moodhairboy · 2017-01-15T06:54:58Z

after copying the config.yml from this thread, I no longer get the site unreachable error. Cool thanks for the code. Much appreciation from Austin, Tx

mtakaki · 2017-01-15T23:33:50Z

Glad to hear it, @moodhairboy!

Exposing the configuration to avoid the current parsing problems. #20

mtakaki · 2017-01-20T09:25:56Z

I've changed the code to print out the parsed configuration yaml. @shaun-ba would you mind trying it again with the latest code? And could you please post here the output?

mtakaki · 2017-02-11T19:16:26Z

I'm closing this issue as there has not been any question popping up anymore. If anyone has still any question, please feel free to reopen this issue.

mtakaki closed this as completed Dec 8, 2016

mtakaki added the question label Dec 8, 2016

mtakaki self-assigned this Dec 8, 2016

mtakaki reopened this Dec 27, 2016

mtakaki added a commit that referenced this issue Jan 20, 2017

Exposing the configuration to avoid the current parsing problems. #20

c59126f

mtakaki added a commit that referenced this issue Jan 20, 2017

Merge pull request #25 from mtakaki/mtakaki_expose_configuration

a147add

Exposing the configuration to avoid the current parsing problems. #20

mtakaki closed this as completed Feb 11, 2017

The URL is unreachable #20

The URL is unreachable #20

Comments

ghost commented Nov 22, 2016

mtakaki commented Nov 23, 2016

ghost commented Nov 23, 2016

mtakaki commented Nov 24, 2016

ghost commented Nov 24, 2016

mtakaki commented Nov 25, 2016

kirtiranjan commented Nov 25, 2016 • edited Loading

mtakaki commented Nov 27, 2016

kirtiranjan commented Nov 28, 2016 • edited Loading

mtakaki commented Nov 28, 2016

kirtiranjan commented Nov 28, 2016

mtakaki commented Nov 28, 2016

waltrinehart commented Nov 29, 2016 • edited Loading

mtakaki commented Nov 30, 2016

waltrinehart commented Nov 30, 2016 • edited Loading

mtakaki commented Dec 1, 2016

mtakaki commented Dec 1, 2016

waltrinehart commented Dec 1, 2016

waltrinehart commented Dec 1, 2016

mtakaki commented Dec 2, 2016

waltrinehart commented Dec 2, 2016

mtakaki commented Dec 2, 2016

waltrinehart commented Dec 2, 2016

mtakaki commented Dec 2, 2016

waltrinehart commented Dec 2, 2016 • edited Loading

mtakaki commented Dec 2, 2016

mtakaki commented Dec 2, 2016

waltrinehart commented Dec 2, 2016 • edited Loading

mtakaki commented Dec 2, 2016

waltrinehart commented Dec 2, 2016 • edited Loading

mtakaki commented Dec 3, 2016 • edited Loading

mtakaki commented Dec 5, 2016

shaun-ba commented Dec 27, 2016

mtakaki commented Dec 27, 2016

shaun-ba commented Dec 27, 2016

moodhairboy commented Jan 15, 2017

mtakaki commented Jan 15, 2017

mtakaki commented Jan 20, 2017 • edited Loading

mtakaki commented Feb 11, 2017

kirtiranjan commented Nov 25, 2016 •

edited

Loading

kirtiranjan commented Nov 28, 2016 •

edited

Loading

waltrinehart commented Nov 29, 2016 •

edited

Loading

waltrinehart commented Nov 30, 2016 •

edited

Loading

waltrinehart commented Dec 2, 2016 •

edited

Loading

waltrinehart commented Dec 2, 2016 •

edited

Loading

waltrinehart commented Dec 2, 2016 •

edited

Loading

mtakaki commented Dec 3, 2016 •

edited

Loading

mtakaki commented Jan 20, 2017 •

edited

Loading