Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

http port not bound #215

Merged
merged 2 commits into from
Jan 30, 2014
Merged

http port not bound #215

merged 2 commits into from
Jan 30, 2014

Conversation

jvshahid
Copy link
Contributor

That's the third time I got this error which but I have no idea how it was caused and how to solve it (since this is a test project for now I just removed the data/raft folders).

When I start the server I get this:

[01/30/14 13:44:55] [INFO] Redirectoring logging to stdout
[01/30/14 13:44:55] [INFO] Starting Influx Server...
[01/30/14 13:44:55] [INFO]
+---------------------------------------------+
|  _____        __ _            _____  ____   |
| |_   _|      / _| |          |  __ \|  _ \  |
|   | |  _ __ | |_| |_   ___  _| |  | | |_) | |
|   | | | '_ \|  _| | | | \ \/ / |  | |  _ <  |
|  _| |_| | | | | | | |_| |>  <| |__| | |_) | |
| |_____|_| |_|_| |_|\__,_/_/\_\_____/|____/  |
+---------------------------------------------+


[01/30/14 13:44:55] [INFO] Opening database at /apps/monitoring/data/metrics
[01/30/14 13:44:55] [INFO] ProtobufServer listening on 127.0.0.1:8099
[01/30/14 13:44:55] [INFO] Initializing Raft HTTP server
[01/30/14 13:44:55] [INFO] Raft Server Listening at http://monitoring-dev:8090
[01/30/14 13:44:55] [INFO] Initializing Raft Server: /apps/monitoring/data/raft 8090
[01/30/14 13:44:55] [INFO] ClusterServer: 1 connecting to: monitoring-dev:8099
[01/30/14 13:44:55] [INFO] connected to monitoring-dev:8099
[01/30/14 13:44:55] [INFO] ProtobufServer: client connected: 127.0.0.1:48726
[01/30/14 13:44:55] [INFO] Recovered from log
[01/30/14 13:44:55] [INFO] Waiting for local server to be added
[01/30/14 13:44:56] [INFO] (raft:d62d2aa) Selected as leader. Starting leader loop.

I think the notable part here is "Recovered from log" but when it starts like this it doe snot appear to do anything: no cpu is used, and the http port is not bound:

$sudo lsof -nPi | grep influxdb
influxdb  22227 monitoring   15u  IPv4 11668631      0t0  TCP 127.0.0.1:8090 (LISTEN)
influxdb  22227 monitoring   16u  IPv4 11666947      0t0  TCP 127.0.0.1:8099 (LISTEN)
influxdb  22227 monitoring   20u  IPv4 11666952      0t0  TCP 127.0.0.1:48727->127.0.0.1:8099 (ESTABLISHED)
influxdb  22227 monitoring   21u  IPv4 11666953      0t0  TCP 127.0.0.1:8099->127.0.0.1:48727 (ESTABLISHED)

Here is my config file:

[logging]
level  = "info"
file   = "stdout"

# Configure the admin server
# [admin]
# port   = 8083
# assets = "./admin"

# Configure the http api
[api]
port = 8086

# Raft configuration
[raft]
port = 8090
dir  = "/apps/monitoring/data/raft"

[storage]
dir = "/apps/monitoring/data/metrics"

[cluster]

protobuf_port = 8099

I am running influxdb 0.4.0 on Linux.
any idea what happenned and how can I restore the service when in this state ?

@schmurfy
Copy link
Contributor Author

For reference here is the log after I moved away to data folder and restarted influxdb:

[01/30/14 13:53:38] [INFO] Redirectoring logging to stdout
[01/30/14 13:53:38] [INFO] Starting Influx Server...
[01/30/14 13:53:38] [INFO]
+---------------------------------------------+
|  _____        __ _            _____  ____   |
| |_   _|      / _| |          |  __ \|  _ \  |
|   | |  _ __ | |_| |_   ___  _| |  | | |_) | |
|   | | | '_ \|  _| | | | \ \/ / |  | |  _ <  |
|  _| |_| | | | | | | |_| |>  <| |__| | |_) | |
| |_____|_| |_|_| |_|\__,_/_/\_\_____/|____/  |
+---------------------------------------------+


[01/30/14 13:53:38] [INFO] Opening database at /apps/monitoring/data/metrics
[01/30/14 13:53:38] [INFO] Using /dev/random to initialize the raft server name
[01/30/14 13:53:38] [INFO] Setting raft name to bf28862
[01/30/14 13:53:38] [INFO] Initializing Raft HTTP server
[01/30/14 13:53:38] [INFO] ProtobufServer listening on 127.0.0.1:8099
[01/30/14 13:53:38] [INFO] Raft Server Listening at http://monitoring-dev:8090
[01/30/14 13:53:38] [INFO] Initializing Raft Server: /apps/monitoring/data/raft 8090
[01/30/14 13:53:38] [INFO] Starting as new Raft leader...
[01/30/14 13:53:38] [INFO] (raft:bf28862) Selected as leader. Starting leader loop.
[01/30/14 13:53:38] [INFO] Added server to cluster config: 1, http://monitoring-dev:8090, monitoring-dev:8099
[01/30/14 13:53:38] [INFO] Added the local server
[01/30/14 13:53:38] [INFO] Waiting for local server to be added
[01/30/14 13:53:38] [INFO] Local server added. Connecting to other nodes in the cluster
[01/30/14 13:53:38] [INFO] Starting admin interface on port 0
[01/30/14 13:53:38] [INFO] Starting Http Api server on port 8086

@pauldix
Copy link
Member

pauldix commented Jan 30, 2014

Ok, I think I can reproduce this problem. It might be cause by the Raft log
compaction. Back later today with more details.

On Thu, Jan 30, 2014 at 8:55 AM, Julien Ammous notifications@github.comwrote:

For reference here is the log after I moved away to data folder and
restarted influxdb:

[01/30/14 13:53:38] [INFO] Redirectoring logging to stdout
[01/30/14 13:53:38] [INFO] Starting Influx Server...
[01/30/14 13:53:38] [INFO]
+---------------------------------------------+
| _____ __ _ _____ ____ |
| |_ | / | | | __ | _ \ |
| | | _ __ | || | ___ | | | | |) | |
| | | | '_ | | | | | \ / / | | | _ < |
| | || | | | | | | || |> <| || | |_) | |
| ||| ||| ||,//_____/|____/ |
+---------------------------------------------+

[01/30/14 13:53:38] [INFO] Opening database at /apps/monitoring/data/metrics
[01/30/14 13:53:38] [INFO] Using /dev/random to initialize the raft server name
[01/30/14 13:53:38] [INFO] Setting raft name to bf28862
[01/30/14 13:53:38] [INFO] Initializing Raft HTTP server
[01/30/14 13:53:38] [INFO] ProtobufServer listening on 127.0.0.1:8099
[01/30/14 13:53:38] [INFO] Raft Server Listening at http://monitoring-dev:8090
[01/30/14 13:53:38] [INFO] Initializing Raft Server: /apps/monitoring/data/raft 8090
[01/30/14 13:53:38] [INFO] Starting as new Raft leader...
[01/30/14 13:53:38] INFO Selected as leader. Starting leader loop.
[01/30/14 13:53:38] [INFO] Added server to cluster config: 1, http://monitoring-dev:8090, monitoring-dev:8099
[01/30/14 13:53:38] [INFO] Added the local server
[01/30/14 13:53:38] [INFO] Waiting for local server to be added
[01/30/14 13:53:38] [INFO] Local server added. Connecting to other nodes in the cluster
[01/30/14 13:53:38] [INFO] Starting admin interface on port 0
[01/30/14 13:53:38] [INFO] Starting Http Api server on port 8086

Reply to this email directly or view it on GitHubhttps://github.com//issues/215#issuecomment-33689271
.

@schmurfy
Copy link
Contributor Author

I will post more informations if I found some but so far I have no idea how I did this.

@jvshahid jvshahid mentioned this pull request Jan 30, 2014
Closed
@jvshahid
make the integration tests less noisy
3d0b527
@jvshahid
Fix #215. Fix hangs on startup after raft compaction
25cb03e 
Because raft compaction compact the entire log in one state, commands in
the log aren't replayed on startup. Instead the entire state is loaded
this prevented the AddPotentialServer from being called which in turn
didn't put anything in the channel
pauldix added a commit that referenced this pull request Jan 30, 2014
@pauldix
Merge pull request #215 from influxdb/fix-215-server-fails-restart-af…
dd547a7 
…ter-raft-compaction

Fix error with server restart after Raft log compaction
@pauldix pauldix merged commit dd547a7 into master Jan 30, 2014
@jvshahid jvshahid deleted the fix-215-server-fails-restart-after-raft-compaction branch January 30, 2014 21:22
@pauldix
Copy link
Member

pauldix commented Jan 30, 2014

@schmurfy ok, it's fixed in 0.4.1. Source tarballs and linux packages have been pushed at http://influxdb.org/download

@schmurfy
Copy link
Contributor Author

awesome ! thanks :)

jvshahid pushed a commit that referenced this pull request Aug 12, 2014
@xiang90
Merge pull request #215 from xiangli-cmu/fix_channel_race
3b1dea1 
fix(server.go) check server state before send event to channel
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@jvshahid @schmurfy @pauldix