Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Travis + apt-get install 4store / 4s-httpd returns with httpd.c:1645 couldn't connect "..." #110

Closed
mwjames opened this issue Jun 24, 2014 · 22 comments

Comments

Projects
None yet
5 participants
@mwjames
Copy link

commented Jun 24, 2014

Using Travis-CI (which is integrated with GitHub) will successfully install the application apt-get install 4store Setting up 4store (1.1.4-1) [0] but will fail to load the KB after executing:

4s-backend-setup db
4store[3130]: backend-setup.c:185 erased files for KB db
4store[3130]: backend-setup.c:310 created RDF metadata for KB db
4s-backend db
4s-httpd -D -p 8000 db
4store[3140]: httpd.c:1634 4store HTTP daemon v1.1.4 started on port 8000
4store[3141]: httpd.c:1645 couldn't connect to “db”
4store[3140]: httpd.c:1694 child 3141 exited with return code 3
4store[3140]:  1: 4s-httpd() [0x406cd2]
4store[3140]:  2: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7fa70722176d]

Running ps auwwx | grep 4s- will only yield

travis    3138  0.0  0.0   9336   928 pts/0    S+   00:53   0:00 grep 4s-
travis    3139  0.0  0.0 107696  1504 ?        S    00:53   0:00 4s-backend db

[0, 1] shows details for the installation and its failure.

[0] https://travis-ci.org/mwjames/SemanticMediaWiki/builds/28274105
[1] https://s3.amazonaws.com/archive.travis-ci.org/jobs/28274106/log.txt

@msalvadores

This comment has been minimized.

Copy link
Collaborator

commented Jun 24, 2014

This is often a firewall issue. Please, try shutting down the firewall (iptables) temporally to see if it works.

@mwjames

This comment has been minimized.

Copy link
Author

commented Jun 24, 2014

try shutting down the firewall (iptables) temporally

sudo iptables -F
sudo iptables -X
sudo iptables -t nat -F
sudo iptables -t nat -X
sudo iptables -t mangle -F
sudo iptables -t mangle -X
sudo iptables -P INPUT ACCEPT
sudo iptables -P FORWARD ACCEPT
sudo iptables -P OUTPUT ACCEPT

Added above commands but the result remains unchanged with couldn't connect to “db” [0].

[0] https://s3.amazonaws.com/archive.travis-ci.org/jobs/28276750/log.txt

@swh

This comment has been minimized.

Copy link
Collaborator

commented Jul 24, 2014

I'm not really familiar with the environment, does it run 4s-backend, then 4s-httpd immediately? Maybe adding a sleep of say 4 seconds after the backend command will help? I'm not sure offhand how long 4s-http will retry for.

@mwjames

This comment has been minimized.

Copy link
Author

commented Jul 24, 2014

Doing an additional sleep 5 [0] does unfortunately not solve the problem.

Setting up 4store (1.1.4-1) ...

+sudo iptables -F
+4s-backend-setup db

4store[3199]: backend-setup.c:185 erased files for KB db
4store[3199]: backend-setup.c:310 created RDF metadata for KB db

+4s-backend db
+sleep 5
+4s-httpd -D -p 8088 db

4store[3215]: httpd.c:1634 4store HTTP daemon v1.1.4 started on port 8088
4store[3216]: httpd.c:1645 couldn't connect to “db”
4store[3215]: httpd.c:1694 child 3216 exited with return code 3
4store[3215]:  1: 4s-httpd() [0x406cd2]
4store[3215]:  2: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7fdeab63176d]
4store[3215]:  3: 4s-httpd() [0x4072d5]
4store[3218]: httpd.c:1645 couldn't connect to “db”

[0] https://s3.amazonaws.com/archive.travis-ci.org/jobs/30780186/log.txt

@swh

This comment has been minimized.

Copy link
Collaborator

commented Jul 25, 2014

OK, thanks - is there anything in /var/log/messages?

Looking at the error, it looks more like something went bang than a networking / firewall issue.

@szarnyasg

This comment has been minimized.

Copy link

commented Aug 29, 2014

Hello, I also ran into a similar issue, even though I am not using 4s-httpd.

The repository is located at https://github.com/szarnyasg/travis-4s-test, while the Travis build results are at https://travis-ci.org/szarnyasg/travis-4s-test.

This setups only aims at doing a single import from a Turtle file (with 4s-import) followed by a query (with 4s-query). I disabled IPv6 as it is a common source of problems for 4store.

I created a VirtualBox VM based on the Travis cookbook (https://github.com/travis-ci/travis-cookbooks) on my local computer and it managed to run the operations fine.

Any ideas on this?

Thanks,
Gabor

@swh

This comment has been minimized.

Copy link
Collaborator

commented Aug 30, 2014

Can you paste the error log (as above) and the relevant contents of /var/log/messages?

@szarnyasg

This comment has been minimized.

Copy link

commented Aug 30, 2014

The latest job is at [1], /var/log/syslog is printed at [2].

+4s-backend-setup -v my_cluster
4store[3008]: backend-setup.c:186 erased files for KB my_cluster
Creating data directoryCreate segment directoriesCreating data directoryCreate segment directoriesWriting metadata.
4store[3008]: backend-setup.c:337 created RDF metadata for KB my_cluster
+4s-backend my_cluster
+sleep 5
+ps auxw
+grep '4s-backen[d]'
travis    3016  0.0  0.0 109840  1540 ?        S    12:20   0:00 4s-backend my_cluster
+ls -la /var/lib/4store/my_cluster
total 8
drwxr-xr-x 4 travis travis 120 Aug 30 12:20 .
drwxr-xr-x 3 travis travis  60 Aug 30 12:20 ..
drwxr-xr-x 3 travis travis 240 Aug 30 12:20 0000
drwxr-xr-x 3 travis travis 240 Aug 30 12:20 0001
-rw------- 1 travis travis 835 Aug 30 12:20 metadata.nt
-rw-r--r-- 1 travis travis   9 Aug 30 12:20 runtime.info
+4s-import my_cluster railway-xform-1.ttl --format turtle --verbose
4store[3020]: 4s-import.c:171 couldn't connect to “my_cluster”
+4s-query my_cluster 'SELECT ?x ?y ?z WHERE { ?x ?y ?z }' -f text --verbose
+head -n 20
4store[3021]: 4s-query.c:183 couldn't connect to “my_cluster”

[1] https://travis-ci.org/szarnyasg/travis-4s-test
[2] https://api.travis-ci.org/jobs/33973567/log.txt

@swh

This comment has been minimized.

Copy link
Collaborator

commented Aug 30, 2014

OK, so neither the import nor the query were able to connect.

Try running the import with -v -v to see why it couldn't connect, it looks like the backend managed to create the socket ok.

Also you could try telnetting to whatever port your 4s-backnd comes up on (it was 6734 on that last run, but it can be other numbers) to rule out firewalls or selinux or anything like that.

@swh

This comment has been minimized.

Copy link
Collaborator

commented Aug 30, 2014

Oh, just seen that you had one level of --verbose, so you should have seen something... is it possible that stderr is being discarded?

@szarnyasg

This comment has been minimized.

Copy link

commented Aug 30, 2014

@swh I added the -v -v flags and redirected stderr to stdout (I don't think it was discarded, but tried this anyway). Unfortunately, no difference in the output.

However, telnet is able to connect: https://travis-ci.org/szarnyasg/travis-4s-test/builds/33976445

@swh

This comment has been minimized.

Copy link
Collaborator

commented Aug 30, 2014

OK, if telnet is able to connect, it's probably a Multicast issue.

Can you try running avahi-browse _4store._tcp - that should show 4store's multicast adverts.

@szarnyasg

This comment has been minimized.

Copy link

commented Aug 30, 2014

@swh

This comment has been minimized.

Copy link
Collaborator

commented Aug 30, 2014

OK, so check the you have an MDNS / Multicast server running, I can't remember exactly what it's called on linux, maybe Avahi Server?

@swh

This comment has been minimized.

Copy link
Collaborator

commented Aug 30, 2014

There's tools in the avahi suite that let you test multicast adverts - avahi-browse to read, can't remember how you write them offhand.

@szarnyasg

This comment has been minimized.

Copy link

commented Aug 30, 2014

Well, avahi-daemon is running, but avahi-browse --all shows no results.

+ps auxw
+grep -i 'avah[i]'
avahi     2916  0.0  0.0  32244  1740 ?        S    14:14   0:00 avahi-daemon: running [testing-worker-linux-6-1-30039-linux-3-33978532.local]
avahi     2917  0.0  0.0  32128   556 ?        S    14:14   0:00 avahi-daemon: chroot helper
+service avahi-daemon status
avahi-daemon start/running, process 2916
+avahi-browse --all --resolve --terminate
@swh

This comment has been minimized.

Copy link
Collaborator

commented Aug 30, 2014

Try deliberately creating something with avahi-publish, if that doesn't show up then something is broken in your avahi / fns / multicast stack. e.g. http://wiki.ros.org/zeroconf_avahi/Tutorials/Debugging%20with%20Avahi%20Command%20Line%20Tools

@szarnyasg

This comment has been minimized.

Copy link

commented Aug 30, 2014

Your guess is correct: the stack is broken somewhere.

+sleep 5
+avahi-publish -s DudeMaster _ros-master._tcp 8889
Established under name 'DudeMaster'
+avahi-browse --all --terminate
+grep -i dude

(Interestingly, the sleep and the avahi-publish commands are shown in reverse order -- https://github.com/szarnyasg/travis-4s-test/blob/48e341aedbc3cee1d7a03559d12929a45ec8e3c3/run.sh).

Thanks for the help, I will file an issue to the travis-ci repository.

@swh

This comment has been minimized.

Copy link
Collaborator

commented Aug 30, 2014

OK, great, hopefully it's a simple issue...

One thing I never checked - I think some cloud systems (e.g. AWS) turn off multicast in the kernel - if that's the problem there's an alternative discovery mechanism which doesn't depend on multicast to discover clusters (see man 4s-boss).

@ianmillard

This comment has been minimized.

Copy link

commented Aug 30, 2014

Since @davechallis did his excellent work on 4s-boss and 4s-admin I've stayed as far away as possible from avahi/MDNS (in both clustered and single node deployments)

I've found it to be much more reliable than avahi-daemon, and easier too

Just create /etc/4store.conf eg

[4s-boss]
discovery = sole
nodes = localhost
#nodes=node1;node2;node3;node4

[my_repository]
port = 7890

and then your command sequence would be

4s-boss
4s-admin create-store --segments=2 my_repository
#4s-admin list-stores
4s-admin start-stores my_repository
4s-import -v my_repository foo.ttl
#4s-httpd my_repository
4s-query my_repository "SELECT ..."

(just seen Steve's reply, I'll shut up now...)

@szarnyasg

This comment has been minimized.

Copy link

commented Aug 30, 2014

@swh, @ianmillard thank you both for your help, I finally managed to get it working: https://travis-ci.org/szarnyasg/travis-4s-test/builds/33986661.

The final trick was that I had to change nodes = localhost to nodes = 127.0.0.1 to enforce IPv4. Without that, localhost was resolved to ::1 based on the /etc/hosts file.

I posted an issue to travis-ci about the problems with Avahi: travis-ci/travis-ci#2739.

@mwjames

This comment has been minimized.

Copy link
Author

commented Jun 17, 2015

After some trial and error setup the following [0] allowed us to run 4store on Travis-CI.

[0] https://github.com/mwjames/SemanticMediaWiki/blob/4store/build/travis/install-services.sh#L67-L115

@mwjames mwjames closed this Jun 17, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.