output_mysql not sending data

[NitroCao]

In the database, only table sessions' has many data, the other tables such asauth,clients,input,ttyloghave no data. Andlog/cowrie.logandlog/cowrie.jsonhave become binary files... What's the matter?... Here are the screenshots: ![zj1 qzbl r0g 4lj a qb](https://cloud.githubusercontent.com/assets/17915615/19677173/9e3e8ea6-9aca-11e6-9c18-53d88e7605e2.png) ![ut1 5_7k 7qrggv4b7pdith](https://cloud.githubusercontent.com/assets/17915615/19677209/c6d128e2-9aca-11e6-9e2e-e5f2b20fadf8.png) ![9tl8 i r pn80bh 1j olr](https://cloud.githubusercontent.com/assets/17915615/19677228/dbc4b962-9aca-11e6-8fec-573ca4a6df38.png) ![ceppb dwca w r4 if 4 d](https://cloud.githubusercontent.com/assets/17915615/19677240/eec24a34-9aca-11e6-9c1f-5eba6fedd259.png) ![cllo45bi hr n14ysdsw_vi](https://cloud.githubusercontent.com/assets/17915615/19677266/0d140bda-9acb-11e6-8503-ebc1fc94d3a2.png) ![gsz g 6 04mj5vp0u n](https://cloud.githubusercontent.com/assets/17915615/19677314/4be3439e-9acb-11e6-8c03-d4bd6132f799.png) In tablesensors`, the ip is my server ip...

[micheloosterhof]

The server IP in the sensor table is correct behaviour.
Sessions has data, this means that at least some data is coming in, and the basic config is good.

Do you get errors in cowrie.log?
Do you use dblog_mysql or output_mysql? There are two ways to write to MySQL.

[NitroCao]

It seemed to have no errors... Here is one part of cowire.log:

And this is my config of mysql:

[micheloosterhof]

Try to use output_mysql instead of database_mysql.

[NitroCao]

If I use output_mysql, it will be nothing in mysql, there was no errors in cowrie.log either...

[erdoking]

Same here!

output_mysql
-> Nothing

database_mysql
-> sessions
-> sensors

CentOS Linux release 7.2.1511
Trying last git commit.
No error in log ...

[NitroCao]

@erdoking Sorry... Could you please describe more detail? How should I configure in cowrie.cfg?...

[erdoking]

@jaycecao: i think we have the same problem! logging to mysql don't work for me too ... (my config looks like your one !)

@micheloosterhof: I started some test's... your commits:

45022b7 ["add service_identity to debian package based install"] -> SQL-Logging works fine with
414569c ["move loggers to tac file"] -> won't start (AttributeError: 'CowrieServiceMaker' object has no attribute 'cfg') -> unrelevant/fixed bug!
414569c ["wip"] -> stop writing to sql

you can try with:
git checkout 45022b7

The problem seems to come with 414569c ...

[NitroCao]

@erdoking So did you solve the problem? Or not?...

[erdoking]

i didn't solve it ...
I just found out the commit where it broke...

You can checkout commit 45022b7, this seems to work!

[NitroCao]

@erdoking @micheloosterhof Yes! It works on commit 45022b7 : ).

[CyberSecGuy]

Confirmed working on my end.

[mattiasgth]

Confirmed working on my end.

[antonio-ossan]

I have the same problem. I have tried all the solutions you provide and nothing work for me.

I have uncommented [Output_mysql] and set up all the parameters, even uncommented "debug" that showed me an error when I ran cowrie. The error said something about the function "getbool" that it doesn't find. So I looked for it and changed it for "getboolean" in mysql.py. It worked and cowrie ran without errors but still not having mysql records.

I realized that if I changed [Output_mysql] for [Database_mysql] cowrie write records on session table and sensor but that was all.

I have no errors log.

[NitroCao]

@antonio-ossan Try use 45022b7, I'm now use it. It doesn't have any problem. I tried changing [output_mysql] to [database_mysql], it would have logs in table sessions, but nothing in table auth...

[antonio-ossan]

@jaycecao I have already installed python-service-identity since I did the installation following recent instruction and no records were writen in the database. Changing "output" to "database" is the only way to some records written in the database.

[micheloosterhof]

I think the conclusion is there are some bugs in output_mysql, because it doesn't seem to write data.
I've installed MySQL again and looking into it now to determine where it's failing.

[micheloosterhof]

There's some issue with the log entries not showing up since I've moved the output module loading.
You can still see what's going on by running twistd -n cowrie and you'll see full errors.

[antonio-ossan]

That seems odd but if I run cowrie with twistd -n cowrie it works fine. I have no errors in log and the records are written in the database.

[rremo]

I have the same problem:
Server: Debian GNU/Linux 8.6 (jessie)
MySQL (remote server) version: 5.6.33
SQL: cowrie/doc/sql/mysql.sql

output_mysql input:
-> nothing :-(

database_mysql input:
-> sessions
-> sensors
-> downloads

output_mysql with DEBUG

2016-12-XX XX:XX:XX+0100 [cowrie.ssh.factory.CowrieSSHFactory] New connection: XXX.XXX.XXX.XXX:61259 (XXX.XXX.XXX.XXX:22) [session: 55c3ad8e]
2016-12-XX XX:XX:XX+0100 [HoneyPotSSHTransport,1,XXX.XXX.XXX.XXX] Remote SSH version: SSH-2.0-libssh-0.6.0
2016-12-XX XX:XX:XX+0100 [HoneyPotSSHTransport,1,XXX.XXX.XXX.XXX] kex alg, key alg: diffie-hellman-group1-sha1 ssh-rsa
2016-12-XX XX:XX:XX+0100 [HoneyPotSSHTransport,1,XXX.XXX.XXX.XXX] outgoing: aes256-ctr hmac-sha1 none
2016-12-XX XX:XX:XX+0100 [HoneyPotSSHTransport,1,XXX.XXX.XXX.XXX] incoming: aes256-ctr hmac-sha1 none
2016-12-XX XX:XX:XX+0100 [HoneyPotSSHTransport,1,XXX.XXX.XXX.XXX] NEW KEYS
2016-12-XX XX:XX:XX+0100 [HoneyPotSSHTransport,1,XXX.XXX.XXX.XXX] starting service ssh-userauth
2016-12-XX XX:XX:XX+0100 [SSHService ssh-userauth on HoneyPotSSHTransport,1,XXX.XXX.XXX.XXX] root trying auth password
2016-12-XX XX:XX:XX+0100 [SSHService ssh-userauth on HoneyPotSSHTransport,1,XXX.XXX.XXX.XXX] first time for XXX.XXX.XXX.XXX, need: 4
2016-12-XX XX:XX:XX+0100 [SSHService ssh-userauth on HoneyPotSSHTransport,1,XXX.XXX.XXX.XXX] login attempt: 1
2016-12-XX XX:XX:XX+0100 [SSHService ssh-userauth on HoneyPotSSHTransport,1,XXX.XXX.XXX.XXX] login attempt [root/111111] failed
2016-12-XX XX:XX:XX+0100 [SSHService ssh-userauth on HoneyPotSSHTransport,1,XXX.XXX.XXX.XXX] output_mysql: MySQL query: INSERT INTO `auth` (`session`, `success`, `username`, `password`, `timestamp`) VALUES (%s, %s, %s, %s, STR_TO_DATE(%s, %s)) ('55c3ad8e', 0, 'root', '111111', '2016-12-XXTXX:XX:XX.323536Z', '%Y-%m-%dT%H:%i:%s.%fZ')
2016-12-XX XX:XX:XX+0100 [-] root failed auth password
2016-12-XX XX:XX:XX+0100 [-] unauthorized login:
2016-12-XX XX:XX:XX+0100 [HoneyPotSSHTransport,1,XXX.XXX.XXX.XXX] Got remote error, code 11 reason: Bye Bye
2016-12-XX XX:XX:XX+0100 [HoneyPotSSHTransport,1,XXX.XXX.XXX.XXX] connection lost
2016-12-XX XX:XX:XX+0100 [HoneyPotSSHTransport,1,XXX.XXX.XXX.XXX] Connection lost after 3 seconds
2016-12-XX XX:XX:XX+0100 [HoneyPotSSHTransport,1,XXX.XXX.XXX.XXX] output_mysql: MySQL query: UPDATE `sessions` SET `endtime` = STR_TO_DATE(%s, %s) WHERE `id` = %s ('2016-12-XXTXX:XX:XX.372645Z', '%Y-%m-%dT%H:%i:%s.%fZ', '55c3ad8e')

database_mysql with DEBUG

2016-12-XX XX:XX:XX+0100 [HoneyPotSSHTransport,26,XXX.XXX.XXX.XXX] starting service ssh-userauth
2016-12-XX XX:XX:XX+0100 [SSHService ssh-userauth on HoneyPotSSHTransport,26,XXX.XXX.XXX.XXX] root trying auth password
2016-12-XX XX:XX:XX+0100 [SSHService ssh-userauth on HoneyPotSSHTransport,26,XXX.XXX.XXX.XXX] already tried this combination
2016-12-XX XX:XX:XX+0100 [SSHService ssh-userauth on HoneyPotSSHTransport,26,XXX.XXX.XXX.XXX] login attempt [root/111111] failed
2016-12-XX XX:XX:XX+0100 [cowrie.ssh.factory.CowrieSSHFactory] New connection: XXX.XXX.XXX.XXX3:37698 (XXX.XXX.XXX.XXX:22) [session: d6d82541]
2016-12-XX XX:XX:XX+0100 [-] root failed auth password
2016-12-XX XX:XX:XX+0100 [-] unauthorized login:
2016-12-XX XX:XX:XX+0100 [HoneyPotSSHTransport,26,XXX.XXX.XXX.XXX] Got remote error, code 11  reason: Bye Bye
2016-12-XX XX:XX:XX+0100 [HoneyPotSSHTransport,26,XXX.XXX.XXX.XXX] connection lost
2016-12-XX XX:XX:XX+0100 [HoneyPotSSHTransport,26,XXX.XXX.XXX.XXX] Connection lost after 3 seconds

[clausing]

So, the issue has been open for more than 3 months, but I don't see any progress being made. Is anyone looking at this? The mysql output is very important to me.

[JSTYL]

I'm also interested in the progress of this issue, could we get a status update?

Has anyone else found a workaround other than the limited functionality of using the [database_mysql] modification in the cowrie.cfg file? Since this change only populates 3 of the tables (sessions, sensors & downloads) it's not really usable.

[katkad]

I have the same result as @antonio-ossan

[ghost]

Somebody working on this? How do we get all the data into MySQL?

[katkad]

@Toontje hello, you can use repo at commit 45022b7 as someone mentioned here (I haven't tried it), or you can use CZ.NIC's fork of kippo/cowrie which uses [database_mysql]

[ghost]

So what should i clone, the master branch? The output-mysql branch? There is no documentation at all about why Michel's branch was cloned, when has changed, etc.
Does the CS-NIC fork work with Kippo-graph?

[micheloosterhof]

It took a while, but I think I've fixed the main bug now that stopped the SQL statements.
Please look at the master branch and reopen this ticket if necessary.