Autodiscover inserted duplicate devices into device table

[Notre1]

I installed LibreNMS on Ubuntu Server 14.04 using the installation instructions in the docs, including creating a cronjob by copying the librenms.nonroot.cron file. Added one initial device and then let autodiscovery add the rest, as I've done before in Observium.

My config.php file is pretty much stock, except for adding a few more values to the snmp community array. (eg. $config['snmp']['community'] = array("public", "ro1", "ro2", "42community", "pub2", "ro3", "ro4");)

When I started working on it this morning, I noticed that I had duplicate devices:

admingab@GAB-LibreNMS:/opt/librenms$ mysql -u librenms -p librenms -h gab-mysql -e 'SELECT device_id, hostname, sysName, snmpver, port, transport, sysObjectID, sysDescr FROM devices WHERE sysName in (SELECT sysName FROM devices GROUP BY sysNAME HAVING COUNT(device_id) > 1) ORDER BY sysName'
Enter password: 
+-----------+-------------------+-------------------+---------+------+-----------+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| device_id | hostname          | sysName           | snmpver | port | transport | sysObjectID          | sysDescr                                                                                                                                                                                                                                               |
+-----------+-------------------+-------------------+---------+------+-----------+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|        12 | gab-idf-2c-cctv-1 | gab-idf-2c-cctv-1 | v2c     |  161 | udp       | .1.3.6.1.4.1.9.1.516 | Cisco IOS Software, C3750 Software (C3750-IPBASEK9-M), Version 12.2(50)SE, RELEASE SOFTWARE (fc1)
Copyright (c) 1986-2013 by Cisco Systems, Inc.
Compiled Wed 26-Jun-13 11:25 by prod_rel_team |
|         7 | gabswitch-idf2b   | gabswitch-idf2b   | v2c     |  161 | udp       | .1.3.6.1.4.1.9.1.516 | Cisco IOS Software, C3750 Software (C3750-IPBASEK9-M), Version 12.2(55)SE8, RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2013 by Cisco Systems, Inc.
Compiled Wed 26-Jun-13 11:25 by prod_rel_team |
|        14 | gabswitch-idfc    | gabswitch-idfc    | v2c     |  161 | udp       | .1.3.6.1.4.1.9.1.516 | Cisco IOS Software, C3750 Software (C3750-IPSERVICESK9-M), Version 12.2(25)SEE4, RELEASE SOFTWARE (fc1)
Copyright (c) 1986-2007 by Cisco Systems, Inc.
Compiled Mon 16-Jul-07 03:24 by myl                                                           |
|        15 | gabswitch-idfc    | gabswitch-idfc    | v2c     |  161 | udp       | .1.3.6.1.4.1.9.1.516 | Cisco IOS Software, C3750 Software (C3750-IPSERVICESK9-M), Version 12.2(25)SEE4, RELEASE SOFTWARE (fc1)
Copyright (c) 1986-2007 by Cisco Systems, Inc.
Compiled Mon 16-Jul-07 03:24 by myl                                                           |
|        38 | gch-mdf-ge1       | gch-mdf-ge1.us    | v2c     |  161 | udp       | .1.3.6.1.4.1.9.1.617 | Cisco IOS Software, C3560 Software (C3560-IPBASE-M), Version 12.2(35)SE5, RELEASE SOFTWARE (fc1)
Copyright (c) 1986-2007 by Cisco Systems, Inc.
Compiled Thu 19-Jul-07 18:15 by nachen                                                               |
|        41 | gch-mdf-ge1.us    | gch-mdf-ge1.us    | v2c     |  161 | udp       | .1.3.6.1.4.1.9.1.617 | Cisco IOS Software, C3560 Software (C3560-IPBASE-M), Version 12.2(35)SE5, RELEASE SOFTWARE (fc1)
Copyright (c) 1986-2007 by Cisco Systems, Inc.
Compiled Thu 19-Jul-07 18:15 by nachen                                                               |
|        16 | mccalla_idf_b     | mccalla_idf_b     | v2c     |  161 | udp       | .1.3.6.1.4.1.9.1.480 | Cisco Internetwork Operating System Software 
IOS (tm) C2950 Software (C2950-I6Q4L2-M), Version 12.1(19)EA1c, RELEASE SOFTWARE (fc2)
Copyright (c) 1986-2004 by cisco Systems, Inc.
Compiled Mon 02-Feb-04 23:29 by yenanh                          |
|        17 | mccalla_idf_b     | mccalla_idf_b     | v2c     |  161 | udp       | .1.3.6.1.4.1.9.1.480 | Cisco Internetwork Operating System Software 
IOS (tm) C2950 Software (C2950-I6Q4L2-M), Version 12.1(19)EA1c, RELEASE SOFTWARE (fc2)
Copyright (c) 1986-2004 by cisco Systems, Inc.
Compiled Mon 02-Feb-04 23:29 by yenanh                          |
|        18 | mccalla_mdf-3     | mccalla_mdf-3     | v2c     |  161 | udp       | .1.3.6.1.4.1.9.1.429 | Cisco Internetwork Operating System Software 
IOS (tm) C2950 Software (C2950-I6Q4L2-M), Version 12.1(19)EA1c, RELEASE SOFTWARE (fc2)
Copyright (c) 1986-2004 by cisco Systems, Inc.
Compiled Mon 02-Feb-04 23:29 by yenanh                          |
|        19 | mccalla_mdf-3     | mccalla_mdf-3     | v2c     |  161 | udp       | .1.3.6.1.4.1.9.1.429 | Cisco Internetwork Operating System Software 
IOS (tm) C2950 Software (C2950-I6Q4L2-M), Version 12.1(19)EA1c, RELEASE SOFTWARE (fc2)
Copyright (c) 1986-2004 by cisco Systems, Inc.
Compiled Mon 02-Feb-04 23:29 by yenanh                          |
|        20 | mccalla_mdf_5     | mccalla_mdf_5     | v2c     |  161 | udp       | .1.3.6.1.4.1.9.1.564 | Cisco IOS Software, C3560 Software (C3560-IPSERVICES-M), Version 12.2(25)SEB4, RELEASE SOFTWARE (fc1)
Copyright (c) 1986-2005 by Cisco Systems, Inc.
Compiled Tue 30-Aug-05 14:19 by yenanh                                                          |
|        21 | mccalla_mdf_5     | mccalla_mdf_5     | v2c     |  161 | udp       | .1.3.6.1.4.1.9.1.564 | Cisco IOS Software, C3560 Software (C3560-IPSERVICES-M), Version 12.2(25)SEB4, RELEASE SOFTWARE (fc1)
Copyright (c) 1986-2005 by Cisco Systems, Inc.
Compiled Tue 30-Aug-05 14:19 by yenanh                                                          |
+-----------+-------------------+-------------------+---------+------+-----------+----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

admingab@GAB-LibreNMS:/opt/librenms$ mysql -u librenms -p librenms -h gab-mysql -e 'SELECT * FROM eventlog WHERE host IN (SELECT device_id FROM devices WHERE sysNAME IN (SELECT sysNAME FROM devices GROUP BY sysNAME HAVING COUNT(device_id) > 1)) AND type = "discovery" ORDER BY datetime'                                                                                            
Enter password: 
+----------+------+---------------------+--------------------------------------------------------------------------------------------------------------+-----------+-----------+
| event_id | host | datetime            | message                                                                                                      | type      | reference |
+----------+------+---------------------+--------------------------------------------------------------------------------------------------------------+-----------+-----------+
|     2275 |   12 | 2015-08-24 18:38:17 | Device $gab-idf-2c-cctv-1 (10.10.110.41)  (port )  autodiscovered through CDP on gabswitch-idf2b             | discovery | NULL      |
|     2276 |   13 | 2015-08-24 18:38:17 | Device $gab-idf-2c-cctv-1 (10.10.110.41)  (port )  autodiscovered through CDP on gabswitch-idf2b             | discovery | NULL      |
|     7740 |    6 | 2015-08-24 18:41:29 | Device $gabswitch-idf2b (10.10.110.39)  (port )  autodiscovered through CDP on mccalla_mdfb                  | discovery | NULL      |
|     7739 |    7 | 2015-08-24 18:41:29 | Device $gabswitch-idf2b (10.10.110.39)  (port )  autodiscovered through CDP on mccalla_mdfb                  | discovery | NULL      |
|     7747 |   14 | 2015-08-24 18:42:19 | Device $gabswitch-idfc (10.10.110.26)  (port GigabitEthernet0/2)  autodiscovered through CDP on mccalla_mdfb | discovery | NULL      |
|     7748 |   15 | 2015-08-24 18:42:21 | Device $gabswitch-idfc (10.10.110.26)  (port GigabitEthernet0/2)  autodiscovered through CDP on mccalla_mdfb | discovery | NULL      |
|     7755 |   16 | 2015-08-24 18:43:07 | Device $mccalla_idf_b (10.10.110.25)  (port GigabitEthernet0/3)  autodiscovered through CDP on mccalla_mdfb  | discovery | NULL      |
|     7756 |   17 | 2015-08-24 18:43:09 | Device $mccalla_idf_b (10.10.110.25)  (port GigabitEthernet0/3)  autodiscovered through CDP on mccalla_mdfb  | discovery | NULL      |
|     7763 |   18 | 2015-08-24 18:44:06 | Device $mccalla_mdf-3 (10.10.110.33)  (port GigabitEthernet0/5)  autodiscovered through CDP on mccalla_mdfb  | discovery | NULL      |
|     7764 |   19 | 2015-08-24 18:44:08 | Device $mccalla_mdf-3 (10.10.110.33)  (port GigabitEthernet0/5)  autodiscovered through CDP on mccalla_mdfb  | discovery | NULL      |
|     9967 |   22 | 2015-08-24 18:45:59 | Device $gab-idf-j (10.10.110.214)  (port )  autodiscovered through CDP on mccalla_mdf_5                      | discovery | NULL      |
|    10722 |   23 | 2015-08-24 18:46:11 | Device $gab-idf-j (10.10.110.214)  (port )  autodiscovered through CDP on mccalla_mdf_5                      | discovery | NULL      |
|    14613 |   24 | 2015-08-24 18:47:21 | Device $gab-idf-i (10.10.110.213)  (port )  autodiscovered through CDP on mccalla_mdf_5                      | discovery | NULL      |
|    14614 |   25 | 2015-08-24 18:47:27 | Device $gab-idf-i (10.10.110.213)  (port )  autodiscovered through CDP on mccalla_mdf_5                      | discovery | NULL      |
|    15009 |   20 | 2015-08-24 18:50:53 | Device $mccalla_mdf_5 (10.10.110.37)  (port GigabitEthernet0/7)  autodiscovered through CDP on mccalla_mdfb  | discovery | NULL      |
|    15714 |   21 | 2015-08-24 18:51:09 | Device $mccalla_mdf_5 (10.10.110.37)  (port GigabitEthernet0/7)  autodiscovered through CDP on mccalla_mdfb  | discovery | NULL      |
|    33164 |   41 | 2015-08-25 09:09:20 | Device $gch-mdf-ge1.us (192.168.176.60)  (port )  autodiscovered through CDP on gch-mdf-ge6                  | discovery | NULL      |
+----------+------+---------------------+--------------------------------------------------------------------------------------------------------------+-----------+-----------+

[laf]

@Notre1 What would be handy is if you could delete a duplicated device (both of them - say gabswitch-idfc ) and then immediately run:

./discovery.php -h mccalla_mdfb -d -m discovery-protocols

And post (and edited version if needs be) of that output pls.

[Notre1]

@laf OK, I did exactly as you said and it only added one of them back.

I created a GIST for the output since it's over 5000 lines. (These are switches)

https://gist.github.com/Notre1/722c8158bc0aa560b3ea

[Notre1]

I wiped the database and configured it fresh again, this time manually running discovery.php -h all from the cli before I enabled the cron job, and it didn't duplicate any devices this time.

Not sure what went on during that initial discovery, but all looks good now.

[laf]

@Notre1 Thanks for the all the debugging. The last thing, if you have the output from /opt/librenms/logs/librenms.log for 2015-08-24 18:30 - 2015-08-24 18:55 that could indicate if multiple discovery processes were running (which really shouldn't happen).

[Notre1]

@laf The log output is below. It does look like there was some overlap in the discovery processes. I guess I should adjust the cron job.

/opt/librenms/discovery.php new 2015-08-24 18:30:02 - 0 devices discovered in 0.014 secs
/opt/librenms/discovery.php new 2015-08-24 18:49:29 - 6 devices discovered in 567.7 secs
/opt/librenms/discovery.php new 2015-08-24 18:51:23 - 11 devices discovered in 981.8 secs
/opt/librenms/discovery.php new 2015-08-24 18:53:03 - 5 devices discovered in 481.7 secs
/opt/librenms/discovery.php new 2015-08-24 18:53:05 - 1 devices discovered in 184.5 secs
/opt/librenms/discovery.php new 2015-08-24 18:55:20 - 1 devices discovered in 18.00 secs
/opt/librenms/discovery.php new 2015-08-24 19:00:25 - 1 devices discovered in 23.31 secs
/opt/librenms/discovery.php all 2015-08-24 19:00:55 - 24 devices discovered in 1673. secs
/opt/librenms/discovery.php new 2015-08-24 19:05:01 - 0 devices discovered in 0.010 secs

[laf]

That does look like its overlapped. The cron by itself is correct though don't alter that.

[MHammett]

I have seen multiple device entries for the same physical device due to multiple IP addresses on the same device.

Can the discovery process have an option to allow or disallow additional devices if that serial number already exists in the database?