feature/540_mysql_table_type_select

[frbattid]

• Implements issue [OrionCKANSink] Allow selecting the table type (by service, servicePath or destination) #540
• 100% unit tests passed:

Results :
Tests run: 54, Failures: 0, Errors: 0, Skipped: 0

• (unofficial) e2e tests passed:

---table_type=table-by-destination, enagle_grouping=false
time=2015-09-24T09:02:00.981CEST | lvl=INFO | trans=1443078104-252-0000000000 | function=getEvents | comp=Cygnus | msg=com.telefonica.iot.cygnus.handlers.OrionRestHandler[150] : Starting transaction (1443078104-252-0000000000)
time=2015-09-24T09:02:00.984CEST | lvl=INFO | trans=1443078104-252-0000000000 | function=getEvents | comp=Cygnus | msg=com.telefonica.iot.cygnus.handlers.OrionRestHandler[232] : Received data ({  "subscriptionId" : "51c0ac9ed714fb3b37d7d5a8",  "originator" : "localhost",  "contextResponses" : [    {      "contextElement" : {        "attributes" : [          {            "name" : "temperature",            "type" : "centigrade",            "value" : "26.5"          }        ],        "type" : "Room",        "isPattern" : "false",        "id" : "Room1"      },      "statusCode" : {        "code" : "200",        "reasonPhrase" : "OK"      }    }  ]})
time=2015-09-24T09:02:00.993CEST | lvl=INFO | trans=1443078104-252-0000000000 | function=getEvents | comp=Cygnus | msg=com.telefonica.iot.cygnus.handlers.OrionRestHandler[255] : Event put in the channel (id=794013466, ttl=10)
time=2015-09-24T09:02:01.046CEST | lvl=INFO | trans=1443078104-252-0000000000 | function=process | comp=Cygnus | msg=com.telefonica.iot.cygnus.sinks.OrionSink[145] : Event got from the channel (id=794013466, headers={default-destinations=room1_room, grouped-fiware-servicepaths=rooms, fiware-servicepath=test_enable_grouping_9, content-type=application/json, grouped-destinations=other_rooms, fiware-service=deleteme3, ttl=10, transactionId=1443078104-252-0000000000, timestamp=1443078120993, default-fiware-servicepaths=test_enable_grouping_9}, bodyLength=460)
time=2015-09-24T09:02:02.537CEST | lvl=INFO | trans=1443078104-252-0000000000 | function=persist | comp=Cygnus | msg=com.telefonica.iot.cygnus.sinks.OrionMySQLSink[250] : [mysql-sink] Persisting data at OrionMySQLSink. Database: deleteme3, Table: test_enable_grouping_9_room1_room, Data: 1443078120,2015-09-24T07:02:00.993,Room1,Room,temperature,Room,26.5,[]
time=2015-09-24T09:02:03.155CEST | lvl=INFO | trans=1443078104-252-0000000000 | function=process | comp=Cygnus | msg=com.telefonica.iot.cygnus.sinks.OrionSink[210] : Finishing transaction (1443078104-252-0000000000)
...
mysql> select * from deleteme3.test_enable_grouping_9_room1_room;
+------------+-------------------------+----------+------------+-------------+------------+-----------+--------+
| recvTimeTs | recvTime                | entityId | entityType | attrName    | attrType   | attrValue | attrMd |
+------------+-------------------------+----------+------------+-------------+------------+-----------+--------+
| 1443078120 | 2015-09-24T07:02:00.993 | Room1    | Room       | temperature | centigrade | 26.5      | []     |
+------------+-------------------------+----------+------------+-------------+------------+-----------+--------+
1 row in set (0.00 sec)
---table_type=table-by-service-path, enable_grouping=false
time=2015-09-24T09:45:46.534CEST | lvl=INFO | trans=1443080736-647-0000000000 | function=getEvents | comp=Cygnus | msg=com.telefonica.iot.cygnus.handlers.OrionRestHandler[150] : Starting transaction (1443080736-647-0000000000)
time=2015-09-24T09:45:46.536CEST | lvl=INFO | trans=1443080736-647-0000000000 | function=getEvents | comp=Cygnus | msg=com.telefonica.iot.cygnus.handlers.OrionRestHandler[232] : Received data ({  "subscriptionId" : "51c0ac9ed714fb3b37d7d5a8",  "originator" : "localhost",  "contextResponses" : [    {      "contextElement" : {        "attributes" : [          {            "name" : "temperature",            "type" : "centigrade",            "value" : "26.5"          }        ],        "type" : "Room",        "isPattern" : "false",        "id" : "Room1"      },      "statusCode" : {        "code" : "200",        "reasonPhrase" : "OK"      }    }  ]})
time=2015-09-24T09:45:46.555CEST | lvl=INFO | trans=1443080736-647-0000000000 | function=getEvents | comp=Cygnus | msg=com.telefonica.iot.cygnus.handlers.OrionRestHandler[255] : Event put in the channel (id=9154917, ttl=10)
time=2015-09-24T09:45:46.671CEST | lvl=INFO | trans=1443080736-647-0000000000 | function=process | comp=Cygnus | msg=com.telefonica.iot.cygnus.sinks.OrionSink[145] : Event got from the channel (id=9154917, headers={default-destinations=room1_room, grouped-fiware-servicepaths=rooms, fiware-servicepath=test_enable_grouping_9, content-type=application/json, grouped-destinations=other_rooms, fiware-service=deleteme3, ttl=10, transactionId=1443080736-647-0000000000, timestamp=1443080746555, default-fiware-servicepaths=test_enable_grouping_9}, bodyLength=460)
time=2015-09-24T09:45:48.411CEST | lvl=INFO | trans=1443080736-647-0000000000 | function=persist | comp=Cygnus | msg=com.telefonica.iot.cygnus.sinks.OrionMySQLSink[250] : [mysql-sink] Persisting data at OrionMySQLSink. Database: deleteme3, Table: test_enable_grouping_9, Data: 1443080746,2015-09-24T07:45:46.555,Room1,Room,temperature,Room,26.5,[]
time=2015-09-24T09:45:48.999CEST | lvl=INFO | trans=1443080736-647-0000000000 | function=process | comp=Cygnus | msg=com.telefonica.iot.cygnus.sinks.OrionSink[210] : Finishing transaction (1443080736-647-0000000000)
...
mysql> select * from deleteme3.test_enable_grouping_9;
+------------+-------------------------+----------+------------+-------------+------------+-----------+--------+
| recvTimeTs | recvTime                | entityId | entityType | attrName    | attrType   | attrValue | attrMd |
+------------+-------------------------+----------+------------+-------------+------------+-----------+--------+
| 1443080746 | 2015-09-24T07:45:46.555 | Room1    | Room       | temperature | centigrade | 26.5      | []     |
+------------+-------------------------+----------+------------+-------------+------------+-----------+--------+
1 row in set (0.00 sec)
---table_type=table-by-destination, enable_grouping=true
{
    "grouping_rules": [
        {
            "id": 1,
            "fields": [
                "entityType"
            ],
            "regex": "Room\\.(\\d*)Room",
            "destination": "numeric_rooms",
            "fiware_service_path": "rooms"
        },
        {
            "id": 2,
            "fields": [
                "entityType"
            ],
            "regex": "Room",
            "destination": "other_rooms",
            "fiware_service_path": "rooms"
        }
    ]
}
...
time=2015-09-24T09:48:33.120CEST | lvl=INFO | trans=1443080907-252-0000000000 | function=getEvents | comp=Cygnus | msg=com.telefonica.iot.cygnus.handlers.OrionRestHandler[150] : Starting transaction (1443080907-252-0000000000)
time=2015-09-24T09:48:33.123CEST | lvl=INFO | trans=1443080907-252-0000000000 | function=getEvents | comp=Cygnus | msg=com.telefonica.iot.cygnus.handlers.OrionRestHandler[232] : Received data ({  "subscriptionId" : "51c0ac9ed714fb3b37d7d5a8",  "originator" : "localhost",  "contextResponses" : [    {      "contextElement" : {        "attributes" : [          {            "name" : "temperature",            "type" : "centigrade",            "value" : "26.5"          }        ],        "type" : "Room",        "isPattern" : "false",        "id" : "Room1"      },      "statusCode" : {        "code" : "200",        "reasonPhrase" : "OK"      }    }  ]})
time=2015-09-24T09:48:33.125CEST | lvl=INFO | trans=1443080907-252-0000000000 | function=getEvents | comp=Cygnus | msg=com.telefonica.iot.cygnus.handlers.OrionRestHandler[255] : Event put in the channel (id=2076554288, ttl=10)
time=2015-09-24T09:48:33.183CEST | lvl=INFO | trans=1443080907-252-0000000000 | function=process | comp=Cygnus | msg=com.telefonica.iot.cygnus.sinks.OrionSink[145] : Event got from the channel (id=2076554288, headers={default-destinations=room1_room, grouped-fiware-servicepaths=rooms, fiware-servicepath=test_enable_grouping_9, content-type=application/json, grouped-destinations=other_rooms, fiware-service=deleteme3, ttl=10, transactionId=1443080907-252-0000000000, timestamp=1443080913125, default-fiware-servicepaths=test_enable_grouping_9}, bodyLength=460)
time=2015-09-24T09:48:34.619CEST | lvl=INFO | trans=1443080907-252-0000000000 | function=persist | comp=Cygnus | msg=com.telefonica.iot.cygnus.sinks.OrionMySQLSink[250] : [mysql-sink] Persisting data at OrionMySQLSink. Database: deleteme3, Table: rooms_other_rooms, Data: 1443080913,2015-09-24T07:48:33.125,Room1,Room,temperature,Room,26.5,[]
time=2015-09-24T09:48:35.212CEST | lvl=INFO | trans=1443080907-252-0000000000 | function=process | comp=Cygnus | msg=com.telefonica.iot.cygnus.sinks.OrionSink[210] : Finishing transaction (1443080907-252-0000000000)
...
mysql> select * from deleteme3.rooms_other_rooms;
+------------+-------------------------+----------+------------+-------------+------------+-----------+--------+
| recvTimeTs | recvTime                | entityId | entityType | attrName    | attrType   | attrValue | attrMd |
+------------+-------------------------+----------+------------+-------------+------------+-----------+--------+
| 1443080913 | 2015-09-24T07:48:33.125 | Room1    | Room       | temperature | centigrade | 26.5      | []     |
+------------+-------------------------+----------+------------+-------------+------------+-----------+--------+
1 row in set (0.00 sec)
--- table_type=table-by-service-path, enable_grouping=true
{
    "grouping_rules": [
        {
            "id": 1,
            "fields": [
                "entityType"
            ],
            "regex": "Room\\.(\\d*)Room",
            "destination": "numeric_rooms",
            "fiware_service_path": "rooms"
        },
        {
            "id": 2,
            "fields": [
                "entityType"
            ],
            "regex": "Room",
            "destination": "other_rooms",
            "fiware_service_path": "rooms"
        }
    ]
}
...
time=2015-09-24T09:50:55.353CEST | lvl=INFO | trans=1443081051-780-0000000000 | function=getEvents | comp=Cygnus | msg=com.telefonica.iot.cygnus.handlers.OrionRestHandler[150] : Starting transaction (1443081051-780-0000000000)
time=2015-09-24T09:50:55.356CEST | lvl=INFO | trans=1443081051-780-0000000000 | function=getEvents | comp=Cygnus | msg=com.telefonica.iot.cygnus.handlers.OrionRestHandler[232] : Received data ({  "subscriptionId" : "51c0ac9ed714fb3b37d7d5a8",  "originator" : "localhost",  "contextResponses" : [    {      "contextElement" : {        "attributes" : [          {            "name" : "temperature",            "type" : "centigrade",            "value" : "26.5"          }        ],        "type" : "Room",        "isPattern" : "false",        "id" : "Room1"      },      "statusCode" : {        "code" : "200",        "reasonPhrase" : "OK"      }    }  ]})
time=2015-09-24T09:50:55.363CEST | lvl=INFO | trans=1443081051-780-0000000000 | function=getEvents | comp=Cygnus | msg=com.telefonica.iot.cygnus.handlers.OrionRestHandler[255] : Event put in the channel (id=167849825, ttl=10)
time=2015-09-24T09:50:55.414CEST | lvl=INFO | trans=1443081051-780-0000000000 | function=process | comp=Cygnus | msg=com.telefonica.iot.cygnus.sinks.OrionSink[145] : Event got from the channel (id=167849825, headers={default-destinations=room1_room, grouped-fiware-servicepaths=rooms, fiware-servicepath=test_enable_grouping_9, content-type=application/json, grouped-destinations=other_rooms, fiware-service=deleteme3, ttl=10, transactionId=1443081051-780-0000000000, timestamp=1443081055363, default-fiware-servicepaths=test_enable_grouping_9}, bodyLength=460)
time=2015-09-24T09:50:56.916CEST | lvl=INFO | trans=1443081051-780-0000000000 | function=persist | comp=Cygnus | msg=com.telefonica.iot.cygnus.sinks.OrionMySQLSink[250] : [mysql-sink] Persisting data at OrionMySQLSink. Database: deleteme3, Table: rooms, Data: 1443081055,2015-09-24T07:50:55.363,Room1,Room,temperature,Room,26.5,[]
time=2015-09-24T09:50:57.520CEST | lvl=INFO | trans=1443081051-780-0000000000 | function=process | comp=Cygnus | msg=com.telefonica.iot.cygnus.sinks.OrionSink[210] : Finishing transaction (1443081051-780-0000000000)
...
mysql> select * from deleteme3.rooms;
+------------+-------------------------+----------+------------+-------------+------------+-----------+--------+
| recvTimeTs | recvTime                | entityId | entityType | attrName    | attrType   | attrValue | attrMd |
+------------+-------------------------+----------+------------+-------------+------------+-----------+--------+
| 1443081055 | 2015-09-24T07:50:55.363 | Room1    | Room       | temperature | centigrade | 26.5      | []     |
+------------+-------------------------+----------+------------+-------------+------------+-----------+--------+
1 row in set (0.00 sec)

• Assignee @fgalan
• DO NOT REMOVE THE BRANCH, SINCE A RPM FOR THE LONG TAIL PROJECT IS GOING TO BE PRODUCED FROM IT

[fgalan]

Number between ( and ) ?

[frbattid]

Fixed in ae24ad3

[fgalan]

LGTM