Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Crash in tcshow Filter Parsing #160

Closed
personalcomputer opened this issue Dec 6, 2021 · 3 comments
Closed

Crash in tcshow Filter Parsing #160

personalcomputer opened this issue Dec 6, 2021 · 3 comments
Labels
bug

Comments

@personalcomputer
Copy link
Contributor

personalcomputer commented Dec 6, 2021 •
edited

Describe the bug

Hi Hombashi-san, thanks for publishing tcconfig. It makes tc a lot more approachable. I have found a crash when parsing filters in tcshow:

$ sudo tcshow wlp3s0
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/simplesqlite/core.py", line 663, in insert_many
    self.connection.executemany(query, records)
sqlite3.IntegrityError: NOT NULL constraint failed: filter.device

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/tcshow", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.8/dist-packages/tcconfig/tcshow.py", line 212, in main
    tc_params = extract_tc_params(options)
  File "/usr/local/lib/python3.8/dist-packages/tcconfig/tcshow.py", line 188, in extract_tc_params
    rule_parser.parse()
  File "/usr/local/lib/python3.8/dist-packages/tcconfig/parser/shaping_rule.py", line 105, in parse
    self.__parse_device(self.ifb_device)
  File "/usr/local/lib/python3.8/dist-packages/tcconfig/parser/shaping_rule.py", line 115, in __parse_device
    self.__parse_tc_filter(device)
  File "/usr/local/lib/python3.8/dist-packages/tcconfig/parser/shaping_rule.py", line 293, in __parse_tc_filter
    self.__filter_parser.parse(
  File "/usr/local/lib/python3.8/dist-packages/tcconfig/parser/_filter.py", line 139, in parse
    Filter.insert(self.__get_filter())
  File "/usr/local/lib/python3.8/dist-packages/simplesqlite/model.py", line 226, in insert
    cls.__connection.insert(cls.get_table_name(), record)
  File "/usr/local/lib/python3.8/dist-packages/simplesqlite/core.py", line 594, in insert
    self.insert_many(table_name, records=[record], attr_names=attr_names)
  File "/usr/local/lib/python3.8/dist-packages/simplesqlite/core.py", line 667, in insert_many
    raise OperationalError(
simplesqlite.error.OperationalError: /usr/local/lib/python3.8/dist-packages/simplesqlite/model.py(226) insert: failed to execute query:
  query=INSERT INTO filter(device,"filter_id",flowid,protocol,priority,[src-network],[dst-network],[src-port],[dst-port],classid,handle) VALUES (?,?,?,?,?,?,?,?,?,?,?)
  msg='NOT NULL constraint failed: filter.device'
  db=:memory:
  records=[[None, '800::801', '1bc4:3', 'ipv6', 6, '0.0.0.0/0', '0.0.0.0/0', None, None, None, None]]

To Reproduce

The simplest way to reproduce that I have found is by running the below two tcset commands in sequence, and then tcshow as third command. These two tcset commands are trying to set up a 100ms delay on both ipv4 and ipv6, so quite a normal usecase I think.

$ sudo tcset --direction incoming --delay 100  wlp3s0 
$ sudo tcset --direction incoming --delay 100  wlp3s0 --ipv6 --add
[ERROR] command execution failed
  command=/usr/sbin/tc qdisc add dev wlp3s0 ingress
  stderr=Error: Exclusivity flag on, cannot modify.

[ERROR] command execution failed
  command=/usr/sbin/tc qdisc add dev ifb7108 root handle 1bc4: htb default 1
  stderr=Error: Exclusivity flag on, cannot modify.
$ sudo tcshow wlp3s0
[... crash ...]

(note: 2nd command gives an error message, but does not crash. Crash only happens when running tcshow.)

Environments

Docker version: I was not using tcconfig in combination with docker.
Installation method: pip

Module Version
uname Linux 5.4.0-90-generic
Python CPython 3.8.10
platform Ubuntu 20.04
tcconfig 0.26.0
docker 4.3.1
SimpleSQLite 1.1.3

Additional context

Full --debug output:

$ sudo tcshow --debug wlp3s0 

DEBUG    | simplesqlite.core:connect:266 - connect to a SQLite database: path=':memory:', mode=w
DEBUG    | simplesqlite.core:create_table:1277 - CREATE TABLE IF NOT EXISTS 'filter' (device TEXT NOT NULL, "filter_id" TEXT, flowid TEXT, protocol TEXT, priority INTEGER, [src-network] TEXT, [dst-network] TEXT, [src-port] INTEGER, [dst-port] INTEGER, classid TEXT, handle INTEGER)
DEBUG    | simplesqlite.core:create_table:1277 - CREATE TABLE IF NOT EXISTS 'qdisc' (device TEXT NOT NULL, "direct_qlen" INTEGER, parent TEXT, handle TEXT, delay TEXT, [delay-distro] TEXT, loss TEXT, duplicate TEXT, corrupt TEXT, reorder TEXT, rate TEXT)
DEBUG    | subprocrunner._subprocess_runner:__debug_print_command:264 - /usr/sbin/tc filter show dev wlp3s0 root
DEBUG    | subprocrunner._subprocess_runner:__debug_print_command:264 - /usr/sbin/tc class show dev wlp3s0
DEBUG    | tcconfig.parser._class:parse:57 - tc class parse result: []
DEBUG    | subprocrunner._subprocess_runner:__debug_print_command:264 - /usr/sbin/tc filter show dev wlp3s0
DEBUG    | subprocrunner._subprocess_runner:__debug_print_command:264 - /usr/sbin/tc qdisc show dev wlp3s0
DEBUG    | subprocrunner._subprocess_runner:__debug_print_command:264 - /usr/sbin/tc class show dev ifb7108
DEBUG    | tcconfig.parser._class:parse:51 - parse a class entry: {'device': 'ifb7108', 'classid': '1bc4:1', 'rate': '32Gbps'}
DEBUG    | tcconfig.parser._class:parse:51 - parse a class entry: {'device': 'ifb7108', 'classid': '1bc4:2', 'rate': '32Gbps'}
DEBUG    | tcconfig.parser._class:parse:51 - parse a class entry: {'device': 'ifb7108', 'classid': '1bc4:3', 'rate': '32Gbps'}
DEBUG    | simplesqlite.core:__create_table_from_tabledata:1726 - __create_table_from_tabledata:
    tbldata=table_name=class, headers=[device, classid, rate], cols=3, rows=3
DEBUG    | tabledata.normalizer:normalize:53 - normalize: SQLiteTableDataSanitizer
DEBUG    | dataproperty._extractor:to_dp_matrix:453 - max_workers=1, preprocessor=strip_str=None, replace_tabs_with_spaces=True, tab_length=2, line_break_handling=LineBreakHandling.NOP, line_break_repl= , escape_html_tag=False, escape_formula_injection=False
DEBUG    | dataproperty._extractor:to_column_dp_list:387 - converting to column dataproperty:
DEBUG    | dataproperty._extractor:to_column_dp_list:415 -   params:
DEBUG    | dataproperty._extractor:to_column_dp_list:415 -     headers=3
DEBUG    | dataproperty._extractor:to_column_dp_list:415 -     prev_col_count=None
DEBUG    | dataproperty._extractor:to_column_dp_list:415 -     matrix_formatting=MatrixFormatting.TRIM
DEBUG    | dataproperty._extractor:to_column_dp_list:415 -     column_type_hints=()
DEBUG    | dataproperty._extractor:to_column_dp_list:417 -   results:
DEBUG    | dataproperty._extractor:to_column_dp_list:447 -     column=0, type=STRING, align=left, ascii_width=7
DEBUG    | dataproperty._extractor:to_column_dp_list:447 -     column=1, type=STRING, align=left, ascii_width=7
DEBUG    | dataproperty._extractor:to_column_dp_list:447 -     column=2, type=STRING, align=left, ascii_width=6
DEBUG    | simplesqlite.core:create_table:1277 - CREATE TABLE IF NOT EXISTS 'class' (device TEXT, classid TEXT, rate TEXT)
DEBUG    | simplesqlite.core:insert_many:631 - insert 3 records into class
DEBUG    | simplesqlite.core:commit:1579 - commit: path=':memory:'
DEBUG    | tcconfig.parser._class:parse:57 - tc class parse result: [
    {
        "device": "ifb7108",
        "classid": "1bc4:1",
        "rate": "32Gbps"
    },
    {
        "device": "ifb7108",
        "classid": "1bc4:2",
        "rate": "32Gbps"
    },
    {
        "device": "ifb7108",
        "classid": "1bc4:3",
        "rate": "32Gbps"
    }
]
DEBUG    | subprocrunner._subprocess_runner:__debug_print_command:264 - /usr/sbin/tc filter show dev ifb7108
DEBUG    | tcconfig.parser._filter:parse:92 - failed to parse mangle: filter parent 1bc4: protocol ip pref 5 u32 chain 0
DEBUG    | tcconfig.parser._filter:parse:126 - failed to parse flow id: filter parent 1bc4: protocol ip pref 5 u32 chain 0
DEBUG    | tcconfig.parser._filter:parse:136 - failed to parse filter: filter parent 1bc4: protocol ip pref 5 u32 chain 0
DEBUG    | tcconfig.parser._filter:parse:92 - failed to parse mangle: filter parent 1bc4: protocol ip pref 5 u32 chain 0 fh 800: ht divisor 1
DEBUG    | tcconfig.parser._filter:parse:126 - failed to parse flow id: filter parent 1bc4: protocol ip pref 5 u32 chain 0 fh 800: ht divisor 1
DEBUG    | tcconfig.parser._filter:parse:136 - failed to parse filter: filter parent 1bc4: protocol ip pref 5 u32 chain 0 fh 800: ht divisor 1
DEBUG    | tcconfig.parser._filter:parse:92 - failed to parse mangle: filter parent 1bc4: protocol ip pref 5 u32 chain 0 fh 800::800 order 2048 key ht 800 bkt 0 flowid 1bc4:2 not_in_hw
DEBUG    | tcconfig.parser._filter:__parse_flow_id:183 - succeed to parse flow id: flow-id=1bc4:2, line=filter parent 1bc4: protocol ip pref 5 u32 chain 0 fh 800::800 order 2048 key ht 800 bkt 0 flowid 1bc4:2 not_in_hw
DEBUG    | tcconfig.parser._filter:__parse_protocol:188 - succeed to parse protocol: protocol=ip, line=filter parent 1bc4: protocol ip pref 5 u32 chain 0 fh 800::800 order 2048 key ht 800 bkt 0 flowid 1bc4:2 not_in_hw
DEBUG    | tcconfig.parser._filter:__parse_priority:195 - succeed to parse priority: priority=5, line=filter parent 1bc4: protocol ip pref 5 u32 chain 0 fh 800::800 order 2048 key ht 800 bkt 0 flowid 1bc4:2 not_in_hw
DEBUG    | tcconfig.parser._filter:__parse_filter_id:202 - succeed to parse filter id: filter-id=800::800, line=filter parent 1bc4: protocol ip pref 5 u32 chain 0 fh 800::800 order 2048 key ht 800 bkt 0 flowid 1bc4:2 not_in_hw
DEBUG    | tcconfig.parser._filter:parse:92 - failed to parse mangle: match 00000000/00000000 at 16
DEBUG    | tcconfig.parser._filter:parse:126 - failed to parse flow id: match 00000000/00000000 at 16
DEBUG    | tcconfig.parser._filter:__parse_filter_ipv4:354 - succeed to parse ipv4 filter: src_network=None, dst_network=0.0.0.0/0, src_port=None, dst_port=None, line=match 00000000/00000000 at 16
DEBUG    | tcconfig.parser._filter:parse:92 - failed to parse mangle: match 00000000/00000000 at 12
DEBUG    | tcconfig.parser._filter:parse:126 - failed to parse flow id: match 00000000/00000000 at 12
DEBUG    | tcconfig.parser._filter:__parse_filter_ipv4:354 - succeed to parse ipv4 filter: src_network=0.0.0.0/0, dst_network=0.0.0.0/0, src_port=None, dst_port=None, line=match 00000000/00000000 at 12
DEBUG    | tcconfig.parser._filter:parse:92 - failed to parse mangle: filter parent 1bc4: protocol ipv6 pref 6 u32 chain 0
DEBUG    | tcconfig.parser._filter:parse:126 - failed to parse flow id: filter parent 1bc4: protocol ipv6 pref 6 u32 chain 0
DEBUG    | tcconfig.parser._filter:parse:136 - failed to parse filter: filter parent 1bc4: protocol ipv6 pref 6 u32 chain 0
DEBUG    | tcconfig.parser._filter:parse:92 - failed to parse mangle: filter parent 1bc4: protocol ipv6 pref 6 u32 chain 0 fh 801: ht divisor 1
DEBUG    | tcconfig.parser._filter:parse:126 - failed to parse flow id: filter parent 1bc4: protocol ipv6 pref 6 u32 chain 0 fh 801: ht divisor 1
DEBUG    | tcconfig.parser._filter:parse:136 - failed to parse filter: filter parent 1bc4: protocol ipv6 pref 6 u32 chain 0 fh 801: ht divisor 1
DEBUG    | tcconfig.parser._filter:parse:92 - failed to parse mangle: filter parent 1bc4: protocol ipv6 pref 6 u32 chain 0 fh 801::800 order 2048 key ht 801 bkt 0 flowid 1bc4:3 not_in_hw
DEBUG    | tcconfig.parser._filter:__parse_flow_id:183 - succeed to parse flow id: flow-id=1bc4:3, line=filter parent 1bc4: protocol ipv6 pref 6 u32 chain 0 fh 801::800 order 2048 key ht 801 bkt 0 flowid 1bc4:3 not_in_hw
DEBUG    | tcconfig.parser._filter:__parse_protocol:188 - succeed to parse protocol: protocol=ipv6, line=filter parent 1bc4: protocol ipv6 pref 6 u32 chain 0 fh 801::800 order 2048 key ht 801 bkt 0 flowid 1bc4:3 not_in_hw
DEBUG    | tcconfig.parser._filter:__parse_priority:195 - succeed to parse priority: priority=6, line=filter parent 1bc4: protocol ipv6 pref 6 u32 chain 0 fh 801::800 order 2048 key ht 801 bkt 0 flowid 1bc4:3 not_in_hw
DEBUG    | tcconfig.parser._filter:__parse_filter_id:202 - succeed to parse filter id: filter-id=801::800, line=filter parent 1bc4: protocol ipv6 pref 6 u32 chain 0 fh 801::800 order 2048 key ht 801 bkt 0 flowid 1bc4:3 not_in_hw
DEBUG    | tcconfig.parser._filter:parse:115 - store filter: Filter (device=ifb7108, filter_id=800::800, flowid=1bc4:2, protocol=ip, priority=5, src-network=0.0.0.0/0, dst-network=0.0.0.0/0)
DEBUG    | simplesqlite.core:insert_many:631 - insert 1 records into filter
DEBUG    | tcconfig.parser._filter:__parse_flow_id:183 - succeed to parse flow id: flow-id=1bc4:3, line=filter parent 1bc4: protocol ipv6 pref 6 u32 chain 0 fh 801::800 order 2048 key ht 801 bkt 0 flowid 1bc4:3 not_in_hw
DEBUG    | tcconfig.parser._filter:__parse_protocol:188 - succeed to parse protocol: protocol=ipv6, line=filter parent 1bc4: protocol ipv6 pref 6 u32 chain 0 fh 801::800 order 2048 key ht 801 bkt 0 flowid 1bc4:3 not_in_hw
DEBUG    | tcconfig.parser._filter:__parse_priority:195 - succeed to parse priority: priority=6, line=filter parent 1bc4: protocol ipv6 pref 6 u32 chain 0 fh 801::800 order 2048 key ht 801 bkt 0 flowid 1bc4:3 not_in_hw
DEBUG    | tcconfig.parser._filter:__parse_filter_id:202 - succeed to parse filter id: filter-id=801::800, line=filter parent 1bc4: protocol ipv6 pref 6 u32 chain 0 fh 801::800 order 2048 key ht 801 bkt 0 flowid 1bc4:3 not_in_hw
DEBUG    | simplesqlite.core:insert_many:631 - insert 1 records into filter
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/simplesqlite/core.py", line 663, in insert_many
    self.connection.executemany(query, records)
sqlite3.IntegrityError: NOT NULL constraint failed: filter.device

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/tcshow", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.8/dist-packages/tcconfig/tcshow.py", line 212, in main
    tc_params = extract_tc_params(options)
  File "/usr/local/lib/python3.8/dist-packages/tcconfig/tcshow.py", line 188, in extract_tc_params
    rule_parser.parse()
  File "/usr/local/lib/python3.8/dist-packages/tcconfig/parser/shaping_rule.py", line 105, in parse
    self.__parse_device(self.ifb_device)
  File "/usr/local/lib/python3.8/dist-packages/tcconfig/parser/shaping_rule.py", line 115, in __parse_device
    self.__parse_tc_filter(device)
  File "/usr/local/lib/python3.8/dist-packages/tcconfig/parser/shaping_rule.py", line 293, in __parse_tc_filter
    self.__filter_parser.parse(
  File "/usr/local/lib/python3.8/dist-packages/tcconfig/parser/_filter.py", line 139, in parse
    Filter.insert(self.__get_filter())
  File "/usr/local/lib/python3.8/dist-packages/simplesqlite/model.py", line 226, in insert
    cls.__connection.insert(cls.get_table_name(), record)
  File "/usr/local/lib/python3.8/dist-packages/simplesqlite/core.py", line 594, in insert
    self.insert_many(table_name, records=[record], attr_names=attr_names)
  File "/usr/local/lib/python3.8/dist-packages/simplesqlite/core.py", line 667, in insert_many
    raise OperationalError(
simplesqlite.error.OperationalError: /usr/local/lib/python3.8/dist-packages/simplesqlite/model.py(226) insert: failed to execute query:
  query=INSERT INTO filter(device,"filter_id",flowid,protocol,priority,[src-network],[dst-network],[src-port],[dst-port],classid,handle) VALUES (?,?,?,?,?,?,?,?,?,?,?)
  msg='NOT NULL constraint failed: filter.device'
  db=:memory:
  records=[[None, '801::800', '1bc4:3', 'ipv6', 6, '0.0.0.0/0', '0.0.0.0/0', None, None, None, None]]

john@x1:~$ sudo tcshow wlp3s0 --debug | copy
DEBUG    | simplesqlite.core:connect:266 - connect to a SQLite database: path=':memory:', mode=w
DEBUG    | simplesqlite.core:create_table:1277 - CREATE TABLE IF NOT EXISTS 'filter' (device TEXT NOT NULL, "filter_id" TEXT, flowid TEXT, protocol TEXT, priority INTEGER, [src-network] TEXT, [dst-network] TEXT, [src-port] INTEGER, [dst-port] INTEGER, classid TEXT, handle INTEGER)
DEBUG    | simplesqlite.core:create_table:1277 - CREATE TABLE IF NOT EXISTS 'qdisc' (device TEXT NOT NULL, "direct_qlen" INTEGER, parent TEXT, handle TEXT, delay TEXT, [delay-distro] TEXT, loss TEXT, duplicate TEXT, corrupt TEXT, reorder TEXT, rate TEXT)
DEBUG    | subprocrunner._subprocess_runner:__debug_print_command:264 - /usr/sbin/tc filter show dev wlp3s0 root
DEBUG    | subprocrunner._subprocess_runner:__debug_print_command:264 - /usr/sbin/tc class show dev wlp3s0
DEBUG    | tcconfig.parser._class:parse:57 - tc class parse result: []
DEBUG    | subprocrunner._subprocess_runner:__debug_print_command:264 - /usr/sbin/tc filter show dev wlp3s0
DEBUG    | subprocrunner._subprocess_runner:__debug_print_command:264 - /usr/sbin/tc qdisc show dev wlp3s0
DEBUG    | subprocrunner._subprocess_runner:__debug_print_command:264 - /usr/sbin/tc class show dev ifb7108
DEBUG    | tcconfig.parser._class:parse:51 - parse a class entry: {'device': 'ifb7108', 'classid': '1bc4:1', 'rate': '32Gbps'}
DEBUG    | tcconfig.parser._class:parse:51 - parse a class entry: {'device': 'ifb7108', 'classid': '1bc4:2', 'rate': '32Gbps'}
DEBUG    | tcconfig.parser._class:parse:51 - parse a class entry: {'device': 'ifb7108', 'classid': '1bc4:3', 'rate': '32Gbps'}
DEBUG    | simplesqlite.core:__create_table_from_tabledata:1726 - __create_table_from_tabledata:
    tbldata=table_name=class, headers=[device, classid, rate], cols=3, rows=3
DEBUG    | tabledata.normalizer:normalize:53 - normalize: SQLiteTableDataSanitizer
DEBUG    | dataproperty._extractor:to_dp_matrix:453 - max_workers=1, preprocessor=strip_str=None, replace_tabs_with_spaces=True, tab_length=2, line_break_handling=LineBreakHandling.NOP, line_break_repl= , escape_html_tag=False, escape_formula_injection=False
DEBUG    | dataproperty._extractor:to_column_dp_list:387 - converting to column dataproperty:
DEBUG    | dataproperty._extractor:to_column_dp_list:415 -   params:
DEBUG    | dataproperty._extractor:to_column_dp_list:415 -     headers=3
DEBUG    | dataproperty._extractor:to_column_dp_list:415 -     prev_col_count=None
DEBUG    | dataproperty._extractor:to_column_dp_list:415 -     matrix_formatting=MatrixFormatting.TRIM
DEBUG    | dataproperty._extractor:to_column_dp_list:415 -     column_type_hints=()
DEBUG    | dataproperty._extractor:to_column_dp_list:417 -   results:
DEBUG    | dataproperty._extractor:to_column_dp_list:447 -     column=0, type=STRING, align=left, ascii_width=7
DEBUG    | dataproperty._extractor:to_column_dp_list:447 -     column=1, type=STRING, align=left, ascii_width=7
DEBUG    | dataproperty._extractor:to_column_dp_list:447 -     column=2, type=STRING, align=left, ascii_width=6
DEBUG    | simplesqlite.core:create_table:1277 - CREATE TABLE IF NOT EXISTS 'class' (device TEXT, classid TEXT, rate TEXT)
DEBUG    | simplesqlite.core:insert_many:631 - insert 3 records into class
DEBUG    | simplesqlite.core:commit:1579 - commit: path=':memory:'
DEBUG    | tcconfig.parser._class:parse:57 - tc class parse result: [
    {
        "device": "ifb7108",
        "classid": "1bc4:1",
        "rate": "32Gbps"
    },
    {
        "device": "ifb7108",
        "classid": "1bc4:2",
        "rate": "32Gbps"
    },
    {
        "device": "ifb7108",
        "classid": "1bc4:3",
        "rate": "32Gbps"
    }
]
DEBUG    | subprocrunner._subprocess_runner:__debug_print_command:264 - /usr/sbin/tc filter show dev ifb7108
DEBUG    | tcconfig.parser._filter:parse:92 - failed to parse mangle: filter parent 1bc4: protocol ip pref 5 u32 chain 0
DEBUG    | tcconfig.parser._filter:parse:126 - failed to parse flow id: filter parent 1bc4: protocol ip pref 5 u32 chain 0
DEBUG    | tcconfig.parser._filter:parse:136 - failed to parse filter: filter parent 1bc4: protocol ip pref 5 u32 chain 0
DEBUG    | tcconfig.parser._filter:parse:92 - failed to parse mangle: filter parent 1bc4: protocol ip pref 5 u32 chain 0 fh 800: ht divisor 1
DEBUG    | tcconfig.parser._filter:parse:126 - failed to parse flow id: filter parent 1bc4: protocol ip pref 5 u32 chain 0 fh 800: ht divisor 1
DEBUG    | tcconfig.parser._filter:parse:136 - failed to parse filter: filter parent 1bc4: protocol ip pref 5 u32 chain 0 fh 800: ht divisor 1
DEBUG    | tcconfig.parser._filter:parse:92 - failed to parse mangle: filter parent 1bc4: protocol ip pref 5 u32 chain 0 fh 800::800 order 2048 key ht 800 bkt 0 flowid 1bc4:2 not_in_hw
DEBUG    | tcconfig.parser._filter:__parse_flow_id:183 - succeed to parse flow id: flow-id=1bc4:2, line=filter parent 1bc4: protocol ip pref 5 u32 chain 0 fh 800::800 order 2048 key ht 800 bkt 0 flowid 1bc4:2 not_in_hw
DEBUG    | tcconfig.parser._filter:__parse_protocol:188 - succeed to parse protocol: protocol=ip, line=filter parent 1bc4: protocol ip pref 5 u32 chain 0 fh 800::800 order 2048 key ht 800 bkt 0 flowid 1bc4:2 not_in_hw
DEBUG    | tcconfig.parser._filter:__parse_priority:195 - succeed to parse priority: priority=5, line=filter parent 1bc4: protocol ip pref 5 u32 chain 0 fh 800::800 order 2048 key ht 800 bkt 0 flowid 1bc4:2 not_in_hw
DEBUG    | tcconfig.parser._filter:__parse_filter_id:202 - succeed to parse filter id: filter-id=800::800, line=filter parent 1bc4: protocol ip pref 5 u32 chain 0 fh 800::800 order 2048 key ht 800 bkt 0 flowid 1bc4:2 not_in_hw
DEBUG    | tcconfig.parser._filter:parse:92 - failed to parse mangle: match 00000000/00000000 at 16
DEBUG    | tcconfig.parser._filter:parse:126 - failed to parse flow id: match 00000000/00000000 at 16
DEBUG    | tcconfig.parser._filter:__parse_filter_ipv4:354 - succeed to parse ipv4 filter: src_network=None, dst_network=0.0.0.0/0, src_port=None, dst_port=None, line=match 00000000/00000000 at 16
DEBUG    | tcconfig.parser._filter:parse:92 - failed to parse mangle: match 00000000/00000000 at 12
DEBUG    | tcconfig.parser._filter:parse:126 - failed to parse flow id: match 00000000/00000000 at 12
DEBUG    | tcconfig.parser._filter:__parse_filter_ipv4:354 - succeed to parse ipv4 filter: src_network=0.0.0.0/0, dst_network=0.0.0.0/0, src_port=None, dst_port=None, line=match 00000000/00000000 at 12
DEBUG    | tcconfig.parser._filter:parse:92 - failed to parse mangle: filter parent 1bc4: protocol ipv6 pref 6 u32 chain 0
DEBUG    | tcconfig.parser._filter:parse:126 - failed to parse flow id: filter parent 1bc4: protocol ipv6 pref 6 u32 chain 0
DEBUG    | tcconfig.parser._filter:parse:136 - failed to parse filter: filter parent 1bc4: protocol ipv6 pref 6 u32 chain 0
DEBUG    | tcconfig.parser._filter:parse:92 - failed to parse mangle: filter parent 1bc4: protocol ipv6 pref 6 u32 chain 0 fh 801: ht divisor 1
DEBUG    | tcconfig.parser._filter:parse:126 - failed to parse flow id: filter parent 1bc4: protocol ipv6 pref 6 u32 chain 0 fh 801: ht divisor 1
DEBUG    | tcconfig.parser._filter:parse:136 - failed to parse filter: filter parent 1bc4: protocol ipv6 pref 6 u32 chain 0 fh 801: ht divisor 1
DEBUG    | tcconfig.parser._filter:parse:92 - failed to parse mangle: filter parent 1bc4: protocol ipv6 pref 6 u32 chain 0 fh 801::800 order 2048 key ht 801 bkt 0 flowid 1bc4:3 not_in_hw
DEBUG    | tcconfig.parser._filter:__parse_flow_id:183 - succeed to parse flow id: flow-id=1bc4:3, line=filter parent 1bc4: protocol ipv6 pref 6 u32 chain 0 fh 801::800 order 2048 key ht 801 bkt 0 flowid 1bc4:3 not_in_hw
DEBUG    | tcconfig.parser._filter:__parse_protocol:188 - succeed to parse protocol: protocol=ipv6, line=filter parent 1bc4: protocol ipv6 pref 6 u32 chain 0 fh 801::800 order 2048 key ht 801 bkt 0 flowid 1bc4:3 not_in_hw
DEBUG    | tcconfig.parser._filter:__parse_priority:195 - succeed to parse priority: priority=6, line=filter parent 1bc4: protocol ipv6 pref 6 u32 chain 0 fh 801::800 order 2048 key ht 801 bkt 0 flowid 1bc4:3 not_in_hw
DEBUG    | tcconfig.parser._filter:__parse_filter_id:202 - succeed to parse filter id: filter-id=801::800, line=filter parent 1bc4: protocol ipv6 pref 6 u32 chain 0 fh 801::800 order 2048 key ht 801 bkt 0 flowid 1bc4:3 not_in_hw
DEBUG    | tcconfig.parser._filter:parse:115 - store filter: Filter (device=ifb7108, filter_id=800::800, flowid=1bc4:2, protocol=ip, priority=5, src-network=0.0.0.0/0, dst-network=0.0.0.0/0)
DEBUG    | simplesqlite.core:insert_many:631 - insert 1 records into filter
DEBUG    | tcconfig.parser._filter:__parse_flow_id:183 - succeed to parse flow id: flow-id=1bc4:3, line=filter parent 1bc4: protocol ipv6 pref 6 u32 chain 0 fh 801::800 order 2048 key ht 801 bkt 0 flowid 1bc4:3 not_in_hw
DEBUG    | tcconfig.parser._filter:__parse_protocol:188 - succeed to parse protocol: protocol=ipv6, line=filter parent 1bc4: protocol ipv6 pref 6 u32 chain 0 fh 801::800 order 2048 key ht 801 bkt 0 flowid 1bc4:3 not_in_hw
DEBUG    | tcconfig.parser._filter:__parse_priority:195 - succeed to parse priority: priority=6, line=filter parent 1bc4: protocol ipv6 pref 6 u32 chain 0 fh 801::800 order 2048 key ht 801 bkt 0 flowid 1bc4:3 not_in_hw
DEBUG    | tcconfig.parser._filter:__parse_filter_id:202 - succeed to parse filter id: filter-id=801::800, line=filter parent 1bc4: protocol ipv6 pref 6 u32 chain 0 fh 801::800 order 2048 key ht 801 bkt 0 flowid 1bc4:3 not_in_hw
DEBUG    | simplesqlite.core:insert_many:631 - insert 1 records into filter
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/simplesqlite/core.py", line 663, in insert_many
    self.connection.executemany(query, records)
sqlite3.IntegrityError: NOT NULL constraint failed: filter.device

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/tcshow", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.8/dist-packages/tcconfig/tcshow.py", line 212, in main
    tc_params = extract_tc_params(options)
  File "/usr/local/lib/python3.8/dist-packages/tcconfig/tcshow.py", line 188, in extract_tc_params
    rule_parser.parse()
  File "/usr/local/lib/python3.8/dist-packages/tcconfig/parser/shaping_rule.py", line 105, in parse
    self.__parse_device(self.ifb_device)
  File "/usr/local/lib/python3.8/dist-packages/tcconfig/parser/shaping_rule.py", line 115, in __parse_device
    self.__parse_tc_filter(device)
  File "/usr/local/lib/python3.8/dist-packages/tcconfig/parser/shaping_rule.py", line 293, in __parse_tc_filter
    self.__filter_parser.parse(
  File "/usr/local/lib/python3.8/dist-packages/tcconfig/parser/_filter.py", line 139, in parse
    Filter.insert(self.__get_filter())
  File "/usr/local/lib/python3.8/dist-packages/simplesqlite/model.py", line 226, in insert
    cls.__connection.insert(cls.get_table_name(), record)
  File "/usr/local/lib/python3.8/dist-packages/simplesqlite/core.py", line 594, in insert
    self.insert_many(table_name, records=[record], attr_names=attr_names)
  File "/usr/local/lib/python3.8/dist-packages/simplesqlite/core.py", line 667, in insert_many
    raise OperationalError(
simplesqlite.error.OperationalError: /usr/local/lib/python3.8/dist-packages/simplesqlite/model.py(226) insert: failed to execute query:
  query=INSERT INTO filter(device,"filter_id",flowid,protocol,priority,[src-network],[dst-network],[src-port],[dst-port],classid,handle) VALUES (?,?,?,?,?,?,?,?,?,?,?)
  msg='NOT NULL constraint failed: filter.device'
  db=:memory:
  records=[[None, '801::800', '1bc4:3', 'ipv6', 6, '0.0.0.0/0', '0.0.0.0/0', None, None, None, None]]

Output of what I think it is failing to parse:

$ tc filter show dev ifb7108
filter parent 1bc4: protocol ip pref 5 u32 chain 0 
filter parent 1bc4: protocol ip pref 5 u32 chain 0 fh 800: ht divisor 1 
filter parent 1bc4: protocol ip pref 5 u32 chain 0 fh 800::800 order 2048 key ht 800 bkt 0 flowid 1bc4:2 not_in_hw 
  match 00000000/00000000 at 16
  match 00000000/00000000 at 12
filter parent 1bc4: protocol ipv6 pref 6 u32 chain 0 
filter parent 1bc4: protocol ipv6 pref 6 u32 chain 0 fh 801: ht divisor 1 
filter parent 1bc4: protocol ipv6 pref 6 u32 chain 0 fh 801::800 order 2048 key ht 801 bkt 0 flowid 1bc4:3 not_in_hw

Output of what the sqlite3 database looks like at the time of the crash:

$ sqlite3 /tmp/tcshow.sqlite3 'select * from filter' -column -header
device      filter_id   flowid      protocol    priority    src-network  dst-network  src-port    dst-port    classid     handle    
----------  ----------  ----------  ----------  ----------  -----------  -----------  ----------  ----------  ----------  ----------
ifb7108     800::800    1bc4:2      ip          5           0.0.0.0/0    0.0.0.0/0
thombashi added a commit that referenced this issue Dec 12, 2021
@thombashi
Fix tcshow parsing: #160
c79d4f2
@thombashi
Copy link
Owner

@personalcomputer
Hi,

Thank you for your detailed report.
It would be a great help.

I had fixed the problem with c79d4f2.
The commit will be included in the next release (probably soon).

@thombashi
Copy link
Owner

Released tcconfig 0.27.1.
Please try to upgrade this version.

Feel free to reopen if you still have any problems about the issue.

@personalcomputer
Copy link
Contributor Author

Thanks @thombashi! I just tested, and I can confirm you've fixed this bug now in 0.27.1. 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@personalcomputer @thombashi