Skip to content

qlog-dancer: improve conversion speed and disk usage and fix garbled output on Windows#2377

Open
csujedihy wants to merge 2 commits intocloudflare:masterfrom
csujedihy:yi/fix-qlog-dancer-win
Open

qlog-dancer: improve conversion speed and disk usage and fix garbled output on Windows#2377
csujedihy wants to merge 2 commits intocloudflare:masterfrom
csujedihy:yi/fix-qlog-dancer-win

Conversation

@csujedihy
Copy link

@csujedihy csujedihy commented Mar 1, 2026

Issues:

  1. On Windows, since tabled's sharp style (UTF8 unicode) is used, when being pipe'd in powershell to a text file, the box-drawing characters are misinterpreted (as ASCII). e.g.
    Γöé 71060.11 Γöé transport Γöé packet_received Γöé 1RTT pn=98831182, ACK {6065814, 6065815, }
  2. The column width automatically adapts to the longest row which leads to massive amount of whitespaces getting printed in almost all rows in the event table. Because of this, the captured text file is unnecessarily large and the conversion consumes a huge amount of memory (converting a 2GB sqlog can crash the qlog-dancer process itself on my 32GB ram laptop). Also, the trailing whitespace makes it hard to share qlog on IMs.

This PR:

  • replaces the tabled style from sharp to ascii.
  • uses simple hand-rolled formatter for event table instead of tabled, which does not output trailing space and trailing separator.

Some benchmarks:

Before:

> time qlog-dancer ~/09899a18f8d329f608474eaec06105376d5ff993.sqlog --report-text > old.qlog
qlog-dancer ~/09899a18f8d329f608474eaec06105376d5ff993.sqlog  >   203.70s user 22.54s system 99% cpu 3:47.63 total

> ls -lsah old.qlog
17G -rw-rw-r-- 1 yi yi 17G Mar  1 14:06 old.qlog

After:

> time ./target/release/qlog-dancer ~/Downloads/09899a18f8d329f608474eaec06105376d5ff993.sqlog --report-text > new.qlog
./target/release/qlog-dancer  --report-text > new.qlog  38.82s user 10.46s system 98% cpu 49.975 total

> ls -lsah new.qlog
1.3G -rw-rw-r-- 1 yi yi 1.3G Mar  1 14:01 new.qlog

Time: 227s to 50s
Space: 17G to 1.3G

Preview of the new format:

    97157.73 | transport  | packet_received           | 1RTT pn=125962,  STREAM {id=2012, off=3685535, len=1303}
    97157.73 | transport  | packet_received           | 1RTT pn=125963,  STREAM {id=2004, off=3704992, len=871}
    97157.74 | transport  | packet_received           | 1RTT pn=125964,  STREAM_DATA_BLOCKED {id=2016, limit=3724965} STREAM {id=1992, off=4238236, len=1296}
    97157.75 | transport  | packet_received           | 1RTT pn=125965,  STREAM {id=1968, off=4737620, len=1303}
    97157.76 | transport  | packet_received           | 1RTT pn=125966,  STREAM {id=2060, off=2981977, len=1303}
    97157.77 | transport  | packet_received           | 1RTT pn=125967,  STREAM {id=2044, off=2995390, len=1303}
    97157.77 | transport  | packet_received           | 1RTT pn=125968,  STREAM {id=2016, off=3698712, len=1303}
    97157.78 | transport  | packet_received           | 1RTT pn=125969,  STREAM {id=1988, off=4191261, len=1303}
    97157.79 | transport  | packet_received           | 1RTT pn=125970,  STREAM {id=1980, off=4295003, len=1303}
    97157.84 | transport  | packet_received           | 1RTT pn=125971,  STREAM {id=2008, off=3687483, len=1303}
    97157.84 | transport  | packet_received           | 1RTT pn=125972,  STREAM {id=2052, off=3006628, len=1303}
    97157.86 | transport  | data_moved                | id=2044, off=2992784, len=3909, from=transport, to=application, 
    97157.88 | transport  | data_moved                | id=2016, off=3696106, len=3909, from=transport, to=application, 
    97157.89 | transport  | data_moved                | id=1988, off=4188655, len=3909, from=transport, to=application, 
    97157.90 | transport  | data_moved                | id=1980, off=4292397, len=3909, from=transport, to=application, 
    97157.91 | transport  | data_moved                | id=2008, off=3684877, len=3909, from=transport, to=application, 
    97160.64 | transport  | data_moved                | id=2052, off=3004022, len=3909, from=transport, to=application, 
    97160.66 | transport  | data_moved                | id=2012, off=3684232, len=2073, from=transport, to=application, 
    97160.66 | transport  | data_moved                | id=2012, off=3686305, len=1, from=transport, to=application, 
    97160.67 | transport  | data_moved                | id=2012, off=3686306, len=1, from=transport, to=application, 
    97160.68 | transport  | data_moved                | id=2012, off=3686307, len=3, from=transport, to=application,

@csujedihy csujedihy requested a review from a team as a code owner March 1, 2026 22:39
@csujedihy csujedihy force-pushed the yi/fix-qlog-dancer-win branch from 1eb4a5c to 014e4c3 Compare March 2, 2026 17:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant