-
Notifications
You must be signed in to change notification settings - Fork 0
/
devlog-wknight.html
497 lines (406 loc) · 16 KB
/
devlog-wknight.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
<html>
<head>
<link rel="stylesheet" type="text/css" href="css/style.css"/>
<script src="js/init.js"></script>
<title>Development Log - wknight - gstep</title>
</head>
<body onload="init()">
<h1>Development Log - wknight - gstep</h1>
<h2>2011-06-01</h2>
<p>
Starting on use of bstrlib in the code.
</p>
<h2>2011-06-02</h2>
<p>
Finished compile fixes yesterday, today, will start debugging (currently hangs, think I know what
the problem is). I'm tempted to use pyclewn, which might take a little more time, but I want to
get comfortable with it.
</p>
<p>
Man, pyclewn is sweet. Now I can really dive into knowing gdb while living in my favorite little
world of vim. So far, I was able to start Pyclewn, send commands to gdb using ':C...', start
and step through the gstep program, examine data in various formats, observe that my suspicion
was correct about where the program was getting stuck (failed to match the gdb terminator sequence
in the read buffer), and also note that the next read from gstep stdout was a blocking read.
Then I was able to send a 'Csigint' through pyclewn to interrupt the program and resume
interaction with gdb. All from the comfort of vim. Great stuff!
</p>
<p>
This is the development environment that RMS and disciples had with EMACS/gdb that I as a VI
apostate have been missing for so long.
</p>
<p>
Still lots of stuff to figure out with pyclewn. Such as, currently the clewn console always wants
to appear on top. But I'll work all that out in time, right now it's very usable as is.
Now is also probably the time to really start learning to use the c quick-fix features of vim.
</p>
<p>
Bug fixed! Program is currently working as expected. Next: start adding some parsing routines
to detect gdb output. First though, I'm adding this project to svn.
</p>
<h2>2011-06-10</h2>
<p>
Completed my first socket program in C for client-server communication. Will now incorporate
that into the gstep program. The initial version will simply server as an interface between
gdb and clients like vim.
</p>
<h2>2011-06-12</h2>
<p>
Sunday morning, have time to code. Communication between client and server is working.
Initial command handling is working. Next: starting on gdb launch code. First, will
check in to subversion.
</p>
<p>
Later in the day, added more debugging, which is helpful, because currently I'm debugging
an issue where the forked gserver process fails and exits on exec gdb (because of bad gdb_args
I believe) and I need to find a good way for the parent process to determine whether or not the
exec of the child process succeeds.
</p>
<p>
One way that occurs to me, after a little thought, is for the parent process to wait around
for the maximum time it would take for the faild child process to die and then check the
state using
<h2>2011-06-13</h2>
<p>
This morning, a little coding, got some Makefile issues worked out and first unit
test running to reveal that the algorithm for timeval subtraction is total bollocks.
Not sure why, I got it from the doc for libc.
</p>
<p>
No matter, I will just implement a simpler version of the algorithm.
</p>
<h2>2011-06-14</h2>
<p>
Another day, a little more coding time. I'm still sick. Will finish up unit tests for
time_sub() and then get back to work on the gdb code.
</p>
<p>
time_sub works, now I can catch the child process failure using waitpid():
</p>
<pre>
[1308054296.547812] TRACE: gdb.c:39: bef waitpid
[1308054296.548250] TRACE: gdb.c:41: aft waitpid: pid: 0, status: 0
[1308054296.549084] TRACE: gdb.c:46: diff_time: 0, 70951
[1308054296.549679] TRACE: gdb.c:39: bef waitpid
[1308054296.550238] TRACE: gdb.c:41: aft waitpid: pid: 68731, status: 256
[1308054296.550735] TRACE: gdb.c:46: diff_time: 0, 72598
[1308054296.551327] TRACE: gdb.c:39: bef waitpid
[1308054296.551770] TRACE: gdb.c:41: aft waitpid: pid: -1, status: 0
[1308054296.552341] TRACE: gdb.c:46: diff_time: 0, 74203
[1308054296.552794] TRACE: gdb.c:39: bef waitpid
[1308054296.553266] TRACE: gdb.c:41: aft waitpid: pid: -1, status: 0
[1308054296.553704] TRACE: gdb.c:46: diff_time: 0, 75570
</p>
<p>
It takes a suprisingly long time to detect the failure, in my opinion, too
close to the 100 msec limit, so I'll increase to 200 msec for now.
</p>
<p>
In addition, I should probably always call waitpid() with WNOHANG before
every access of the child process and make sure it returns 0 to check
if the child process died at some point. Think I'll just create an
is_child_running() function.
</p>
<p>
Got gdb running. Next: add code to terminate gdb when gserver exits (currently
doesn't work).
</p>
<p>
After work, little coding, added kill, looks like it works:
</p>
<pre>
mhackslab:gstep $ ./gclient start_gdb
--- recvd from server:
200 OK
mhackslab:gstep $ sudo lsof -i -P | grep gdb
gdb-i386- 72021 williamknight 3u IPv4 0x5317270 0t0 TCP *:6667 (LISTEN)
gdb-i386- 72021 williamknight 4u IPv4 0x531766c 0t0 TCP localhost:6667->localhost:50021 (CLOSE_WAIT)
mhackslab:gstep $ sudo lsof -i -P | grep 6667
gserver 72012 williamknight 3u IPv4 0x5317270 0t0 TCP *:6667 (LISTEN)
gserver 72012 williamknight 4u IPv4 0x531766c 0t0 TCP localhost:6667->localhost:50021 (CLOSE_WAIT)
gdb-i386- 72021 williamknight 3u IPv4 0x5317270 0t0 TCP *:6667 (LISTEN)
gdb-i386- 72021 williamknight 4u IPv4 0x531766c 0t0 TCP localhost:6667->localhost:50021 (CLOSE_WAIT)
mhackslab:gstep $ ./gclient quit_gserver
--- recvd from server:
200 OK
mhackslab:gstep $ sudo lsof -i -P | grep 6667
mhackslab:gstep $ sudo lsof -i -P | grep gdb
mhackslab:gstep $
</pre>
<h2>2011-06-15</h2>
<p>
Only a little time to code this morning, I want to make progress. Will try to finish up all the start/stop
stuff and then move on to relaying actual gdb commands.
</p>
<p>
Changed a bunch of return codes, now detection of child gdb startup is failing.
Ah, duh, I changed the meaning of my return codes. Must be consistent and use ret==0 for success
always.
</p>
<p>
Hm, ok, that's fixed, but how we have a zombie:
</p>
<pre>
mhackslab:gstep $ ./gclient quit_gdb
--- recvd from server:
200 OK
mhackslab:gstep $ ps aux | grep gdb
williamknight 72528 0.0 0.0 599820 460 s002 R+ 7:38AM 0:00.00 grep gdb
williamknight 72515 0.0 0.0 0 0 s001 Z+ 7:38AM 0:00.00 (gdb-i386-apple-d)
mhackslab:gstep $
</pre>
<h2>2011-06-16</h2>
<p>
Woke up very early this morning, but only got up at 5:10 am, so just a moderate amount of
time for programming. Reading about process termination first.
</p>
<p>
Getting there, I just remembered that a terminated child will become a zombie until its
termination status is reported to the parent (I think). So one possibility is for the
gserver app to wait for a bit after killing the gdb process, before quitting.
</p>
<p>
Yeah, but, after reading this, why was it a zombie after the gserver process terminated
(from linux programming interface):
</p>
<pre>
On the other
hand, if the parent terminates without doing a wait(), then the init process adopts
the child and automatically performs a wait(), thus removing the zombie process
from the system.
</pre>
<p>
Uh, because you left the parent process still running, dumbkopf. Ok, this is could,
I am finally understanding the nature of zombies and how to deal with them.
</p>
<h2>2011-06-17</h2>
<p>
Truly no time for code, but want to do a tiny bit anyhow.
</p>
<p>
gdb shutdown is working.
</p>
<h2>2011-06-19</h2>
<p>
Sunday morning, back from Lymefest.
</p>
<p>
Working on 'gdb cmd'. Looks like gdb i/o is working from the server end, now
probs on the client. Adding more logging there.
</p>
<p>
Identified the problem: current client code expects empty newline (as done
by gserver responses which end with HTTP status). Need to consider how to
handle responses from gdb process.
</p>
<p>
Thoughts in the shower: the client should read a response from the server
until the socket is closed. The HTTP status line should be sent first,
followed by the output from the gdb process.
</p>
<p>
Next, working on fixing detection of end of gdb output. When gdb outputs
to console stdout and input is from keyboard stdin, an input of an empty
line with only newline produces my current spec for gdb terminator:
</p>
<pre>
...
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-apple-darwin".
(gdb)
(gdb)
</pre>
<p>
However, when the input to gdb comes from one pipe and the output goes out
another, here's what the same input produces in the output stream:
</p>
<pre>
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-apple-darwin".
(gdb) (gdb)
</pre>
<p>
For now, I can easily fix this by dropping the leading '\n' from the gdb_terminator
spec. And I won't worry about possible false terminator that could show up in dumping
code because at some point I'll switch to the mi interpreter and detect the much
less ambiguous output termination sequences such as:
</p>
<pre>
show version
&"show version\n"
~"GNU gdb 6.3.50-20050815 (Apple version gdb-966) (Tue Mar 10 02:43:13 UTC 2009)\n"
~"Copyright 2004 Free Software Foundation, Inc.\n"
~"GDB is free software, covered by the GNU General Public License, and you are\nwelcome to change it and/or distribute copies of it under certain conditions.\nType \"show copying\" to see the conditions.\nThere is absolutely no warranty for GDB. Type \"show warranty\" for details.\n"
~"This GDB was configured as \"i386-apple-darwin\".\n"
^done
(gdb)
</pre>
<p>
Next, I added code to close the socket on the server side, but the client side is
still hanging in recv(). More to learn about sockets I guess...
</p>
<p>
From the logs I can see that the client makes the recv call
before the serve closes the socket, but I don't understand why
it wouldn't return right after the server closes it.
</p>
<p>
Next, I'll re-read the docs on recv().
</p>
<p>
Did that, no clues. The socket shows as still connected by lsof
</p>
<pre>
Google 73890 williamknight 123u IPv4 0x614b66c 0t0 TCP mhackslab:64623->pz-in-f106.1e100.net:80 (ESTABLISHED)
Google 73890 williamknight 124u IPv4 0x685466c 0t0 TCP mhackslab:64624->pz-in-f106.1e100.net:80 (ESTABLISHED)
gserver 75730 williamknight 3u IPv4 0xc83b66c 0t0 TCP *:6667 (LISTEN)
gclient 75736 williamknight 3u IPv4 0xc841a68 0t0 TCP localhost:64495->localhost:6667 (ESTABLISHED)
gdb-i386- 75737 williamknight 3u IPv4 0xc83b66c 0t0 TCP *:6667 (LISTEN)
gdb-i386- 75737 williamknight 4u IPv4 0x6812a68 0t0 TCP localhost:6667->localhost:64495 (ESTABLISHED)
</pre>
<p>
Next, will probably start logging more info on connected socket and ports.
</p>
<p>
Reading and thinking - what I'm reading tells me that closing the socker on the server side should cause
a 0 to be returned to the client in a recv() call. Is closing the 'listen' socket required? That doesn't
seem to make any sense.
</p>
<p>
I may start making calls to getsockopt() and investigating the current values I find.
</p>
<p>
Another thing I might do is investigate the current behavior in Linux and see if it's the same.
</p>
<p>
Maybe I have to do some kind of flush() call?
</p>
<p>
Later, more read. Everything I'm reading says that 'close' should shut the damn thing down. Will
try 'shutdown()' instead. I'm thinking that after close, the server side state should definitely
be 'FIN_WAIT_...', not 'ESTABLISHED' like it's currently showing:
</p>
<pre>
gserver 76902 williamknight 3u IPv4 0x5345e64 0t0 TCP *:6667 (LISTEN)
gclient 76909 williamknight 3u IPv4 0x533de64 0t0 TCP localhost:50885->localhost:6667 (ESTABLISHED)
gdb-i386- 76910 williamknight 3u IPv4 0x5345e64 0t0 TCP *:6667 (LISTEN)
gdb-i386- 76910 williamknight 4u IPv4 0x533d270 0t0 TCP localhost:6667->localhost:50885 (ESTABLISHED)
</pre>
<p>
Wait, maybe that's the answer? Is the child gdb process inheriting the ESTABLISHED socket connections
and preventing the close somehow? I can understand it inheriting the 'listening' socket, but
it doesn't make sense to me right now that it inherits the 'accepted' socket. Think I might add
a debug command 'client wait' where the server will respond with nothing and the client will
wait for a response, just to take a look at the state of the connections in that situation.
</p>
<p>
w/o gdb running:
</p>
<pre>
gserver 77139 williamknight 3u IPv4 0x5658e64 0t0 TCP *:6667 (LISTEN)
gserver 77139 williamknight 4u IPv4 0x5658270 0t0 TCP localhost:6667->localhost:52409 (ESTABLISHED)
gclient 77146 williamknight 3u IPv4 0xc834a68 0t0 TCP localhost:52409->localhost:6667 (ESTABLISHED)
</pre>
<p>
w/ gdb running:
</p>
<pre>
gserver 77159 williamknight 3u IPv4 0x5658e64 0t0 TCP *:6667 (LISTEN)
gserver 77159 williamknight 4u IPv4 0x5592e64 0t0 TCP localhost:6667->localhost:52412 (ESTABLISHED)
gclient 77163 williamknight 3u IPv4 0x5658270 0t0 TCP localhost:52412->localhost:6667 (ESTABLISHED)
gdb-i386- 77164 williamknight 3u IPv4 0x5658e64 0t0 TCP *:6667 (LISTEN)
gdb-i386- 77164 williamknight 4u IPv4 0x5592e64 0t0 TCP localhost:6667->localhost:52412 (ESTABLISHED)
</pre>
<h2>2011-06-20</h2>
<p>
Little monday-morning coding. Wrote a test server/client echo test that confirmed that closing
the 'accept' socket on server side shuts down client receive. Next, need to combine that with
creating a child process.
</p>
<p>
Excellent, forked a child process after the accept but before the echo, confirmed that it holds
the client in recv() call even after the parent process closes both sockets. So that's it.
</p>
<h2>2011-06-21</h2>
<p>
Up early, will try to get this thing working. Will just start by closing the socket fd in
any gdb forked process that occurs.
</p>
<p>
Did that, I believe I also need to close the accept socket.
</p>
<p>
That did it, I'm finally getting output from gdb to the client!
</p>
<p>
Need a little more cleanup on the server side (closing sockets), and then next:
start passing actual gdb_cmd args to gdb.
</p>
<h2>2011-06-22</h2>
<p>
Will try to add stderr to gdb stdout pipe.
</p>
<p>
Did it, looks like it works. With 4 split windows in vim, this is really starting to get
useful. Next, will add stripping of headers and status line from gclient.
</p>
<p>
Did that, not quite working, will debug gclient using gserver!
</p>
<p>
Hmm, will probably get a port conflict though. No, port wasn't a problem, but
it didn't work. No bigs.
</p>
<h2>2011-06-23</h2>
<p>
So I actually started using this tool for debugging yesterday, and it is useful and works.
However, it would be much nicer if the auto-line-highlighting works and so that's
what I'll start on next.
</p>
<p>
Debugging mail on mac, looks like similar problem to deb6, so will debug that today.
</p>
check out: "The Book of IMAP: Building a Mail Server with Courier and Cyrus"
<h2>2011-07-09</h2>
<p>
On vacation - hallellujah. Hacking on BosWars and learning scons, I want to get
gstep to work with pdb as well as gdb. Since it's such a simple project, I'm
inclined to just fork and dup code at this point, maybe consolidate/share later.
</p>
<p>
Actually, could keep single project, just add pserver.c and pclient.c. Simple.
</p>
<p>
And a more general approach that I'll take with pserver is to ditch the extra
commands that start and stop the debugger. Instead, the user specifies
the command-line arg of the process to launch. This will allow me to debug
scons using its debug flag, as I'm not sure at this time how to correctly
load and debug it from python on the command-line.
</p>
<h2>2011-08-06</h2>
<p>
I want to finish adding the python implementation (pclient/pserver) but also
realize that I need to add docs to the existing gclient/gserver project.
</p>
<p>
Added man docs and command-line help.
</p>
<p>
Next: pserver (runs python and pdb)
</p>
<h2>2011-08-25</h2>
<p>
Added more docs for line-by-line debugging setup in vim.
</p>
<h2>2011-12-15</h2>
<p>
I'm going to add a help command today.
</p>
<h2>2012-04-06</h2>
<p>
Moving to github.
</p>
</body>
</html>