dosemu2 crashes when running DataRescue IDA 3.7 freeware installer

[tkchia]

Describe the bug
dosemu2 crashes when I try to run the installer for DataRescue IDA 3.7 freeware.

ERROR: Fault in dosemu code, in_dpmi=0
ERROR: cpu exception in dosemu code outside of VM86()!
sig: 8 trapno: 0x00  errorcode: 0x00000000  cr2: 0x00000000
eip: 0x7fd3dde9db61  esp: 0x7fd3c48a6c10  eflags: 0x00010246
cs: 0x0033  ds: 0x0000  es: 0x0000  ss: 0x0000
fs: 0x0000  gs: 0x0000
fsbase: 0x7fd3dfc37700 gsbase: (nil)

ERROR: Please install gdb, update dosemu from git, compile it with debug
info and make a bug report with the content of ~/.dosemu/boot.log at
https://github.com/stsp/dosemu2/issues
Please provide any additional info you can, like the test-cases,
URLs and all the rest that fits.

To Reproduce

• Unpack ida37fw.zip (below).
• Run install.exe under dosemu2 like so: dosemu -D9+D -t -quiet -K install.exe.
• Under install.exe, select "Install Program".

Attach the binaries

• ida37fw.zip (also available at textfiles.com).

Attach the log

• boot.log.

[stsp]

Reading symbols from /usr/bin/dosemu.bin...(no debugging symbols found)...done.

You built w/o debug info.
Also this is an fdpp bug, so its better to
open it in fdpp tracker. But I'll take a look
as its easily reproducible.

[stsp]

I added the work-around.
This installer installs the int24 handler that
always returns IGNORE status. Then it probes
all drives. The error from missing floppy drive
gets ignored, fdpp thinks its there, and crash on
trying to calculate the geometry.
I made it so the int24 handler is ignored.
Dunno if its right or wrong, but at least it works...

[stsp]

By the way, Andrew, thanks for the ivec stuff,
was very handy when debugging this. :)
One thing I miss, is a process name taken from
mcb, so that we know who set the ivec.
Would it be possible to implement?

[andrewbird]

So you want to scan the MCB chain, and see which allocated block the interrupt vector's target falls in, or something else?

[stsp]

Something like this, but walking an MCB is nasty.
How about the probabilistic approach:

• It the vector is hooked by TSR, its likely "tiny", so
  we look for its PSP at ISEG:0 and then MCB at (ISEG-1):0.
• If not, then it is likely the current prog is hooking,
  so we take its PSP from SDA, then get its MCB and
  see if it owns the vector.
• If still not - go away.

IMHO this strategy is quite simple and will catch
most of the real-life cases.

[tkchia]

Hello @stsp,

I made it so the int24 handler is ignored.
Dunno if its right or wrong, but at least it works...

Thanks for the fix!

In case it might be useful, I was able to run the IDA installer (without crashes) under DOSBox, and also the old dosemu (version 1.4.0) + FreeDOS. Perhaps they were handling int 0x24's return status in a different way.

Thank you!

[stsp]

The crash was because of division by zero,
which can easily go unnoticed on freedos.
And of course on dosbox there is a different
handling. So I don't think anything conclusive
can be said from your experiment.
Unfortunately the freedos architecture is too
complex (and far from being clean) for me to
tell for sure whether the bug was where I fixed
it, or it just shouldn't have been calculating the
drive geometry as a result of that event flow.
Namely, truename() calls media_check(),
which, in turn, calls rqblockio(C_BLDBPB, dpbp);,
which crashes. Of course for me it is quite
surprising that truename() ends up building
BPB, but if you look into InitializeAllBPBs(),
you'll see it exploits exactly that "feature"
to re-build all BPBs...
I really don't think I want to maintain this
1Mb of C code mess (and now also with
C++ mess on top). I'd like to get it working
anyhow and stop.

[stsp]

During the course of general code review, I've
fixed a few bugs, and surprisingly, ida bug seems
to have been fixed by this:
23a04f8
I reverted the initial hack, and it still seems to work!
What is much more surprising, is that also #478
is fixed by that very same commit, even though it was
reported against freedos, not fdpp!
So I am really puzzled... And perhaps this is a result
of whole-weekend coding, I am probably messing up
things.
So would be nice if you check that the problem is
fixed and the original hack is reverted.

[tkchia]

Hello @stsp,

Yes --- I tried the latest Ubuntu PPA builds (dosemu2_2.0~pre8-5386+0e04ad3~ubuntu16.04.1_amd64.deb with fdpp_0.1~beta3-0~201901132213~ubuntu16.04.1_amd64.deb), and the IDA 3.7 installer still works. Thank you!

[stsp]

Hmm!
Poof and this ticket is in an fdpp repo. :)
OKey, at least MS brings some interesting
new features to github.