[kernel] [libc] [elksemu] (ABI CHANGE) Make _signal syscall accept a far pointer to a far signal handler

[tkchia]

This is in preparation for adding support for "medium" memory model (far code + near data) programs to ELKS (see https://github.com/tkchia/build-ia16/issues/14).

Currently the kernel assumes that, when a user process handles an asynchronous signal, the signal handler can simply return to a near (16-bit) address. But this may not be true once ELKS allows a user process to run code in a far text segment.

This patch changes the semantics of the _signal syscall so that it accepts a far (32-bit) signal handler pointer which returns to a far address.

To save data space, I also changed the kernel's internal signal handling structures.

[ghaerr]

Hello @tkchia,

Wow, nice work! Impressive.

I like the approach of having the kernel only handle dispositions and a single process-wide far handler, which also saves task data space.

In exec.c, instead of setting a bogus signal handler, why not just reset the process handler to 0, and then handle the very unlikely no-handler case in do_signal with a printk? I feel that would be cleaner, keep more signal code out of exec, and printk instead of stopping the system with panic.

Did you test with DEBUG_SIG on, I am curious if that helped with your implementation.

[ghaerr]

A side topic: I happened to be thinking of implementing ftruncate last night, which takes two arguments, an int and a long, and ran into the same issue with syscall.dat.

It seems there is no way to specify long arguments. How does this work, the argument count isn't checked, do the scripts just push the total argument count of 'int's and the kernel do_ function just "matches up"?

[tkchia]

Hello @ghaerr,

In exec.c, instead of setting a bogus signal handler, why not just reset the process handler to 0, and then handle the very unlikely no-handler case in do_signal with a printk? I feel that would be cleaner, keep more signal code out of exec, and printk instead of stopping the system with panic.

The bogus_sig_handler is more of a debugging aid (perhaps I should guard it with DEBUG_SIG or some such).

From my understanding, if everything works properly, currentp->sig.handler should never be dereferenced or used if a program has never set any signal handlers. If a program ends up invoking currentp->sig.handler without actually setting any signal handler, then it means the whole system is in a corrupt state.

Did you test with DEBUG_SIG on, I am curious if that helped with your implementation.

Yes, I did. :-)

It seems there is no way to specify long arguments. How does this work, the argument count isn't checked, do the scripts just push the total argument count of 'int's and the kernel do_ function just "matches up"?

Basically yes, I would say. The libc syscall routines simply pick up the syscall arguments from the user stack, and stuff then into registers %bx, %cx, %dx, etc., before saying int $0x80. On the kernel side, the int $0x80 handler (see irqtab.S and entry.S) pushes the register arguments onto the kernel stack, then invokes the appropriate sys_... routines.

Thank you!

[ghaerr]

From my understanding, if everything works properly, currentp->sig.handler should never be dereferenced or used if a program has never set any signal handlers. If a program ends up invoking currentp->sig.handler without actually setting any signal handler, then it means the whole system is in a corrupt state.

Yep and yep.

The bogus_sig_handler is more of a debugging aid (perhaps I should guard it with DEBUG_SIG or some such).

That's fine. After debugging is completed, I think it should be removed from exec.c. I am increasingly concerned about kernel code/data space, and keeping things compartmentalized. Also, I have removed many panic's and changed them to printk's where possible, as in general panics hinder the ability to find the problem as the system is inaccessible.

[ghaerr]

@tkchia - I am fine with this being committed now, thanks!!

[tkchia]

Hello @ghaerr,

After debugging is completed, I think it should be removed from exec.c.

OK.

Also, I have removed many panic's and changed them to printk's where possible, as in general panics hinder the ability to find the problem as the system is inaccessible.

I think a longer-term solution to this might be to add some more debugging and introspection facilities to the panic routine, so that one can see what went wrong with the system even after normal operations are frozen. (This will probably be after we manage to magic up some additional code space for the kernel.)

Thank you!

[Mellvik]

ELKS no longer compiles after latest merge. /Users/helge/src/mellvik/elks/libc/system/signal.c: In function ‘signal’: /Users/helge/src/mellvik/elks/libc/system/signal.c:50:10: error: cannot take address of far static variable rv = _signal(number, (__kern_sighandler_t) _syscall_signal); ~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /Users/helge/src/mellvik/elks/libc/system/signal.c:50:10: note: use ‘-msegment-relocation-stuff’ to allow this —Mellvik

…

13. mai 2020 kl. 11:52 skrev tkchia ***@***.***>: Merged #629 <#629> into master. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#629 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA3WGOEN76DCLSB3FHUDFX3RRJUUBANCNFSM4M65MBZQ>.

[tkchia]

Hello @Mellvik,

ELKS no longer compiles after latest merge.

The changes require an updated gcc-ia16 to build properly --- you will need to arrange to download the updated toolchain and rebuild ELKS. E.g.

git clean -x -f
./build.sh

Thank you!

[Mellvik]

Thank you @tkchia, I'm getting build errors in the toolchain too: /Applications/Xcode.app/Contents/Developer/usr/bin/make -C /Users/helge/src/mellvik/elks/cross/build/emu86 all cc -g -c -o op-common.o op-common.c cc -g -c -o op-id-name.o op-id-name.c cc -g -c -o op-class.o op-class.c cc -g -c -o emu-mem-io.o emu-mem-io.c cc -g -c -o emu-proc.o emu-proc.c cc -g -c -o emu-serial.o emu-serial.c emu-serial.c:68:10: warning: implicit declaration of function 'getpt' is invalid in C99 [-Wimplicit-function-declaration] _ptm = getpt (); ^ 1 warning generated. cc -g -c -o emu-int.o emu-int.c cc -g -c -o op-exec.o op-exec.c cc -g -c -o emu-main.o emu-main.c emu-main.c:45:37: warning: format specifies type 'unsigned long' but the argument has type 'off_t' (aka 'long long') [-Wformat] printf ("info: file size=%lXh\n", size); ~~~ ^~~~ %llX 1 warning generated. cc -o emu86 op-common.o op-id-name.o op-class.o emu-mem-io.o emu-proc.o emu-serial.o emu-int.o op-exec.o emu-main.o Undefined symbols for architecture x86_64: "_getpt", referenced from: _serial_init in emu-serial.o ld: symbol(s) not found for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) make[1]: *** [emu86] Error 1 make: *** [/Users/helge/src/mellvik/elks/cross/build/.emu86.build] Error 2 Build script has terminated with error 1 What did I miss? —Mellvik

…

13. mai 2020 kl. 13:25 skrev tkchia ***@***.***>: Hello @Mellvik <https://github.com/Mellvik>, ELKS no longer compiles after latest merge. The changes require an updated gcc-ia16 to build properly --- you will need to arrange to download the updated toolchain and rebuild ELKS. E.g. git clean -x -f ./build.sh Thank you! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#629 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA3WGOBESPLSGB45YS3I4ETRRJ7S7ANCNFSM4M65MBZQ>.

[tkchia]

Hello @Mellvik,

Undefined symbols for architecture x86_64:
  "_getpt", referenced from:
      _serial_init in emu-serial.o
ld: symbol(s) not found for architecture x86_64

Thanks for the report. This seems to come from @mfld-fr's emu86 project, which is not really part of the gcc-ia16 toolchain (but might be useful for testing ELKS).

I guess the error is because there is no getpt () function on OS X, which I presume you are using.

I will see if I can fix this and submit a pull request to @mfld-fr. Meanwhile you can try and modify the line

_ptm = getpt ();

to

_ptm = posix_openpt (O_RDWR);

and rerun ./build.sh. Do let me know if this still does not work.

Thank you!

[Mellvik]

Thanks @tkchia, that did the trick. —Mellvik

…

13. mai 2020 kl. 14:59 skrev tkchia ***@***.***>: _ptm = posix_openpt (O_RDWR);

[ghaerr]

Hello @tkchia,

The changes require an updated gcc-ia16 to build properly --- you will need to arrange to download the updated toolchain and rebuild ELKS.
git clean -x -f
./build.sh

It used to be that running ./build.sh (or perhaps the previous .sh that built the the toolchain alone), was able to be update the toolchain without deeply cleaning the other portion of the source tree. Now, as @Mellvik has indicated, it doesn't realize the compiler chain has been updated and just runs menuconfig, make clean; make.

In my case, I can't afford to clean all untracked files, as I have many helper scripts being used for current projects. Thus, I have to clone a new elks tree, ./build.sh, then mv the cross/ chain over.

Can you suggest a modification that would (re)allow just the build chain to be upgraded?
There is value in having either two scripts, or having build.sh rebuild the toolchain, for scenarios just like this one.

Undefined symbols for architecture x86_64:
"_getpt", referenced from:
_serial_init in emu-serial.o
ld: symbol(s) not found for architecture x86_64

I had been getting this error the last few times (OS X also), but have ignored it, since the toolchain build is usable at this point, and the emulator won't run on OS X anyways. Thank you for the fix suggestion, I should have submitted an issue at that time but didn't think to do so.

Thank you!

[tkchia]

Hello @ghaerr,

Now, as @Mellvik has indicated, it doesn't realize the compiler chain has been updated and just runs menuconfig, make clean; make.

I think this is because tools/Makefile now does some caching to avoid rebuilding toolchains that have already been built.

Can you suggest a modification that would (re)allow just the build chain to be upgraded?

Looks like an issue worth looking into. Apparently tools/Makefile has a prune rule that will clean away an old build of the toolchain. Perhaps add a script, similar to tools/build.sh, that does a make ... prune, then a make ... all?

Thank you!

[ghaerr]

@tkchia,

On another point regarding the toolchain build, I decided to clone from scratch to build another cross/ directory aside from my existing work.

FYI, After cloning, build.sh was run, with the following result:

--2020-05-13 10:03:33--  ftp://gcc.gnu.org/pub/gcc/infrastructure/gmp-4.3.2.tar.bz2
           => ‘gmp-4.3.2.tar.bz2’
Resolving gcc.gnu.org... 2620:52:3:1::246e:9693:128c, 8.43.85.97
Connecting to gcc.gnu.org|2620:52:3:1::246e:9693:128c|:21... connected.
Logging in as anonymous ... Logged in!
==> SYST ... done.    ==> PWD ... done.
==> TYPE I ... done.  ==> CWD (1) /pub/gcc/infrastructure ... done.
==> SIZE gmp-4.3.2.tar.bz2 ... 1897483
==> EPSV ... couldn't connect to 2620:52:3:1::246e:9693:128c port 57552: Connection refused
make: *** [/Users/greg/net/elks-gh/cross/build/.gcc.src] Error 1
Build script has terminated with error 1

Running ./build.sh again (without clean), started compilation, rather than trying to pull down gmp. It then ran for awhile, then errored in compilation:

../../gcc-src/mpfr/mpfr.h:293:40: error: unknown type name '__gmp_const'
  mpfr_set_str _MPFR_PROTO ((mpfr_ptr, __gmp_const char *, int, mpfr_rnd_t));
                                       ^
../../gcc-src/mpfr/mpfr.h:293:52: error: expected ')'
  mpfr_set_str _MPFR_PROTO ((mpfr_ptr, __gmp_const char *, int, mpfr_rnd_t));
                                                   ^
../../gcc-src/mpfr/mpfr.h:293:29: note: to match this '('
  mpfr_set_str _MPFR_PROTO ((mpfr_ptr, __gmp_const char *, int, mpfr_rnd_t));
                            ^
../../gcc-src/mpfr/mpfr.h:295:45: error: unknown type name '__gmp_const'
  mpfr_init_set_str _MPFR_PROTO ((mpfr_ptr, __gmp_const char *, int,
                                            ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
1 warning and 20 errors generated.
make[4]: *** [exceptions.lo] Error 1
make[3]: *** [all-recursive] Error 1
make[2]: *** [all-mpfr] Error 2
make[1]: *** [all] Error 2
make: *** [/Users/greg/net/elks-gh/cross/build/.gcc.build] Error 2
Build script has terminated with error 1

Would there be a way that build.sh or another script could restart or check the required .zip files in cross/dist instead of having to start from scratch?

I suppose I'll 'rm -rf cross/build' and try again.

[ghaerr]

Perhaps add a script, similar to tools/build.sh, that does a make ... prune, then a make ... all?

That's not bad, and will certainly force a new build.

I was thinking something along the lines of either fixing that build.sh doesn't recognize that a updated build chain is required, or having a separate .sh file that determined the tool chain was out of date, then updated it only, without requiring any dangerous git cleaning. It seems that tools/Makefile has version numbers of gcc and binutils, but they apparently aren't checked by build.sh.

I suppose I'll 'rm -rf cross/build' and try again.

That worked - please regard my previous email as informative only.

Thank you!