Collapse OS in Forth

[mistergibson]

Please see this: http://www.camelforth.com/page.php?5

Forth is one of my past loves. It can be used for most things rather well and is the precursor to PostScript found in printers. Using Forth one could conceivably construct a Display PostScript like system to build a rudimentary GUI system, and provide print support.

Any chance of including a version of CamelForth with CollapseOS? I'd like to hear your thoughts.

Cheers, and great work on this project!

G.

[nastepnym]

and mruby ;)
400KiB are needed

[nastepnym]

and mruby ;)
400KiB are needed

[Skrylar]

I would suggest changing the question to one about including a FORTH at all.

FORTH having a low burden to get an environment running on exotic hardware is one of its most significant benefits.

Though mostly in that FORTH code is just barely a veneer over stack machine assembly. It might also be worth looking at old Z80 interpreters (should be able to find them around old infocom fandoms, since they like to keep that stuff alive) or various small VMs (RetroFORTH's Nga and Ngaro are well documented and tiny, Lua's VM design is very battle tested, and I think Janet's is basically Lua but on a stack machine)

You just need a VM which is easy to port, and then you can run whatever compilers on top of that 😉 Amusingly, the need to be portable across multiple different computing platforms (even though they were still mostly Z80's) is one of the reasons scripting used to be so prevalent in the Amstraad days.

[agsb]

I agree, a minimal stack forth can support a environment OS and be used to make compilers and "primal" archive functions.

[hsoft]

FORTH came a lot in the different comments I got. I knew it by name, but I've been convinced to look more into it, which I started to. We'll probably want a FORTH interpreter, yes.

[paulidale]

Forth is often denser than assembly code. It's very portable and only takes a tiny kernel. It seems absolutely ideal for this project. For example: the book Threaded Interpretive Languages covers most of the implementation details.

The big problem with Forth is that it can often be difficult to understand. Easer to understand but not quite so compact and somewhat slower is LISP. It would be another good language for this kind of project -- again it only requires a tiny kernel.

[Skrylar]

Lisp has more compiler overhead, kind of. The dictionary would be bigger (because they use full names like read-file instead of something like F<) but compiled code would ultimately be pointers in both. And having to read an entire function definition before compiling it (I suspect Forth can just 1:1 words with machine code, which does work even if the result is sub-optimal.)

Not much of a reason it should run slower afterwards though. You could probably compile the Lisp to Forth in the background.

[lmamakos]

Including a variation on Jones FORTH targeted to the Z80 would be a nice addition.

See https://github.com/nornagon/jonesforth/blob/master/jonesforth.S for most of the machine-specific code and what I think is the best example of literate programming I've ever seen! You start reading the comments at the top, with more and more bits of code interspersed and then by the time you reach the end of the file, you have the core of the FORTH kernel.

The mode of presentation makes a great tool for someone that then would want to target some other hardware architecture.

[lmamakos]

Any chance of including a version of CamelForth with CollapseOS? I'd like to hear your thoughts.

The other nice resource that accompany's CamelForth is a series of wonderful articles written about its design and implementation choices. Starting at https://www.bradrodriguez.com/papers/moving1.htm and following parts. Again, another great thing to have at your disposal when considering porting the FORTH kernel to a different CPU architecture.

[jes]

We'll probably want a FORTH interpreter, yes.

How does this help preserve humanity's ability to program microcontrollers with low-tech tools?

I'm not saying there shouldn't be a FORTH interpreter! Just perhaps that the goal of the project isn't as clear to many of us as it could be.

[hsoft]

@jes from what I understand (still exploring this field), bootstrapped FORTH could be as compact if not more compact than straight ASM. This makes it an interesting path to explore.

[agsb]

2 cents. I understood the Z80 as a basic model for development and testing, to be easy translated into many cpu; A minimal system, must have a VT100 emulator, and parameters for boot and devices. As tools, a forth interpreter is a bonus to make a C compiler and then all tools.

…

On Wed, Oct 16, 2019 at 2:02 PM Virgil Dupras ***@***.***> wrote: @jes <https://github.com/jes> from what I understand (still exploring this field), bootstrapped FORTH could be as compact if not more compact than straight ASM. This makes it an interesting path to explore. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#4?email_source=notifications&email_token=ADR73T4GFCEHPV26CH5PVATQO5CLDA5CNFSM4I6UHPSKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBNG7LI#issuecomment-542797741>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADR73T57FUSMRATXBPKBLXTQO5CLDANCNFSM4I6UHPSA> .

-- *"Sling and move continents, a rock at a time.*"

[jodavaho]

Every programmer (or very nearly) learns some assembly. It's ugly and painful but easy enough to grok once you realize how dumb a computer is (that's the point of learning assembly, I think). A quick poll around (large company of programmers) reveals nobody knows any forth. Perhaps it is mainstream in some community, and perhaps it is the best tool for this job, but if accessibility is the goal, then this goes against the utility to any non-specialized technologist / programmer post-collapse.

This is not an argument against FORTH, which might be the shiniest bit of bits around, but an argument against the accessibility penalty of collapseos. Granted, I'm a non-contributing spectator at the moment, so take this with a grain of salt.

[hsoft]

Thanks @lmamakos for the Moving Forth link. It's very interesting.

I've been exploring Forth and it's a fascinating field, but too time consuming to explore and I'll think I'll stop there. So far, what I've read on that subject made me come to a few conclusions. Please, someone more knowledgable correct me if I'm wrong.

Forth only deals with 16-bit numbers

Even on 8-bit CPUs, it doesn't seem imaginable to have a forth with 8-bit cells. Making all operations of a 8-bit CPU 16-bit seems a bit wasteful to me. Doesn't Forth incur a heavy performance penalty compared to hand-written assembly?

Forth has many layers of indirection

I looked at how threads work in Forth and although we can't call this an interpreted language, it seems to me that there's much more indirection than in assembly language we see in current Collapse OS code. That would, again, incur a heavy performance penalty.

Is Camel Forth self-hosting?

Unless I'm missing something, CamelFoth is written in z80 assembly, but it doesn't include a z80 assembler. To make CamelFoth self-hosting, a z80 assembler would need to be written in Forth.

Conclusion

Unless I'm missing something (I probably am, but please correct me), these points above would definitely rule out Forth as being the "foundation language" for Collapse OS: too wasteful with regards to resources.

If someone wanted to port CamelForth to Collapse OS' (which would probably be a minor task considering that it can be trivially made self standalone), I'd say it would be a nice addition, but as @jes says, I'm not sure how it brings Collapse OS' closer to its goal.

Side note: when trying to answer "does someone know of a z80 assembler in Forth?" question, I came across https://github.com/siraben/zkeme80 which looks nice. But it seems that its assembler is written in Scheme, so the question still stands.

[hsoft]

Some further research I've made led me to https://www.bradrodriguez.com/papers/tcjassem.txt (Brad again!) which would seem to indicate that Forth is well suited for writing assemblers.

[Allisontheolder]

I've seen stand alone forth that fits in something like 8k rom.
The 1802 based ELF2000 system has Forth in eprom, along with a
monitor, bios, library routines, and even tiny basic and an editor.
Forth is a very compact language.

I'm not a forth fan it makes my head hurt but have used it and
it does seem efficient. Much more so than NASCOM basic.

Allison

[agsb]

And a formal C compiler, with cpp c1 c2, could be made in forth.

…

On Tue, Nov 12, 2019 at 1:40 AM Allisontheolder ***@***.***> wrote: I've seen stand alone forth that fits in something like 8k rom. The 1802 based ELF2000 system has Forth in eprom, along with a monitor, bios, library routines, and even tiny basic and an editor. Forth is a very compact language. I'm not a forth fan it makes my head hurt but have used it and it does seem efficient. Much more so than NASCOM basic. Allison — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#4?email_source=notifications&email_token=ADR73T6INBR5L6CZCYLL7QLQTIXSDA5CNFSM4I6UHPSKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDZAEPA#issuecomment-552731196>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADR73TYZSL7YORLM5T2X24DQTIXSDANCNFSM4I6UHPSA> .

-- *"Sling and move continents, a rock at a time.*"

[Allisontheolder]

A lot of it will not fit stuff seems to come from people only used to PC (winders/linux/Mac)
space where things are huge and bloated. The micro world (6502/1802/8080/z80/6800)
people were used to limited ram and efficient code was the rule so much of the stuff from
then will fit easily. Forth was one such item of code.

Allison

[Allisontheolder]

Looked in my Forth folder and:
Camel Forth Z80 for CP/M, NS*dos and a standalone version
with a bag and tag file system for a simple 765 based FDC.
All three came form the Teserect BBS around '84ish.
The largest is the standalone and its under 8K.
They were PD then.

There was an 8080, PDP-11(RT-11 ) and 6502 (apple prodos) version as well.

[PeterForth]

Openboot Openbios and OpenBsd are examples of using Forth as OS. Camelforth will start a Forth from hex code, as also any FIG forth for Z80, there are many free listings. Have a look at my collection of videos on Youtube so you can learn more about forth. This is my contact form you can reach me by email and I will answer your questions and try to help. https://sites.google.com/view/win32forth/home/contact (Forth was 16 bit till 90´s then when the 32 bit CPUs arrived, Forth became the facto 32 bit as standart. There are also 64bits Forth, and no reason to make 128 bits forths or higher, since forth is very flexible and real easy to implement on any hardware) Another good example of the Z80 Forth systems is the Jupiter Ace https://en.wikipedia.org/wiki/Jupiter_Ace

[gitjeff2]

I just want to bring up the fact that lbForth Lite exists. It's a Forth cross-compiler for tiny devices. The cross-compiler targets many of the CPU's @hsoft appears to be interested in getting CollapseOS up and running on. It looks like adding additional CPU targets such as the Z80 wouldn't be all that difficult for someone who has a decent grasp of that architecture.

From what I can see the current list of CPU's is:

• MOS 6502
• C8051
• AVR
• MSP430
• PDP8
• PIC
• STM8
• THUMB

While this won't directly help you with a native Forth compiler, getting cross-compiler going is likely to be a huge help every time you add a new architecture. To help you wrap your head around Forth there is an excellent 10 article series that walks the reader through writing a Forth compiler. This is basically the "guts" of CamelForth (GPL3) which someone else already mentioned. Here is the link to the CamelForth source code.

[hsoft]

I'd like to let people know that I started, as an experiment, a reboot of Collapse OS as a Forth: https://github.com/hsoft/collapseforth . The goal is too bootstrap this Collapse OS-to-be entirely through that evolving Forth.

I'm learning Forth as I go. I'd be curious to hear about what people experienced in Forth think of this approach.

[hsoft]

The approach I took with that "collapseforth" reboot was a dead end, but it made me learn a bit more about Forth and I like it. I've added apps/forth, a new z80 implementation of Forth, in Collapse OS directly. With time, I can very well see it take a central place and make Collapse OS self-hosting in an even compacter way.

[hsoft]

I'm changing the title of this issue because I can see the distinct possibility of making Forth take over Collapse OS. My expectation is that, because higher order words are more powerful, they are also more compact.

With any other higher order language, this high level comes at a prohibitive cost in terms of implementation complexity (and thus RAM usage), but not with Forth, which is what makes it so interesting.

Collapse OS currently self-hosts with about 10K of ROM and 8K of RAM. The vast majority of that space is zasm's complexity. I believe that with Forth very quickly (in terms of complexity) bootstrapping into higher order words, it makes its "self complexity" (the complexity it needs to self-host entirely) much lower, I can achieve similar or lower requirements and get a high-level language (and arch-independent!) as a bonus. This makes Forth very exciting.

I'm not certain of success yet, but I'm optimistic and excited. Expect Forth to "eat away" (I will not do a reboot, just an incremental "code cancer") current ASM code.in the following days.

[hsoft]

By the way, sorry to Forth enthusiasts for not having seen its potential early. It's difficult to see it without diving into it. It's such a wicked way to approach computing in general. I'll have to supply Collapse OS doc with good Forth introductory material.

[agsb]

seek "thinking forth", http://thinking-forth.sourceforge.net/ all 2 cents.

…

On Thu, Mar 12, 2020 at 9:14 AM Virgil Dupras ***@***.***> wrote: By the way, sorry to Forth enthusiasts for not having seen its potential early. It's difficult to see it without diving into it. It's such a wicked way to approach computing in general. I'll have to supply Collapse OS doc with good Forth introductory material. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#4 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADR73T3YGT34PTOIGWABAKLRHDG33ANCNFSM4I6UHPSA> .

-- *"Sling and move continents, a rock at a time.*"

[hsoft]

@agsb yes, it's been on my reading list. I've started reading it, but my hands are too dirty playing with code. If I were reasonable, I'd take the time to step back and read this...

[piranna]

Awesome! :-D

[ghost]

It doesn't seem very compact.

I thought it was interesting that a (very light) lisp-like or imperative language might by able to be evolved out of the work you're doing creating forth, possibly for scripting within a minimal shell environment.

[ghost]

Corresponding HN discussion : https://news.ycombinator.com/item?id=22935650

[gitjeff2]

I'm not sure how much this applies to the Z80, which you're targeting first for entirely understandable reasons. But, there is a very interesting article that will help increase the performance of the 6502 port when it gets here: Zero-Overhead Forth Interrupt Service on 65C02.

I'm not sure if it applies in anyway to the Z80.

[hsoft]

It looks like the concept can be applied to z80 as well. It is also very relevant to Collapse OS' needs: I wrote the ACIA interrupt routine in assembly precisely for the reasons cited.

However, calling it zero-overhead is a bit of a misnomer: the NEXT routine has to poll for interrupt requests. The question is whether that polling is worth it.

Right now, only the ACIA drivers use interrupt requests, I'll wait until there's a few more of them before asking myself the question seriously. But thanks for the reference, it's certainly a good one.

[f4grx]

Also usable in my own implementation on HC11 and soon-to-be on msp430, which is now able to find words!

Thank you for the awesome IRQ management link.

So basically, native IRQ routines raise a flag and the inner interpreter just EXECUTE a specific forth word if the flag is set. The next word will naturally get executed on the next call to NEXT without even having to save/restore IP! This is a close as a native implementation as it could be, I love it!

[gitjeff2]

Thank you for the awesome IRQ management link.

Sure @f4grx! I don't have the cycles to submit code just now, which sucks because I have a few ideas I'd really like to implement. But, I can send useful resources and do other things to stay active and help the project.

[gitjeff2]

Given the anticipated hardware ecosystem of CollapseOS, something like 6502 binary relocation format might be useful. Much like the so-called "Zero-Overhead Forth Interrupt Service" article, this article is 6502 specific. I hope it's useful in helping to reduce assumptions about a given system's memory map regardless of architecture and increase code portability.

Memory map assumptions plagued CP/M again and again. Wildly incompatible disk formats were an issue too, but it sounds like you're focusing on FAT or ex-FAT for now. So, that's not an issue.

[gitjeff2]

The RTX2010, an 8-bit processor that supports direct execution of Forth code. Lots of technical details are readily documented. Effectively, Forth is its machine code. I wonder if this can be recreated efficiently in purely TTL. Apparently, the RTX2010 chip has been used by NASA because it can be easily hardened.

[PeterForth]

Jeff , RTX family from harris was a 32bits architecture. Microcode is forth words, but this are stack processors, so the architecture of data adress and processing internals is a Forth Machine, no registers, all happends on the stacks. No assembler, the machine code are Forth high level words. It can not be repeated on TTL it would be overkill. You can get other similar architectures like the Novix and RTX chips (Charles Moore invention) for FPGA. There are other MPUs that came with Forth on board, on internal rom, even beying harvard architecture 8bits processors. Like the rockwell 65f11 processor and others from zilog and other manufacturers. But I do not see any relation of Forth CPUs to the topic collapse/os !<div id="DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2"><br /> <table style="border-top: 1px solid #D3D4DE;"> <tr> <td style="width: 55px; padding-top: 18px;"><a href="http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail" target="_blank"><img src="https://ipmcdn.avast.com/images/icons/icon-envelope-tick-green-avg-v1.png" alt="" width="46" height="29" style="width: 46px; height: 29px;" /></a></td> <td style="width: 470px; padding-top: 17px; color: #41424e; font-size: 13px; font-family: Arial, Helvetica, sans-serif; line-height: 18px;">Livre de vírus. <a href="http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail" target="_blank" style="color: #4453ea;">www.avg.com</a>. </td> </tr> </table> <a href="#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2" width="1" height="1"></a></div>

…

On 5/1/20, gitjeff2 ***@***.***> wrote: The [RTX2010](https://en.wikipedia.org/wiki/RTX2010), an 8-bit processor that supports direct execution of Forth code. Lots of [technical details are readily documented](http://soton.mpeforth.com/flag/jfar/vol6/no1/article1.pdf). Effectively, Forth is its machine code. I wonder if this can be recreated efficiently in purely TTL. Apparently, the RTX2010 chip has been used by NASA because it can be easily hardened. -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #4 (comment)

[Picatout]

@PeterForth, Except for GA144 do you know any CPU or MCU with a native FORTH core still available as 'active part' today? (No FPGA).

[PeterForth]

Hi Jacques, From manufacturer available today only the GA144. I would use the term "Forth Machine Architecture", to refer to processors, because we are talking of the internal arrangement of the blocks =functions that build the CPU. "Native" could also be called an internal Forth in Rom, which is not a Forth machine, still a Harvard architecture that runs Forth over machine code mnemonics (as we normally do, when we load a Forth in any system). No Forth in hardware= microcode.are : 65f11 and ZilogSuper8 and other CPUs I mentioned earlier. Propeller or Pharlap (propforth and others) still delivers a modern Forth native CPU, but not Forth architecture CPU. https://en.wikipedia.org/wiki/Parallax_Propeller It is really an impressive chip for automation and controll, is also very fast even not = to a Forth-architecture cpu. That CPU is also free available as FPGA Verilog core. If some of you get interested in this theme , one of my favourite places is Andrews TTL forth processor. http://www.aholme.co.uk/Mk1/Architecture.htm Go also to my book links https://sites.google.com/view/forth-books/home/forth-books/forth-books1 on page 14, Dr. Ting has lots of explanations on Forth CPUs. We must thank very much Dr. Ting for his great effort to make Forth popular all over the world, he recenly placed all his books for free download ! Please add links + cooperate to this projekt if possible. I am open to add more themes on my channel dedicated to forth on Youtube, ask me for any themes you would like to see, I will try to include in future editions. Cheers Peter <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> Livre de vírus. www.avg.com <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>. <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

…

On Sat, May 2, 2020 at 9:52 AM Jacques ***@***.***> wrote: @PeterForth <https://github.com/PeterForth>, Except for GA144 do you know any CPU or MCU with a native FORTH core still available as 'active part' today? (No FPGA). — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#4 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKLYKHQVDQQKTLT2M5YKYKTRPQJQFANCNFSM4I6UHPSA> .

[piranna]

Propeller or Pharlap (propforth and others) still delivers a modern Forth native CPU, but not Forth architecture CPU. en.wikipedia.org/wiki/Parallax_Propeller It is really an impressive chip for automation and controll, is also very fast even not = to a Forth-architecture cpu. That CPU is also free available as FPGA Verilog core.

Propeller Parallax is available as a Verilog core?!??! Where?!?!?!?! 😲 What license does it have?

[piranna]

On August 6, 2014, Parallax Inc., released all of the Propeller 1 P8X32A hardware and tools as open-source hardware and software under the GNU General Public License (GPL) 3.0. This included the Verilog code, top-level hardware description language (HDL) files, Spin interpreter, PropellerIDE and SimpleIDE programming tools and compilers.[3]

OMG, that Verilog files would be really nice to read, I remember to read that the cogs where created using basic gates instead of high level code, making the DIE layout smaller and faster :-D

[Picatout]

@PeterForth ,
Thanks for information and links.

[hsoft]

Things are getting deliciously serious on the TRS-80+Forth side. With the new indirections I've added to KEY and EMIT, I can easily take control of the prompt from RS-232, and with the new blkup tool I've just pushed, I can send the blkfs directly to floppy. It works, I now have a floppy with the first 10 blocks of Collapse OS fully working on the TRS-80!

With a completed editor and some smart management on the blocks side (to avoid having to swap disks all the time), with the addition of some kind of daemon on the "modern side" listening to the serial com for incoming blocks to spit to blk/, I could easily see myself in a setup where I work on Collapse OS from within Collapse OS on a TRS-80 4P. sexy...

[Picatout]

hsoft, "Things are getting deliciously serious on the TRS-80+Forth side. "
Do own the hardware or is it an emulator?

[hsoft]

On the real thing of course. I'd have a hard time qualifying running Collapse OS on an emulated TRS-80 4P as "delicious" :)

Yup, Collapse OS now spans on 5 1/4 floppies and my last commit, the one about ACC initialization, was first fixed on the machine itself (I don't notice these initialization issues on emulator because memory is zeroed in there, something that doesn't happen on the real thing) using the in-system editor to write it to floppy and then re-LOAD it t confirm the fix... I'm getting there.

[hsoft]

TI-84 recipe is now working on the Forth branch. There's not much left to implement before Forth is on par with master and can thus replace it: SMS recipe and AVR assembler.

[hsoft]

@TalosThoren your video at https://twitter.com/talosthoren/status/1259825070509297664 looks very good! thanks for making it. I haven't finished watching it, but I presume that you end up being able to boot Collapse OS :) I'll link it in the project's README. It's so far one of the best introductory material I've seen so far.

EDIT: oh, I thought you got to Collapse OS prompt. You still have this whole write-to-EEPROM challenge to meet. Let me know if you find the documentation lacking. I'm interested in knowing how newcomers can manage following the recipe.

[hsoft]

Collapse OS now cross-compiles in a single stage! On april 12, I was commenting that I managed to bootstrap Collapse OS using a hybrid xcomp+relink approach, which was the approach I took until now. Cross compiling the whole base set of words was too hairy.

However, now that things are stabilizing, I took a second look at the situation and with careful threading, I could manage to organize core words in a way that makes them 100% cross compilable.

This makes the bootstrapping process much leaner.

[f4grx]

That's great!
What about text editing?

On my part I have a working compiler, which is also great for me.

[hsoft]

The editor is partly implemented, following Starting Forth's description. Inconvenient but usable.

[hsoft]

Forth Collapse OS runs on my Genesis with a gamepad!

This brings me really close to parity with the master branch. The PS/2 sub-recipes from SMS and RC2014, the AVR assembler, and I'm done!

[piranna]

Forth Collapse OS runs on my Genesis with a gamepad!

How are you using it? An on screen keyboard or something similar? :-)

[hsoft]

The recipe describes how it works.

[hsoft]

I've completed the conversion of the PS/2 interface recipes in RC2014 and SMS. This completes the conversion to Forth! Now I only need to clean up things a bit, beef up the usage docs section, and switch the master branch.

[hsoft]

Forth takeover is complete! closing.

[f4grx]

Congrats and thank you for making me dive in forth. I dont regret it.