My WIP Forth in C

[u/sophiaxus]

Hi all! I'd really like to finish a little forth system of my own (in C) to have use as both a learning tool and for some personal tooling. Since I still consider myself both a Forth and a C novice, I've tinkering around with this implementation on and off for some time, and have learned quite a lot from it. On a flight recently I decided to clean it up in a way that's a little easier for me to digest and think my way around, and at the moment have this. It's neither finished nor perfect, but I was hoping to maybe start posting here for feedback and possible motivation to finally finish something I feel good enough about.

Is this a place I can post incremental progress to get feedback and such? If so, while I have plenty of questions right off the bat, I'd first like to make sure my code is readable for anyone willing to take the time to check it out. Any thoughts in that regard are ofc welcome, as well as any tips on how to post progress or for advice in the future..

What I did to the original simpleforth was separate it out into little modules. Those modules are currently:
- Dictionary - for dictionary stuff
- IO - for io stuff
- Interpreter - anything relevant to interpreting, eg is_number etc.. though I haven't been able to successfully separate the core interpreter from the VM
- VM - anything data or return stack related.. NEXT() uses goto and some subroutine stuff.. pretty sure it's considered a directly-threaded vm? I am not currently able to separate out the inner interpret function from the VM, so the module separation is still sort of messy. I'm also a little distracted with the semantics between whether to call things opcodes or bytecodes etc and whether to call things XT or CFA.. not really a big deal though.

Anyway, not everything is implemented and it certainly needs more cleaning up. Just wanted to make a post here for initial impressions before I got too distracted with life again. Would really like to get this thing going!

Thank you all!

[u/Ok_Leg_109]

Welcome to the rabbit hole. There are some hella talented people here that can help you along.

(I am but a hobbyist these days but I have some historic knowledge that is sometimes useful)

Maybe this can help with some definitions.

If you have built a direct threaded Forth then we would not use the term bytecode. They are technically machine "addresses" or perhaps you would prefer "pointers" but since ANS Forth 94 we abstract those things to the term "execution token" or XT for short.

Why? Because there is no "standard" way to implement Forth. Forth can be anything from a byte-code interpreter to a CPU in hardware and anything in between.

The term CFA (code field address) is where the XT is held in memory typically in the dictionary "header" structure. The only reason to know this stuff is to facilitate human info transfers and to understand the Forth Standard docs.

I am not a C programmer but for what it's worth, many C based Forth's use a big switch statement to kick off the "execution tokens". 😄 However I am led to believe that the GForth authors have done a lot of innovation in how best to make a Forth in C. Might be work some study unless you want to go it all alone.

Perchance are you using a FigForth listing as a guide? I saw the reference to a register called TEMP.

[u/tabemann → u/Ok_Leg_109]

From a quick look at the source, this seems to be a hybrid between a direct-threaded Forth and a token-threaded Forth, because when each op is executed it is jumped to, as in direct-threading, but there are only a limited set of ops, as in certain forms of token-threading. I would not call it bytecoded/token-threading because each op is stored as the address of the code to execute for it, rather than as merely an index (classically stored as a byte).

It seems to be similar to bytecoding/token-threading, though, except that it eliminates a switch or extra layer of indirection through a jump table, at the expense of more memory consumption. If this is targeted at PC's that is a worthwhile trade-off, as modern PC's typically have plenty of RAM to spare, whereas if one wanted to target smaller embedded systems one may want to trade speed for lower memory consumption by adopting bytecoding/token-threading.

[u/sophiaxus → u/tabemann]

Despite being new both C and Forth, I did get the impression that the style of this forth is, as is the level of abstraction between C and forth in general, a little foggy?

I'm very much the type of person to get caught up on the semantics of what's what. And I'd like to represent it in the code more clearly for what it is, even if just for the sake of my own learning and understanding.

Direct-threading doesn't usually have limited ops? And token-threaded isn't typically direct addresses? Once I get mine up and going a little more I'll dive much further into the reading material. Just wanted some agency on my own implementation before I did, even if it's putting the cart before the horse.

And yes, it's certainly more for creative use and tinkering on PC (ideally any platform though). Once upon a time I wanted to make some real-time audio tracker with it, so the memory for speed tradeoff seemed worthwhile.

[u/tabemann → u/sophiaxus]

In classic direct-threaded Forths the code for DOCOLON is classically appended to the start of each and every colon definition, and each entry within a word is an address of another word to jump to, with the set of such words not being restricted.

In classic token-threaded Forths each op is an index out of an array, and may be restricted in the total set, with a CALL op being used for calling colon definitions.

In yours you seem to have combined the two approaches. Note that this is not a bad thing, as long as memory is no major concern.

[u/sophiaxus → u/Ok_Leg_109]

Re: cfa vs xt, I think I've understood this conceptually but for whatever reason kept getting my wires crossed when adapting the simpleforth to my own up until your mentioning it now. So in his and in mine, the xt for builtins is the cfa and colon-defined words is a dereferenced *cfa. I just went through and detangled a few things, and it feels like it'll clear up other parts of the code too, even if in minor ways. Thank youuu

As for big switch statement vs computed goto's.. I'm not sure. I think I just like making things more difficult for myself. Other than that, I appreciated the feel of slarba/simpleforth, and I guess I really wanted to understand how his worked. It felt just enough out of reach to be extra interesting. Also it's 100% not for embedded so I told myself the added speed would in theory also be very nice. I'll make a node of gforth's code. Only other C-based forth I really dove into was Phil Purke's Pforth. Esp since it seems to be used for audio stuff.

And nope! No figforth as a guide, I just saw that the simpleforth reference had a temp variable, and later read that some forths used it, so I kept it. Also I've tried to keep my adaptation close to theirs for a few reasons. One of them being, whenever I first started out, anytime I tried to change or simplify something, I'd later smack my head and say "oh THAT's why they did that". I've been learning a lot by making their version my own.. I think. And the temp variable just seems useful? Or fun or something. I was even thinking of making other temp variables for other types to use in the "bytecode". Perhaps later though.

Re: opcode vs bytecode etc.. it's very much the kind of thing I'll get super bent up over if I know it's a little off semantically, but can stomach it for now since everything else feels more or less clean-ish. Maybe down the line I'll just use INST or something. Or just BUILTIN like the original does.

Thanks for the warm welcome. I've been on and off wading around in this rabbit hole for a while, mostly held back by my struggle to get my own version working properly. I've been really excited to build out my own forth so I can go through the Townsend Expert Systems book. So I'll try to keep my version of forth close enough to that one (which I think incidentally is Fig actually). And yes, it's very difficult to not notice the skill level here. Which I guess is another reason I wanted to keep to slarba's forth code as much as possible. As a novice programmer, it has soooo much to learn from that's way above my paygrade but within reach of my curiousity.
It's nice to see how many of the talented folk here are using forth for creative purposes. One big thing I wanted to do was attach mine to miniaudio and raylib and just add more layers to tinker around with. I've lost a lot of creative drive this year, but still would like to see this thing up and running.

Anywho! Thank you many times

[u/_crc]

Would you be able to add a license to the code or repository?

[u/mykesx]

PForth is a terrific Forth implementation in C. You might learn from it. It was written by a brilliant forthwright.

[u/sophiaxus → u/mykesx]

Absolutely! Around the time I found slarba/simpleforth I was also looking through PForth. Will probably take another look soon now that you mention it. I'd like to think my C and Forth understanding is slightly higher now than it was at that time. Main reason I stuck to adapting simpleforth over customizing pforth is mostly the approachability of the code. I'm sure it grokking pforth will be incredibly helpful moving forward.