refactoring of the codebase

[YashasSamaga]

This issue is for listing down possible improvements for the compiler's codebase.

Current list of suggestions:

• grow literal queue exponentially (currently linear)
• inconsistent naming of preprocessor tokens
• use enum instead of #define for assign token numbers.
• scope mechanism
• unused error ids
• pushstk crash (copying to null pointer)
• iREFFUNC is unused

[YashasSamaga]

grow literal queue exponentially (currently linear)

In chk_grow_litq, the compiler does the following to resize the literal queue:

litmax +=sDEF_LITMAX;

This causes O(n^2) cells to be copied but this method is space efficient: in the worst case, max wastage will be sDEF_LITMAX - 1 cells.

The alternative solution is to:

litmax *= 2;

This causes O(n) cells to be copied but is less space efficient: in the worst case, half of the literal queue will be wasted.

Tests:

static arr[10000000] = {1, 2, 3, ...};
main () {
    arr[0] = 123;
}

litmax+=sDEF_LITMAX; takes on average around 13s (compiled continuously)
litmax*=2; takes on average around 1.5s (compiled continuously)

The space efficiency is probably not a problem as gamemodes doesn't exceed few MBs (at most maybe 100MB). The amount of memory wasted in the worst case would be 100MB in that scenario. I am not sure if someone would ever have a situation like this though.

[YashasSamaga]

#define flgDEPRECATED 0x01  /* symbol is deprecated (avoid use) */
#define flagNAKED     0x10  /* function is naked */
#define flagPREDEF    0x20  /* symbol is pre-defined; successor of uPREDEF */

The flags member of s_symbol use just the above 3 bits. Is there a reason why they aren't continuous? (why isn't 0x02, 0x04, etc. there?)

---

unsigned char hasdefault; /* bit0: is there a default value? bit6: "tagof"; bit7: "sizeof" */

What happened to bits 1-5? Nowhere are those bits used as far as I could check.

[Daniel-Cortez]

What happened to bits 1-5? Nowhere are those bits used as far as I could check.

Maybe they were never even been used because they were supposed to be reserved backwards (7, 6, 5, ...) for some reason. At least, in Small 1.8.4 only bits 0 and 7 are used, and bit 6 isn't
char hasdefault; /* bit0: is there a default value? bit7: "sizeof" */

---

Also, since low memory use doesn't seem to be the first priority for this project, I think we could probably get rid from the numerous #if defined SC_LIGHT branches in order to clean up the codebase, just like with the packed string data in sc5.scp and sc7.scp.

[Daniel-Cortez]

inconsistent naming of preprocessor tokens

Just noticed that the constant for the #include directive is defined as tINCLUDE, with a t prefix, while the names of the other directive-related constants are prefixed with tp

#define tpERROR     323
#define tpFILE      324
#define tpIF        325 /* #if */
#define tINCLUDE    326
#define tpLINE      327
#define tpPRAGMA    328

so we should probably rename tINCLUDE to tpINCLUDE.

Symbol flags are named inconsistently too: while flgDEPRECATED starts with flg, flagNAKED and flagPREDEF are prefixed with flag.

[YashasSamaga]

use enum instead of #define for assign token numbers

#define taMULT      256 /* *= */
#define taDIV       257 /* /= */
#define taMOD       258 /* %= */
#define taADD       259 /* += */
#define taSUB       260 /* -= */
.
.
.

enum {
     taMULT = 256, /* *= */
     taDIV, /* /= */
     taMOD, /* %= */
     taADD, /* += */
     taSUB  /* -= */
};

The constants should not be hard-coded. Not hard-coding makes adding new entry easier. As of now, to add a new entry, all the defines which follow from the place of insertion has to be manually updated. It also feels insecure because people wouldn't know if there is code which relies on the numbers being the way they are.

[YashasSamaga]

scope mechanism

We could use a new concept of scope number instead of the file number. This would be useful for implementing the #section directive (and probably new features). It also makes the code legible (the current system is very confusing and highly error-prone) and easier to maintain.