Adding support for more proprocessor directives (ifs, block comments, ...)

[jgOhYeah]

I have been having a bit of a play around with getting ifs and block comments to work. Hopefully this will be helpful to someone else if they need it.

It still probably needs a bit more testing to make sure it isn't doing something too strange with certain edge cases or common events.

The main additions / changes are:

• Adding a --nocolor option to disable colour in the warning and error messages for shells that do not support it through ansi colours (cough cough versions of Windows).
• A string replacement function that will only substitute entire words that are not in strings or comments.
• Support for block comments using #REM and #ENDREM and adding the comment symbol ; to the start of all lines in between them.
• Implementing #IF, #IFDEF, #IFNDEF, #ELSE, #ELSEIF, #ELSEIFDEF, #ELSEIFNDEF and #ENDIF in a way that is hopefully moderately compatible with PE6. This is done by using a list as something similar to a stack that keeps a record of whether the current level of code should be included and what its parent is when it jumps back to it upon reaching an #ENDIF.
• Implementing #UNDEF to undefine and #ERROR to user the preprocessor_error function to display the error message.
• Adding an error message if a directory is attempted to be included instead of a file.

[Patronics]

Thanks for the pull request, good work on it! Have you tested its behavior when used with AxePad? I know that AxePad also implements the #IF, #ENDIF, etc, so I'm hesitant to merge this until it has been verified not to conflict with the built-in implementation, or, even better, add an option to disable the preprocessing options that are already implemented in AxePad.

[jgOhYeah]

That is interesting that AxePad has it. I didn't know AxePad had it built in, but assumed it didn't as the compilers themselves seemed to nearly always evaluate everything to True and include it regardless of any if or else statement. To be honest I haven't been able to get AxePad to run on linux, however I will try in the next few days on Windows and have another go on linux (I think it is something to do with 32 bit support packages missing).

This implementation comments out most preprocessor directives, so there shouldn't be too much of a chance for a second preprocessor to misbehave, but will check and add a switch to disable it.

As another side project in another branch, I have also been working on an extension to this preprocessor to automatically store and print strings from table memory in supported chips to save program memory. To use the extension, the directive ;#sertxd("Text", cr, lf) needs to be used, allowing preprocessors without this extension to treat it as a comment and ignore it. The string will be stored inside a table command and a subroutine called with the correct parameters to read and print the correct data instead of using sertxd or manually trying to keep track of string lengths and offsets.

This needs a bit of cleaning up, finishing off and testing, so if they turn out to be reliable, I might open another pull request for them later on. The extension is non-standard, so it would definitely have an opt in switch lest it breaks something in certain situations.

Thanks for looking at this pull request and finding things to take into account.

[jgOhYeah]

I have done a bit of testing with this preprocessor in Windows and Linux, as well as AxePad and PE6 on Windows. Using this script (the files in the include folder contain a fair bit of what I would use the preprocessor for). In all editors, preprocessors and compilers, the code has always compiled to be 2005 bytes, hopefully indicating that all are evaluating the #IFs and #INCLUDEs the same. AxePad on its own wouldn't compile this due to the includes, however it will happily compile the preprocessed output and also gives 2005 bytes. Preprocessing the preprocessed output a couple if times before compiling also doesn't seem to affect it, which is good.

It seems that the compilers and axepad have a small implementation of #IFDEF and can handle simple things such as:

; 45 bytes when defined, 185 bytes when not defined
; #DEFINE WRITE_MSG
#IFDEF WRITE_MSG
    sertxd("WRITE_MSG is defined!!!", cr, lf)
#ELSE
    sertxd("WRITE_MSG is NOT defined, but am writing a message anyway to se memory to demonstrate this has been selected.", cr, lf)
#ENDIF

However, the compiler cannot handle numbers as using #DEFINE WRITE_MSG 76 in the example above treats WRITE_MSG as not defined. It also cannot handle #IF:

#DEFINE WRITE_MSG 76
#IF WRITE_MSG > 40 ; Syntax error thrown here
    sertxd("WRITE_MSG is > 40!!!", cr, lf)
#ELSE
    sertxd("WRITE_MSG is <= 40", cr, lf)
#ENDIF

; ERROR:
; #IF WRITE_MSG > 40
;  ^
; line# 3, col# 1
;
; Error: syntax error

Because my implementation uses a stack to keep track of whether the current code should be included (appending when an #IFDEF or #IF is found, popping for #ENDIF and modifying the top element for everything else), it has to evaluate #IFDEF as well, even though the compiler can already partly handle it.

I have also added a --noifs flag that should break the recognition and handling of any #IF, #ELSE, #ENDIF... lines, so if there is some edge case (or common usage that I didn't even think about), this functionality can be disabled.

[Patronics]

Looks good, I added a few bugfixes and minor tweaks, thanks for the contributions!