Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding Luna Autocode scripting support #255

Merged
merged 97 commits into from Jan 30, 2022
Merged

Conversation

Wohlstand
Copy link
Owner

@Wohlstand Wohlstand commented Jan 16, 2022

This is the major work that adds the support for LunaDLL Autocode language which allows playing of episodes that originally used it.

Additionally:

  • Demos counter which can be enabled by the compat.ini or by gameinfo.ini
  • Memory emulator which converts SMBX memory requests into the safe reading and writing of internal variables

Things left incomplete:

  • Find, why Raocow gets fixed frame instead of the proper animation when running "Science final battle" level on an Android device
  • Repair the bank of variables module which should save all reminded variables at SAVX's entry (previously it used the LunaSavedVar#.txt file at the game directory, now just a native part of SAVX file)
  • Add the build option to disable the Luna build (to produce the game with the disabled Luna Script at all)
  • Try to port the Sprite API that allows load and draw images using Autocode
  • Verify more episodes in a work to make sure that the Autocode thing works properly Can be polished later
  • Keep track of the "abstract controls" (Wip abstract controls #245) branch, and once it gets merged, tune out the memory emulator to handle updated globals properly.

P.S.:

New addons of gameinfo.ini:

[death-counter] ; Demos counter globally
enabled = true    ; Enable the counter
title = "DEMOS"  ; title shown at the counter ("DEMOS" is default)

[luna-script]
enable-engine = true   ; Enable luna script globally

New addons of compat.ini:

[death-counter]
enabled = true    ; Enable the counter of Demos at an episode or at a level even it's disabled globally

[luna-script]
enable-engine = unspecified / enable / true / disable / false    ; Enable luna script locally at episode (mainly intended if episode requires the script engine enabled)
allow-level-codes = false     ; Allow using of built-in C++ codes for levels, this should been enabled at "A2MBXT: Episode 1: Analog Funk" only

NOTE: If you see certain LunaDLL-based episode or level works improperly, please check the log for possible warning messages (against attempts to access any unknown addresses), etc.

Until start developing the memory emulator, I want to implement the backend of the Luna Autocode engine to get an ability run existng scripts for verifications and debugs.
To test out the thing in action
Also, ported the set of hardcoded level codes for the Analog Funk
Fixed workflow at Kood's stage where Twomps do fall up or down depending on switch's state
TODO: Try to implement an ability to override values
(To allow put various custom settings files)
They will work in condition luna engine is enabled
- Get rid of the new/delete, use std::move() when need to move
- Added copy constructor into Autocode class
- Other clean-ups
To don't conuse all content of deaths counter, make it save all stuff per game save only
For the case episode creator wants to enable the death counter at episode side, add the compat.ini option
Using any file manager, you can open one of level files to play
@Wohlstand
Copy link
Owner Author

Sorts like "SaveData" table in X2B4+!!

This is the "grand grandpa" of the SaveData table. LunaDLL Autocode has the only global bank and was able to store integers and double number values only, with no strings and other types. SAVX itself allows to store multiple data banks, and it's able to store ganks divided by access scopes (global, specific level only, all levels, world map only, etc.), and is able to store more data types than numbers (as it uses the string container, it can store just numbers, strings, JSON data blocks, and more also). Another side needs to be implemented to handle these nifty features of SAVX.

Copy link
Collaborator

@ds-sloth ds-sloth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've read everything that changes the main game and left comments on the things that I noticed.

From running size, it looks like this adds ~400KB of statically allocated memory. That's fine on all of our current clients, but I do wonder where it comes from.

Do you intend to support no-autocode builds or will this be a permanently-enabled subsystem?

Since this code seems to not affect the main gameplay, I would encourage you to merge it into master as soon as possible so that our code doesn't diverge when I begin merging features from devel.

@@ -44,6 +45,8 @@ RangeArr<std::string, 1, maxEvents> NewEvent;
RangeArrI<int, 1, maxEvents, 0> newEventDelay;
int newEventNum = 0;

static std::set<std::string> recentlyTriggeredEvents;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be easy to update to std::set<eventindex_t> when we begin merging the devel branch improvements. But it may be even better to just add a "WasRecentlyTriggered" member to Event_t since events are now referenced exclusively through their indices, not their names. (For the Luna, we will probably want to use a Name -> index hash map.)

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, this thing is just an initial experiment, and I think, it can be improved later easily. So indeed, it needs to convert the raw name into the index number easily, that can be done on the script parsing stage easily (we can store all "MyString" fields at the string collections on the parsing stage, and just re-use them, easily).

{
// double* pCameraY = (double*)GM_CAMERA_Y;
// double* pCameraX = (double*)GM_CAMERA_X;
double cam_y = -vScreenY[1];
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this correct?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the original code at LunaLua side is:

    double* pCameraY = (double*)GM_CAMERA_Y;
    double* pCameraX = (double*)GM_CAMERA_X;
    double cam_y = -pCameraY[1];
    double cam_x = -pCameraX[1];

It computes the location of the player from the global to screen local coordinates. But right, there is LunaLua-side fault as it didn't properly care about multiplayer, and therefore there is lots of direct use of player 1 and screen 1, etc. That needs to be polished later to support this on multiplayer.

@ChristianSilvermoon
Copy link

I've read everything that changes the main game and left comments on the things that I noticed.

From running size, it looks like this adds ~400KB of statically allocated memory. That's fine on all of our current clients, but I do wonder where it comes from.

Do you intend to support no-autocode builds or will this be a permanently-enabled subsystem?

Since this code seems to not affect the main gameplay, I would encourage you to merge it into master as soon as possible so that our code doesn't diverge when I begin merging features from devel.

I do think it would be neat to see one of these personally:

  • -DENABLE_LUNA_AUTO build flag (defaulting to true)
  • Runtime Option to enable/disable the system
  • If possible, enabling it automatically, but only for episodes that use it

If a build lacked support for it, but an episode needed it, it would also be neat to see the game somehow inform you with a prompt of some kind such as "This (level|episode) requires Luna Autocode Support, which is not available in your current build. Do you want to continue? (Yes|No)"

@Wohlstand
Copy link
Owner Author

Do you intend to support no-autocode builds or will this be a permanently-enabled subsystem?

Right now, the Autocode engine can be enabled or disabled by runtime (by gameinfo.ini or by compat.ini). I guess, for some experiments that could be useful to disable the Autocode engine building for some special cases.

-DENABLE_LUNA_AUTO build flag (defaulting to true)

Yeah, I can easily add this, no problem 😉

Runtime Option to enable/disable the system

Already done a gameinfo.ini and compat.ini :fox: 👍

If possible, enabling it automatically, but only for episodes that use it

By default, Autocode is not running when no scripts are loaded. Once any script appears, it starts to work. There are built-in C++-side level codes that are required at Analog Funk to work, disabled by default, and they were enabled when compat.ini enables them.

From running size, it looks like this adds ~400KB of statically allocated memory. That's fine on all of our current clients, but I do wonder where it comes from.

The size will also depend on CPU architecture, because of long and size_t types, alignment, and other cases.

Since this code seems to not affect the main gameplay, I would encourage you to merge it into master as soon as possible so that our code doesn't diverge when I begin merging features from devel.

The whole thing is an addition to the main, it doesn't introduce any significant core changes (there are only minor add-ons that were done to insert Luna hooks to handle the script engine). I also had to optimize the code of Luna: I had replaced it with hash tables, indexed tables, and some other cases to avoid linear loops as much as possible. There is Luna itself can get hurt from other changes as the memory emulator needs to be tuned to handle the updated environment to be used properly (for example, replace mouse-related fields with lambdas and references of other fields, etc.). So, absolutely no worry about this. 😉

@ds-sloth
Copy link
Collaborator

ds-sloth commented Jan 18, 2022

-DENABLE_LUNA_AUTO build flag (defaulting to true)

This is what I was hoping to see, only because it might help us squeeze the game onto devices with 32 MB of RAM.

There is Luna itself can get hurt from other changes as the memory emulator needs to be tuned to handle the updated environment to be used properly (for example, replace mouse-related fields with lambdas and references of other fields, etc.).

Right, I want to reduce this pain for you as much as possible by merging the deep-level changes as quickly as possible. (Mainly, the updated layer and event systems and the sync_layers hooks.)

I had replaced it with hash tables, indexed tables, and some other cases to avoid linear loops as much as possible.

This is great! I did notice that the code looked really good, clean, and optimized as I was reading it. I wondered how much of that was the old Luna code and how much was yours.

I made the `-DTHEXTECH_ENABLE_LUNA_AUTOCODE=ON/OFF`
@Wohlstand
Copy link
Owner Author

Added just now, the -DTHEXTECH_ENABLE_LUNA_AUTOCODE it is to be consistent with other similar options

@Wohlstand
Copy link
Owner Author

Right, I want to reduce this pain for you as much as possible by merging the deep-level changes as quickly as possible. (Mainly, the updated layer and event systems and the sync_layers hooks.)

Also, a little note, that there are some C++ codes that will need to be updated for this: they use index values for given layers (the example is Docopoper-TheFloorIsLave.cpp). So, replace array index requests with the name requests once the map will be done, or something also

I am not sure, was it actually used anywhere or not, because it was not documented and it was a sort of the experimental thing in the past, or something also.
@ds-sloth
Copy link
Collaborator

Docopoper-TheFloorIsLave.cpp

I have read the code and it looks good to me. Accessing Layers by index is still allowed and good. The thing that is not good is setting object Layer attributes as a string rather than as an index.

We will probably want to intercept memory writes that write to the Layer attribute of any object so we can (1) find the layer index and write it instead of the string, and (2) call the syncLayers_NPC (_block, etc) hook for it.

@Wohlstand
Copy link
Owner Author

Any string-related tricks can be replaced from raw strings into indexes on the parse stage, and the autocode runner will use these indeces instead of raw strings (see MyString and commands where it's used)

Added the revision number
It's no longer needed to specify the version manually
TODO: Hook the build number too
(and attempt to fix the Emscripten build)
@Wohlstand
Copy link
Owner Author

Okay, after abstract-controls got been merged, I'll need to fix up the breakage to make sure it's compatible

Fixed compatibility with the updated set of global variables (GamePaused is now enum, and needs the lambda-function to mimicry as old boolean value)
@Wohlstand
Copy link
Owner Author

Done, I fixed the compatibility with the abstract controls now. I had to add the lambda-overlay for some global variables to mimicry old variables after their major changes.

@Wohlstand Wohlstand marked this pull request as ready for review January 30, 2022 04:21
@Wohlstand
Copy link
Owner Author

Okay, gonna merge this now.

NOTE: @0lhi, If you see certain LunaDLL-based episode or level works improperly, please check the log for possible warning messages (against attempts to access any unknown addresses), etc. That means I didn't added into memory emulator certain maps for some fields, and that means, I shoud add these maps into the memory emulator to fix the behaviour.

@Wohlstand Wohlstand merged commit dc2ffe1 into master Jan 30, 2022
@Wohlstand Wohlstand deleted the wip-memory-emulator branch January 30, 2022 20:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New quality-of-life feature or request (does not change gameplay)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants