Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compiling with Clang on Windows fails at libcurl.rc #7765

Closed
VitalyVaryvdin opened this issue Sep 23, 2021 · 18 comments
Closed

Compiling with Clang on Windows fails at libcurl.rc #7765

VitalyVaryvdin opened this issue Sep 23, 2021 · 18 comments
Labels
build Windows Windows-specific

Comments

@VitalyVaryvdin
Copy link

VitalyVaryvdin commented Sep 23, 2021

I did this

Compiling on Windows 10 with Clang 12.0.1. Ninja build files generated by CMake.
Getting error at curl/lib/libcurl.rc.
llvm-rc: error in versioninfo statement (id 1): [build] non-ascii 8-bit codepoint (169) can't occur in a non-unicode string

Removing everything coming after #define RC_VERSION results in successful compilation.

I expected the following

Successful compilation

curl/libcurl version

7.79.1-DEV

operating system

Windows 10.

UPD:
The issue is here VALUE "LegalCopyright", "\xa9 " LIBCURL_COPYRIGHT "\0" /* a9: Copyright symbol */. Removing this results in successful compilation.
To be exact it doesn't like "\xa9 " part.

UPD:
Adding L before the string fixes the issue.
VALUE "LegalCopyright", "\xa9 " LIBCURL_COPYRIGHT "\0" /* a9: Copyright symbol */
Becomes
VALUE "LegalCopyright", L"\xa9 " LIBCURL_COPYRIGHT "\0" /* a9: Copyright symbol */

Copyright symbol is properly displayed in .dll info

Tested under Clang 12.0.1 and MSVC 16.9

@bagder
Copy link
Member

bagder commented Sep 23, 2021

Is this perhaps what's needed? It seems it assumes a certain dir tree layout that may not be true

--- a/lib/libcurl.rc
+++ b/lib/libcurl.rc
@@ -18,11 +18,11 @@
  * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
  * KIND, either express or implied.
  *
  ***************************************************************************/
 #include <winver.h>
-#include "../include/curl/curlver.h"
+#include <curl/curlver.h>
 
 LANGUAGE  0, 0
 
 #define RC_VERSION  LIBCURL_VERSION_MAJOR, LIBCURL_VERSION_MINOR, LIBCURL_VERSION_PATCH, 0
 

@bagder bagder added build Windows Windows-specific labels Sep 23, 2021
bagder added a commit that referenced this issue Sep 23, 2021
Reported-by: Vitaly Varyvdin
Fixes #7765
@VitalyVaryvdin
Copy link
Author

VitalyVaryvdin commented Sep 23, 2021

Nope, doesn't fix the issue.

The issue is here VALUE "LegalCopyright", "\xa9 " LIBCURL_COPYRIGHT "\0" /* a9: Copyright symbol */. Removing this results in successful compilation.

To be exact it doesn't like "\xa9 " part.

@bagder
Copy link
Member

bagder commented Sep 23, 2021

Ok, that's beyond me. @vszakats, any idea?

@VitalyVaryvdin
Copy link
Author

VitalyVaryvdin commented Sep 23, 2021

Not sure if that's correct way to fix that but adding L before the string fixes the issue.

VALUE "LegalCopyright", "\xa9 " LIBCURL_COPYRIGHT "\0" /* a9: Copyright symbol */
Becomes
VALUE "LegalCopyright", L"\xa9 " LIBCURL_COPYRIGHT "\0" /* a9: Copyright symbol */

Copyright symbol is properly displayed in .dll info

Upd: works under MSVC 16.9 as well

@vszakats
Copy link
Member

vszakats commented Sep 23, 2021

Codepage requirements/expectations of various compilers and of Windows resource files in general are generally mysterious, so what about replacing that copyright symbol with: (c)? /cc @bagder

Prefixing with L looks interesting, though I have no clue why it works and how portable that is.

UPDATE: L also fixes it in my (non-Windows) LLVM/clang 12.0.1 + mingw-w64 env with command: x86_64-w64-mingw32-windres -c65001. Another thing to try is a literal © and/or the -c65001 (or /C 65001) option.

@VitalyVaryvdin
Copy link
Author

VitalyVaryvdin commented Sep 23, 2021

According to https://docs.microsoft.com/en-us/windows/win32/menurc/stringtable-resource

To encode Unicode characters, use an "L" followed by the Unicode characters enclosed by quotes. See the Examples section for an example.

IDS_RUSSIANSTRING L"\x0421\x043f\x0440\x0430\x0432\x043a\x0430"

I might assume this being pretty portable and common practice.

@VitalyVaryvdin VitalyVaryvdin changed the title Compiling with Clang on Windows fails at libcurlc.rc Compiling with Clang on Windows fails at libcurl.rc Sep 23, 2021
@vszakats
Copy link
Member

@VitalyVaryvdin Thanks for the link! It looks good to me. This method may even make any -c codepage options unnecessary.

@bagder
Copy link
Member

bagder commented Sep 24, 2021

I personally wouldn't mind using the ascii (C) version either. Whatever makes this work better.

@VitalyVaryvdin
Copy link
Author

VitalyVaryvdin commented Sep 24, 2021

You're welcome!

Using (C) would probably be better option, since it presumable would never bring any issues.

bagder added a commit that referenced this issue Sep 24, 2021
Reported-by: Vitaly Varyvdin
Assisted-by: Viktor Szakats
Fixes #7765
@dfandrich
Copy link
Contributor

dfandrich commented Sep 25, 2021 via email

@vszakats
Copy link
Member

So the options are:

  1. Replacing \xa9 with Copyright (C)
  2. Prefixing current string with L
  3. Adding -c65001 option to all resource compiler invocations (this is suppored by clang/msvc/mingw/pocc/icc/bcc), indicating UTF-8 strings in the .rc input. (and possibly change \xa9 to © in libcurl.rc)
  • I think the third option gives the most future-proofing (meaning: any future UTF-8 string will continue to work there without further issues), but it is also the one requiring the most work.
  • Prefixing with L is nice and light, but I'm not sure how to apply it to non-literal strings (ones coming via macros.)
  • Avoiding UTF-8 is the most risk-averse with the cost of a longer and uglier-looking text and no provisions for any future UTF-8 text.

This adds the flag for the Makefile.m32 build-system:

diff --git a/docs/examples/Makefile.m32 b/docs/examples/Makefile.m32
index 45ec8679c..261dc03be 100644
--- a/docs/examples/Makefile.m32
+++ b/docs/examples/Makefile.m32
@@ -118,7 +118,7 @@ CFLAGS += -fno-strict-aliasing
 # comment LDFLAGS below to keep debug info
 LDFLAGS = $(CURL_LDFLAG_EXTRAS) $(CURL_LDFLAG_EXTRAS_EXE) -s
 RC = $(CROSSPREFIX)windres
-RCFLAGS = --include-dir=$(PROOT)/include -O coff
+RCFLAGS = --include-dir=$(PROOT)/include -O coff -c65001
 
 # Set environment var ARCH to your architecture to override autodetection.
 ifndef ARCH
diff --git a/lib/Makefile.m32 b/lib/Makefile.m32
index 9f1f5963d..0cd903e61 100644
--- a/lib/Makefile.m32
+++ b/lib/Makefile.m32
@@ -115,7 +115,7 @@ LDFLAGS = $(CURL_LDFLAG_EXTRAS) $(CURL_LDFLAG_EXTRAS_DLL) -s
 AR = $(CURL_AR)
 RANLIB = $(CURL_RANLIB)
 RC = $(CROSSPREFIX)windres
-RCFLAGS = --include-dir=$(PROOT)/include -DDEBUGBUILD=0 -O coff
+RCFLAGS = --include-dir=$(PROOT)/include -DDEBUGBUILD=0 -O coff -c65001
 STRIP   = $(CROSSPREFIX)strip -g
 
 # Set environment var ARCH to your architecture to override autodetection.
diff --git a/src/Makefile.m32 b/src/Makefile.m32
index 380d264e7..23afbe308 100644
--- a/src/Makefile.m32
+++ b/src/Makefile.m32
@@ -119,7 +119,7 @@ CFLAGS += -fno-strict-aliasing
 LDFLAGS = $(CURL_LDFLAG_EXTRAS) $(CURL_LDFLAG_EXTRAS_EXE) -s
 AR = $(CURL_AR)
 RC = $(CROSSPREFIX)windres
-RCFLAGS = --include-dir=$(PROOT)/include -O coff -DCURL_EMBED_MANIFEST
+RCFLAGS = --include-dir=$(PROOT)/include -O coff -c65001 -DCURL_EMBED_MANIFEST
 STRIP   = $(CROSSPREFIX)strip -g
 
 # We may need these someday

@VitalyVaryvdin
Copy link
Author

VitalyVaryvdin commented Sep 25, 2021

The symbol is widely recognized but, under the Berne Convention, is no longer required in most nations to assert a new copyright.
The majority of nations now belong to Berne, and thus do not require copyright notices to obtain copyright.

The symbol itself was just an alternative to "Copyright" word in the documents. So probably going with Copyright (C) or even plain Copyright ... is the most elegant solution.

@bagder
Copy link
Member

bagder commented Sep 25, 2021

"Copyright (C)" is what we use in hundreds of sources files already so it should be fine I think.

@gvanem
Copy link
Contributor

gvanem commented Sep 25, 2021

To be exact it doesn't like "\xa9 " part.

I assume llvm-rc is the default selection done by CMake. But could it not use rc instead?

@bagder
Copy link
Member

bagder commented Sep 26, 2021

@gvanem what selection is that? Is that even something we control?

@gvanem
Copy link
Contributor

gvanem commented Sep 26, 2021

@bagder In man cmake-env-variables, there is a mention of CMAKE_RC_COMPILER. Could perhaps override the built-in RC for clang?

@bagder
Copy link
Member

bagder commented Sep 26, 2021

It seems that in order to use that variable someone would have to patch lib/CMakeLists.txt to compile libcurl.rc separately from the other sources with that compiler. I think we can go with the ASCII version until someone finds the time and inspiration to do that change...

@bagder bagder closed this as completed in 1ca62bb Sep 26, 2021
@vszakats
Copy link
Member

vszakats commented Sep 26, 2021

rc belongs to MSVC, so it's not available in every LLVM build-scenario. Also the MSVC's rc is also sensitive to codepage issues. So, this may create different problems.

vszakats added a commit to vszakats/libssh2 that referenced this issue Mar 2, 2023
vszakats added a commit to libssh2/libssh2 that referenced this issue Mar 3, 2023
- in `hostkey.c` check the result of `libssh2_sha256_init()` and
  `libssh2_sha512_init()` calls. This avoid the warning that we're
  ignoring the return values.

- fix code using `int` (or `SOCKET`) for sockets. Use libssh2's
  dedicated `libssh2_socket_t` and `LIBSSH2_INVALID_SOCKET` instead.

- fix compiler warnings due to `STATUS_*` macro redefinitions between
  `ntstatus.h` / `winnt.h`. Solve it by manually defining the single
  `STATUS` value we need from `ntstatus.h` and stop including the whole
  header.
  Fixes #733

- improve Windows UWP/WinRT builds by detecting it with code copied
  from the curl project. Then excluding problematic libssh2 parts
  according to PR by Dmitry Kostjučenko.
  Fixes #734

- always use `SecureZeroMemory()` on Windows.

  We can tweak this if not found or not inlined by a C compiler which
  we otherwise support. Same if it causes issues with UWP apps.

  Ref: https://learn.microsoft.com/en-us/previous-versions/windows/desktop/legacy/aa366877(v=vs.85)
  Ref: https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/wdm/nf-wdm-rtlsecurezeromemory

- always enable `LIBSSH2_CLEAR_MEMORY` on Windows. CMake and
  curl-for-win builds already did that. Delete `SecureZeroMemory()`
  detection from autotools' WinCNG backend logic, that this
  setting used to depend on.

  TODO: Enable it for all platforms in a separate PR.
  TODO: For clearing buffers in WinCNG, call `_libssh2_explicit_zero()`,
        insead of a local function or explicit `SecureZeroMemory()`.

- Makefile.inc: move `os400qc3.h` to `HEADERS`. This fixes
  compilation on non-unixy platforms. Recent regression.

- `libssh2.rc`: replace copyright with plain ASCII, as in curl.

  Ref: curl/curl@1ca62bb
  Ref: curl/curl#7765
  Ref: curl/curl#7776

- CMake fixes and improvements:

  - enable warnings with llvm/clang.
  - enable more comprehensive warnings with gcc and llvm/clang.
    Logic copied from curl:
    https://github.com/curl/curl/blob/233810bb5f6c5e7bedfc10bdd36607b958c0cfe4/CMakeLists.txt#L131-L148
  - fix `Policy CMP0080` CMake warning by deleting that reference.
  - add `ENABLE_WERROR` (default: `OFF`) option. Ported from curl.
  - add `PICKY_COMPILER` (default: `ON`) option, as known from curl.

    It controls both the newly added picky warnings for llvm/clang and
    gcc, and also the pre-existing ones for MSVC.

- `win32/GNUmakefile` fixes and improvements:

  - delete `_AMD64_` and add missing `-m64` for x64 builds under test.
  - add support for `ARCH=custom`.
    It disables hardcoded Intel 64-bit and Intel 32-bit options,
    allowing ARM64 builds.
  - add support for `LIBSSH2_RCFLAG_EXTRAS`.
    To pass custom options to windres, e.g. in ARM64 builds.
  - add support for `LIBSSH2_RC`. To override `windres`.
  - delete support for Metrowerks C. Last released in 2004.

- `win32/libssh2_config.h`: delete unnecessary socket #includes

  `src/libssh2_priv.h` includes `winsock2.h` and `ws2tcpip.h` further
   down the line, triggered by `HAVE_WINSOCK2_H`.

  `mswsock.h` does not seem to be necessary anymore.

  Double-including these (before `windows.h`) caused compiler failures
  when building against BoringSSL and warnings with LibreSSL. We could
  work this around by passing `-DNOCRYPT`. Deleting the duplicates
  fixes these issues.

  Timeline:
  2013: c910cd3 deleted `mswsock.h` from `src/libssh2_priv.h`
  2008: 8c43bc5 added `winsock2.h` and `ws2tcpip.h` to `src/libssh2_priv.h`
  2005: dc4bb1a added the now deleted #includes

- delete or replace `LIBSSH2_WIN32` with `WIN32`.

- replace hand-rolled `HAVE_WINDOWS_H` macro with `WIN32`. Also delete
  its detections/definitions.

- delete unused `LIBSSH2_DARWIN` macro.

- delete unused `writev()` Windows implementation

  There is no reference to `writev()` since 2007-02-02, commit
  9d55db6.

- fix a bunch of MSVC / llvm/clang / gcc compiler warnings:

  - `warning C4100: '...': unreferenced formal parameter`
  - using value of undefined PP macro `LIBSSH2DEBUG`
  - missing void from function definition
  - `if()` block missing in non-debug builds
  - unreferenced variable in non-debug builds
  - `warning: must specify at least one argument for '...' parameter of variadic macro [-Wgnu-zero-variadic-macro-arguments]`
    in `_libssh2_debug()`
  - `warning C4295: 'ciphertext' : array is too small to include a terminating null character`
  - `warning C4706: assignment within conditional expression`
  - `warning C4996: 'inet_addr': Use inet_pton() or InetPton() instead or
      define _WINSOCK_DEPRECATED_NO_WARNINGS to disable deprecated API warnings`
    By suppressning it. Would be best to use inet_pton() as suggested.
    On Windows this needs Vista though.
  - `warning C4152: nonstandard extension, function/data pointer conversion in expression`
    (silenced locally)
  - `warning C4068: unknown pragma`

  Ref: https://ci.appveyor.com/project/libssh2org/libssh2/builds/46354480/job/j7d0m34qgq8rag5w

Closes #808
water111 pushed a commit to open-goal/jak-project that referenced this issue Oct 15, 2023
#3088)

This fix is for the following error when building with llvm on windows

```
[build] llvm-rc: Error in VERSIONINFO statement (ID 1): 
[proc] The command: "C:\Program Files\CMake\bin\cmake.exe" --build C:/.../git/jak-project-test/out/build/Release --parallel 8 exited with code: 1
[build] Non-ASCII 8-bit codepoint (´┐¢) can't be interpreted in the current codepage
[build] ninja: build stopped: subcommand failed.
```

It's caused by llvm-rc not handling non 8-bit characters.
The same issue is discussed in the curl project here
curl/curl#7765

This commit copies the fix from curl, replacing '©' with '(C)'.
Zedb0T added a commit to OpenGOAL-Mods/OG-Mod-Base that referenced this issue Oct 26, 2023
* improve `send-event` for jak 2  (#2828)

Fixes #2825

* [jak2] Work-in-progress texture animations (#2819)

* [jak2] speed up the sky texture animation (#2829)

This saved about 1.6 ms per frame in the city for me (~1.3 saved from
not doing sky twice, 0.3 saved in format lookup tables).

The big texture animator is about 1.0 ms.

![image](https://github.com/open-goal/jak-project/assets/48171810/c7bc7743-308c-4425-ad14-118e2d483fad)

* Added bound check for blend vertex count (#2830)

Co-authored-by: animalstyletaco <animalstyletaco95@gmail.com>

* Get the project compiling on Apple Silicon macOS natively (arm64) (#2827)

I havn't tested it yet, but I can almost guarantee that atleast `goalc`
will not work in the slightest!

But the project is atleast fully compiling. My hope is to start
translating some AVX to NEON next / get `goalc` working...eventually.

* [jak2] More texture animations (#2831)

Added framework to do texture animations entirely in C++. Currently only
works on relatively simple ones, and doesn't handle updating all
parameters - only the speeds.

Connected texture animations to merc and tfrag for skull gems, dark
bomb, and scrolling conveyors.

Cleaned up Tfragment/Tfrag3, which used to be two classes. This was one
of the first C++ renderers, so it had a weird design.

* CI: Periodic Controller Database Update (#2832)

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* [jak2] Fix overlord related crash (#2834)

This fixes the crash reported in
open-goal/jak-project#2833

There was a memory bug here for a long time where our array of `VagCmd`
in `iso_queue.cpp` was too small. This caused GetVagCommand to return
bogus pointers, and sound code would write over other parts of memory.

* [jak2] More progress on texture animations (#2835)

- Add security wall animation
- Add waterfall animations
- Add lava animations
- Update layer values from the game to fix the security wall
- Remove leftover debug in `level.gc` that would break level-specific
animations on the second time you visited the level
- Optionally load animated slot textures to the pool so generic can use
them (fixes skull gems in UI)

* [jak2] Add static textures for the progress menu (#2838)

The progress menu loads its icon textures from a .STR file that we were
previously ignoring.

This change:
- updates the decompiler so it can process a .STR file containing a
texture
- adds a feature to force an entire page to always be loaded in the PC
renderer by putting all textures in the GAME.FR3 file.
- regenerates the texture offset map file for jak 2 with these new
textures

For now, I've just put the icon textures in GAME.FR3. The downside is
that these will always stay on the GPU, using up VRAM even when they
aren't needed. But the entire GAME.FR3 file is under 3 MB so I think
it's ok.


![image](https://github.com/open-goal/jak-project/assets/48171810/39f075b5-7cc5-4168-872a-33026342afab)

* game: add a clear text label for how to hide the top bar (#2845)

* [jak2] Fix dark jak anim and low res skull gems (#2842)

Fixes skull gems using a low resolution texture.

Fixes issue where jak in cutscenes is dark after watching oracle-level-1
(and likely other bugs with texture animations getting stuck)

---------

Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>

* [jak2] make progress menu work for widescreen + various other fixes (#2843)

Lots of manual fixes to make the text and other UI elements in the
progress fit within the small 4x3 view.


![image](https://cdn.discordapp.com/attachments/995787558816595968/1132986451664056423/image.png)

![image](https://cdn.discordapp.com/attachments/995787558816595968/1133064077460131840/image.png)

![image](https://cdn.discordapp.com/attachments/995787558816595968/1133074288564510780/image.png)

![image](https://cdn.discordapp.com/attachments/995787558816595968/1133117633898762280/image.png)

![image](https://cdn.discordapp.com/attachments/995787558816595968/1133117633470922842/image.png)

![image](https://cdn.discordapp.com/attachments/995787558816595968/1133135949610627225/image.png)

![image](https://cdn.discordapp.com/attachments/995787558816595968/1133146823968706621/image.png)

![image](https://cdn.discordapp.com/attachments/995787558816595968/1133153504417042593/image.png)

I've also changed the select start menus to not have that giant space in
each option. It's behind a toggle in the code right now.

![image](https://cdn.discordapp.com/attachments/995787558816595968/1133146382006489138/image.png)

Additionally:
- fixed city billboard particles
- fix stadium crash from original game
- fix an accidental regression

* Fix small buzz motor not vibrating correctly (#2846)

* fix stad-samos regression from refactor (#2796) + change imgui toggle to left-alt and make it customizable (#2848)

* [jak2] More texture animations, and fixes (#2847)

* Fix possible crash when loading files (#2850)

There was a single static path buffer being shared between multiple file
i/o threads. So sometimes you would end up using the wrong path for the
file, and getting size/data for the wrong file.

I think the original reason to have this buffer was just me being lazy
when we changed how project paths works a long time ago. It was a bad
idea in the first place.

* [jak1] Fix ice walking animation bug and crash (#2851)

This should fix a crash and animation bug in snowy. The way to trigger
the bug:
- go on ice
- move forward slowly
- stop moving forward. Reach zero speed when the frame number isn't
between 30 and 35 of the ice walking animation
- Due to this bug, the animation gets stuck at frame 60
- Take damage (due to normal lurker or ice lurker)
- Sidekick eye animation crashes because a frame number is NaN.

* [jak2] graphic options menu proof of concept (#2849)

![imagem](https://github.com/open-goal/jak-project/assets/7569514/b8eaa0b6-7c51-4bd8-96ac-5fb2e0d08c4a)

![imagem](https://github.com/open-goal/jak-project/assets/7569514/d2742cef-98d4-43e4-a48d-cc1996e41612)

(The "Custom" option does not do anything right now.)

* `force-actors?` -> `ps2-actor-vis?` (#2852)

supersedes #2839

* [jak2] workaround for missing yakow textures (#2854)

The yakow texture is actually missing from the real game too. 

This adds a special case in the extractor to replace it with a
similar-ish fur texture:

![image](https://github.com/open-goal/jak-project/assets/48171810/db429e70-e5c5-4302-824a-238e94cf3d69)

* [jak2] fix transparent shrub blending (#2853)

![image](https://github.com/open-goal/jak-project/assets/48171810/ae4d40bf-b061-45af-ba20-c546fc935f11)

Should probably wait until the release just in case. But I'm 99% sure
this line was just a mistake in the original implementation.

* [jak2] Fix ocean culling when using camera-other (#2858)

When drawing the spinning palace, there's a terrible hack that lets some
stuff be drawn with different camera matrices.

The ocean is drawn with camera-other (which spins), but was being culled
with camera (doesn't spin). This changes ocean to use the right camera
matrix.

Note that the ocean is special when it comes to camera-other - it always
uses camera-other if there is one. Other uses are set per-level and
should already be handled correctly.

* [jak2] fix compile error (#2862)

not sure how this passed tests....

* [jak2] use current buffer for blit-displays (#2855)

This changes how `BlitDisplays.cpp` works so it looks at the current
render buffer, rather than the back buffer.

This approach is a bit faster because we avoid copying the back buffer
on every single frame.
It also removes the black frames when the transition starts/stops.

The remaining issues are:
- there's still a single frame of weirdness with the sprite glow
renderer.
- when changing resolutions, it doesn't work super well.

* [jak2] disable envmap when it should be (#2864)

In jak 2, there's an option to disable envmap. It's used on krew
holograms, hiphog trophies, and baron bosses in palace/tomb.

* [gfx] Clean up background renderer matrices, fix "hole covers" (#2866)

The way we got/stored background matrices is a bit weird and full of
leftovers from the first attempts at porting renderers. This doesn't
work well with the Jak 2 "other camera" system where some stuff is
rendered with a different camera matrix.

This cleans most of it up. The exception is that the collide mesh
renderer and the additional sprite culling I added still need to peek at
some cached camera matrices.

This fixes the problem where etie uses the wrong matrices for "other
camera" levels. Now the "hole covers" go in the holes in the background
of the throne room.

![image](https://github.com/open-goal/jak-project/assets/48171810/73a88f7b-05d4-4e9c-bb34-5b45efffcb69)

* ci: strip binaries in their respective runners (#2868)

* New Crowdin updates (#2869)

* fix texture anim shader on macOS (#2870)

* fix collision renderer filters not working (#2872)

fixes #2867

* [goalc] Sign extend objects when loading them (#2863)

Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>

* Anim Tester X - Quality of Life improvements (#2682)

Co-authored-by: Tyler Wilding <xTVaser@users.noreply.github.com>
Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>

* Basic hash table and jump table (#1837)

Co-authored-by: Tyler Wilding <xtvaser@gmail.com>

* github: update issue templates, move general feature requests to discussions preferably (#2879)

* jak1: start a new game correctly when speedrunner mode is enabled (#2873)

* formatter: support formatting bindings, for example in a `let` (#2883)

* [jak2] get new cheats working (#2885)

* [jak2] progress: make custom aspect ratio menu (#2888)

New menu for setting a custom aspect ratio.


![image](https://github.com/open-goal/jak-project/assets/7569514/33f81bc4-0219-4aba-b4ed-81394e4e4ceb)

![image](https://github.com/open-goal/jak-project/assets/7569514/9ddc3059-3ddf-4b35-afa5-11b9005e087c)

![image](https://github.com/open-goal/jak-project/assets/7569514/f3a16342-b38d-4082-ac18-1f82db349191)

![image](https://github.com/open-goal/jak-project/assets/7569514/69963cfe-2963-44f8-bde9-c0205c467d6c)

* CI: Periodic Controller Database Update (#2889)

Updating Controller Database

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* [jak2] prepare `scf-get-territory` usage (#2890)

* log: rotate log files with timestamps and add flag to disable ANSI colors (#2886)

Rotates the log files with a timestamp instead of copying all files and
incrementing an integer. Increases the amount of info you have when
looking at user's log files (ie. when looking at all the files, the file
creation dates are accurate).


![image](https://github.com/open-goal/jak-project/assets/13153231/61bcdf51-f0f6-4eee-b1e5-140aede5d19e)

Also simplifies the API for setting the log file, and `gk` logs are now
game specific with `jak1` or `jak2`. Which should be useful going
forward.

Lastly, added a flag to all CLIs to disable ansi colors for people that
want to do so. Though at the same time, there is finally a workaround in
jenkins to fix ANSI colors in the truncated log view -- so I'm not sure
why anyone would want to get rid of the color information. You can even
setup text editors to display the color info making log parsing much
easier. Fixes #1917

---------

Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>

* custom levels: fix crash when more than one ambient is present (#2891)

* log: don't recursively iterate when rotating log files (#2892)

* cleanup our cmake and build warnings (#2876)

* cmake: fix regression for building SDL statically (#2894)

* input: fix double press issue when assigning a bind (#2895)

* Update test-zone.jsonc to use lowercase for nickname (#2896)

* [jak1] Fix texture-related custom level crash (#2898)

Force FR3 file name for custom levels to be uppercase, fixes the crash
described in
open-goal/jak-project#2534
and 
open-goal/jak-project#2897

* [shrub] Fix bug with gs-prim settings (#2899)

Fix the bug described in
open-goal/jak-project#2882 where some shrubs
are transparent when they shouldn't be. The problem was that we never
carefully looked at the settings in `gs-prim`, which has a bit to
enable/disable alpha blending entirely. Now it should be correct for
both jak 1 and jak 2. To see this change, you'll need to re-extract.

Also adds a setting to disable saving texture .pngs, to speed up
decompilation. I left it on for jak 1 (to avoid confusion for texture
swapping(, but off for jak 2 for now.

* [glow] fix clipping bug (#2902)

Now you can be blinded by the sun:

![image](https://github.com/open-goal/jak-project/assets/48171810/caf3903c-8333-4daa-89ba-ff83ca7d6b2f)

* log: ensure the `log/` directory exists (#2904)

* 989snd: implement SetVolPan for midi sounds (#2905)

Fixes #2820

* [jak 2] bigmap, fix texture filtering on map icons (#2906)

![image](https://github.com/open-goal/jak-project/assets/48171810/4c285f31-c874-424a-8510-d181ba0f88d5)


![image](https://github.com/open-goal/jak-project/assets/48171810/6baa29ef-df92-435e-ad2f-9a42d56e6f17)

and the minimap has filtering now:

![image](https://github.com/open-goal/jak-project/assets/48171810/7cb2b0c9-5c86-426e-b028-dddcc3d649f7)

It's mostly implemented in C++ using the texture animator.

* [jak2] fix bad blerc on objects with warp (#2907)

fixes the weird mirror offset stuff

![image](https://github.com/open-goal/jak-project/assets/48171810/9a36c0bd-7350-46d7-b14f-d2f876582b03)

* [jak2] music player cheat menu (#2900)

* [jak2] small minor fixes (#2909)

- fix deci2 hang when closing the game in retail mode.
- change bigmap to always filter because the pixels look really ugly.
- don't start the game in fullscreen by default if we're debugging.

* [jak2] fix bug in direct in texture handler (#2910)

* handle texture of 0 in extract_tie (#2911)

Fixes texture difference shown in issue
open-goal/jak-project#2908

* [jak2] fix glow sprite flickering (#2912)

oops, forgot to set `glDepthMask`, so sometimes it was enabled, and the
drawing for the clear was writing into the depth buffer.
Fixes open-goal/jak-project#2735

* Fix alpha test > vs >= (minimap arrow fix) (#2913)

Fixes open-goal/jak-project#2814

Previously we didn't handle `GREATER` vs `GEQUAL` correctly. I just left
this as a TODO by accident.


![image](https://github.com/open-goal/jak-project/assets/48171810/bfa7bd98-e40e-4850-af9c-ed6954c00958)

* [jak2] Support Japanese Subtitles (#2914)

![image](https://github.com/open-goal/jak-project/assets/48171810/7b5edd00-4193-47ab-92f6-8f270edbcafe)

* [jak2] Support `draw-raw-image` (#2915)

![image](https://github.com/open-goal/jak-project/assets/48171810/9a94ea6e-99b8-4589-8210-86e63df31dd6)


![image](https://github.com/open-goal/jak-project/assets/48171810/e441225b-214c-46b9-ad01-0160590d30ba)

* [jak2] widescreen fix for splash/jp subtitles (#2916)

* [jak2] fix a few silly texture bugs (#2917)

Fix an issue where the commit sha would not use the right blending mode
if `draw-raw-image` is running at the same time.
Fix an issue where japanese subtitles would accidentally overwrite other
textures, leading to random textures missing. (in particular, glows
would disappear after watching a cutscene with the subtitles on)

* jak1: Consolidate art-elts into a single file as is done in jak 2 (#2887)

Fixes #2167

Reduces test flakiness if ran on multiple threads and gets rid of a few
hundred files from the source tree

I believe this also makes #1434 irrelevant and it can be closed.

---------

Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>

* [jak 2] Clouds V2, Clean up texture animator (#2921)

Some general improvements for the texture animator:
- Clouds are special cased, saving about 1 ms per frame
- Adjusting the amount of clouds now actually works.
- Fixed an issue with the brightness of clouds, and the way that they
fade out around the edges.

* [jak2] Toxic Slime texture animations (#2922)

https://github.com/open-goal/jak-project/assets/48171810/85e68808-09bf-4f74-a16a-6310e2b81cd3

* [jak2] Fix depth testing issues on glow sprites (#2923)

https://github.com/open-goal/jak-project/assets/48171810/0d3064f9-3f27-4be1-b6a4-55fdfb681313

Fixes open-goal/jak-project#2328

* [jak2] a few more subtitles + L1/R1 to move *much* faster through debug menu (#2925)

* [jak2] fill in a lot of flags for decomp + mouse macros (#2927)

Also changed the default type of enums to `int64` (same as `int`).

* [jak2] minimap improvements (#2929)

Smoothens minimap icon motion and fixes the target icon being squished
depending on the angle it's facing.

* g/j1: add support for missing polish alphabet characters (#2931)

* ckernel: fix file paths for art groups and tpages (#2932)

* custom levels: add support for packing models and textures into the FR3 file (#2936)

* [jak2] fix bugs with incorrect music playing sometimes (#2938)

- fixes flava and mode not being correct after a new music starts
playback by setting them every frame (there is at most 1 frame where
it's wrong).
- fixes city pursuit theme not playing if the city alarm was triggered
while the music wasn't playing yet.

* various subtitle editor fixes (#2941)

- fix speaker names and time frames being uneditable in Jak 1
- added toggle to auto-selected a newly created scene as current
- changed the subtitle summary format slightly.
- current scene's name now appears in the UI

* ci: macOS build with big sur to increase compatibility (#2944)

* New Crowdin updates (#2945)

* Placeholder strings for future Jak 2 features + minor subtitle editor fixes (#2943)

* [extractor] fix territory being set to wrong value (#2946)

Also fixes a minor issue where the JP sound bank wouldn't work (Jak 1).

Fixes #2793

* [sprite3] fix max sprite number being wrong (#2949)

I don't think this fixes anything, but the number was wrong so might as
well fix it.

* [jak2] fix bug in `outro-port` script (#2952)

* [jak2] higher resolution cloud textures (#2951)

Adds a new "hires" sky texture animation that makes the sky use 512x512
textures instead of 128x128. The clouds have higher detail, are slightly
more noisy, and are considerably less blocky. There is also a less
noticeable "dot crawl" effect.


![image](https://cdn.discordapp.com/attachments/995787558816595968/1146960820320878643/image.png)

![image](https://cdn.discordapp.com/attachments/995787558816595968/1146960819750449264/image.png)

* [sprite] Adjust allocation randomization (#2953)

This tries to match the original behavior of the sprite allocation
"randomness", which should reduce the number of very empty sprite blocks
sent to C++.

* Make  cloud_lookup return float (#2958)

* rewrite `sp-get-particle` and fix some jak 2 region decomp (#2955)

* goalc: use iso_data `build_info` to inform custom level build process (#2959)

* [gltf export] Export TIE wind models (#2960)

Export models with "wind".  The levels with wind models are:
firecanyon (9), beach (5), village1 (7), lavatube (2).

Sometimes a single object is made up of multiple models - for example
the tree in sandover is actually several meshes.

* CI: Periodic Controller Database Update (#2926)

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* entity-debug: Show path and volume data. (#2942)

* Update subtitle_en-US.json (#2763)

Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>
Co-authored-by: Tyler Wilding <xtvaser@gmail.com>

* g/j2: Some more work on the SQL editor - dump and seed `light` data (#2954)

* decomp: handle dumping jak 2 VAG file assets (#2969)

There are potentially still some minor issues with the resulting files.
Some of them appear to have minor artifacts that playing through the
actual game do not -- but this is a much better starting point for
someone to iterate from if they are interested in improving things.

* jak2: add frame rate option to graphics menu (#2962)

This PR adds a frame rate option to the graphics menu for some of the
most common refresh rates.

Jak 2 has much better support for variable frame rates than Jak 1 out of
the box, but there are still some edge cases, most prominently the fact
that sprites are still limited to the 300 tick system, which is most
noticeable on glow sprites. For this, I abused the glow boost debug
setting to scale the glow based on the frame rate.

While testing, I noticed two other cases that I have also patched,
there's likely to be many more that are yet to be found, but aside from
that, the game is playable as normal.



https://github.com/open-goal/jak-project/assets/6624576/ad4db24f-cd27-4237-a155-0db7008160f3

* [jak2] Fix hover-formation decomp (#2971)

I think this is very likely to fix
open-goal/jak-project#2970

We had somebody report a stacktrace from the debugger, and it was
immediately after calling `gen-perms`.
I found that `gen-perms` writes past the end of a stack array during
this mission, and at the same time as the reported freezes.

I was unable to recreate the original freeze after making this change.

* [jak1] Fix duplicated tie extract (#2972)

Fix a bug where meshes are extracted too many times.

* g/j2: Start adding game options to jak 2's menu (#2968)

* Save entities to JSON, and make custom level building a little faster (#2973)

This will create a folder like `decompiler_out/jak1/entities` and save a
JSON file per level with all the actors.

Also, it should hopefully make custom level building a little faster.

* jak2: macro detection for `launch-particles` and `seconds-per-frame`, add `og:preserve-this` comments for manual patches (#2974)

This PR adds detection of the `launch-particles` and `seconds-per-frame`
macros to the decompiler, removing a lot of bloat and hiding many
process register uses.

I also added `og:preserve-this` comments to as many manual patches and
comments as I could, which will soon be used in conjunction with CI to
hopefully catch any regressions in future big decomp update PRs.

I have some concerns about the `launch-particles` macro (more details in
`sparticle-launcher.gc`) , but thus far, I have not seen anything break
yet.

---------

Co-authored-by: water <awaterford111445@gmail.com>

* g/j2: revert signature changes to `cam-setting-data`, fix camera regressions (#2975)

Co-authored-by: Hat Kid <6624576+Hat-Kid@users.noreply.github.com>

* [jak 2] Export the collision mesh (#2977)

![image](https://github.com/open-goal/jak-project/assets/48171810/266d014b-f6c5-440f-9c3e-ab7c5eafe771)

* g/j1: resolve memory leak around display/input dynamic strings from C++ (#2979)

* CI: Periodic Controller Database Update (#2980)

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* Polish translation for Jak 1 (#2961)

* CI: check for removed goal_src code that we want or need to perserve (#2987)

* [jak2] placeholder subtitle files (#2986)

fixes #2985

* [decompiler] decomp jak2 `static-attack-info` (#2992)

Fixes #2993

* [jak2] fix `task-arrow-spawn` decomp (#2995)

* gsrc: add more `og:preserve-this` comments (#2996)

* g/j2: Implement speedrunner mode in jak 2 (#2976)

* Update game_base_text_pl-PL.json (#2997)

* jak2: fix `metalkor` regression (#2999)

* goalc: add `macro-expand` form (#3000)

* ci: separate lint checks into a required and optional workflow (#3002)

* [jak2] ckernel: implement `loado` and `load_and_link` (#3005)

* decomp: fix some `enemy` decomp (#3008)

* [opengoal] make `none` a child of `object` (#3001)

Previously, `object` and `none` were both top-level types. This made
decompilation rather messy as they have no LCA and resulted in a lot of
variables coming out as type `none` which is very very wrong and
additionally there were plenty of casts to `object`. This changes it so
`none` becomes a child of `object` (it is still represented by
`NullType` which remains unusable in compilation).

This change makes `object` the sole top-level type, and the type that
can represent *any* GOAL object. I believe this matches the original
GOAL built-in type structure. A function that has a return type of
`object` can now return an integer or a `none` at the same time.
However, keep in mind that the return value of `(none)` is still
undefined, just as before. This also makes a cast to `object`
meaningless in 90% of the situations it showed up in (as every single
thing is already an `object`) and the decompiler will no longer emit
them. Casts to `none` are also reduced. Yay!

Additionally, state handlers also don't get the final `(none)` printed
out anymore. The return type of a state handler is completely
meaningless outside the event handler (which is return type `object`
anyway) so there are no limitations on what the last form needs to be. I
did this instead of making them return `object` to trick the decompiler
into not trying to output a variable to be used as a return value
(internally, in the decompiler they still have return type `none`, but
they have `object` elsewhere).

Fixes #1703 
Fixes #830 
Fixes #928

* [jak2] fully implement collide mesh renderer (#3010)

Fixes #2983

* CI: Periodic Controller Database Update (#3006)

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* improve decomp of state handlers and art groups (#3014)

- state handlers that are not inlined lambdas have smarter type
checking, getting rid of 99.9% of the casts emitted (they were not
useful)
- art groups were not being properly linked to their "master" groups.
- `max` in `ja` in Jak 2 was not being detected.

Another huge PR...

* g/j2: speed up level-select menu scrolling (#3013)

* [jak2] also allow analog for play select fast scroll (#3015)

* g/j2: Fill out `game options` menu (#3003)

* scripts: remove unused `code_retention` script code (#3019)

* New Crowdin updates (#3025)

* fix `(font-flags left)` name + fix `right` flag not reflecting in debug text box (#3020)

Fixes #3016 
Fixes #3017

* CI: Periodic Controller Database Update (#3028)

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* [compiler] fix `compile_condition` not attempting macro expansion (#3030)

Ancient bug! Resulted in a 1%-2% decrease in code object file size (and
more importantly the asm is more readable).

Fixes #3029

* decompiler: `obj` -> `this`, `set-time!` and `time-elapsed?` macros (#3026)

This renames the method object in `defmethod`s to `this` and adds
detection for the `set-time!` and `time-elapsed?` macros.

Definitely my biggest PR yet...

* [decompiler] do not use `time-elapsed?` on unsigned comps (#3031)

* misc low hanging fruit fixes (#3034)

Fixes #2235 
Fixes #2236 
Fixes #2963 
Fixes #3027

* Jak 2 controller LED implementation (#3035)

Adds controller LED features to Jak 2:
- progressive flickering denoting health
- copies tomb simon says puzzle colors
- unique colors for each gun
- orange color for being indax
- yellow color for being in mech
- purple color for being darkjak
- blue color for being in board
- red flash when wanted.

May add more features later?

Also did some minor clean-up on some types.

* [jak2] un-mips2c `draw-string` (#3040)

It's not an asm function.

* Remove dupe line in discord-rpc .clang-format (#3039)

* [decompiler] Handle `find-parent-method` (#3018)

This change adds a few new features:
- Decompiler automatically knows the type of `find-parent-method` use in
jak 1 and jak2 when used in a method or virtual state handler.
- Decompiler inserts a call to `call-parent-method` or
`find-parent-state`
- Removed most casts related to these functions

There are still a few minor issues around this:
- There are still some casts needed when using `post` methods, as `post`
is just a `function`, and needs a cast to `(function none)` or similar.
It didn't seem easy to change the type of `post`, so I'm not going to
worry about it for this PR. It only shows up in like 3 places in jak 2.
(and 0 in jak 1)
- If "call the handler if it's not #f" logic should probably be another
macro.

Fixes #805

* [jak2] scalable debug font + fix debug menu widescreen bugs  (#3044)

also hides empty (no commands) regions from the viewers by default now.

* fix draw-string-xy-scaled cropping (#3041)

fixes #3038 


https://github.com/open-goal/jak-project/assets/2515356/5b7a490f-e1fb-4a12-a3d9-b576a2c1d574

* [jak2] fully implement `*user*` (#3046)

Fixes #1918

* Bump actions/checkout from 3 to 4 (#3047)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* g/j2: don't force auto-save on in jak 2 when starting a speedrun (#3048)

* fix eco pickup time bug (#3056)

* CI: Periodic Controller Database Update (#3060)

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* decompiler: better automatic detection of art groups and `joint-node-index` macro detection (#3061)

* [jak2] Set up extractor (#3042)

This sets up the extractor for jak 2. I was expecting that I'd have to
make some more significant changes to the decompiler/compiler path
stuff, but this was not the case!

The only real change is that you can now provide multiple ISO hashes for
an entry in `ISOMetadata`. This is needed for the two different NTSC
versions, which have the same configs, serials, and ELF hashes, but
slightly different contents.

I also didn't add the korean version because I don't have the info for
it.

---------

Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>

* [goalc] Cleaned up speedups (#3066)

Started at 349,880,038 allocations and 42s

- Switched to making `Symbol` in GOOS be a "fixed type", just a wrapper
around a `const char*` pointing to the string in the symbol table. This
is a step toward making a lot of things better, but by itself not a huge
improvement. Some things may be worse due to more temp `std::string`
allocations, but one day all these can be removed. On linux it saved
allocations (347,685,429), and saved a second or two (41 s).
- cache `#t` and `#f` in interpreter, better lookup for special
forms/builtins (hashtable of pointers instead of strings, vector for the
small special form list). Dropped time to 38s.
- special-case in quasiquote when splicing is the last thing in a list.
Allocation dropped to 340,603,082
- custom hash table for environment lookups (lexical vars). Dropped to
36s and 314,637,194
- less allocation in `read_list` 311,613,616. Time about the same.
- `let` and `let*` in Interpreter.cpp 191,988,083, time down to 28s.

* [jak2] new glow fix (#3067)

Fix a typo in the shader. (fixes
open-goal/jak-project#3045, and fixes fadeout
when glows move offscreen on the bottom).

Also, try a different format for the depth blit, in an attempt to fix
this issue open-goal/jak-project#3065.

* decompiler: some hacks to allow running decompiler on jak 3 v5 code files, improve `all-types` generation (#2526)

Co-authored-by: water <awaterford111445@gmail.com>

* [glow] potentially fix bad glow texture (#3071)

Might fix #3065 

fixes #3070

* [jak2] fix minor original game visual bugs (#3072)

Fixes #2994 the underport and consite icons being green (same color as
the goal, likely copy-pasta).

Fixes the hundredths timer in the race timer actually being a 60ths
timer.

* github: add issue template for jak 2 (#3073)

* jak3: add `goal_src` skeleton (#3069)

* [decompiler] Support v5 data file link data (#3076)

Fix the implementation of `link_v5` so it works on "data" files for jak
3.

* decompiler: fix for v5 art group info dump, update `taskfile` for jak 3 (#3077)

* decompiler: allow jak 3 texture and model extraction (#3080)

Added some hacks and stubs to allow extracting textures and models.

* [jak2] Fix collision renderer extract (#3081)

Fixes missing collision geometry reported in
open-goal/jak-project#3011

The issue happens when there are 256 polygons. In this case `num-polys`
is 0 (it's a u8). There are actual cases where there are 0 polygons, so
we have to do a more complicated check to get the real count. I should
have done this in the first place, but it seemed to work...

* g/j2: Integrate highscores with Speedrun.com/JakSpeedruns.com when speedrunner mode is enabled (#3037)

* [jak1] Finnish base game strings (#3058)

Finnish translations for the remaining UI texts

---------

Co-authored-by: Tyler Wilding <xtvaser@gmail.com>

* jak3: add missing files and implement `pexcw` (#3084)

Some files were in the `banned_objects` list and were thus excluded from
the `all_objs` file.

Also implements the `pexcw` instruction which is only used in `hfrag`
code.

* decompiler: add texture merging feature (#3083)

* fix `*jak1-full-game*` being flipped (#3085)

* [glb export] Export bones. (#3087)

![image](https://github.com/open-goal/jak-project/assets/48171810/0f26e77b-af68-4450-882e-762a501bdef4)

* custom levels: support for packing textures (#3089)

* Fix build failure at llvm-rc on windows due to codepage error on zydis (#3088)

This fix is for the following error when building with llvm on windows

```
[build] llvm-rc: Error in VERSIONINFO statement (ID 1): 
[proc] The command: "C:\Program Files\CMake\bin\cmake.exe" --build C:/.../git/jak-project-test/out/build/Release --parallel 8 exited with code: 1
[build] Non-ASCII 8-bit codepoint (´┐¢) can't be interpreted in the current codepage
[build] ninja: build stopped: subcommand failed.
```

It's caused by llvm-rc not handling non 8-bit characters.
The same issue is discussed in the curl project here
curl/curl#7765

This commit copies the fix from curl, replacing '©' with '(C)'.

* jak2: add warning text when changing frame rate for the first time (#3092)

* CI: Periodic Controller Database Update (#3091)

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* Add label_types file merge for PAL version (#3086)

* ci: Workaround CMake/Perl regression in recent windows-2022 images (#3097)

* formatter: rewrite and refactor, address more edge-cases, begin documenting my work (#3096)

* custom levels: refactor level building code and jak 2 support (#3090)

Co-authored-by: water <awaterford111445@gmail.com>

* CI: Periodic Controller Database Update (#3101)

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* Get compiler working

* patches

* Fix custom level textures

---------

Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>
Co-authored-by: water111 <48171810+water111@users.noreply.github.com>
Co-authored-by: animalstyletaco <mcc-108@hotmail.com>
Co-authored-by: animalstyletaco <animalstyletaco95@gmail.com>
Co-authored-by: Tyler Wilding <xTVaser@users.noreply.github.com>
Co-authored-by: OpenGOAL Bot <99294829+OpenGOALBot@users.noreply.github.com>
Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>
Co-authored-by: Luminar Light <18116946+LuminarLight@users.noreply.github.com>
Co-authored-by: Brent Hickey <brent.hickey@icloud.com>
Co-authored-by: Tyler Wilding <xtvaser@gmail.com>
Co-authored-by: Hat Kid <6624576+Hat-Kid@users.noreply.github.com>
Co-authored-by: Matt Dallmeyer <mattdallmeyer@gmail.com>
Co-authored-by: Ziemas <ziemas@ziemas.se>
Co-authored-by: atylmo <48929878+atylmo@users.noreply.github.com>
Co-authored-by: water <awaterford111445@gmail.com>
Co-authored-by: rafalekkB <144032866+rafalekkB@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Aloqas <102683375+Aloqas@users.noreply.github.com>
Co-authored-by: jabermony <748988+jabermony@users.noreply.github.com>
Zedb0T added a commit to my-opengoal-mods/OG-Jak-The-Chicken-Reborn that referenced this issue Oct 29, 2023
* Add python script for custom levels (#25)

* Bump actions/checkout from 3 to 4 (#31)

Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@v3...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Merge vanilla 10/25 (#37)

* improve `send-event` for jak 2  (#2828)

Fixes #2825

* [jak2] Work-in-progress texture animations (#2819)

* [jak2] speed up the sky texture animation (#2829)

This saved about 1.6 ms per frame in the city for me (~1.3 saved from
not doing sky twice, 0.3 saved in format lookup tables).

The big texture animator is about 1.0 ms.

![image](https://github.com/open-goal/jak-project/assets/48171810/c7bc7743-308c-4425-ad14-118e2d483fad)

* Added bound check for blend vertex count (#2830)

Co-authored-by: animalstyletaco <animalstyletaco95@gmail.com>

* Get the project compiling on Apple Silicon macOS natively (arm64) (#2827)

I havn't tested it yet, but I can almost guarantee that atleast `goalc`
will not work in the slightest!

But the project is atleast fully compiling. My hope is to start
translating some AVX to NEON next / get `goalc` working...eventually.

* [jak2] More texture animations (#2831)

Added framework to do texture animations entirely in C++. Currently only
works on relatively simple ones, and doesn't handle updating all
parameters - only the speeds.

Connected texture animations to merc and tfrag for skull gems, dark
bomb, and scrolling conveyors.

Cleaned up Tfragment/Tfrag3, which used to be two classes. This was one
of the first C++ renderers, so it had a weird design.

* CI: Periodic Controller Database Update (#2832)

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* [jak2] Fix overlord related crash (#2834)

This fixes the crash reported in
open-goal/jak-project#2833

There was a memory bug here for a long time where our array of `VagCmd`
in `iso_queue.cpp` was too small. This caused GetVagCommand to return
bogus pointers, and sound code would write over other parts of memory.

* [jak2] More progress on texture animations (#2835)

- Add security wall animation
- Add waterfall animations
- Add lava animations
- Update layer values from the game to fix the security wall
- Remove leftover debug in `level.gc` that would break level-specific
animations on the second time you visited the level
- Optionally load animated slot textures to the pool so generic can use
them (fixes skull gems in UI)

* [jak2] Add static textures for the progress menu (#2838)

The progress menu loads its icon textures from a .STR file that we were
previously ignoring.

This change:
- updates the decompiler so it can process a .STR file containing a
texture
- adds a feature to force an entire page to always be loaded in the PC
renderer by putting all textures in the GAME.FR3 file.
- regenerates the texture offset map file for jak 2 with these new
textures

For now, I've just put the icon textures in GAME.FR3. The downside is
that these will always stay on the GPU, using up VRAM even when they
aren't needed. But the entire GAME.FR3 file is under 3 MB so I think
it's ok.


![image](https://github.com/open-goal/jak-project/assets/48171810/39f075b5-7cc5-4168-872a-33026342afab)

* game: add a clear text label for how to hide the top bar (#2845)

* [jak2] Fix dark jak anim and low res skull gems (#2842)

Fixes skull gems using a low resolution texture.

Fixes issue where jak in cutscenes is dark after watching oracle-level-1
(and likely other bugs with texture animations getting stuck)

---------

Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>

* [jak2] make progress menu work for widescreen + various other fixes (#2843)

Lots of manual fixes to make the text and other UI elements in the
progress fit within the small 4x3 view.


![image](https://cdn.discordapp.com/attachments/995787558816595968/1132986451664056423/image.png)

![image](https://cdn.discordapp.com/attachments/995787558816595968/1133064077460131840/image.png)

![image](https://cdn.discordapp.com/attachments/995787558816595968/1133074288564510780/image.png)

![image](https://cdn.discordapp.com/attachments/995787558816595968/1133117633898762280/image.png)

![image](https://cdn.discordapp.com/attachments/995787558816595968/1133117633470922842/image.png)

![image](https://cdn.discordapp.com/attachments/995787558816595968/1133135949610627225/image.png)

![image](https://cdn.discordapp.com/attachments/995787558816595968/1133146823968706621/image.png)

![image](https://cdn.discordapp.com/attachments/995787558816595968/1133153504417042593/image.png)

I've also changed the select start menus to not have that giant space in
each option. It's behind a toggle in the code right now.

![image](https://cdn.discordapp.com/attachments/995787558816595968/1133146382006489138/image.png)

Additionally:
- fixed city billboard particles
- fix stadium crash from original game
- fix an accidental regression

* Fix small buzz motor not vibrating correctly (#2846)

* fix stad-samos regression from refactor (#2796) + change imgui toggle to left-alt and make it customizable (#2848)

* [jak2] More texture animations, and fixes (#2847)

* Fix possible crash when loading files (#2850)

There was a single static path buffer being shared between multiple file
i/o threads. So sometimes you would end up using the wrong path for the
file, and getting size/data for the wrong file.

I think the original reason to have this buffer was just me being lazy
when we changed how project paths works a long time ago. It was a bad
idea in the first place.

* [jak1] Fix ice walking animation bug and crash (#2851)

This should fix a crash and animation bug in snowy. The way to trigger
the bug:
- go on ice
- move forward slowly
- stop moving forward. Reach zero speed when the frame number isn't
between 30 and 35 of the ice walking animation
- Due to this bug, the animation gets stuck at frame 60
- Take damage (due to normal lurker or ice lurker)
- Sidekick eye animation crashes because a frame number is NaN.

* [jak2] graphic options menu proof of concept (#2849)

![imagem](https://github.com/open-goal/jak-project/assets/7569514/b8eaa0b6-7c51-4bd8-96ac-5fb2e0d08c4a)

![imagem](https://github.com/open-goal/jak-project/assets/7569514/d2742cef-98d4-43e4-a48d-cc1996e41612)

(The "Custom" option does not do anything right now.)

* `force-actors?` -> `ps2-actor-vis?` (#2852)

supersedes #2839

* [jak2] workaround for missing yakow textures (#2854)

The yakow texture is actually missing from the real game too. 

This adds a special case in the extractor to replace it with a
similar-ish fur texture:

![image](https://github.com/open-goal/jak-project/assets/48171810/db429e70-e5c5-4302-824a-238e94cf3d69)

* [jak2] fix transparent shrub blending (#2853)

![image](https://github.com/open-goal/jak-project/assets/48171810/ae4d40bf-b061-45af-ba20-c546fc935f11)

Should probably wait until the release just in case. But I'm 99% sure
this line was just a mistake in the original implementation.

* [jak2] Fix ocean culling when using camera-other (#2858)

When drawing the spinning palace, there's a terrible hack that lets some
stuff be drawn with different camera matrices.

The ocean is drawn with camera-other (which spins), but was being culled
with camera (doesn't spin). This changes ocean to use the right camera
matrix.

Note that the ocean is special when it comes to camera-other - it always
uses camera-other if there is one. Other uses are set per-level and
should already be handled correctly.

* [jak2] fix compile error (#2862)

not sure how this passed tests....

* [jak2] use current buffer for blit-displays (#2855)

This changes how `BlitDisplays.cpp` works so it looks at the current
render buffer, rather than the back buffer.

This approach is a bit faster because we avoid copying the back buffer
on every single frame.
It also removes the black frames when the transition starts/stops.

The remaining issues are:
- there's still a single frame of weirdness with the sprite glow
renderer.
- when changing resolutions, it doesn't work super well.

* [jak2] disable envmap when it should be (#2864)

In jak 2, there's an option to disable envmap. It's used on krew
holograms, hiphog trophies, and baron bosses in palace/tomb.

* [gfx] Clean up background renderer matrices, fix "hole covers" (#2866)

The way we got/stored background matrices is a bit weird and full of
leftovers from the first attempts at porting renderers. This doesn't
work well with the Jak 2 "other camera" system where some stuff is
rendered with a different camera matrix.

This cleans most of it up. The exception is that the collide mesh
renderer and the additional sprite culling I added still need to peek at
some cached camera matrices.

This fixes the problem where etie uses the wrong matrices for "other
camera" levels. Now the "hole covers" go in the holes in the background
of the throne room.

![image](https://github.com/open-goal/jak-project/assets/48171810/73a88f7b-05d4-4e9c-bb34-5b45efffcb69)

* ci: strip binaries in their respective runners (#2868)

* New Crowdin updates (#2869)

* fix texture anim shader on macOS (#2870)

* fix collision renderer filters not working (#2872)

fixes #2867

* [goalc] Sign extend objects when loading them (#2863)

Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>

* Anim Tester X - Quality of Life improvements (#2682)

Co-authored-by: Tyler Wilding <xTVaser@users.noreply.github.com>
Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>

* Basic hash table and jump table (#1837)

Co-authored-by: Tyler Wilding <xtvaser@gmail.com>

* github: update issue templates, move general feature requests to discussions preferably (#2879)

* jak1: start a new game correctly when speedrunner mode is enabled (#2873)

* formatter: support formatting bindings, for example in a `let` (#2883)

* [jak2] get new cheats working (#2885)

* [jak2] progress: make custom aspect ratio menu (#2888)

New menu for setting a custom aspect ratio.


![image](https://github.com/open-goal/jak-project/assets/7569514/33f81bc4-0219-4aba-b4ed-81394e4e4ceb)

![image](https://github.com/open-goal/jak-project/assets/7569514/9ddc3059-3ddf-4b35-afa5-11b9005e087c)

![image](https://github.com/open-goal/jak-project/assets/7569514/f3a16342-b38d-4082-ac18-1f82db349191)

![image](https://github.com/open-goal/jak-project/assets/7569514/69963cfe-2963-44f8-bde9-c0205c467d6c)

* CI: Periodic Controller Database Update (#2889)

Updating Controller Database

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* [jak2] prepare `scf-get-territory` usage (#2890)

* log: rotate log files with timestamps and add flag to disable ANSI colors (#2886)

Rotates the log files with a timestamp instead of copying all files and
incrementing an integer. Increases the amount of info you have when
looking at user's log files (ie. when looking at all the files, the file
creation dates are accurate).


![image](https://github.com/open-goal/jak-project/assets/13153231/61bcdf51-f0f6-4eee-b1e5-140aede5d19e)

Also simplifies the API for setting the log file, and `gk` logs are now
game specific with `jak1` or `jak2`. Which should be useful going
forward.

Lastly, added a flag to all CLIs to disable ansi colors for people that
want to do so. Though at the same time, there is finally a workaround in
jenkins to fix ANSI colors in the truncated log view -- so I'm not sure
why anyone would want to get rid of the color information. You can even
setup text editors to display the color info making log parsing much
easier. Fixes #1917

---------

Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>

* custom levels: fix crash when more than one ambient is present (#2891)

* log: don't recursively iterate when rotating log files (#2892)

* cleanup our cmake and build warnings (#2876)

* cmake: fix regression for building SDL statically (#2894)

* input: fix double press issue when assigning a bind (#2895)

* Update test-zone.jsonc to use lowercase for nickname (#2896)

* [jak1] Fix texture-related custom level crash (#2898)

Force FR3 file name for custom levels to be uppercase, fixes the crash
described in
open-goal/jak-project#2534
and 
open-goal/jak-project#2897

* [shrub] Fix bug with gs-prim settings (#2899)

Fix the bug described in
open-goal/jak-project#2882 where some shrubs
are transparent when they shouldn't be. The problem was that we never
carefully looked at the settings in `gs-prim`, which has a bit to
enable/disable alpha blending entirely. Now it should be correct for
both jak 1 and jak 2. To see this change, you'll need to re-extract.

Also adds a setting to disable saving texture .pngs, to speed up
decompilation. I left it on for jak 1 (to avoid confusion for texture
swapping(, but off for jak 2 for now.

* [glow] fix clipping bug (#2902)

Now you can be blinded by the sun:

![image](https://github.com/open-goal/jak-project/assets/48171810/caf3903c-8333-4daa-89ba-ff83ca7d6b2f)

* log: ensure the `log/` directory exists (#2904)

* 989snd: implement SetVolPan for midi sounds (#2905)

Fixes #2820

* [jak 2] bigmap, fix texture filtering on map icons (#2906)

![image](https://github.com/open-goal/jak-project/assets/48171810/4c285f31-c874-424a-8510-d181ba0f88d5)


![image](https://github.com/open-goal/jak-project/assets/48171810/6baa29ef-df92-435e-ad2f-9a42d56e6f17)

and the minimap has filtering now:

![image](https://github.com/open-goal/jak-project/assets/48171810/7cb2b0c9-5c86-426e-b028-dddcc3d649f7)

It's mostly implemented in C++ using the texture animator.

* [jak2] fix bad blerc on objects with warp (#2907)

fixes the weird mirror offset stuff

![image](https://github.com/open-goal/jak-project/assets/48171810/9a36c0bd-7350-46d7-b14f-d2f876582b03)

* [jak2] music player cheat menu (#2900)

* [jak2] small minor fixes (#2909)

- fix deci2 hang when closing the game in retail mode.
- change bigmap to always filter because the pixels look really ugly.
- don't start the game in fullscreen by default if we're debugging.

* [jak2] fix bug in direct in texture handler (#2910)

* handle texture of 0 in extract_tie (#2911)

Fixes texture difference shown in issue
open-goal/jak-project#2908

* [jak2] fix glow sprite flickering (#2912)

oops, forgot to set `glDepthMask`, so sometimes it was enabled, and the
drawing for the clear was writing into the depth buffer.
Fixes open-goal/jak-project#2735

* Fix alpha test > vs >= (minimap arrow fix) (#2913)

Fixes open-goal/jak-project#2814

Previously we didn't handle `GREATER` vs `GEQUAL` correctly. I just left
this as a TODO by accident.


![image](https://github.com/open-goal/jak-project/assets/48171810/bfa7bd98-e40e-4850-af9c-ed6954c00958)

* [jak2] Support Japanese Subtitles (#2914)

![image](https://github.com/open-goal/jak-project/assets/48171810/7b5edd00-4193-47ab-92f6-8f270edbcafe)

* [jak2] Support `draw-raw-image` (#2915)

![image](https://github.com/open-goal/jak-project/assets/48171810/9a94ea6e-99b8-4589-8210-86e63df31dd6)


![image](https://github.com/open-goal/jak-project/assets/48171810/e441225b-214c-46b9-ad01-0160590d30ba)

* [jak2] widescreen fix for splash/jp subtitles (#2916)

* [jak2] fix a few silly texture bugs (#2917)

Fix an issue where the commit sha would not use the right blending mode
if `draw-raw-image` is running at the same time.
Fix an issue where japanese subtitles would accidentally overwrite other
textures, leading to random textures missing. (in particular, glows
would disappear after watching a cutscene with the subtitles on)

* jak1: Consolidate art-elts into a single file as is done in jak 2 (#2887)

Fixes #2167

Reduces test flakiness if ran on multiple threads and gets rid of a few
hundred files from the source tree

I believe this also makes #1434 irrelevant and it can be closed.

---------

Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>

* [jak 2] Clouds V2, Clean up texture animator (#2921)

Some general improvements for the texture animator:
- Clouds are special cased, saving about 1 ms per frame
- Adjusting the amount of clouds now actually works.
- Fixed an issue with the brightness of clouds, and the way that they
fade out around the edges.

* [jak2] Toxic Slime texture animations (#2922)

https://github.com/open-goal/jak-project/assets/48171810/85e68808-09bf-4f74-a16a-6310e2b81cd3

* [jak2] Fix depth testing issues on glow sprites (#2923)

https://github.com/open-goal/jak-project/assets/48171810/0d3064f9-3f27-4be1-b6a4-55fdfb681313

Fixes open-goal/jak-project#2328

* [jak2] a few more subtitles + L1/R1 to move *much* faster through debug menu (#2925)

* [jak2] fill in a lot of flags for decomp + mouse macros (#2927)

Also changed the default type of enums to `int64` (same as `int`).

* [jak2] minimap improvements (#2929)

Smoothens minimap icon motion and fixes the target icon being squished
depending on the angle it's facing.

* g/j1: add support for missing polish alphabet characters (#2931)

* ckernel: fix file paths for art groups and tpages (#2932)

* custom levels: add support for packing models and textures into the FR3 file (#2936)

* [jak2] fix bugs with incorrect music playing sometimes (#2938)

- fixes flava and mode not being correct after a new music starts
playback by setting them every frame (there is at most 1 frame where
it's wrong).
- fixes city pursuit theme not playing if the city alarm was triggered
while the music wasn't playing yet.

* various subtitle editor fixes (#2941)

- fix speaker names and time frames being uneditable in Jak 1
- added toggle to auto-selected a newly created scene as current
- changed the subtitle summary format slightly.
- current scene's name now appears in the UI

* ci: macOS build with big sur to increase compatibility (#2944)

* New Crowdin updates (#2945)

* Placeholder strings for future Jak 2 features + minor subtitle editor fixes (#2943)

* [extractor] fix territory being set to wrong value (#2946)

Also fixes a minor issue where the JP sound bank wouldn't work (Jak 1).

Fixes #2793

* [sprite3] fix max sprite number being wrong (#2949)

I don't think this fixes anything, but the number was wrong so might as
well fix it.

* [jak2] fix bug in `outro-port` script (#2952)

* [jak2] higher resolution cloud textures (#2951)

Adds a new "hires" sky texture animation that makes the sky use 512x512
textures instead of 128x128. The clouds have higher detail, are slightly
more noisy, and are considerably less blocky. There is also a less
noticeable "dot crawl" effect.


![image](https://cdn.discordapp.com/attachments/995787558816595968/1146960820320878643/image.png)

![image](https://cdn.discordapp.com/attachments/995787558816595968/1146960819750449264/image.png)

* [sprite] Adjust allocation randomization (#2953)

This tries to match the original behavior of the sprite allocation
"randomness", which should reduce the number of very empty sprite blocks
sent to C++.

* Make  cloud_lookup return float (#2958)

* rewrite `sp-get-particle` and fix some jak 2 region decomp (#2955)

* goalc: use iso_data `build_info` to inform custom level build process (#2959)

* [gltf export] Export TIE wind models (#2960)

Export models with "wind".  The levels with wind models are:
firecanyon (9), beach (5), village1 (7), lavatube (2).

Sometimes a single object is made up of multiple models - for example
the tree in sandover is actually several meshes.

* CI: Periodic Controller Database Update (#2926)

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* entity-debug: Show path and volume data. (#2942)

* Update subtitle_en-US.json (#2763)

Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>
Co-authored-by: Tyler Wilding <xtvaser@gmail.com>

* g/j2: Some more work on the SQL editor - dump and seed `light` data (#2954)

* decomp: handle dumping jak 2 VAG file assets (#2969)

There are potentially still some minor issues with the resulting files.
Some of them appear to have minor artifacts that playing through the
actual game do not -- but this is a much better starting point for
someone to iterate from if they are interested in improving things.

* jak2: add frame rate option to graphics menu (#2962)

This PR adds a frame rate option to the graphics menu for some of the
most common refresh rates.

Jak 2 has much better support for variable frame rates than Jak 1 out of
the box, but there are still some edge cases, most prominently the fact
that sprites are still limited to the 300 tick system, which is most
noticeable on glow sprites. For this, I abused the glow boost debug
setting to scale the glow based on the frame rate.

While testing, I noticed two other cases that I have also patched,
there's likely to be many more that are yet to be found, but aside from
that, the game is playable as normal.



https://github.com/open-goal/jak-project/assets/6624576/ad4db24f-cd27-4237-a155-0db7008160f3

* [jak2] Fix hover-formation decomp (#2971)

I think this is very likely to fix
open-goal/jak-project#2970

We had somebody report a stacktrace from the debugger, and it was
immediately after calling `gen-perms`.
I found that `gen-perms` writes past the end of a stack array during
this mission, and at the same time as the reported freezes.

I was unable to recreate the original freeze after making this change.

* [jak1] Fix duplicated tie extract (#2972)

Fix a bug where meshes are extracted too many times.

* g/j2: Start adding game options to jak 2's menu (#2968)

* Save entities to JSON, and make custom level building a little faster (#2973)

This will create a folder like `decompiler_out/jak1/entities` and save a
JSON file per level with all the actors.

Also, it should hopefully make custom level building a little faster.

* jak2: macro detection for `launch-particles` and `seconds-per-frame`, add `og:preserve-this` comments for manual patches (#2974)

This PR adds detection of the `launch-particles` and `seconds-per-frame`
macros to the decompiler, removing a lot of bloat and hiding many
process register uses.

I also added `og:preserve-this` comments to as many manual patches and
comments as I could, which will soon be used in conjunction with CI to
hopefully catch any regressions in future big decomp update PRs.

I have some concerns about the `launch-particles` macro (more details in
`sparticle-launcher.gc`) , but thus far, I have not seen anything break
yet.

---------

Co-authored-by: water <awaterford111445@gmail.com>

* g/j2: revert signature changes to `cam-setting-data`, fix camera regressions (#2975)

Co-authored-by: Hat Kid <6624576+Hat-Kid@users.noreply.github.com>

* [jak 2] Export the collision mesh (#2977)

![image](https://github.com/open-goal/jak-project/assets/48171810/266d014b-f6c5-440f-9c3e-ab7c5eafe771)

* g/j1: resolve memory leak around display/input dynamic strings from C++ (#2979)

* CI: Periodic Controller Database Update (#2980)

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* Polish translation for Jak 1 (#2961)

* CI: check for removed goal_src code that we want or need to perserve (#2987)

* [jak2] placeholder subtitle files (#2986)

fixes #2985

* [decompiler] decomp jak2 `static-attack-info` (#2992)

Fixes #2993

* [jak2] fix `task-arrow-spawn` decomp (#2995)

* gsrc: add more `og:preserve-this` comments (#2996)

* g/j2: Implement speedrunner mode in jak 2 (#2976)

* Update game_base_text_pl-PL.json (#2997)

* jak2: fix `metalkor` regression (#2999)

* goalc: add `macro-expand` form (#3000)

* ci: separate lint checks into a required and optional workflow (#3002)

* [jak2] ckernel: implement `loado` and `load_and_link` (#3005)

* decomp: fix some `enemy` decomp (#3008)

* [opengoal] make `none` a child of `object` (#3001)

Previously, `object` and `none` were both top-level types. This made
decompilation rather messy as they have no LCA and resulted in a lot of
variables coming out as type `none` which is very very wrong and
additionally there were plenty of casts to `object`. This changes it so
`none` becomes a child of `object` (it is still represented by
`NullType` which remains unusable in compilation).

This change makes `object` the sole top-level type, and the type that
can represent *any* GOAL object. I believe this matches the original
GOAL built-in type structure. A function that has a return type of
`object` can now return an integer or a `none` at the same time.
However, keep in mind that the return value of `(none)` is still
undefined, just as before. This also makes a cast to `object`
meaningless in 90% of the situations it showed up in (as every single
thing is already an `object`) and the decompiler will no longer emit
them. Casts to `none` are also reduced. Yay!

Additionally, state handlers also don't get the final `(none)` printed
out anymore. The return type of a state handler is completely
meaningless outside the event handler (which is return type `object`
anyway) so there are no limitations on what the last form needs to be. I
did this instead of making them return `object` to trick the decompiler
into not trying to output a variable to be used as a return value
(internally, in the decompiler they still have return type `none`, but
they have `object` elsewhere).

Fixes #1703 
Fixes #830 
Fixes #928

* [jak2] fully implement collide mesh renderer (#3010)

Fixes #2983

* CI: Periodic Controller Database Update (#3006)

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* improve decomp of state handlers and art groups (#3014)

- state handlers that are not inlined lambdas have smarter type
checking, getting rid of 99.9% of the casts emitted (they were not
useful)
- art groups were not being properly linked to their "master" groups.
- `max` in `ja` in Jak 2 was not being detected.

Another huge PR...

* g/j2: speed up level-select menu scrolling (#3013)

* [jak2] also allow analog for play select fast scroll (#3015)

* g/j2: Fill out `game options` menu (#3003)

* scripts: remove unused `code_retention` script code (#3019)

* New Crowdin updates (#3025)

* fix `(font-flags left)` name + fix `right` flag not reflecting in debug text box (#3020)

Fixes #3016 
Fixes #3017

* CI: Periodic Controller Database Update (#3028)

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* [compiler] fix `compile_condition` not attempting macro expansion (#3030)

Ancient bug! Resulted in a 1%-2% decrease in code object file size (and
more importantly the asm is more readable).

Fixes #3029

* decompiler: `obj` -> `this`, `set-time!` and `time-elapsed?` macros (#3026)

This renames the method object in `defmethod`s to `this` and adds
detection for the `set-time!` and `time-elapsed?` macros.

Definitely my biggest PR yet...

* [decompiler] do not use `time-elapsed?` on unsigned comps (#3031)

* misc low hanging fruit fixes (#3034)

Fixes #2235 
Fixes #2236 
Fixes #2963 
Fixes #3027

* Jak 2 controller LED implementation (#3035)

Adds controller LED features to Jak 2:
- progressive flickering denoting health
- copies tomb simon says puzzle colors
- unique colors for each gun
- orange color for being indax
- yellow color for being in mech
- purple color for being darkjak
- blue color for being in board
- red flash when wanted.

May add more features later?

Also did some minor clean-up on some types.

* [jak2] un-mips2c `draw-string` (#3040)

It's not an asm function.

* Remove dupe line in discord-rpc .clang-format (#3039)

* [decompiler] Handle `find-parent-method` (#3018)

This change adds a few new features:
- Decompiler automatically knows the type of `find-parent-method` use in
jak 1 and jak2 when used in a method or virtual state handler.
- Decompiler inserts a call to `call-parent-method` or
`find-parent-state`
- Removed most casts related to these functions

There are still a few minor issues around this:
- There are still some casts needed when using `post` methods, as `post`
is just a `function`, and needs a cast to `(function none)` or similar.
It didn't seem easy to change the type of `post`, so I'm not going to
worry about it for this PR. It only shows up in like 3 places in jak 2.
(and 0 in jak 1)
- If "call the handler if it's not #f" logic should probably be another
macro.

Fixes #805

* [jak2] scalable debug font + fix debug menu widescreen bugs  (#3044)

also hides empty (no commands) regions from the viewers by default now.

* fix draw-string-xy-scaled cropping (#3041)

fixes #3038 


https://github.com/open-goal/jak-project/assets/2515356/5b7a490f-e1fb-4a12-a3d9-b576a2c1d574

* [jak2] fully implement `*user*` (#3046)

Fixes #1918

* Bump actions/checkout from 3 to 4 (#3047)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* g/j2: don't force auto-save on in jak 2 when starting a speedrun (#3048)

* fix eco pickup time bug (#3056)

* CI: Periodic Controller Database Update (#3060)

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* decompiler: better automatic detection of art groups and `joint-node-index` macro detection (#3061)

* [jak2] Set up extractor (#3042)

This sets up the extractor for jak 2. I was expecting that I'd have to
make some more significant changes to the decompiler/compiler path
stuff, but this was not the case!

The only real change is that you can now provide multiple ISO hashes for
an entry in `ISOMetadata`. This is needed for the two different NTSC
versions, which have the same configs, serials, and ELF hashes, but
slightly different contents.

I also didn't add the korean version because I don't have the info for
it.

---------

Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>

* [goalc] Cleaned up speedups (#3066)

Started at 349,880,038 allocations and 42s

- Switched to making `Symbol` in GOOS be a "fixed type", just a wrapper
around a `const char*` pointing to the string in the symbol table. This
is a step toward making a lot of things better, but by itself not a huge
improvement. Some things may be worse due to more temp `std::string`
allocations, but one day all these can be removed. On linux it saved
allocations (347,685,429), and saved a second or two (41 s).
- cache `#t` and `#f` in interpreter, better lookup for special
forms/builtins (hashtable of pointers instead of strings, vector for the
small special form list). Dropped time to 38s.
- special-case in quasiquote when splicing is the last thing in a list.
Allocation dropped to 340,603,082
- custom hash table for environment lookups (lexical vars). Dropped to
36s and 314,637,194
- less allocation in `read_list` 311,613,616. Time about the same.
- `let` and `let*` in Interpreter.cpp 191,988,083, time down to 28s.

* [jak2] new glow fix (#3067)

Fix a typo in the shader. (fixes
open-goal/jak-project#3045, and fixes fadeout
when glows move offscreen on the bottom).

Also, try a different format for the depth blit, in an attempt to fix
this issue open-goal/jak-project#3065.

* decompiler: some hacks to allow running decompiler on jak 3 v5 code files, improve `all-types` generation (#2526)

Co-authored-by: water <awaterford111445@gmail.com>

* [glow] potentially fix bad glow texture (#3071)

Might fix #3065 

fixes #3070

* [jak2] fix minor original game visual bugs (#3072)

Fixes #2994 the underport and consite icons being green (same color as
the goal, likely copy-pasta).

Fixes the hundredths timer in the race timer actually being a 60ths
timer.

* github: add issue template for jak 2 (#3073)

* jak3: add `goal_src` skeleton (#3069)

* [decompiler] Support v5 data file link data (#3076)

Fix the implementation of `link_v5` so it works on "data" files for jak
3.

* decompiler: fix for v5 art group info dump, update `taskfile` for jak 3 (#3077)

* decompiler: allow jak 3 texture and model extraction (#3080)

Added some hacks and stubs to allow extracting textures and models.

* [jak2] Fix collision renderer extract (#3081)

Fixes missing collision geometry reported in
open-goal/jak-project#3011

The issue happens when there are 256 polygons. In this case `num-polys`
is 0 (it's a u8). There are actual cases where there are 0 polygons, so
we have to do a more complicated check to get the real count. I should
have done this in the first place, but it seemed to work...

* g/j2: Integrate highscores with Speedrun.com/JakSpeedruns.com when speedrunner mode is enabled (#3037)

* [jak1] Finnish base game strings (#3058)

Finnish translations for the remaining UI texts

---------

Co-authored-by: Tyler Wilding <xtvaser@gmail.com>

* jak3: add missing files and implement `pexcw` (#3084)

Some files were in the `banned_objects` list and were thus excluded from
the `all_objs` file.

Also implements the `pexcw` instruction which is only used in `hfrag`
code.

* decompiler: add texture merging feature (#3083)

* fix `*jak1-full-game*` being flipped (#3085)

* [glb export] Export bones. (#3087)

![image](https://github.com/open-goal/jak-project/assets/48171810/0f26e77b-af68-4450-882e-762a501bdef4)

* custom levels: support for packing textures (#3089)

* Fix build failure at llvm-rc on windows due to codepage error on zydis (#3088)

This fix is for the following error when building with llvm on windows

```
[build] llvm-rc: Error in VERSIONINFO statement (ID 1): 
[proc] The command: "C:\Program Files\CMake\bin\cmake.exe" --build C:/.../git/jak-project-test/out/build/Release --parallel 8 exited with code: 1
[build] Non-ASCII 8-bit codepoint (´┐¢) can't be interpreted in the current codepage
[build] ninja: build stopped: subcommand failed.
```

It's caused by llvm-rc not handling non 8-bit characters.
The same issue is discussed in the curl project here
curl/curl#7765

This commit copies the fix from curl, replacing '©' with '(C)'.

* jak2: add warning text when changing frame rate for the first time (#3092)

* CI: Periodic Controller Database Update (#3091)

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* Add label_types file merge for PAL version (#3086)

* ci: Workaround CMake/Perl regression in recent windows-2022 images (#3097)

* formatter: rewrite and refactor, address more edge-cases, begin documenting my work (#3096)

* custom levels: refactor level building code and jak 2 support (#3090)

Co-authored-by: water <awaterford111445@gmail.com>

* CI: Periodic Controller Database Update (#3101)

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* Get compiler working

* patches

* Fix custom level textures

---------

Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>
Co-authored-by: water111 <48171810+water111@users.noreply.github.com>
Co-authored-by: animalstyletaco <mcc-108@hotmail.com>
Co-authored-by: animalstyletaco <animalstyletaco95@gmail.com>
Co-authored-by: Tyler Wilding <xTVaser@users.noreply.github.com>
Co-authored-by: OpenGOAL Bot <99294829+OpenGOALBot@users.noreply.github.com>
Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>
Co-authored-by: Luminar Light <18116946+LuminarLight@users.noreply.github.com>
Co-authored-by: Brent Hickey <brent.hickey@icloud.com>
Co-authored-by: Tyler Wilding <xtvaser@gmail.com>
Co-authored-by: Hat Kid <6624576+Hat-Kid@users.noreply.github.com>
Co-authored-by: Matt Dallmeyer <mattdallmeyer@gmail.com>
Co-authored-by: Ziemas <ziemas@ziemas.se>
Co-authored-by: atylmo <48929878+atylmo@users.noreply.github.com>
Co-authored-by: water <awaterford111445@gmail.com>
Co-authored-by: rafalekkB <144032866+rafalekkB@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Aloqas <102683375+Aloqas@users.noreply.github.com>
Co-authored-by: jabermony <748988+jabermony@users.noreply.github.com>

* mod ver placeholder

* merge test 2

* skel effect hack

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>
Co-authored-by: water111 <48171810+water111@users.noreply.github.com>
Co-authored-by: animalstyletaco <mcc-108@hotmail.com>
Co-authored-by: animalstyletaco <animalstyletaco95@gmail.com>
Co-authored-by: Tyler Wilding <xTVaser@users.noreply.github.com>
Co-authored-by: OpenGOAL Bot <99294829+OpenGOALBot@users.noreply.github.com>
Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>
Co-authored-by: Luminar Light <18116946+LuminarLight@users.noreply.github.com>
Co-authored-by: Brent Hickey <brent.hickey@icloud.com>
Co-authored-by: Tyler Wilding <xtvaser@gmail.com>
Co-authored-by: Hat Kid <6624576+Hat-Kid@users.noreply.github.com>
Co-authored-by: Matt Dallmeyer <mattdallmeyer@gmail.com>
Co-authored-by: Ziemas <ziemas@ziemas.se>
Co-authored-by: atylmo <48929878+atylmo@users.noreply.github.com>
Co-authored-by: water <awaterford111445@gmail.com>
Co-authored-by: rafalekkB <144032866+rafalekkB@users.noreply.github.com>
Co-authored-by: Aloqas <102683375+Aloqas@users.noreply.github.com>
Co-authored-by: jabermony <748988+jabermony@users.noreply.github.com>
Co-authored-by: dallmeyer <2515356+dallmeyer@users.noreply.github.com>
Zedb0T added a commit to my-opengoal-mods/OG-Jak-The-Chicken-Reborn that referenced this issue Oct 30, 2023
* Add python script for custom levels (#25)

* Bump actions/checkout from 3 to 4 (#31)

Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@v3...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Merge vanilla 10/25 (#37)

* improve `send-event` for jak 2  (#2828)

Fixes #2825

* [jak2] Work-in-progress texture animations (#2819)

* [jak2] speed up the sky texture animation (#2829)

This saved about 1.6 ms per frame in the city for me (~1.3 saved from
not doing sky twice, 0.3 saved in format lookup tables).

The big texture animator is about 1.0 ms.

![image](https://github.com/open-goal/jak-project/assets/48171810/c7bc7743-308c-4425-ad14-118e2d483fad)

* Added bound check for blend vertex count (#2830)

Co-authored-by: animalstyletaco <animalstyletaco95@gmail.com>

* Get the project compiling on Apple Silicon macOS natively (arm64) (#2827)

I havn't tested it yet, but I can almost guarantee that atleast `goalc`
will not work in the slightest!

But the project is atleast fully compiling. My hope is to start
translating some AVX to NEON next / get `goalc` working...eventually.

* [jak2] More texture animations (#2831)

Added framework to do texture animations entirely in C++. Currently only
works on relatively simple ones, and doesn't handle updating all
parameters - only the speeds.

Connected texture animations to merc and tfrag for skull gems, dark
bomb, and scrolling conveyors.

Cleaned up Tfragment/Tfrag3, which used to be two classes. This was one
of the first C++ renderers, so it had a weird design.

* CI: Periodic Controller Database Update (#2832)

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* [jak2] Fix overlord related crash (#2834)

This fixes the crash reported in
open-goal/jak-project#2833

There was a memory bug here for a long time where our array of `VagCmd`
in `iso_queue.cpp` was too small. This caused GetVagCommand to return
bogus pointers, and sound code would write over other parts of memory.

* [jak2] More progress on texture animations (#2835)

- Add security wall animation
- Add waterfall animations
- Add lava animations
- Update layer values from the game to fix the security wall
- Remove leftover debug in `level.gc` that would break level-specific
animations on the second time you visited the level
- Optionally load animated slot textures to the pool so generic can use
them (fixes skull gems in UI)

* [jak2] Add static textures for the progress menu (#2838)

The progress menu loads its icon textures from a .STR file that we were
previously ignoring.

This change:
- updates the decompiler so it can process a .STR file containing a
texture
- adds a feature to force an entire page to always be loaded in the PC
renderer by putting all textures in the GAME.FR3 file.
- regenerates the texture offset map file for jak 2 with these new
textures

For now, I've just put the icon textures in GAME.FR3. The downside is
that these will always stay on the GPU, using up VRAM even when they
aren't needed. But the entire GAME.FR3 file is under 3 MB so I think
it's ok.


![image](https://github.com/open-goal/jak-project/assets/48171810/39f075b5-7cc5-4168-872a-33026342afab)

* game: add a clear text label for how to hide the top bar (#2845)

* [jak2] Fix dark jak anim and low res skull gems (#2842)

Fixes skull gems using a low resolution texture.

Fixes issue where jak in cutscenes is dark after watching oracle-level-1
(and likely other bugs with texture animations getting stuck)

---------

Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>

* [jak2] make progress menu work for widescreen + various other fixes (#2843)

Lots of manual fixes to make the text and other UI elements in the
progress fit within the small 4x3 view.


![image](https://cdn.discordapp.com/attachments/995787558816595968/1132986451664056423/image.png)

![image](https://cdn.discordapp.com/attachments/995787558816595968/1133064077460131840/image.png)

![image](https://cdn.discordapp.com/attachments/995787558816595968/1133074288564510780/image.png)

![image](https://cdn.discordapp.com/attachments/995787558816595968/1133117633898762280/image.png)

![image](https://cdn.discordapp.com/attachments/995787558816595968/1133117633470922842/image.png)

![image](https://cdn.discordapp.com/attachments/995787558816595968/1133135949610627225/image.png)

![image](https://cdn.discordapp.com/attachments/995787558816595968/1133146823968706621/image.png)

![image](https://cdn.discordapp.com/attachments/995787558816595968/1133153504417042593/image.png)

I've also changed the select start menus to not have that giant space in
each option. It's behind a toggle in the code right now.

![image](https://cdn.discordapp.com/attachments/995787558816595968/1133146382006489138/image.png)

Additionally:
- fixed city billboard particles
- fix stadium crash from original game
- fix an accidental regression

* Fix small buzz motor not vibrating correctly (#2846)

* fix stad-samos regression from refactor (#2796) + change imgui toggle to left-alt and make it customizable (#2848)

* [jak2] More texture animations, and fixes (#2847)

* Fix possible crash when loading files (#2850)

There was a single static path buffer being shared between multiple file
i/o threads. So sometimes you would end up using the wrong path for the
file, and getting size/data for the wrong file.

I think the original reason to have this buffer was just me being lazy
when we changed how project paths works a long time ago. It was a bad
idea in the first place.

* [jak1] Fix ice walking animation bug and crash (#2851)

This should fix a crash and animation bug in snowy. The way to trigger
the bug:
- go on ice
- move forward slowly
- stop moving forward. Reach zero speed when the frame number isn't
between 30 and 35 of the ice walking animation
- Due to this bug, the animation gets stuck at frame 60
- Take damage (due to normal lurker or ice lurker)
- Sidekick eye animation crashes because a frame number is NaN.

* [jak2] graphic options menu proof of concept (#2849)

![imagem](https://github.com/open-goal/jak-project/assets/7569514/b8eaa0b6-7c51-4bd8-96ac-5fb2e0d08c4a)

![imagem](https://github.com/open-goal/jak-project/assets/7569514/d2742cef-98d4-43e4-a48d-cc1996e41612)

(The "Custom" option does not do anything right now.)

* `force-actors?` -> `ps2-actor-vis?` (#2852)

supersedes #2839

* [jak2] workaround for missing yakow textures (#2854)

The yakow texture is actually missing from the real game too. 

This adds a special case in the extractor to replace it with a
similar-ish fur texture:

![image](https://github.com/open-goal/jak-project/assets/48171810/db429e70-e5c5-4302-824a-238e94cf3d69)

* [jak2] fix transparent shrub blending (#2853)

![image](https://github.com/open-goal/jak-project/assets/48171810/ae4d40bf-b061-45af-ba20-c546fc935f11)

Should probably wait until the release just in case. But I'm 99% sure
this line was just a mistake in the original implementation.

* [jak2] Fix ocean culling when using camera-other (#2858)

When drawing the spinning palace, there's a terrible hack that lets some
stuff be drawn with different camera matrices.

The ocean is drawn with camera-other (which spins), but was being culled
with camera (doesn't spin). This changes ocean to use the right camera
matrix.

Note that the ocean is special when it comes to camera-other - it always
uses camera-other if there is one. Other uses are set per-level and
should already be handled correctly.

* [jak2] fix compile error (#2862)

not sure how this passed tests....

* [jak2] use current buffer for blit-displays (#2855)

This changes how `BlitDisplays.cpp` works so it looks at the current
render buffer, rather than the back buffer.

This approach is a bit faster because we avoid copying the back buffer
on every single frame.
It also removes the black frames when the transition starts/stops.

The remaining issues are:
- there's still a single frame of weirdness with the sprite glow
renderer.
- when changing resolutions, it doesn't work super well.

* [jak2] disable envmap when it should be (#2864)

In jak 2, there's an option to disable envmap. It's used on krew
holograms, hiphog trophies, and baron bosses in palace/tomb.

* [gfx] Clean up background renderer matrices, fix "hole covers" (#2866)

The way we got/stored background matrices is a bit weird and full of
leftovers from the first attempts at porting renderers. This doesn't
work well with the Jak 2 "other camera" system where some stuff is
rendered with a different camera matrix.

This cleans most of it up. The exception is that the collide mesh
renderer and the additional sprite culling I added still need to peek at
some cached camera matrices.

This fixes the problem where etie uses the wrong matrices for "other
camera" levels. Now the "hole covers" go in the holes in the background
of the throne room.

![image](https://github.com/open-goal/jak-project/assets/48171810/73a88f7b-05d4-4e9c-bb34-5b45efffcb69)

* ci: strip binaries in their respective runners (#2868)

* New Crowdin updates (#2869)

* fix texture anim shader on macOS (#2870)

* fix collision renderer filters not working (#2872)

fixes #2867

* [goalc] Sign extend objects when loading them (#2863)

Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>

* Anim Tester X - Quality of Life improvements (#2682)

Co-authored-by: Tyler Wilding <xTVaser@users.noreply.github.com>
Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>

* Basic hash table and jump table (#1837)

Co-authored-by: Tyler Wilding <xtvaser@gmail.com>

* github: update issue templates, move general feature requests to discussions preferably (#2879)

* jak1: start a new game correctly when speedrunner mode is enabled (#2873)

* formatter: support formatting bindings, for example in a `let` (#2883)

* [jak2] get new cheats working (#2885)

* [jak2] progress: make custom aspect ratio menu (#2888)

New menu for setting a custom aspect ratio.


![image](https://github.com/open-goal/jak-project/assets/7569514/33f81bc4-0219-4aba-b4ed-81394e4e4ceb)

![image](https://github.com/open-goal/jak-project/assets/7569514/9ddc3059-3ddf-4b35-afa5-11b9005e087c)

![image](https://github.com/open-goal/jak-project/assets/7569514/f3a16342-b38d-4082-ac18-1f82db349191)

![image](https://github.com/open-goal/jak-project/assets/7569514/69963cfe-2963-44f8-bde9-c0205c467d6c)

* CI: Periodic Controller Database Update (#2889)

Updating Controller Database

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* [jak2] prepare `scf-get-territory` usage (#2890)

* log: rotate log files with timestamps and add flag to disable ANSI colors (#2886)

Rotates the log files with a timestamp instead of copying all files and
incrementing an integer. Increases the amount of info you have when
looking at user's log files (ie. when looking at all the files, the file
creation dates are accurate).


![image](https://github.com/open-goal/jak-project/assets/13153231/61bcdf51-f0f6-4eee-b1e5-140aede5d19e)

Also simplifies the API for setting the log file, and `gk` logs are now
game specific with `jak1` or `jak2`. Which should be useful going
forward.

Lastly, added a flag to all CLIs to disable ansi colors for people that
want to do so. Though at the same time, there is finally a workaround in
jenkins to fix ANSI colors in the truncated log view -- so I'm not sure
why anyone would want to get rid of the color information. You can even
setup text editors to display the color info making log parsing much
easier. Fixes #1917

---------

Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>

* custom levels: fix crash when more than one ambient is present (#2891)

* log: don't recursively iterate when rotating log files (#2892)

* cleanup our cmake and build warnings (#2876)

* cmake: fix regression for building SDL statically (#2894)

* input: fix double press issue when assigning a bind (#2895)

* Update test-zone.jsonc to use lowercase for nickname (#2896)

* [jak1] Fix texture-related custom level crash (#2898)

Force FR3 file name for custom levels to be uppercase, fixes the crash
described in
open-goal/jak-project#2534
and 
open-goal/jak-project#2897

* [shrub] Fix bug with gs-prim settings (#2899)

Fix the bug described in
open-goal/jak-project#2882 where some shrubs
are transparent when they shouldn't be. The problem was that we never
carefully looked at the settings in `gs-prim`, which has a bit to
enable/disable alpha blending entirely. Now it should be correct for
both jak 1 and jak 2. To see this change, you'll need to re-extract.

Also adds a setting to disable saving texture .pngs, to speed up
decompilation. I left it on for jak 1 (to avoid confusion for texture
swapping(, but off for jak 2 for now.

* [glow] fix clipping bug (#2902)

Now you can be blinded by the sun:

![image](https://github.com/open-goal/jak-project/assets/48171810/caf3903c-8333-4daa-89ba-ff83ca7d6b2f)

* log: ensure the `log/` directory exists (#2904)

* 989snd: implement SetVolPan for midi sounds (#2905)

Fixes #2820

* [jak 2] bigmap, fix texture filtering on map icons (#2906)

![image](https://github.com/open-goal/jak-project/assets/48171810/4c285f31-c874-424a-8510-d181ba0f88d5)


![image](https://github.com/open-goal/jak-project/assets/48171810/6baa29ef-df92-435e-ad2f-9a42d56e6f17)

and the minimap has filtering now:

![image](https://github.com/open-goal/jak-project/assets/48171810/7cb2b0c9-5c86-426e-b028-dddcc3d649f7)

It's mostly implemented in C++ using the texture animator.

* [jak2] fix bad blerc on objects with warp (#2907)

fixes the weird mirror offset stuff

![image](https://github.com/open-goal/jak-project/assets/48171810/9a36c0bd-7350-46d7-b14f-d2f876582b03)

* [jak2] music player cheat menu (#2900)

* [jak2] small minor fixes (#2909)

- fix deci2 hang when closing the game in retail mode.
- change bigmap to always filter because the pixels look really ugly.
- don't start the game in fullscreen by default if we're debugging.

* [jak2] fix bug in direct in texture handler (#2910)

* handle texture of 0 in extract_tie (#2911)

Fixes texture difference shown in issue
open-goal/jak-project#2908

* [jak2] fix glow sprite flickering (#2912)

oops, forgot to set `glDepthMask`, so sometimes it was enabled, and the
drawing for the clear was writing into the depth buffer.
Fixes open-goal/jak-project#2735

* Fix alpha test > vs >= (minimap arrow fix) (#2913)

Fixes open-goal/jak-project#2814

Previously we didn't handle `GREATER` vs `GEQUAL` correctly. I just left
this as a TODO by accident.


![image](https://github.com/open-goal/jak-project/assets/48171810/bfa7bd98-e40e-4850-af9c-ed6954c00958)

* [jak2] Support Japanese Subtitles (#2914)

![image](https://github.com/open-goal/jak-project/assets/48171810/7b5edd00-4193-47ab-92f6-8f270edbcafe)

* [jak2] Support `draw-raw-image` (#2915)

![image](https://github.com/open-goal/jak-project/assets/48171810/9a94ea6e-99b8-4589-8210-86e63df31dd6)


![image](https://github.com/open-goal/jak-project/assets/48171810/e441225b-214c-46b9-ad01-0160590d30ba)

* [jak2] widescreen fix for splash/jp subtitles (#2916)

* [jak2] fix a few silly texture bugs (#2917)

Fix an issue where the commit sha would not use the right blending mode
if `draw-raw-image` is running at the same time.
Fix an issue where japanese subtitles would accidentally overwrite other
textures, leading to random textures missing. (in particular, glows
would disappear after watching a cutscene with the subtitles on)

* jak1: Consolidate art-elts into a single file as is done in jak 2 (#2887)

Fixes #2167

Reduces test flakiness if ran on multiple threads and gets rid of a few
hundred files from the source tree

I believe this also makes #1434 irrelevant and it can be closed.

---------

Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>

* [jak 2] Clouds V2, Clean up texture animator (#2921)

Some general improvements for the texture animator:
- Clouds are special cased, saving about 1 ms per frame
- Adjusting the amount of clouds now actually works.
- Fixed an issue with the brightness of clouds, and the way that they
fade out around the edges.

* [jak2] Toxic Slime texture animations (#2922)

https://github.com/open-goal/jak-project/assets/48171810/85e68808-09bf-4f74-a16a-6310e2b81cd3

* [jak2] Fix depth testing issues on glow sprites (#2923)

https://github.com/open-goal/jak-project/assets/48171810/0d3064f9-3f27-4be1-b6a4-55fdfb681313

Fixes open-goal/jak-project#2328

* [jak2] a few more subtitles + L1/R1 to move *much* faster through debug menu (#2925)

* [jak2] fill in a lot of flags for decomp + mouse macros (#2927)

Also changed the default type of enums to `int64` (same as `int`).

* [jak2] minimap improvements (#2929)

Smoothens minimap icon motion and fixes the target icon being squished
depending on the angle it's facing.

* g/j1: add support for missing polish alphabet characters (#2931)

* ckernel: fix file paths for art groups and tpages (#2932)

* custom levels: add support for packing models and textures into the FR3 file (#2936)

* [jak2] fix bugs with incorrect music playing sometimes (#2938)

- fixes flava and mode not being correct after a new music starts
playback by setting them every frame (there is at most 1 frame where
it's wrong).
- fixes city pursuit theme not playing if the city alarm was triggered
while the music wasn't playing yet.

* various subtitle editor fixes (#2941)

- fix speaker names and time frames being uneditable in Jak 1
- added toggle to auto-selected a newly created scene as current
- changed the subtitle summary format slightly.
- current scene's name now appears in the UI

* ci: macOS build with big sur to increase compatibility (#2944)

* New Crowdin updates (#2945)

* Placeholder strings for future Jak 2 features + minor subtitle editor fixes (#2943)

* [extractor] fix territory being set to wrong value (#2946)

Also fixes a minor issue where the JP sound bank wouldn't work (Jak 1).

Fixes #2793

* [sprite3] fix max sprite number being wrong (#2949)

I don't think this fixes anything, but the number was wrong so might as
well fix it.

* [jak2] fix bug in `outro-port` script (#2952)

* [jak2] higher resolution cloud textures (#2951)

Adds a new "hires" sky texture animation that makes the sky use 512x512
textures instead of 128x128. The clouds have higher detail, are slightly
more noisy, and are considerably less blocky. There is also a less
noticeable "dot crawl" effect.


![image](https://cdn.discordapp.com/attachments/995787558816595968/1146960820320878643/image.png)

![image](https://cdn.discordapp.com/attachments/995787558816595968/1146960819750449264/image.png)

* [sprite] Adjust allocation randomization (#2953)

This tries to match the original behavior of the sprite allocation
"randomness", which should reduce the number of very empty sprite blocks
sent to C++.

* Make  cloud_lookup return float (#2958)

* rewrite `sp-get-particle` and fix some jak 2 region decomp (#2955)

* goalc: use iso_data `build_info` to inform custom level build process (#2959)

* [gltf export] Export TIE wind models (#2960)

Export models with "wind".  The levels with wind models are:
firecanyon (9), beach (5), village1 (7), lavatube (2).

Sometimes a single object is made up of multiple models - for example
the tree in sandover is actually several meshes.

* CI: Periodic Controller Database Update (#2926)

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* entity-debug: Show path and volume data. (#2942)

* Update subtitle_en-US.json (#2763)

Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>
Co-authored-by: Tyler Wilding <xtvaser@gmail.com>

* g/j2: Some more work on the SQL editor - dump and seed `light` data (#2954)

* decomp: handle dumping jak 2 VAG file assets (#2969)

There are potentially still some minor issues with the resulting files.
Some of them appear to have minor artifacts that playing through the
actual game do not -- but this is a much better starting point for
someone to iterate from if they are interested in improving things.

* jak2: add frame rate option to graphics menu (#2962)

This PR adds a frame rate option to the graphics menu for some of the
most common refresh rates.

Jak 2 has much better support for variable frame rates than Jak 1 out of
the box, but there are still some edge cases, most prominently the fact
that sprites are still limited to the 300 tick system, which is most
noticeable on glow sprites. For this, I abused the glow boost debug
setting to scale the glow based on the frame rate.

While testing, I noticed two other cases that I have also patched,
there's likely to be many more that are yet to be found, but aside from
that, the game is playable as normal.



https://github.com/open-goal/jak-project/assets/6624576/ad4db24f-cd27-4237-a155-0db7008160f3

* [jak2] Fix hover-formation decomp (#2971)

I think this is very likely to fix
open-goal/jak-project#2970

We had somebody report a stacktrace from the debugger, and it was
immediately after calling `gen-perms`.
I found that `gen-perms` writes past the end of a stack array during
this mission, and at the same time as the reported freezes.

I was unable to recreate the original freeze after making this change.

* [jak1] Fix duplicated tie extract (#2972)

Fix a bug where meshes are extracted too many times.

* g/j2: Start adding game options to jak 2's menu (#2968)

* Save entities to JSON, and make custom level building a little faster (#2973)

This will create a folder like `decompiler_out/jak1/entities` and save a
JSON file per level with all the actors.

Also, it should hopefully make custom level building a little faster.

* jak2: macro detection for `launch-particles` and `seconds-per-frame`, add `og:preserve-this` comments for manual patches (#2974)

This PR adds detection of the `launch-particles` and `seconds-per-frame`
macros to the decompiler, removing a lot of bloat and hiding many
process register uses.

I also added `og:preserve-this` comments to as many manual patches and
comments as I could, which will soon be used in conjunction with CI to
hopefully catch any regressions in future big decomp update PRs.

I have some concerns about the `launch-particles` macro (more details in
`sparticle-launcher.gc`) , but thus far, I have not seen anything break
yet.

---------

Co-authored-by: water <awaterford111445@gmail.com>

* g/j2: revert signature changes to `cam-setting-data`, fix camera regressions (#2975)

Co-authored-by: Hat Kid <6624576+Hat-Kid@users.noreply.github.com>

* [jak 2] Export the collision mesh (#2977)

![image](https://github.com/open-goal/jak-project/assets/48171810/266d014b-f6c5-440f-9c3e-ab7c5eafe771)

* g/j1: resolve memory leak around display/input dynamic strings from C++ (#2979)

* CI: Periodic Controller Database Update (#2980)

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* Polish translation for Jak 1 (#2961)

* CI: check for removed goal_src code that we want or need to perserve (#2987)

* [jak2] placeholder subtitle files (#2986)

fixes #2985

* [decompiler] decomp jak2 `static-attack-info` (#2992)

Fixes #2993

* [jak2] fix `task-arrow-spawn` decomp (#2995)

* gsrc: add more `og:preserve-this` comments (#2996)

* g/j2: Implement speedrunner mode in jak 2 (#2976)

* Update game_base_text_pl-PL.json (#2997)

* jak2: fix `metalkor` regression (#2999)

* goalc: add `macro-expand` form (#3000)

* ci: separate lint checks into a required and optional workflow (#3002)

* [jak2] ckernel: implement `loado` and `load_and_link` (#3005)

* decomp: fix some `enemy` decomp (#3008)

* [opengoal] make `none` a child of `object` (#3001)

Previously, `object` and `none` were both top-level types. This made
decompilation rather messy as they have no LCA and resulted in a lot of
variables coming out as type `none` which is very very wrong and
additionally there were plenty of casts to `object`. This changes it so
`none` becomes a child of `object` (it is still represented by
`NullType` which remains unusable in compilation).

This change makes `object` the sole top-level type, and the type that
can represent *any* GOAL object. I believe this matches the original
GOAL built-in type structure. A function that has a return type of
`object` can now return an integer or a `none` at the same time.
However, keep in mind that the return value of `(none)` is still
undefined, just as before. This also makes a cast to `object`
meaningless in 90% of the situations it showed up in (as every single
thing is already an `object`) and the decompiler will no longer emit
them. Casts to `none` are also reduced. Yay!

Additionally, state handlers also don't get the final `(none)` printed
out anymore. The return type of a state handler is completely
meaningless outside the event handler (which is return type `object`
anyway) so there are no limitations on what the last form needs to be. I
did this instead of making them return `object` to trick the decompiler
into not trying to output a variable to be used as a return value
(internally, in the decompiler they still have return type `none`, but
they have `object` elsewhere).

Fixes #1703 
Fixes #830 
Fixes #928

* [jak2] fully implement collide mesh renderer (#3010)

Fixes #2983

* CI: Periodic Controller Database Update (#3006)

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* improve decomp of state handlers and art groups (#3014)

- state handlers that are not inlined lambdas have smarter type
checking, getting rid of 99.9% of the casts emitted (they were not
useful)
- art groups were not being properly linked to their "master" groups.
- `max` in `ja` in Jak 2 was not being detected.

Another huge PR...

* g/j2: speed up level-select menu scrolling (#3013)

* [jak2] also allow analog for play select fast scroll (#3015)

* g/j2: Fill out `game options` menu (#3003)

* scripts: remove unused `code_retention` script code (#3019)

* New Crowdin updates (#3025)

* fix `(font-flags left)` name + fix `right` flag not reflecting in debug text box (#3020)

Fixes #3016 
Fixes #3017

* CI: Periodic Controller Database Update (#3028)

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* [compiler] fix `compile_condition` not attempting macro expansion (#3030)

Ancient bug! Resulted in a 1%-2% decrease in code object file size (and
more importantly the asm is more readable).

Fixes #3029

* decompiler: `obj` -> `this`, `set-time!` and `time-elapsed?` macros (#3026)

This renames the method object in `defmethod`s to `this` and adds
detection for the `set-time!` and `time-elapsed?` macros.

Definitely my biggest PR yet...

* [decompiler] do not use `time-elapsed?` on unsigned comps (#3031)

* misc low hanging fruit fixes (#3034)

Fixes #2235 
Fixes #2236 
Fixes #2963 
Fixes #3027

* Jak 2 controller LED implementation (#3035)

Adds controller LED features to Jak 2:
- progressive flickering denoting health
- copies tomb simon says puzzle colors
- unique colors for each gun
- orange color for being indax
- yellow color for being in mech
- purple color for being darkjak
- blue color for being in board
- red flash when wanted.

May add more features later?

Also did some minor clean-up on some types.

* [jak2] un-mips2c `draw-string` (#3040)

It's not an asm function.

* Remove dupe line in discord-rpc .clang-format (#3039)

* [decompiler] Handle `find-parent-method` (#3018)

This change adds a few new features:
- Decompiler automatically knows the type of `find-parent-method` use in
jak 1 and jak2 when used in a method or virtual state handler.
- Decompiler inserts a call to `call-parent-method` or
`find-parent-state`
- Removed most casts related to these functions

There are still a few minor issues around this:
- There are still some casts needed when using `post` methods, as `post`
is just a `function`, and needs a cast to `(function none)` or similar.
It didn't seem easy to change the type of `post`, so I'm not going to
worry about it for this PR. It only shows up in like 3 places in jak 2.
(and 0 in jak 1)
- If "call the handler if it's not #f" logic should probably be another
macro.

Fixes #805

* [jak2] scalable debug font + fix debug menu widescreen bugs  (#3044)

also hides empty (no commands) regions from the viewers by default now.

* fix draw-string-xy-scaled cropping (#3041)

fixes #3038 


https://github.com/open-goal/jak-project/assets/2515356/5b7a490f-e1fb-4a12-a3d9-b576a2c1d574

* [jak2] fully implement `*user*` (#3046)

Fixes #1918

* Bump actions/checkout from 3 to 4 (#3047)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* g/j2: don't force auto-save on in jak 2 when starting a speedrun (#3048)

* fix eco pickup time bug (#3056)

* CI: Periodic Controller Database Update (#3060)

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* decompiler: better automatic detection of art groups and `joint-node-index` macro detection (#3061)

* [jak2] Set up extractor (#3042)

This sets up the extractor for jak 2. I was expecting that I'd have to
make some more significant changes to the decompiler/compiler path
stuff, but this was not the case!

The only real change is that you can now provide multiple ISO hashes for
an entry in `ISOMetadata`. This is needed for the two different NTSC
versions, which have the same configs, serials, and ELF hashes, but
slightly different contents.

I also didn't add the korean version because I don't have the info for
it.

---------

Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>

* [goalc] Cleaned up speedups (#3066)

Started at 349,880,038 allocations and 42s

- Switched to making `Symbol` in GOOS be a "fixed type", just a wrapper
around a `const char*` pointing to the string in the symbol table. This
is a step toward making a lot of things better, but by itself not a huge
improvement. Some things may be worse due to more temp `std::string`
allocations, but one day all these can be removed. On linux it saved
allocations (347,685,429), and saved a second or two (41 s).
- cache `#t` and `#f` in interpreter, better lookup for special
forms/builtins (hashtable of pointers instead of strings, vector for the
small special form list). Dropped time to 38s.
- special-case in quasiquote when splicing is the last thing in a list.
Allocation dropped to 340,603,082
- custom hash table for environment lookups (lexical vars). Dropped to
36s and 314,637,194
- less allocation in `read_list` 311,613,616. Time about the same.
- `let` and `let*` in Interpreter.cpp 191,988,083, time down to 28s.

* [jak2] new glow fix (#3067)

Fix a typo in the shader. (fixes
open-goal/jak-project#3045, and fixes fadeout
when glows move offscreen on the bottom).

Also, try a different format for the depth blit, in an attempt to fix
this issue open-goal/jak-project#3065.

* decompiler: some hacks to allow running decompiler on jak 3 v5 code files, improve `all-types` generation (#2526)

Co-authored-by: water <awaterford111445@gmail.com>

* [glow] potentially fix bad glow texture (#3071)

Might fix #3065 

fixes #3070

* [jak2] fix minor original game visual bugs (#3072)

Fixes #2994 the underport and consite icons being green (same color as
the goal, likely copy-pasta).

Fixes the hundredths timer in the race timer actually being a 60ths
timer.

* github: add issue template for jak 2 (#3073)

* jak3: add `goal_src` skeleton (#3069)

* [decompiler] Support v5 data file link data (#3076)

Fix the implementation of `link_v5` so it works on "data" files for jak
3.

* decompiler: fix for v5 art group info dump, update `taskfile` for jak 3 (#3077)

* decompiler: allow jak 3 texture and model extraction (#3080)

Added some hacks and stubs to allow extracting textures and models.

* [jak2] Fix collision renderer extract (#3081)

Fixes missing collision geometry reported in
open-goal/jak-project#3011

The issue happens when there are 256 polygons. In this case `num-polys`
is 0 (it's a u8). There are actual cases where there are 0 polygons, so
we have to do a more complicated check to get the real count. I should
have done this in the first place, but it seemed to work...

* g/j2: Integrate highscores with Speedrun.com/JakSpeedruns.com when speedrunner mode is enabled (#3037)

* [jak1] Finnish base game strings (#3058)

Finnish translations for the remaining UI texts

---------

Co-authored-by: Tyler Wilding <xtvaser@gmail.com>

* jak3: add missing files and implement `pexcw` (#3084)

Some files were in the `banned_objects` list and were thus excluded from
the `all_objs` file.

Also implements the `pexcw` instruction which is only used in `hfrag`
code.

* decompiler: add texture merging feature (#3083)

* fix `*jak1-full-game*` being flipped (#3085)

* [glb export] Export bones. (#3087)

![image](https://github.com/open-goal/jak-project/assets/48171810/0f26e77b-af68-4450-882e-762a501bdef4)

* custom levels: support for packing textures (#3089)

* Fix build failure at llvm-rc on windows due to codepage error on zydis (#3088)

This fix is for the following error when building with llvm on windows

```
[build] llvm-rc: Error in VERSIONINFO statement (ID 1): 
[proc] The command: "C:\Program Files\CMake\bin\cmake.exe" --build C:/.../git/jak-project-test/out/build/Release --parallel 8 exited with code: 1
[build] Non-ASCII 8-bit codepoint (´┐¢) can't be interpreted in the current codepage
[build] ninja: build stopped: subcommand failed.
```

It's caused by llvm-rc not handling non 8-bit characters.
The same issue is discussed in the curl project here
curl/curl#7765

This commit copies the fix from curl, replacing '©' with '(C)'.

* jak2: add warning text when changing frame rate for the first time (#3092)

* CI: Periodic Controller Database Update (#3091)

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* Add label_types file merge for PAL version (#3086)

* ci: Workaround CMake/Perl regression in recent windows-2022 images (#3097)

* formatter: rewrite and refactor, address more edge-cases, begin documenting my work (#3096)

* custom levels: refactor level building code and jak 2 support (#3090)

Co-authored-by: water <awaterford111445@gmail.com>

* CI: Periodic Controller Database Update (#3101)

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* Get compiler working

* patches

* Fix custom level textures

---------

Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>
Co-authored-by: water111 <48171810+water111@users.noreply.github.com>
Co-authored-by: animalstyletaco <mcc-108@hotmail.com>
Co-authored-by: animalstyletaco <animalstyletaco95@gmail.com>
Co-authored-by: Tyler Wilding <xTVaser@users.noreply.github.com>
Co-authored-by: OpenGOAL Bot <99294829+OpenGOALBot@users.noreply.github.com>
Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>
Co-authored-by: Luminar Light <18116946+LuminarLight@users.noreply.github.com>
Co-authored-by: Brent Hickey <brent.hickey@icloud.com>
Co-authored-by: Tyler Wilding <xtvaser@gmail.com>
Co-authored-by: Hat Kid <6624576+Hat-Kid@users.noreply.github.com>
Co-authored-by: Matt Dallmeyer <mattdallmeyer@gmail.com>
Co-authored-by: Ziemas <ziemas@ziemas.se>
Co-authored-by: atylmo <48929878+atylmo@users.noreply.github.com>
Co-authored-by: water <awaterford111445@gmail.com>
Co-authored-by: rafalekkB <144032866+rafalekkB@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Aloqas <102683375+Aloqas@users.noreply.github.com>
Co-authored-by: jabermony <748988+jabermony@users.noreply.github.com>

* mod ver placeholder

* merge test 2

* skel effect hack

* block portal use if a chicken is walking to it

* Refactor collectables.gc

* refactor some stuff softlock prevention hack

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: ZedB0T <89345505+Zedb0T@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>
Co-authored-by: water111 <48171810+water111@users.noreply.github.com>
Co-authored-by: animalstyletaco <mcc-108@hotmail.com>
Co-authored-by: animalstyletaco <animalstyletaco95@gmail.com>
Co-authored-by: Tyler Wilding <xTVaser@users.noreply.github.com>
Co-authored-by: OpenGOAL Bot <99294829+OpenGOALBot@users.noreply.github.com>
Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>
Co-authored-by: Luminar Light <18116946+LuminarLight@users.noreply.github.com>
Co-authored-by: Brent Hickey <brent.hickey@icloud.com>
Co-authored-by: Tyler Wilding <xtvaser@gmail.com>
Co-authored-by: Hat Kid <6624576+Hat-Kid@users.noreply.github.com>
Co-authored-by: Ziemas <ziemas@ziemas.se>
Co-authored-by: atylmo <48929878+atylmo@users.noreply.github.com>
Co-authored-by: water <awaterford111445@gmail.com>
Co-authored-by: rafalekkB <144032866+rafalekkB@users.noreply.github.com>
Co-authored-by: Aloqas <102683375+Aloqas@users.noreply.github.com>
Co-authored-by: jabermony <748988+jabermony@users.noreply.github.com>
Zedb0T added a commit to OpenGOAL-Mods/OG-Mod-Base that referenced this issue Nov 4, 2023
* improve `send-event` for jak 2  (#2828)

Fixes #2825

* [jak2] Work-in-progress texture animations (#2819)

* [jak2] speed up the sky texture animation (#2829)

This saved about 1.6 ms per frame in the city for me (~1.3 saved from
not doing sky twice, 0.3 saved in format lookup tables).

The big texture animator is about 1.0 ms.

![image](https://github.com/open-goal/jak-project/assets/48171810/c7bc7743-308c-4425-ad14-118e2d483fad)

* Added bound check for blend vertex count (#2830)

Co-authored-by: animalstyletaco <animalstyletaco95@gmail.com>

* Get the project compiling on Apple Silicon macOS natively (arm64) (#2827)

I havn't tested it yet, but I can almost guarantee that atleast `goalc`
will not work in the slightest!

But the project is atleast fully compiling. My hope is to start
translating some AVX to NEON next / get `goalc` working...eventually.

* [jak2] More texture animations (#2831)

Added framework to do texture animations entirely in C++. Currently only
works on relatively simple ones, and doesn't handle updating all
parameters - only the speeds.

Connected texture animations to merc and tfrag for skull gems, dark
bomb, and scrolling conveyors.

Cleaned up Tfragment/Tfrag3, which used to be two classes. This was one
of the first C++ renderers, so it had a weird design.

* CI: Periodic Controller Database Update (#2832)

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* [jak2] Fix overlord related crash (#2834)

This fixes the crash reported in
open-goal/jak-project#2833

There was a memory bug here for a long time where our array of `VagCmd`
in `iso_queue.cpp` was too small. This caused GetVagCommand to return
bogus pointers, and sound code would write over other parts of memory.

* [jak2] More progress on texture animations (#2835)

- Add security wall animation
- Add waterfall animations
- Add lava animations
- Update layer values from the game to fix the security wall
- Remove leftover debug in `level.gc` that would break level-specific
animations on the second time you visited the level
- Optionally load animated slot textures to the pool so generic can use
them (fixes skull gems in UI)

* [jak2] Add static textures for the progress menu (#2838)

The progress menu loads its icon textures from a .STR file that we were
previously ignoring.

This change:
- updates the decompiler so it can process a .STR file containing a
texture
- adds a feature to force an entire page to always be loaded in the PC
renderer by putting all textures in the GAME.FR3 file.
- regenerates the texture offset map file for jak 2 with these new
textures

For now, I've just put the icon textures in GAME.FR3. The downside is
that these will always stay on the GPU, using up VRAM even when they
aren't needed. But the entire GAME.FR3 file is under 3 MB so I think
it's ok.


![image](https://github.com/open-goal/jak-project/assets/48171810/39f075b5-7cc5-4168-872a-33026342afab)

* game: add a clear text label for how to hide the top bar (#2845)

* [jak2] Fix dark jak anim and low res skull gems (#2842)

Fixes skull gems using a low resolution texture.

Fixes issue where jak in cutscenes is dark after watching oracle-level-1
(and likely other bugs with texture animations getting stuck)

---------

Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>

* [jak2] make progress menu work for widescreen + various other fixes (#2843)

Lots of manual fixes to make the text and other UI elements in the
progress fit within the small 4x3 view.


![image](https://cdn.discordapp.com/attachments/995787558816595968/1132986451664056423/image.png)

![image](https://cdn.discordapp.com/attachments/995787558816595968/1133064077460131840/image.png)

![image](https://cdn.discordapp.com/attachments/995787558816595968/1133074288564510780/image.png)

![image](https://cdn.discordapp.com/attachments/995787558816595968/1133117633898762280/image.png)

![image](https://cdn.discordapp.com/attachments/995787558816595968/1133117633470922842/image.png)

![image](https://cdn.discordapp.com/attachments/995787558816595968/1133135949610627225/image.png)

![image](https://cdn.discordapp.com/attachments/995787558816595968/1133146823968706621/image.png)

![image](https://cdn.discordapp.com/attachments/995787558816595968/1133153504417042593/image.png)

I've also changed the select start menus to not have that giant space in
each option. It's behind a toggle in the code right now.

![image](https://cdn.discordapp.com/attachments/995787558816595968/1133146382006489138/image.png)

Additionally:
- fixed city billboard particles
- fix stadium crash from original game
- fix an accidental regression

* Fix small buzz motor not vibrating correctly (#2846)

* fix stad-samos regression from refactor (#2796) + change imgui toggle to left-alt and make it customizable (#2848)

* [jak2] More texture animations, and fixes (#2847)

* Fix possible crash when loading files (#2850)

There was a single static path buffer being shared between multiple file
i/o threads. So sometimes you would end up using the wrong path for the
file, and getting size/data for the wrong file.

I think the original reason to have this buffer was just me being lazy
when we changed how project paths works a long time ago. It was a bad
idea in the first place.

* [jak1] Fix ice walking animation bug and crash (#2851)

This should fix a crash and animation bug in snowy. The way to trigger
the bug:
- go on ice
- move forward slowly
- stop moving forward. Reach zero speed when the frame number isn't
between 30 and 35 of the ice walking animation
- Due to this bug, the animation gets stuck at frame 60
- Take damage (due to normal lurker or ice lurker)
- Sidekick eye animation crashes because a frame number is NaN.

* [jak2] graphic options menu proof of concept (#2849)

![imagem](https://github.com/open-goal/jak-project/assets/7569514/b8eaa0b6-7c51-4bd8-96ac-5fb2e0d08c4a)

![imagem](https://github.com/open-goal/jak-project/assets/7569514/d2742cef-98d4-43e4-a48d-cc1996e41612)

(The "Custom" option does not do anything right now.)

* `force-actors?` -> `ps2-actor-vis?` (#2852)

supersedes #2839

* [jak2] workaround for missing yakow textures (#2854)

The yakow texture is actually missing from the real game too. 

This adds a special case in the extractor to replace it with a
similar-ish fur texture:

![image](https://github.com/open-goal/jak-project/assets/48171810/db429e70-e5c5-4302-824a-238e94cf3d69)

* [jak2] fix transparent shrub blending (#2853)

![image](https://github.com/open-goal/jak-project/assets/48171810/ae4d40bf-b061-45af-ba20-c546fc935f11)

Should probably wait until the release just in case. But I'm 99% sure
this line was just a mistake in the original implementation.

* [jak2] Fix ocean culling when using camera-other (#2858)

When drawing the spinning palace, there's a terrible hack that lets some
stuff be drawn with different camera matrices.

The ocean is drawn with camera-other (which spins), but was being culled
with camera (doesn't spin). This changes ocean to use the right camera
matrix.

Note that the ocean is special when it comes to camera-other - it always
uses camera-other if there is one. Other uses are set per-level and
should already be handled correctly.

* [jak2] fix compile error (#2862)

not sure how this passed tests....

* [jak2] use current buffer for blit-displays (#2855)

This changes how `BlitDisplays.cpp` works so it looks at the current
render buffer, rather than the back buffer.

This approach is a bit faster because we avoid copying the back buffer
on every single frame.
It also removes the black frames when the transition starts/stops.

The remaining issues are:
- there's still a single frame of weirdness with the sprite glow
renderer.
- when changing resolutions, it doesn't work super well.

* [jak2] disable envmap when it should be (#2864)

In jak 2, there's an option to disable envmap. It's used on krew
holograms, hiphog trophies, and baron bosses in palace/tomb.

* [gfx] Clean up background renderer matrices, fix "hole covers" (#2866)

The way we got/stored background matrices is a bit weird and full of
leftovers from the first attempts at porting renderers. This doesn't
work well with the Jak 2 "other camera" system where some stuff is
rendered with a different camera matrix.

This cleans most of it up. The exception is that the collide mesh
renderer and the additional sprite culling I added still need to peek at
some cached camera matrices.

This fixes the problem where etie uses the wrong matrices for "other
camera" levels. Now the "hole covers" go in the holes in the background
of the throne room.

![image](https://github.com/open-goal/jak-project/assets/48171810/73a88f7b-05d4-4e9c-bb34-5b45efffcb69)

* ci: strip binaries in their respective runners (#2868)

* New Crowdin updates (#2869)

* fix texture anim shader on macOS (#2870)

* fix collision renderer filters not working (#2872)

fixes #2867

* [goalc] Sign extend objects when loading them (#2863)

Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>

* Anim Tester X - Quality of Life improvements (#2682)

Co-authored-by: Tyler Wilding <xTVaser@users.noreply.github.com>
Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>

* Basic hash table and jump table (#1837)

Co-authored-by: Tyler Wilding <xtvaser@gmail.com>

* github: update issue templates, move general feature requests to discussions preferably (#2879)

* jak1: start a new game correctly when speedrunner mode is enabled (#2873)

* formatter: support formatting bindings, for example in a `let` (#2883)

* [jak2] get new cheats working (#2885)

* [jak2] progress: make custom aspect ratio menu (#2888)

New menu for setting a custom aspect ratio.


![image](https://github.com/open-goal/jak-project/assets/7569514/33f81bc4-0219-4aba-b4ed-81394e4e4ceb)

![image](https://github.com/open-goal/jak-project/assets/7569514/9ddc3059-3ddf-4b35-afa5-11b9005e087c)

![image](https://github.com/open-goal/jak-project/assets/7569514/f3a16342-b38d-4082-ac18-1f82db349191)

![image](https://github.com/open-goal/jak-project/assets/7569514/69963cfe-2963-44f8-bde9-c0205c467d6c)

* CI: Periodic Controller Database Update (#2889)

Updating Controller Database

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* [jak2] prepare `scf-get-territory` usage (#2890)

* log: rotate log files with timestamps and add flag to disable ANSI colors (#2886)

Rotates the log files with a timestamp instead of copying all files and
incrementing an integer. Increases the amount of info you have when
looking at user's log files (ie. when looking at all the files, the file
creation dates are accurate).


![image](https://github.com/open-goal/jak-project/assets/13153231/61bcdf51-f0f6-4eee-b1e5-140aede5d19e)

Also simplifies the API for setting the log file, and `gk` logs are now
game specific with `jak1` or `jak2`. Which should be useful going
forward.

Lastly, added a flag to all CLIs to disable ansi colors for people that
want to do so. Though at the same time, there is finally a workaround in
jenkins to fix ANSI colors in the truncated log view -- so I'm not sure
why anyone would want to get rid of the color information. You can even
setup text editors to display the color info making log parsing much
easier. Fixes #1917

---------

Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>

* custom levels: fix crash when more than one ambient is present (#2891)

* log: don't recursively iterate when rotating log files (#2892)

* cleanup our cmake and build warnings (#2876)

* cmake: fix regression for building SDL statically (#2894)

* input: fix double press issue when assigning a bind (#2895)

* Update test-zone.jsonc to use lowercase for nickname (#2896)

* [jak1] Fix texture-related custom level crash (#2898)

Force FR3 file name for custom levels to be uppercase, fixes the crash
described in
open-goal/jak-project#2534
and 
open-goal/jak-project#2897

* [shrub] Fix bug with gs-prim settings (#2899)

Fix the bug described in
open-goal/jak-project#2882 where some shrubs
are transparent when they shouldn't be. The problem was that we never
carefully looked at the settings in `gs-prim`, which has a bit to
enable/disable alpha blending entirely. Now it should be correct for
both jak 1 and jak 2. To see this change, you'll need to re-extract.

Also adds a setting to disable saving texture .pngs, to speed up
decompilation. I left it on for jak 1 (to avoid confusion for texture
swapping(, but off for jak 2 for now.

* [glow] fix clipping bug (#2902)

Now you can be blinded by the sun:

![image](https://github.com/open-goal/jak-project/assets/48171810/caf3903c-8333-4daa-89ba-ff83ca7d6b2f)

* log: ensure the `log/` directory exists (#2904)

* 989snd: implement SetVolPan for midi sounds (#2905)

Fixes #2820

* [jak 2] bigmap, fix texture filtering on map icons (#2906)

![image](https://github.com/open-goal/jak-project/assets/48171810/4c285f31-c874-424a-8510-d181ba0f88d5)


![image](https://github.com/open-goal/jak-project/assets/48171810/6baa29ef-df92-435e-ad2f-9a42d56e6f17)

and the minimap has filtering now:

![image](https://github.com/open-goal/jak-project/assets/48171810/7cb2b0c9-5c86-426e-b028-dddcc3d649f7)

It's mostly implemented in C++ using the texture animator.

* [jak2] fix bad blerc on objects with warp (#2907)

fixes the weird mirror offset stuff

![image](https://github.com/open-goal/jak-project/assets/48171810/9a36c0bd-7350-46d7-b14f-d2f876582b03)

* [jak2] music player cheat menu (#2900)

* [jak2] small minor fixes (#2909)

- fix deci2 hang when closing the game in retail mode.
- change bigmap to always filter because the pixels look really ugly.
- don't start the game in fullscreen by default if we're debugging.

* [jak2] fix bug in direct in texture handler (#2910)

* handle texture of 0 in extract_tie (#2911)

Fixes texture difference shown in issue
open-goal/jak-project#2908

* [jak2] fix glow sprite flickering (#2912)

oops, forgot to set `glDepthMask`, so sometimes it was enabled, and the
drawing for the clear was writing into the depth buffer.
Fixes open-goal/jak-project#2735

* Fix alpha test > vs >= (minimap arrow fix) (#2913)

Fixes open-goal/jak-project#2814

Previously we didn't handle `GREATER` vs `GEQUAL` correctly. I just left
this as a TODO by accident.


![image](https://github.com/open-goal/jak-project/assets/48171810/bfa7bd98-e40e-4850-af9c-ed6954c00958)

* [jak2] Support Japanese Subtitles (#2914)

![image](https://github.com/open-goal/jak-project/assets/48171810/7b5edd00-4193-47ab-92f6-8f270edbcafe)

* [jak2] Support `draw-raw-image` (#2915)

![image](https://github.com/open-goal/jak-project/assets/48171810/9a94ea6e-99b8-4589-8210-86e63df31dd6)


![image](https://github.com/open-goal/jak-project/assets/48171810/e441225b-214c-46b9-ad01-0160590d30ba)

* [jak2] widescreen fix for splash/jp subtitles (#2916)

* [jak2] fix a few silly texture bugs (#2917)

Fix an issue where the commit sha would not use the right blending mode
if `draw-raw-image` is running at the same time.
Fix an issue where japanese subtitles would accidentally overwrite other
textures, leading to random textures missing. (in particular, glows
would disappear after watching a cutscene with the subtitles on)

* jak1: Consolidate art-elts into a single file as is done in jak 2 (#2887)

Fixes #2167

Reduces test flakiness if ran on multiple threads and gets rid of a few
hundred files from the source tree

I believe this also makes #1434 irrelevant and it can be closed.

---------

Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>

* [jak 2] Clouds V2, Clean up texture animator (#2921)

Some general improvements for the texture animator:
- Clouds are special cased, saving about 1 ms per frame
- Adjusting the amount of clouds now actually works.
- Fixed an issue with the brightness of clouds, and the way that they
fade out around the edges.

* [jak2] Toxic Slime texture animations (#2922)

https://github.com/open-goal/jak-project/assets/48171810/85e68808-09bf-4f74-a16a-6310e2b81cd3

* [jak2] Fix depth testing issues on glow sprites (#2923)

https://github.com/open-goal/jak-project/assets/48171810/0d3064f9-3f27-4be1-b6a4-55fdfb681313

Fixes open-goal/jak-project#2328

* [jak2] a few more subtitles + L1/R1 to move *much* faster through debug menu (#2925)

* [jak2] fill in a lot of flags for decomp + mouse macros (#2927)

Also changed the default type of enums to `int64` (same as `int`).

* [jak2] minimap improvements (#2929)

Smoothens minimap icon motion and fixes the target icon being squished
depending on the angle it's facing.

* g/j1: add support for missing polish alphabet characters (#2931)

* ckernel: fix file paths for art groups and tpages (#2932)

* custom levels: add support for packing models and textures into the FR3 file (#2936)

* [jak2] fix bugs with incorrect music playing sometimes (#2938)

- fixes flava and mode not being correct after a new music starts
playback by setting them every frame (there is at most 1 frame where
it's wrong).
- fixes city pursuit theme not playing if the city alarm was triggered
while the music wasn't playing yet.

* various subtitle editor fixes (#2941)

- fix speaker names and time frames being uneditable in Jak 1
- added toggle to auto-selected a newly created scene as current
- changed the subtitle summary format slightly.
- current scene's name now appears in the UI

* ci: macOS build with big sur to increase compatibility (#2944)

* New Crowdin updates (#2945)

* Placeholder strings for future Jak 2 features + minor subtitle editor fixes (#2943)

* [extractor] fix territory being set to wrong value (#2946)

Also fixes a minor issue where the JP sound bank wouldn't work (Jak 1).

Fixes #2793

* [sprite3] fix max sprite number being wrong (#2949)

I don't think this fixes anything, but the number was wrong so might as
well fix it.

* [jak2] fix bug in `outro-port` script (#2952)

* [jak2] higher resolution cloud textures (#2951)

Adds a new "hires" sky texture animation that makes the sky use 512x512
textures instead of 128x128. The clouds have higher detail, are slightly
more noisy, and are considerably less blocky. There is also a less
noticeable "dot crawl" effect.


![image](https://cdn.discordapp.com/attachments/995787558816595968/1146960820320878643/image.png)

![image](https://cdn.discordapp.com/attachments/995787558816595968/1146960819750449264/image.png)

* [sprite] Adjust allocation randomization (#2953)

This tries to match the original behavior of the sprite allocation
"randomness", which should reduce the number of very empty sprite blocks
sent to C++.

* Make  cloud_lookup return float (#2958)

* rewrite `sp-get-particle` and fix some jak 2 region decomp (#2955)

* goalc: use iso_data `build_info` to inform custom level build process (#2959)

* [gltf export] Export TIE wind models (#2960)

Export models with "wind".  The levels with wind models are:
firecanyon (9), beach (5), village1 (7), lavatube (2).

Sometimes a single object is made up of multiple models - for example
the tree in sandover is actually several meshes.

* CI: Periodic Controller Database Update (#2926)

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* entity-debug: Show path and volume data. (#2942)

* Update subtitle_en-US.json (#2763)

Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>
Co-authored-by: Tyler Wilding <xtvaser@gmail.com>

* g/j2: Some more work on the SQL editor - dump and seed `light` data (#2954)

* decomp: handle dumping jak 2 VAG file assets (#2969)

There are potentially still some minor issues with the resulting files.
Some of them appear to have minor artifacts that playing through the
actual game do not -- but this is a much better starting point for
someone to iterate from if they are interested in improving things.

* jak2: add frame rate option to graphics menu (#2962)

This PR adds a frame rate option to the graphics menu for some of the
most common refresh rates.

Jak 2 has much better support for variable frame rates than Jak 1 out of
the box, but there are still some edge cases, most prominently the fact
that sprites are still limited to the 300 tick system, which is most
noticeable on glow sprites. For this, I abused the glow boost debug
setting to scale the glow based on the frame rate.

While testing, I noticed two other cases that I have also patched,
there's likely to be many more that are yet to be found, but aside from
that, the game is playable as normal.



https://github.com/open-goal/jak-project/assets/6624576/ad4db24f-cd27-4237-a155-0db7008160f3

* [jak2] Fix hover-formation decomp (#2971)

I think this is very likely to fix
open-goal/jak-project#2970

We had somebody report a stacktrace from the debugger, and it was
immediately after calling `gen-perms`.
I found that `gen-perms` writes past the end of a stack array during
this mission, and at the same time as the reported freezes.

I was unable to recreate the original freeze after making this change.

* [jak1] Fix duplicated tie extract (#2972)

Fix a bug where meshes are extracted too many times.

* g/j2: Start adding game options to jak 2's menu (#2968)

* Save entities to JSON, and make custom level building a little faster (#2973)

This will create a folder like `decompiler_out/jak1/entities` and save a
JSON file per level with all the actors.

Also, it should hopefully make custom level building a little faster.

* jak2: macro detection for `launch-particles` and `seconds-per-frame`, add `og:preserve-this` comments for manual patches (#2974)

This PR adds detection of the `launch-particles` and `seconds-per-frame`
macros to the decompiler, removing a lot of bloat and hiding many
process register uses.

I also added `og:preserve-this` comments to as many manual patches and
comments as I could, which will soon be used in conjunction with CI to
hopefully catch any regressions in future big decomp update PRs.

I have some concerns about the `launch-particles` macro (more details in
`sparticle-launcher.gc`) , but thus far, I have not seen anything break
yet.

---------

Co-authored-by: water <awaterford111445@gmail.com>

* g/j2: revert signature changes to `cam-setting-data`, fix camera regressions (#2975)

Co-authored-by: Hat Kid <6624576+Hat-Kid@users.noreply.github.com>

* [jak 2] Export the collision mesh (#2977)

![image](https://github.com/open-goal/jak-project/assets/48171810/266d014b-f6c5-440f-9c3e-ab7c5eafe771)

* g/j1: resolve memory leak around display/input dynamic strings from C++ (#2979)

* CI: Periodic Controller Database Update (#2980)

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* Polish translation for Jak 1 (#2961)

* CI: check for removed goal_src code that we want or need to perserve (#2987)

* [jak2] placeholder subtitle files (#2986)

fixes #2985

* [decompiler] decomp jak2 `static-attack-info` (#2992)

Fixes #2993

* [jak2] fix `task-arrow-spawn` decomp (#2995)

* gsrc: add more `og:preserve-this` comments (#2996)

* g/j2: Implement speedrunner mode in jak 2 (#2976)

* Update game_base_text_pl-PL.json (#2997)

* jak2: fix `metalkor` regression (#2999)

* goalc: add `macro-expand` form (#3000)

* ci: separate lint checks into a required and optional workflow (#3002)

* [jak2] ckernel: implement `loado` and `load_and_link` (#3005)

* decomp: fix some `enemy` decomp (#3008)

* [opengoal] make `none` a child of `object` (#3001)

Previously, `object` and `none` were both top-level types. This made
decompilation rather messy as they have no LCA and resulted in a lot of
variables coming out as type `none` which is very very wrong and
additionally there were plenty of casts to `object`. This changes it so
`none` becomes a child of `object` (it is still represented by
`NullType` which remains unusable in compilation).

This change makes `object` the sole top-level type, and the type that
can represent *any* GOAL object. I believe this matches the original
GOAL built-in type structure. A function that has a return type of
`object` can now return an integer or a `none` at the same time.
However, keep in mind that the return value of `(none)` is still
undefined, just as before. This also makes a cast to `object`
meaningless in 90% of the situations it showed up in (as every single
thing is already an `object`) and the decompiler will no longer emit
them. Casts to `none` are also reduced. Yay!

Additionally, state handlers also don't get the final `(none)` printed
out anymore. The return type of a state handler is completely
meaningless outside the event handler (which is return type `object`
anyway) so there are no limitations on what the last form needs to be. I
did this instead of making them return `object` to trick the decompiler
into not trying to output a variable to be used as a return value
(internally, in the decompiler they still have return type `none`, but
they have `object` elsewhere).

Fixes #1703 
Fixes #830 
Fixes #928

* [jak2] fully implement collide mesh renderer (#3010)

Fixes #2983

* CI: Periodic Controller Database Update (#3006)

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* improve decomp of state handlers and art groups (#3014)

- state handlers that are not inlined lambdas have smarter type
checking, getting rid of 99.9% of the casts emitted (they were not
useful)
- art groups were not being properly linked to their "master" groups.
- `max` in `ja` in Jak 2 was not being detected.

Another huge PR...

* g/j2: speed up level-select menu scrolling (#3013)

* [jak2] also allow analog for play select fast scroll (#3015)

* g/j2: Fill out `game options` menu (#3003)

* scripts: remove unused `code_retention` script code (#3019)

* New Crowdin updates (#3025)

* fix `(font-flags left)` name + fix `right` flag not reflecting in debug text box (#3020)

Fixes #3016 
Fixes #3017

* CI: Periodic Controller Database Update (#3028)

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* [compiler] fix `compile_condition` not attempting macro expansion (#3030)

Ancient bug! Resulted in a 1%-2% decrease in code object file size (and
more importantly the asm is more readable).

Fixes #3029

* decompiler: `obj` -> `this`, `set-time!` and `time-elapsed?` macros (#3026)

This renames the method object in `defmethod`s to `this` and adds
detection for the `set-time!` and `time-elapsed?` macros.

Definitely my biggest PR yet...

* [decompiler] do not use `time-elapsed?` on unsigned comps (#3031)

* misc low hanging fruit fixes (#3034)

Fixes #2235 
Fixes #2236 
Fixes #2963 
Fixes #3027

* Jak 2 controller LED implementation (#3035)

Adds controller LED features to Jak 2:
- progressive flickering denoting health
- copies tomb simon says puzzle colors
- unique colors for each gun
- orange color for being indax
- yellow color for being in mech
- purple color for being darkjak
- blue color for being in board
- red flash when wanted.

May add more features later?

Also did some minor clean-up on some types.

* [jak2] un-mips2c `draw-string` (#3040)

It's not an asm function.

* Remove dupe line in discord-rpc .clang-format (#3039)

* [decompiler] Handle `find-parent-method` (#3018)

This change adds a few new features:
- Decompiler automatically knows the type of `find-parent-method` use in
jak 1 and jak2 when used in a method or virtual state handler.
- Decompiler inserts a call to `call-parent-method` or
`find-parent-state`
- Removed most casts related to these functions

There are still a few minor issues around this:
- There are still some casts needed when using `post` methods, as `post`
is just a `function`, and needs a cast to `(function none)` or similar.
It didn't seem easy to change the type of `post`, so I'm not going to
worry about it for this PR. It only shows up in like 3 places in jak 2.
(and 0 in jak 1)
- If "call the handler if it's not #f" logic should probably be another
macro.

Fixes #805

* [jak2] scalable debug font + fix debug menu widescreen bugs  (#3044)

also hides empty (no commands) regions from the viewers by default now.

* fix draw-string-xy-scaled cropping (#3041)

fixes #3038 


https://github.com/open-goal/jak-project/assets/2515356/5b7a490f-e1fb-4a12-a3d9-b576a2c1d574

* [jak2] fully implement `*user*` (#3046)

Fixes #1918

* Bump actions/checkout from 3 to 4 (#3047)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* g/j2: don't force auto-save on in jak 2 when starting a speedrun (#3048)

* fix eco pickup time bug (#3056)

* CI: Periodic Controller Database Update (#3060)

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* decompiler: better automatic detection of art groups and `joint-node-index` macro detection (#3061)

* [jak2] Set up extractor (#3042)

This sets up the extractor for jak 2. I was expecting that I'd have to
make some more significant changes to the decompiler/compiler path
stuff, but this was not the case!

The only real change is that you can now provide multiple ISO hashes for
an entry in `ISOMetadata`. This is needed for the two different NTSC
versions, which have the same configs, serials, and ELF hashes, but
slightly different contents.

I also didn't add the korean version because I don't have the info for
it.

---------

Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>

* [goalc] Cleaned up speedups (#3066)

Started at 349,880,038 allocations and 42s

- Switched to making `Symbol` in GOOS be a "fixed type", just a wrapper
around a `const char*` pointing to the string in the symbol table. This
is a step toward making a lot of things better, but by itself not a huge
improvement. Some things may be worse due to more temp `std::string`
allocations, but one day all these can be removed. On linux it saved
allocations (347,685,429), and saved a second or two (41 s).
- cache `#t` and `#f` in interpreter, better lookup for special
forms/builtins (hashtable of pointers instead of strings, vector for the
small special form list). Dropped time to 38s.
- special-case in quasiquote when splicing is the last thing in a list.
Allocation dropped to 340,603,082
- custom hash table for environment lookups (lexical vars). Dropped to
36s and 314,637,194
- less allocation in `read_list` 311,613,616. Time about the same.
- `let` and `let*` in Interpreter.cpp 191,988,083, time down to 28s.

* [jak2] new glow fix (#3067)

Fix a typo in the shader. (fixes
open-goal/jak-project#3045, and fixes fadeout
when glows move offscreen on the bottom).

Also, try a different format for the depth blit, in an attempt to fix
this issue open-goal/jak-project#3065.

* decompiler: some hacks to allow running decompiler on jak 3 v5 code files, improve `all-types` generation (#2526)

Co-authored-by: water <awaterford111445@gmail.com>

* [glow] potentially fix bad glow texture (#3071)

Might fix #3065 

fixes #3070

* [jak2] fix minor original game visual bugs (#3072)

Fixes #2994 the underport and consite icons being green (same color as
the goal, likely copy-pasta).

Fixes the hundredths timer in the race timer actually being a 60ths
timer.

* github: add issue template for jak 2 (#3073)

* jak3: add `goal_src` skeleton (#3069)

* [decompiler] Support v5 data file link data (#3076)

Fix the implementation of `link_v5` so it works on "data" files for jak
3.

* decompiler: fix for v5 art group info dump, update `taskfile` for jak 3 (#3077)

* decompiler: allow jak 3 texture and model extraction (#3080)

Added some hacks and stubs to allow extracting textures and models.

* [jak2] Fix collision renderer extract (#3081)

Fixes missing collision geometry reported in
open-goal/jak-project#3011

The issue happens when there are 256 polygons. In this case `num-polys`
is 0 (it's a u8). There are actual cases where there are 0 polygons, so
we have to do a more complicated check to get the real count. I should
have done this in the first place, but it seemed to work...

* g/j2: Integrate highscores with Speedrun.com/JakSpeedruns.com when speedrunner mode is enabled (#3037)

* [jak1] Finnish base game strings (#3058)

Finnish translations for the remaining UI texts

---------

Co-authored-by: Tyler Wilding <xtvaser@gmail.com>

* jak3: add missing files and implement `pexcw` (#3084)

Some files were in the `banned_objects` list and were thus excluded from
the `all_objs` file.

Also implements the `pexcw` instruction which is only used in `hfrag`
code.

* decompiler: add texture merging feature (#3083)

* fix `*jak1-full-game*` being flipped (#3085)

* [glb export] Export bones. (#3087)

![image](https://github.com/open-goal/jak-project/assets/48171810/0f26e77b-af68-4450-882e-762a501bdef4)

* custom levels: support for packing textures (#3089)

* Fix build failure at llvm-rc on windows due to codepage error on zydis (#3088)

This fix is for the following error when building with llvm on windows

```
[build] llvm-rc: Error in VERSIONINFO statement (ID 1): 
[proc] The command: "C:\Program Files\CMake\bin\cmake.exe" --build C:/.../git/jak-project-test/out/build/Release --parallel 8 exited with code: 1
[build] Non-ASCII 8-bit codepoint (´┐¢) can't be interpreted in the current codepage
[build] ninja: build stopped: subcommand failed.
```

It's caused by llvm-rc not handling non 8-bit characters.
The same issue is discussed in the curl project here
curl/curl#7765

This commit copies the fix from curl, replacing '©' with '(C)'.

* jak2: add warning text when changing frame rate for the first time (#3092)

* CI: Periodic Controller Database Update (#3091)

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* Add label_types file merge for PAL version (#3086)

* ci: Workaround CMake/Perl regression in recent windows-2022 images (#3097)

* formatter: rewrite and refactor, address more edge-cases, begin documenting my work (#3096)

* custom levels: refactor level building code and jak 2 support (#3090)

Co-authored-by: water <awaterford111445@gmail.com>

* CI: Periodic Controller Database Update (#3101)

Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>

* Fix starting continue point for Hub 2/3 category extensions (#3103)

* [compiler] asm-only disasm output + fix spacing bug (#3104)

* `deftype` and `defmethod` syntax major changes (#3094)

Major change to how `deftype` shows up in our code:
- the decompiler will no longer emit the `offset-assert`,
`method-count-assert`, `size-assert` and `flag-assert` parameters. There
are extremely few cases where having this in the decompiled code is
helpful, as the types there come from `all-types` which already has
those parameters. This also doesn't break type consistency because:
  - the asserts aren't compared.
- the first step of the test uses `all-types`, which has the asserts,
which will throw an error if they're bad.
- the decompiler won't emit the `heap-base` parameter unless necessary
now.
- the decompiler will try its hardest to turn a fixed-offset field into
an `overlay-at` field. It falls back to the old offset if all else
fails.
- `overlay-at` now supports field "dereferencing" to specify the offset
that's within a field that's a structure, e.g.:
```lisp
(deftype foobar (structure)
  ((vec    vector  :inline)
   (flags  int32   :overlay-at (-> vec w))
   )
  )
```
in this structure, the offset of `flags` will be 12 because that is the
final offset of `vec`'s `w` field within this structure.
- **removed ID from all method declarations.** IDs are only ever
automatically assigned now. Fixes #3068.
- added an `:overlay` parameter to method declarations, in order to
declare a new method that goes on top of a previously-defined method.
Syntax is `:overlay <method-name>`. Please do not ever use this.
- added `state-methods` list parameter. This lets you quickly specify a
list of states to be put in the method table. Same syntax as the
`states` list parameter. The decompiler will try to put as many states
in this as it can without messing with the method ID order.

Also changes `defmethod` to make the first type definition (before the
arguments) optional. The type can now be inferred from the first
argument. Fixes #3093.

---------

Co-authored-by: Hat Kid <6624576+Hat-Kid@users.noreply.github.com>

* [jak2] simplify fps disclaimer (#3108)

* fps selection fixes i missed (#3109)

* rename joint node and art element macros (#3111)

* New Crowdin updates (#3113)

* ci/windows: stop Strawberry perl from changing resulting built artifacts in CI (#3114)

* docs: Re-order root README a bit and add new link to new tutorial (#3119)

* fix build warnings

* remove art groups from default test-zone levels

* switch slime lut to a texture

* ci: properly statically link on macOS (#3127)

* g/jak2: allow `L1 + R1 + Start` to open speedrun menu as well (#3131)

* g/jak2: disable speech volume when starting a new speedrun (#3132)

* Add fallback text functionality to Jak 2 (#3128)

* cleanup

* sky build

---------

Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>
Co-authored-by: water111 <48171810+water111@users.noreply.github.com>
Co-authored-by: animalstyletaco <mcc-108@hotmail.com>
Co-authored-by: animalstyletaco <animalstyletaco95@gmail.com>
Co-authored-by: Tyler Wilding <xTVaser@users.noreply.github.com>
Co-authored-by: OpenGOAL Bot <99294829+OpenGOALBot@users.noreply.github.com>
Co-authored-by: OpenGOALBot <OpenGOALBot@users.noreply.github.com>
Co-authored-by: Luminar Light <18116946+LuminarLight@users.noreply.github.com>
Co-authored-by: Brent Hickey <brent.hickey@icloud.com>
Co-authored-by: Tyler Wilding <xtvaser@gmail.com>
Co-authored-by: Hat Kid <6624576+Hat-Kid@users.noreply.github.com>
Co-authored-by: Ziemas <ziemas@ziemas.se>
Co-authored-by: ZedB0T <89345505+Zedb0T@users.noreply.github.com>
Co-authored-by: atylmo <48929878+atylmo@users.noreply.github.com>
Co-authored-by: water <awaterford111445@gmail.com>
Co-authored-by: rafalekkB <144032866+rafalekkB@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Aloqas <102683375+Aloqas@users.noreply.github.com>
Co-authored-by: jabermony <748988+jabermony@users.noreply.github.com>
ShuangLiu1992 added a commit to ShuangLiu1992/assimp that referenced this issue Jan 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build Windows Windows-specific
5 participants