JIT operations should return the current exception in a return GPR wh…

…en it's free.

https://bugs.webkit.org/show_bug.cgi?id=273264
rdar://127065985

Reviewed by Yusuke Suzuki.

Reland: Move operationExceptionRegister to CCallHelpers because including GPRInfo.h in
OperationResult.h cause downstream build issues.

This patch makes a lot of changes so that we can pass the current exception in a return register.

1) There's now a new set of macros for JIT_OPERATIONS that wrap the return type in a templated
   struct containing the true result and a pointer to the current exception if there's an extra
   return register in the C++ calling convention (and returning a `struct` won't push everything
   to the stack... armv7...).

2) Exception checking macro assembler functions take an optional register parameter, which is
   the register the exception is in, typically returnGPR2 but sometimes returnGPR.

3) Started moving FTL towards the same templated operation signature deduction of arguments used
   in the other JITs.

4) exception checks have been pushed into callOperation in DFG since that has the signature
   of the operation being called already.

5) We don't support exceptions in registers for double on ARM64 since that causes the double to
   be returned in x0 rather than d0. Also, FTL doesn't support `FastOperationResult<double/float>`
   since B3 can't handle that yet.

6) Add FTL::Output::verify which behaves like an assert by using a Check node.

7) FTL has a list of tuples for operation returns since we only use pairs this is just an LType[].

* Source/JavaScriptCore/CMakeLists.txt:
* Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj:
* Source/JavaScriptCore/assembler/MacroAssembler.h:
(JSC::MacroAssembler::branchTestPtr):
* Source/JavaScriptCore/assembler/MacroAssemblerARM64.cpp:
* Source/JavaScriptCore/assembler/MacroAssemblerARMv7.cpp:
* Source/JavaScriptCore/assembler/MacroAssemblerRISCV64.cpp:
* Source/JavaScriptCore/assembler/MacroAssemblerX86Common.cpp:
* Source/JavaScriptCore/b3/B3Type.h:
* Source/JavaScriptCore/b3/B3Validate.cpp:
* Source/JavaScriptCore/b3/air/AirCCallingConvention.cpp:
(JSC::B3::Air::cCallResultCount):
(JSC::B3::Air::cCallResult):
* Source/JavaScriptCore/b3/testb3.h:
* Source/JavaScriptCore/b3/testb3_5.cpp:
(JSC_DEFINE_NOEXCEPT_JIT_OPERATION):
(JSC_DEFINE_JIT_OPERATION): Deleted.
* Source/JavaScriptCore/b3/testb3_6.cpp:
(JSC_DEFINE_NOEXCEPT_JIT_OPERATION):
(JSC_DEFINE_JIT_OPERATION): Deleted.
* Source/JavaScriptCore/b3/testb3_7.cpp:
(JSC_DEFINE_NOEXCEPT_JIT_OPERATION):
(JSC_DEFINE_JIT_OPERATION): Deleted.
* Source/JavaScriptCore/dfg/DFGAbstractInterpreter.h:
* Source/JavaScriptCore/dfg/DFGAbstractInterpreterInlines.h:
(JSC::DFG::AbstractInterpreter<AbstractStateType>::executeEffects):
(JSC::DFG::AbstractInterpreter<AbstractStateType>::executeDoubleUnaryOpEffects):
* Source/JavaScriptCore/dfg/DFGArithMode.h:
* Source/JavaScriptCore/dfg/DFGArrayifySlowPathGenerator.h:
* Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::handleDOMJITGetter):
(JSC::DFG::ByteCodeParser::handleGetById):
* Source/JavaScriptCore/dfg/DFGCallArrayAllocatorSlowPathGenerator.h:
* Source/JavaScriptCore/dfg/DFGCallCreateDirectArgumentsSlowPathGenerator.h:
* Source/JavaScriptCore/dfg/DFGJITCompiler.h:
(JSC::DFG::JITCompiler::appendCall):
(JSC::DFG::JITCompiler::appendCallWithUGPRPair):
* Source/JavaScriptCore/dfg/DFGOSRExit.cpp:
(JSC::DFG::JSC_DEFINE_NOEXCEPT_JIT_OPERATION):
(JSC::DFG::JSC_DEFINE_JIT_OPERATION): Deleted.
* Source/JavaScriptCore/dfg/DFGOSRExit.h:
* Source/JavaScriptCore/dfg/DFGOperations.cpp:
(JSC::DFG::JSC_DEFINE_JIT_OPERATION):
(JSC::DFG::JSC_DEFINE_NOEXCEPT_JIT_OPERATION):
* Source/JavaScriptCore/dfg/DFGOperations.h:
* Source/JavaScriptCore/dfg/DFGSaneStringGetByValSlowPathGenerator.h:
* Source/JavaScriptCore/dfg/DFGSlowPathGenerator.h:
(JSC::DFG::slowPathMove):
(JSC::DFG::CallSlowPathGenerator::tearDown):
* Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp:
(JSC::DFG::SpeculativeJIT::exceptionCheck):
(JSC::DFG::SpeculativeJIT::compileDeleteById):
(JSC::DFG::SpeculativeJIT::compileDeleteByVal):
(JSC::DFG::SpeculativeJIT::compilePushWithScope):
(JSC::DFG::SpeculativeJIT::compileStringSubstring):
(JSC::DFG::SpeculativeJIT::compileToLowerCase):
(JSC::DFG::SpeculativeJIT::compilePutByVal):
(JSC::DFG::SpeculativeJIT::compileFromCharCode):
(JSC::DFG::SpeculativeJIT::compileValueToInt32):
* Source/JavaScriptCore/dfg/DFGSpeculativeJIT.h:
(JSC::DFG::SpeculativeJIT::operationExceptionCheck):
(JSC::DFG::SpeculativeJIT::callOperation):
(JSC::DFG::SpeculativeJIT::callOperationWithoutExceptionCheck):
(JSC::DFG::SpeculativeJIT::appendCall):
(JSC::DFG::SpeculativeJIT::appendCallWithUGPRPair):
(JSC::DFG::SpeculativeJIT::appendCallSetResult):
* Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp:
(JSC::DFG::SpeculativeJIT::nonSpeculativePeepholeStrictEq):
(JSC::DFG::SpeculativeJIT::genericJSValueNonPeepholeStrictEq):
(JSC::DFG::SpeculativeJIT::emitCall):
(JSC::DFG::SpeculativeJIT::compile):
(JSC::DFG::SpeculativeJIT::compileGetByValWithThis):
(JSC::DFG::SpeculativeJIT::compileCreateClonedArguments):
* Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp:
(JSC::DFG::SpeculativeJIT::nonSpeculativePeepholeStrictEq):
(JSC::DFG::SpeculativeJIT::genericJSValueNonPeepholeStrictEq):
(JSC::DFG::SpeculativeJIT::emitCall):
(JSC::DFG::SpeculativeJIT::compileGetByVal):
(JSC::DFG::SpeculativeJIT::compileRegExpTestInline):
(JSC::DFG::SpeculativeJIT::compile):
(JSC::DFG::SpeculativeJIT::compileFunctionBind):
(JSC::DFG::SpeculativeJIT::compileCreateClonedArguments):
* Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp:
(JSC::FTL::DFG::JSC_DEFINE_NOEXCEPT_JIT_OPERATION_WITH_ATTRIBUTES):
(JSC::FTL::DFG::LowerDFGToB3::lower):
(JSC::FTL::DFG::LowerDFGToB3::compileBlock):
(JSC::FTL::DFG::LowerDFGToB3::validateAIState):
(JSC::FTL::DFG::LowerDFGToB3::compileNode):
(JSC::FTL::DFG::LowerDFGToB3::compileInt52Constant):
(JSC::FTL::DFG::LowerDFGToB3::compileDoubleRep):
(JSC::FTL::DFG::LowerDFGToB3::compileValueRep):
(JSC::FTL::DFG::LowerDFGToB3::compileInt52Rep):
(JSC::FTL::DFG::LowerDFGToB3::compileValueToInt32):
(JSC::FTL::DFG::LowerDFGToB3::compileBooleanToNumber):
(JSC::FTL::DFG::LowerDFGToB3::compileGetStack):
(JSC::FTL::DFG::LowerDFGToB3::compilePutStack):
(JSC::FTL::DFG::LowerDFGToB3::compileValueSub):
(JSC::FTL::DFG::LowerDFGToB3::compileValueMul):
(JSC::FTL::DFG::LowerDFGToB3::compileBinaryMathIC):
(JSC::FTL::DFG::LowerDFGToB3::compileArithAddOrSub):
(JSC::FTL::DFG::LowerDFGToB3::compileArithMul):
(JSC::FTL::DFG::LowerDFGToB3::compileValueDiv):
(JSC::FTL::DFG::LowerDFGToB3::compileArithDiv):
(JSC::FTL::DFG::LowerDFGToB3::compileArithMod):
(JSC::FTL::DFG::LowerDFGToB3::compileArithMinOrMax):
(JSC::FTL::DFG::LowerDFGToB3::compileArithAbs):
(JSC::FTL::DFG::LowerDFGToB3::compileValuePow):
(JSC::FTL::DFG::LowerDFGToB3::compileArithNegate):
(JSC::FTL::DFG::LowerDFGToB3::compileValueBitAnd):
(JSC::FTL::DFG::LowerDFGToB3::compileValueBitLShift):
(JSC::FTL::DFG::LowerDFGToB3::compileCheckStructure):
(JSC::FTL::DFG::LowerDFGToB3::compileCheckIsConstant):
(JSC::FTL::DFG::LowerDFGToB3::compileArrayify):
(JSC::FTL::DFG::LowerDFGToB3::compilePutStructure):
(JSC::FTL::DFG::LowerDFGToB3::compileGetById):
(JSC::FTL::DFG::LowerDFGToB3::compileGetByIdWithThis):
(JSC::FTL::DFG::LowerDFGToB3::getPrivateName):
(JSC::FTL::DFG::LowerDFGToB3::compileAtomicsReadModifyWrite):
(JSC::FTL::DFG::LowerDFGToB3::compileAtomicsIsLockFree):
(JSC::FTL::DFG::LowerDFGToB3::compileGetIndexedPropertyStorage):
(JSC::FTL::DFG::LowerDFGToB3::compileCheckArray):
(JSC::FTL::DFG::LowerDFGToB3::compileGetArrayLength):
(JSC::FTL::DFG::LowerDFGToB3::compileGetByValImpl):
(JSC::FTL::DFG::LowerDFGToB3::compileGetMyArgumentByVal):
(JSC::FTL::DFG::LowerDFGToB3::compilePutByVal):
(JSC::FTL::DFG::LowerDFGToB3::compileArrayPush):
(JSC::FTL::DFG::LowerDFGToB3::compileArrayPop):
(JSC::FTL::DFG::LowerDFGToB3::compileCreateActivation):
(JSC::FTL::DFG::LowerDFGToB3::compileNewFunction):
(JSC::FTL::DFG::LowerDFGToB3::compileCreateDirectArguments):
(JSC::FTL::DFG::LowerDFGToB3::compileCreateScopedArguments):
(JSC::FTL::DFG::LowerDFGToB3::compileGetRestLength):
(JSC::FTL::DFG::LowerDFGToB3::compileNewArray):
(JSC::FTL::DFG::LowerDFGToB3::compileNewArrayBuffer):
(JSC::FTL::DFG::LowerDFGToB3::compileNewArrayWithSize):
(JSC::FTL::DFG::LowerDFGToB3::compileNewTypedArray):
(JSC::FTL::DFG::LowerDFGToB3::compileReallocatePropertyStorage):
(JSC::FTL::DFG::LowerDFGToB3::compileToNumeric):
(JSC::FTL::DFG::LowerDFGToB3::compileToStringOrCallStringConstructorOrStringValueOf):
(JSC::FTL::DFG::LowerDFGToB3::compileToPrimitive):
(JSC::FTL::DFG::LowerDFGToB3::compileMakeRope):
(JSC::FTL::DFG::LowerDFGToB3::compileStringCharCodeAt):
(JSC::FTL::DFG::LowerDFGToB3::compileStringFromCharCode):
(JSC::FTL::DFG::LowerDFGToB3::compileGetByOffset):
(JSC::FTL::DFG::LowerDFGToB3::compileMultiGetByOffset):
(JSC::FTL::DFG::LowerDFGToB3::compilePutByOffset):
(JSC::FTL::DFG::LowerDFGToB3::compileMultiPutByOffset):
(JSC::FTL::DFG::LowerDFGToB3::compileMatchStructure):
(JSC::FTL::DFG::LowerDFGToB3::compileNotifyWrite):
(JSC::FTL::DFG::LowerDFGToB3::compileCompareEq):
(JSC::FTL::DFG::LowerDFGToB3::compileCompareStrictEq):
(JSC::FTL::DFG::JSC_DEFINE_JIT_OPERATION_WITH_ATTRIBUTES): Deleted.
* Source/JavaScriptCore/ftl/FTLOSRExitCompiler.cpp:
(JSC::FTL::JSC_DEFINE_NOEXCEPT_JIT_OPERATION):
(JSC::FTL::JSC_DEFINE_JIT_OPERATION): Deleted.
* Source/JavaScriptCore/ftl/FTLOSRExitCompiler.h:
* Source/JavaScriptCore/ftl/FTLOperations.cpp:
(JSC::FTL::JSC_DEFINE_NOEXCEPT_JIT_OPERATION):
(JSC::FTL::JSC_DEFINE_NOEXCEPT_JIT_OPERATION_WITH_ATTRIBUTES):
(JSC::FTL::JSC_DEFINE_JIT_OPERATION): Deleted.
(JSC::FTL::JSC_DEFINE_JIT_OPERATION_WITH_ATTRIBUTES): Deleted.
* Source/JavaScriptCore/ftl/FTLOperations.h:
* Source/JavaScriptCore/ftl/FTLOutput.cpp:
(JSC::FTL::Output::verify):
* Source/JavaScriptCore/ftl/FTLOutput.h:
* Source/JavaScriptCore/jit/AssemblyHelpers.cpp:
(JSC::AssemblyHelpers::emitExceptionCheck):
(JSC::AssemblyHelpers::emitNonPatchableExceptionCheck):
* Source/JavaScriptCore/jit/AssemblyHelpers.h:
* Source/JavaScriptCore/jit/CCallHelpers.h:
(JSC::CCallHelpers::setupResults):
(JSC::CCallHelpers::operationExceptionRegister):
* Source/JavaScriptCore/jit/JIT.h:
* Source/JavaScriptCore/jit/JITCall.cpp:
(JSC::JIT::compileSetupFrame):
* Source/JavaScriptCore/jit/JITInlines.h:
(JSC::JIT::appendCallWithExceptionCheck):
(JSC::JIT::appendCallSetJSValueResult):
(JSC::JIT::appendCallWithExceptionCheckSetJSValueResult):
(JSC::JIT::appendCallWithExceptionCheckSetJSValueResultWithProfile):
* Source/JavaScriptCore/jit/JITOperations.cpp:
(JSC::JSC_DEFINE_NOEXCEPT_JIT_OPERATION):
(JSC::JSC_DEFINE_JIT_OPERATION):
* Source/JavaScriptCore/jit/JITOperations.h:
* Source/JavaScriptCore/jit/OperationResult.h: Added.
(JSC::ExceptionOperationImplicitResult::operator ExceptionOperationResult<To>):
(JSC::ExceptionOperationImplicitResult::operator To):
(JSC::ExceptionOperationImplicitResult<void>::operator ExceptionOperationResult<void>):
(JSC::makeOperationResult):
* Source/JavaScriptCore/llint/LLIntThunks.h:
* Source/JavaScriptCore/runtime/AtomicsObject.cpp:
(JSC::JSC_DEFINE_JIT_OPERATION):
* Source/JavaScriptCore/runtime/CommonSlowPaths.h:
* Source/JavaScriptCore/runtime/ExceptionScope.h:
(JSC::ExceptionScope::exception const):
* Source/JavaScriptCore/runtime/HashMapImplInlines.h:
(JSC::HashMapImpl<HashMapBucketType>::addNormalized):
* Source/JavaScriptCore/runtime/MathCommon.cpp:
(JSC::JSC_DEFINE_NOEXCEPT_JIT_OPERATION):
(JSC::Math::JSC_DEFINE_NOEXCEPT_JIT_OPERATION):
(JSC::JSC_DEFINE_JIT_OPERATION): Deleted.
(JSC::Math::JSC_DEFINE_JIT_OPERATION): Deleted.
* Source/JavaScriptCore/runtime/MathCommon.h:
* Source/JavaScriptCore/runtime/StringPrototype.cpp:
(JSC::JSC_DEFINE_JIT_OPERATION):
* Source/JavaScriptCore/tools/JSDollarVM.cpp:
* Source/JavaScriptCore/wasm/WasmOperations.cpp:
(JSC::Wasm::JSC_DEFINE_NOEXCEPT_JIT_OPERATION):
(JSC::Wasm::JSC_DEFINE_JIT_OPERATION): Deleted.
* Source/JavaScriptCore/wasm/WasmOperations.h:
* Source/JavaScriptCore/yarr/YarrJIT.cpp:
(JSC::Yarr::JSC_DEFINE_NOEXCEPT_JIT_OPERATION):
(JSC::Yarr::JSC_DEFINE_JIT_OPERATION): Deleted.
* Source/WTF/wtf/PlatformCallingConventions.h:
* Source/WTF/wtf/Threading.cpp:
(WTF::Thread::mayBeGCThread):
* Source/WebCore/bindings/scripts/CodeGeneratorJS.pm:
(GenerateOperationDefinition):
* Source/WebCore/bindings/scripts/test/JS/JSTestDOMJIT.cpp:
(WebCore::JSC_DEFINE_JIT_OPERATION):
* Source/WebCore/cssjit/SelectorCompiler.cpp:
(WebCore::SelectorCompiler::JSC_DEFINE_NOEXCEPT_JIT_OPERATION):
(WebCore::SelectorCompiler::JSC_DEFINE_JIT_OPERATION): Deleted.
* Source/WebCore/domjit/JSDocumentDOMJIT.cpp:
(WebCore::DOMJIT::JSC_DEFINE_JIT_OPERATION):

Canonical link: https://commits.webkit.org/278477@main

Source/JavaScriptCore/CMakeLists.txt

@@ -878,6 +878,7 @@ set(JavaScriptCore_PRIVATE_FRAMEWORK_HEADERS
     jit/JITOperations.h
     jit/JITStubRoutine.h
     jit/JITThunks.h
+    jit/OperationResult.h
     jit/PCToCodeOriginMap.h
     jit/PolymorphicCallStubRoutine.h
     jit/Reg.h

Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj

@@ -1028,6 +1028,7 @@
 		535557141D9D9EA5006D583B /* WasmMemory.h in Headers */ = {isa = PBXBuildFile; fileRef = 535557131D9D9EA5006D583B /* WasmMemory.h */; settings = {ATTRIBUTES = (Private, ); }; };
 		535C24611F78928E006EC40E /* create_regex_tables in Headers */ = {isa = PBXBuildFile; fileRef = A718F8211178EB4B002465A7 /* create_regex_tables */; };
 		535C246C1F7A1778006EC40E /* UnifiedSource136.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 535C246B1F7A1777006EC40E /* UnifiedSource136.cpp */; };
+		535EC0972BCEF71900CCC5F7 /* OperationResult.h in Headers */ = {isa = PBXBuildFile; fileRef = 535EC0962BCEF71900CCC5F7 /* OperationResult.h */; settings = {ATTRIBUTES = (Private, ); }; };
 		53663FDA23562F96005EA68C /* BytecodeIndex.h in Headers */ = {isa = PBXBuildFile; fileRef = 53663FD923562F90005EA68C /* BytecodeIndex.h */; settings = {ATTRIBUTES = (Private, ); }; };
 		536B310C1F71C5990037FC33 /* UnifiedSource119.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 536B30871F71C5380037FC33 /* UnifiedSource119.cpp */; };
 		536B310D1F71C5990037FC33 /* UnifiedSource125.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 536B30881F71C5380037FC33 /* UnifiedSource125.cpp */; };
@@ -4083,6 +4084,7 @@
 		535557151D9DFA32006D583B /* WasmMemory.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = WasmMemory.cpp; sourceTree = "<group>"; };
 		535C246B1F7A1777006EC40E /* UnifiedSource136.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = UnifiedSource136.cpp; path = "DerivedSources/JavaScriptCore/unified-sources/UnifiedSource136.cpp"; sourceTree = BUILT_PRODUCTS_DIR; };
 		535E08C9225460AB00DF00CA /* postprocess-header-rule */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.script.sh; path = "postprocess-header-rule"; sourceTree = "<group>"; };
+		535EC0962BCEF71900CCC5F7 /* OperationResult.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = OperationResult.h; sourceTree = "<group>"; };
 		5360DABB2356ADCA003F6AB8 /* BytecodeIndex.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = BytecodeIndex.cpp; sourceTree = "<group>"; };
 		53663FD923562F90005EA68C /* BytecodeIndex.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = BytecodeIndex.h; sourceTree = "<group>"; };
 		53696E5720A3A70200D7E01E /* BytecodeStructs.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = BytecodeStructs.h; sourceTree = "<group>"; };
@@ -6914,6 +6916,7 @@
 				726B91AC26581653008E6F82 /* JITWorklistThread.cpp */,
 				726B91A826581652008E6F82 /* JITWorklistThread.h */,
 				A76C51741182748D00715B05 /* JSInterfaceJIT.h */,
+				535EC0962BCEF71900CCC5F7 /* OperationResult.h */,
 				792CB3471C4EED5C00D13AF3 /* PCToCodeOriginMap.cpp */,
 				792CB3481C4EED5C00D13AF3 /* PCToCodeOriginMap.h */,
 				0FE834151A6EF97B00D04847 /* PolymorphicCallStubRoutine.cpp */,
@@ -11475,6 +11478,7 @@
 				14A4680D216FA56A000D2B1A /* OpcodeSize.h in Headers */,
 				0F2BDC2C151FDE9100CD8910 /* Operands.h in Headers */,
 				A70447EA17A0BD4600F5898E /* OperandsInlines.h in Headers */,
+				535EC0972BCEF71900CCC5F7 /* OperationResult.h in Headers */,
 				BC18C4480E16F5CD00B34460 /* Operations.h in Headers */,
 				0FE228ED1436AB2700196C48 /* Options.h in Headers */,
 				FE3842332324D51B009DD445 /* OptionsList.h in Headers */,

Source/JavaScriptCore/assembler/MacroAssembler.h

@@ -936,6 +936,11 @@ class MacroAssembler : public MacroAssemblerBase {
         return branchTest32(cond, address, mask);
     }
 
+    Jump branchTestPtr(ResultCondition cond, AbsoluteAddress address, TrustedImm32 mask = TrustedImm32(-1))
+    {
+        return branchTest32(cond, address, mask);
+    }
+
     Jump branchAddPtr(ResultCondition cond, RegisterID src, RegisterID dest)
     {
         return branchAdd32(cond, src, dest);

Source/JavaScriptCore/assembler/MacroAssemblerARM64.cpp

@@ -29,6 +29,7 @@
 #include "MacroAssembler.h"
 
 #include "JSCPtrTag.h"
+#include "OperationResult.h"
 #include "ProbeContext.h"
 #include <wtf/InlineASM.h>
 #include <wtf/TZoneMallocInlines.h>
@@ -60,9 +61,9 @@ namespace JSC {
 
 WTF_MAKE_TZONE_ALLOCATED_IMPL(MacroAssemblerARM64);
 
-JSC_DECLARE_JIT_OPERATION(ctiMasmProbeTrampoline, void, ());
+JSC_DECLARE_NOEXCEPT_JIT_OPERATION(ctiMasmProbeTrampoline, void, ());
 JSC_ANNOTATE_JIT_OPERATION_PROBE(ctiMasmProbeTrampoline);
-JSC_DECLARE_JIT_OPERATION(ctiMasmProbeTrampolineSIMD, void, ());
+JSC_DECLARE_NOEXCEPT_JIT_OPERATION(ctiMasmProbeTrampolineSIMD, void, ());
 JSC_ANNOTATE_JIT_OPERATION_PROBE(ctiMasmProbeTrampolineSIMD);
 
 using namespace ARM64Registers;

Source/JavaScriptCore/assembler/MacroAssemblerARMv7.cpp

@@ -28,12 +28,13 @@
 #if ENABLE(ASSEMBLER) && CPU(ARM_THUMB2)
 #include "MacroAssembler.h"
 
+#include "OperationResult.h"
 #include "ProbeContext.h"
 #include <wtf/InlineASM.h>
 
 namespace JSC {
 
-JSC_DECLARE_JIT_OPERATION(ctiMasmProbeTrampoline, void, ());
+JSC_DECLARE_NOEXCEPT_JIT_OPERATION(ctiMasmProbeTrampoline, void, ());
 JSC_ANNOTATE_JIT_OPERATION_PROBE(ctiMasmProbeTrampoline);
 
 using namespace ARMRegisters;

Source/JavaScriptCore/assembler/MacroAssemblerRISCV64.cpp

@@ -28,12 +28,13 @@
 
 #if ENABLE(ASSEMBLER) && CPU(RISCV64)
 
+#include "OperaitonResult.h"
 #include "ProbeContext.h"
 #include <wtf/InlineASM.h>
 
 namespace JSC {
 
-JSC_DECLARE_JIT_OPERATION(ctiMasmProbeTrampoline, void, ());
+JSC_DECLARE_NOEXCEPT_JIT_OPERATION(ctiMasmProbeTrampoline, void, ());
 JSC_ANNOTATE_JIT_OPERATION_PROBE(ctiMasmProbeTrampoline);
 
 using namespace RISCV64Registers;

Source/JavaScriptCore/assembler/MacroAssemblerX86Common.cpp

@@ -28,6 +28,7 @@
 #if ENABLE(ASSEMBLER) && (CPU(X86) || CPU(X86_64))
 #include "MacroAssembler.h"
 
+#include "OperationResult.h"
 #include "ProbeContext.h"
 #include <wtf/InlineASM.h>
 
@@ -41,10 +42,10 @@
 
 namespace JSC {
 
-JSC_DECLARE_JIT_OPERATION(ctiMasmProbeTrampoline, void, ());
+JSC_DECLARE_NOEXCEPT_JIT_OPERATION(ctiMasmProbeTrampoline, void, ());
 JSC_ANNOTATE_JIT_OPERATION_PROBE(ctiMasmProbeTrampoline);
 #if CPU(X86_64)
-JSC_DECLARE_JIT_OPERATION(ctiMasmProbeTrampolineSIMD, void, ());
+JSC_DECLARE_NOEXCEPT_JIT_OPERATION(ctiMasmProbeTrampolineSIMD, void, ());
 JSC_ANNOTATE_JIT_OPERATION_PROBE(ctiMasmProbeTrampolineSIMD);
 #endif
 

Source/JavaScriptCore/b3/B3Type.h

@@ -65,6 +65,7 @@ class Type {
 
     ~Type() = default;
 
+    static const unsigned numberOfPrimitiveTypes = V128 + 1;
     static Type tupleFromIndex(unsigned index) { ASSERT(!(index & tupleFlag)); return bitwise_cast<Type>(index | tupleFlag); }
 
     TypeKind kind() const { return m_kind & tupleFlag ? Tuple : m_kind; }

Source/JavaScriptCore/b3/B3Validate.cpp

@@ -699,10 +699,13 @@ class Validater {
                 VALIDATE(value->numChildren() >= 1, ("At ", *value));
                 VALIDATE(value->child(0)->type() == pointerType(), ("At ", *value));
                 if (value->type().isTuple()) {
-                    // FIXME: Right now we only support a pair of register sized values since on every calling
+                    // FIXME: Right now we only support a pair of two GPR values since on every calling
                     // convention we support that's returned in returnValueGPR/returnValueGPR2, respectively.
                     VALIDATE(m_procedure.resultCount(value->type()) == 2, ("At ", *value));
-                    VALIDATE(m_procedure.typeAtOffset(value->type(), 0) == registerType(), ("At ", *value));
+                    if (is32Bit())
+                        VALIDATE(m_procedure.typeAtOffset(value->type(), 0) == registerType(), ("At ", *value));
+                    else
+                        VALIDATE(m_procedure.typeAtOffset(value->type(), 0).isInt(), ("At ", *value));
                     VALIDATE(m_procedure.typeAtOffset(value->type(), 1) == registerType(), ("At ", *value));
                 }
 

Source/JavaScriptCore/b3/air/AirCCallingConvention.cpp

@@ -114,10 +114,13 @@ size_t cCallResultCount(Code& code, CCallValue* value)
             return 2;
         return 1;
     case Tuple:
-        // We only support tuples that return exactly two register sized ints.
+        // We only support functions that return each parameter in its own register for now.
         UNUSED_PARAM(code);
         ASSERT(code.proc().resultCount(value->type()) == 2);
-        ASSERT(code.proc().typeAtOffset(value->type(), 0) == pointerType());
+        if (is32Bit())
+            ASSERT(code.proc().typeAtOffset(value->type(), 0) == pointerType());
+        else
+            ASSERT(code.proc().typeAtOffset(value->type(), 0).isInt());
         ASSERT(code.proc().typeAtOffset(value->type(), 1) == pointerType());
         return 2;
     default:
@@ -171,7 +174,10 @@ Tmp cCallResult(Code& code, CCallValue* value, unsigned index)
     case Tuple:
         ASSERT_UNUSED(code, code.proc().resultCount(value->type()) == 2);
         // We only support functions that return each parameter in its own register for now.
-        ASSERT(code.proc().typeAtOffset(value->type(), 0) == registerType());
+        if (is32Bit())
+            ASSERT(code.proc().typeAtOffset(value->type(), 0) == registerType());
+        else
+            ASSERT(code.proc().typeAtOffset(value->type(), 0).isInt());
         ASSERT(code.proc().typeAtOffset(value->type(), 1) == registerType());
         return index ? Tmp(GPRInfo::returnValueGPR2) : Tmp(GPRInfo::returnValueGPR);
     case V128:

Source/JavaScriptCore/b3/testb3.h

@@ -68,6 +68,7 @@
 #include "JITCompilation.h"
 #include "JSCInlines.h"
 #include "LinkBuffer.h"
+#include "OperationResult.h"
 #include "PureNaN.h"
 #include <cmath>
 #include <regex>

Source/JavaScriptCore/b3/testb3_6.cpp

@@ -1613,8 +1613,8 @@ void testLateRegister()
     CHECK(invoke<uint64_t>(*code) == result);
 }
 
-JSC_DECLARE_JIT_OPERATION(interpreterPrint, void, (Vector<intptr_t>* stream, intptr_t value));
-JSC_DEFINE_JIT_OPERATION(interpreterPrint, void, (Vector<intptr_t>* stream, intptr_t value))
+JSC_DECLARE_NOEXCEPT_JIT_OPERATION(interpreterPrint, void, (Vector<intptr_t>* stream, intptr_t value));
+JSC_DEFINE_NOEXCEPT_JIT_OPERATION(interpreterPrint, void, (Vector<intptr_t>* stream, intptr_t value))
 {
     stream->append(value);
 }
@@ -2840,9 +2840,9 @@ void testMoveConstants()
 }
 
 extern "C" {
-static JSC_DECLARE_JIT_OPERATION_WITHOUT_WTF_INTERNAL(testMoveConstantsWithLargeOffsetsFunc, double, (double));
+static JSC_DECLARE_NOEXCEPT_JIT_OPERATION_WITHOUT_WTF_INTERNAL(testMoveConstantsWithLargeOffsetsFunc, double, (double));
 }
-JSC_DEFINE_JIT_OPERATION(testMoveConstantsWithLargeOffsetsFunc, double, (double a))
+JSC_DEFINE_NOEXCEPT_JIT_OPERATION(testMoveConstantsWithLargeOffsetsFunc, double, (double a))
 {
     return a;
 }

Source/JavaScriptCore/b3/testb3_7.cpp

@@ -587,18 +587,18 @@ void generateLoopNotBackwardsDominant(Procedure& proc, std::array<int, 100>& arr
 }
 
 extern "C" {
-static JSC_DECLARE_JIT_OPERATION_WITHOUT_WTF_INTERNAL(oneFunction, int, (int* callCount));
+static JSC_DECLARE_NOEXCEPT_JIT_OPERATION_WITHOUT_WTF_INTERNAL(oneFunction, int, (int* callCount));
 }
-JSC_DEFINE_JIT_OPERATION(oneFunction, int, (int* callCount))
+JSC_DEFINE_NOEXCEPT_JIT_OPERATION(oneFunction, int, (int* callCount))
 {
     (*callCount)++;
     return 1;
 }
 
 extern "C" {
-static JSC_DECLARE_JIT_OPERATION_WITHOUT_WTF_INTERNAL(noOpFunction, void, ());
+static JSC_DECLARE_NOEXCEPT_JIT_OPERATION_WITHOUT_WTF_INTERNAL(noOpFunction, void, ());
 }
-JSC_DEFINE_JIT_OPERATION(noOpFunction, void, ())
+JSC_DEFINE_NOEXCEPT_JIT_OPERATION(noOpFunction, void, ())
 {
 }
 
@@ -1415,9 +1415,9 @@ void testFloatEqualOrUnorderedDontFold()
 }
 
 extern "C" {
-static JSC_DECLARE_JIT_OPERATION_WITHOUT_WTF_INTERNAL(functionNineArgs, void, (int32_t, void*, void*, void*, void*, void*, void*, void*, void*));
+static JSC_DECLARE_NOEXCEPT_JIT_OPERATION_WITHOUT_WTF_INTERNAL(functionNineArgs, void, (int32_t, void*, void*, void*, void*, void*, void*, void*, void*));
 }
-JSC_DEFINE_JIT_OPERATION(functionNineArgs, void, (int32_t, void*, void*, void*, void*, void*, void*, void*, void*))
+JSC_DEFINE_NOEXCEPT_JIT_OPERATION(functionNineArgs, void, (int32_t, void*, void*, void*, void*, void*, void*, void*, void*))
 {
 }
 

Source/JavaScriptCore/dfg/DFGAbstractInterpreter.h

@@ -271,7 +271,7 @@ class AbstractInterpreter {
 
     void verifyEdge(Node*, Edge);
     void verifyEdges(Node*);
-    void executeDoubleUnaryOpEffects(Node*, double(*equivalentFunction)(double));
+    void executeDoubleUnaryOpEffects(Node*, const auto& functor);
 
     bool handleConstantDivOp(Node*);
 

Source/JavaScriptCore/dfg/DFGAbstractInterpreterInlines.h

@@ -1450,7 +1450,7 @@ bool AbstractInterpreter<AbstractStateType>::executeEffects(unsigned clobberLimi
     }
 
     case ArithSqrt:
-        executeDoubleUnaryOpEffects(node, sqrt);
+        executeDoubleUnaryOpEffects(node, [](double value) -> double { return sqrt(value); });
         break;
 
     case ArithFRound:
@@ -5504,7 +5504,7 @@ FiltrationResult AbstractInterpreter<AbstractStateType>::filterClassInfo(
 }
 
 template<typename AbstractStateType>
-void AbstractInterpreter<AbstractStateType>::executeDoubleUnaryOpEffects(Node* node, double(*equivalentFunction)(double))
+void AbstractInterpreter<AbstractStateType>::executeDoubleUnaryOpEffects(Node* node, const auto& equivalentFunction)
 {
     JSValue child = forNode(node->child1()).value();
     if (std::optional<double> number = child.toNumberFromPrimitive()) {

Source/JavaScriptCore/dfg/DFGArithMode.h

@@ -26,6 +26,7 @@
 #pragma once
 
 #include "MathCommon.h"
+#include "OperationResult.h"
 
 #if ENABLE(DFG_JIT)
 
@@ -62,7 +63,7 @@ enum class UnaryType : uint32_t {
 };
 
 using UnaryFunction = double(JIT_OPERATION_ATTRIBUTES*)(double);
-using UnaryOperation = double(JIT_OPERATION_ATTRIBUTES*)(JSGlobalObject*, EncodedJSValue);
+using UnaryOperation = OperationReturnType<double>(JIT_OPERATION_ATTRIBUTES*)(JSGlobalObject*, EncodedJSValue);
 
 } // namespace Arith
 

Source/JavaScriptCore/dfg/DFGArrayifySlowPathGenerator.h

@@ -114,7 +114,6 @@ class ArrayifySlowPathGenerator final : public JumpingSlowPathGenerator<MacroAss
         }
         for (unsigned i = m_plans.size(); i--;)
             jit->silentFill(m_plans[i]);
-        jit->exceptionCheck();
 
         if (m_op == ArrayifyToStructure) {
             ASSERT(m_structure.get());

Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp

@@ -4420,6 +4420,7 @@ bool ByteCodeParser::handleDOMJITGetter(Operand result, const GetByVariant& vari
     CallDOMGetterData* callDOMGetterData = m_graph.m_callDOMGetterData.add();
     callDOMGetterData->customAccessorGetter = variant.customAccessorGetter();
     ASSERT(callDOMGetterData->customAccessorGetter);
+    // JITOperationList::assertIsJITOperation(callDOMGetterData->customAccessorGetter);
     callDOMGetterData->requiredClassInfo = domAttribute->classInfo;
 
     if (const auto* domJIT = domAttribute->domJIT) {
@@ -5221,7 +5222,7 @@ void ByteCodeParser::handleGetById(
 #endif
     }
 
-    // Special path for custom accessors since custom's offset does not have any meanings.
+    // Special path for custom accessors since custom's offset does not have any meaning.
     // So, this is completely different from Simple one. But we have a chance to optimize it when we use DOMJIT.
     if (is64Bit()) {
         if (getByStatus.numVariants() == 1) {

Source/JavaScriptCore/dfg/DFGCallArrayAllocatorSlowPathGenerator.h

@@ -60,7 +60,6 @@ class CallArrayAllocatorSlowPathGenerator final : public JumpingSlowPathGenerato
         jit->callOperation(m_function, m_resultGPR, SpeculativeJIT::TrustedImmPtr(&jit->vm()), m_structure, m_size, m_storageGPR);
         for (unsigned i = m_plans.size(); i--;)
             jit->silentFill(m_plans[i]);
-        jit->exceptionCheck();
         jit->loadPtr(MacroAssembler::Address(m_resultGPR, JSObject::butterflyOffset()), m_storageGPR);
         jumpTo(jit);
     }
@@ -110,7 +109,6 @@ class CallArrayAllocatorWithVariableSizeSlowPathGenerator final : public Jumping
         jit->callOperation(m_function, m_resultGPR, m_globalObject, scratchGPR, m_sizeGPR, m_storageGPR);
         for (unsigned i = m_plans.size(); i--;)
             jit->silentFill(m_plans[i]);
-        jit->exceptionCheck();
         jumpTo(jit);
     }
 
@@ -150,7 +148,6 @@ class CallArrayAllocatorWithVariableStructureVariableSizeSlowPathGenerator final
         jit->callOperation(m_function, m_resultGPR, m_globalObject, m_structureGPR, m_sizeGPR, m_storageGPR);
         for (unsigned i = m_plans.size(); i--;)
             jit->silentFill(m_plans[i]);
-        jit->exceptionCheck();
         jumpTo(jit);
     }
 

Source/JavaScriptCore/dfg/DFGCallCreateDirectArgumentsSlowPathGenerator.h

@@ -61,7 +61,6 @@ class CallCreateDirectArgumentsSlowPathGenerator final : public JumpingSlowPathG
             operationCreateDirectArguments, m_resultGPR, SpeculativeJIT::TrustedImmPtr(&jit->vm()), m_structure, m_lengthGPR, m_minCapacity);
         for (unsigned i = m_plans.size(); i--;)
             jit->silentFill(m_plans[i]);
-        jit->exceptionCheck();
         jit->loadPtr(
             MacroAssembler::Address(m_resultGPR, DirectArguments::offsetOfLength()), m_lengthGPR);
         jumpTo(jit);

Source/JavaScriptCore/dfg/DFGJITCompiler.h

@@ -141,18 +141,27 @@ class JITCompiler : public CCallHelpers {
     }
 
     // Add a call out from JIT code, without an exception check.
-    Call appendCall(const CodePtr<CFunctionPtrTag> function)
+    template<PtrTag tag>
+    requires (tag != NoPtrTag)
+    Call appendCall(const CodePtr<tag> function)
     {
         Call functionCall = call(OperationPtrTag);
-        m_calls.append(CallLinkRecord(functionCall, function.retagged<OperationPtrTag>()));
+        // FIXME: If we had CustomGetters in JITOperationList we could just call retagged on all
+        // code paths but since we don't register them retagging triggers an ASSERT.
+        if constexpr (tag == OperationPtrTag)
+            m_calls.append(CallLinkRecord(functionCall, function));
+        else
+            m_calls.append(CallLinkRecord(functionCall, function.template retagged<OperationPtrTag>()));
         return functionCall;
     }
 
 #if OS(WINDOWS) && CPU(X86_64)
-    JITCompiler::Call appendCallWithUGPRPair(const CodePtr<CFunctionPtrTag> function)
+    template<PtrTag tag>
+    requires (tag != NoPtrTag)
+    JITCompiler::Call appendCallWithUGPRPair(const CodePtr<tag> function)
     {
         Call functionCall = callWithUGPRPair(OperationPtrTag);
-        m_calls.append(CallLinkRecord(functionCall, function.retagged<OperationPtrTag>()));
+        m_calls.append(CallLinkRecord(functionCall, function.template retagged<OperationPtrTag>()));
         return functionCall;
     }
 #endif