Skip to content
Permalink
Browse files
[JSC] Make DFG::OSRExit data unlinked
https://bugs.webkit.org/show_bug.cgi?id=239828

Reviewed by Saam Barati.

This patch makes DFG::OSRExit unlinked. While generated OSR exit code is linked version,
we no longer put linked data to DFG::OSRExit so that unlinked DFG can use DFG::OSRExit.
Key changes are two things.

1. Now, we always store compiled MacroAssemblerCodeRef in DFG::JITData regardless of whether
   we have linked / unlinked DFG. While linked DFG uses repatching to jump to this code,
   unlinked DFG looks into this vector in JITData and jump to that.
2. MethodOfGettingAValueProfile was including CodeBlock*, ValueProfile* in CodeBlock* etc.,
   so it was linked data structure which unlinked DFG cannot use. Instead, we encode how to
   retrieve these pointers when generating OSR exit code actually, and just storing CodeOrigin,
   type, and Operand to make MethodOfGettingAValueProfile unlinked data structure. While
   CodeOrigin can include InlineCallFrame, but our first version of unlinked DFG will not perform
   inlining thus we will not include it. It also makes sizeof(MethodOfGettingAValueProfile) smaller
   from 32 bytes to 16 bytes (50% reduction).

* Source/JavaScriptCore/assembler/MacroAssemblerCodeRef.h:
(JSC::MacroAssemblerCodeRef::offsetOfCodePtr):
* Source/JavaScriptCore/bytecode/CodeBlock.cpp:
(JSC::CodeBlock::updateOSRExitCounterAndCheckIfNeedToReoptimize): Deleted.
* Source/JavaScriptCore/bytecode/CodeBlock.h:
* Source/JavaScriptCore/bytecode/MethodOfGettingAValueProfile.cpp:
(JSC::MethodOfGettingAValueProfile::emitReportValue const):
(JSC::MethodOfGettingAValueProfile::fromLazyOperand): Deleted.
(JSC::MethodOfGettingAValueProfile::reportValue): Deleted.
* Source/JavaScriptCore/bytecode/MethodOfGettingAValueProfile.h:
(JSC::MethodOfGettingAValueProfile::unaryArithProfile):
(JSC::MethodOfGettingAValueProfile::binaryArithProfile):
(JSC::MethodOfGettingAValueProfile::argumentValueProfile):
(JSC::MethodOfGettingAValueProfile::bytecodeValueProfile):
(JSC::MethodOfGettingAValueProfile::lazyOperandValueProfile):
(JSC::MethodOfGettingAValueProfile::operator bool const):
(JSC::MethodOfGettingAValueProfile::MethodOfGettingAValueProfile): Deleted.
* Source/JavaScriptCore/dfg/DFGGraph.cpp:
(JSC::DFG::Graph::methodOfGettingAValueProfileFor):
* Source/JavaScriptCore/dfg/DFGJITCode.cpp:
(JSC::DFG::JITCode::JITCode):
(JSC::DFG::JITCode::findPC):
* Source/JavaScriptCore/dfg/DFGJITCode.h:
* Source/JavaScriptCore/dfg/DFGJITCompiler.cpp:
(JSC::DFG::JITCompiler::JITCompiler):
(JSC::DFG::JITCompiler::linkOSRExits):
(JSC::DFG::JITCompiler::link):
* Source/JavaScriptCore/dfg/DFGJITFinalizer.cpp:
(JSC::DFG::JITFinalizer::finalize):
* Source/JavaScriptCore/dfg/DFGOSRExit.cpp:
(JSC::DFG::JSC_DEFINE_JIT_OPERATION):
(JSC::DFG::OSRExit::compileExit):
(JSC::DFG::OSRExit::codeLocationForRepatch const): Deleted.
* Source/JavaScriptCore/dfg/DFGOSRExit.h:
(JSC::DFG::OSRExit::codeLocationForRepatch const):
(JSC::DFG::OSRExitState::OSRExitState): Deleted.
* Source/JavaScriptCore/dfg/DFGPlan.cpp:
(JSC::DFG::Plan::finalizeJITData):
* Source/JavaScriptCore/dfg/DFGPlan.h:
* Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp:
(JSC::FTL::DFG::LowerDFGToB3::lower):
* Source/JavaScriptCore/ftl/FTLOSRExitCompiler.cpp:
(JSC::FTL::compileStub):
* Source/WTF/wtf/FixedVector.h:

Canonical link: https://commits.webkit.org/250111@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@293605 268f45cc-cd09-0410-ab3c-d52691b4dbfc
  • Loading branch information
Constellation committed Apr 29, 2022
1 parent 10294b1 commit 6eaf4a53b9fd4000b762e0fb9e246025b2b58f91
Show file tree
Hide file tree
Showing 19 changed files with 271 additions and 258 deletions.
@@ -1,3 +1,69 @@
2022-04-27 Yusuke Suzuki <ysuzuki@apple.com>

[JSC] Make DFG::OSRExit data unlinked
https://bugs.webkit.org/show_bug.cgi?id=239828

Reviewed by Saam Barati.

This patch makes DFG::OSRExit unlinked. While generated OSR exit code is linked version,
we no longer put linked data to DFG::OSRExit so that unlinked DFG can use DFG::OSRExit.
Key changes are two things.

1. Now, we always store compiled MacroAssemblerCodeRef in DFG::JITData regardless of whether
we have linked / unlinked DFG. While linked DFG uses repatching to jump to this code,
unlinked DFG looks into this vector in JITData and jump to that.
2. MethodOfGettingAValueProfile was including CodeBlock*, ValueProfile* in CodeBlock* etc.,
so it was linked data structure which unlinked DFG cannot use. Instead, we encode how to
retrieve these pointers when generating OSR exit code actually, and just storing CodeOrigin,
type, and Operand to make MethodOfGettingAValueProfile unlinked data structure. While
CodeOrigin can include InlineCallFrame, but our first version of unlinked DFG will not perform
inlining thus we will not include it. It also makes sizeof(MethodOfGettingAValueProfile) smaller
from 32 bytes to 16 bytes (50% reduction).

* assembler/MacroAssemblerCodeRef.h:
(JSC::MacroAssemblerCodeRef::offsetOfCodePtr):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::updateOSRExitCounterAndCheckIfNeedToReoptimize): Deleted.
* bytecode/CodeBlock.h:
* bytecode/MethodOfGettingAValueProfile.cpp:
(JSC::MethodOfGettingAValueProfile::emitReportValue const):
(JSC::MethodOfGettingAValueProfile::fromLazyOperand): Deleted.
(JSC::MethodOfGettingAValueProfile::reportValue): Deleted.
* bytecode/MethodOfGettingAValueProfile.h:
(JSC::MethodOfGettingAValueProfile::unaryArithProfile):
(JSC::MethodOfGettingAValueProfile::binaryArithProfile):
(JSC::MethodOfGettingAValueProfile::argumentValueProfile):
(JSC::MethodOfGettingAValueProfile::bytecodeValueProfile):
(JSC::MethodOfGettingAValueProfile::lazyOperandValueProfile):
(JSC::MethodOfGettingAValueProfile::operator bool const):
(JSC::MethodOfGettingAValueProfile::MethodOfGettingAValueProfile): Deleted.
* dfg/DFGGraph.cpp:
(JSC::DFG::Graph::methodOfGettingAValueProfileFor):
* dfg/DFGJITCode.cpp:
(JSC::DFG::JITCode::JITCode):
(JSC::DFG::JITCode::findPC):
* dfg/DFGJITCode.h:
* dfg/DFGJITCompiler.cpp:
(JSC::DFG::JITCompiler::JITCompiler):
(JSC::DFG::JITCompiler::linkOSRExits):
(JSC::DFG::JITCompiler::link):
* dfg/DFGJITFinalizer.cpp:
(JSC::DFG::JITFinalizer::finalize):
* dfg/DFGOSRExit.cpp:
(JSC::DFG::JSC_DEFINE_JIT_OPERATION):
(JSC::DFG::OSRExit::compileExit):
(JSC::DFG::OSRExit::codeLocationForRepatch const): Deleted.
* dfg/DFGOSRExit.h:
(JSC::DFG::OSRExit::codeLocationForRepatch const):
(JSC::DFG::OSRExitState::OSRExitState): Deleted.
* dfg/DFGPlan.cpp:
(JSC::DFG::Plan::finalizeJITData):
* dfg/DFGPlan.h:
* ftl/FTLLowerDFGToB3.cpp:
(JSC::FTL::DFG::LowerDFGToB3::lower):
* ftl/FTLOSRExitCompiler.cpp:
(JSC::FTL::compileStub):

2022-04-28 Yusuke Suzuki <ysuzuki@apple.com>

Support C files in Unified Builds
@@ -526,6 +526,8 @@ class MacroAssemblerCodeRef : private MacroAssemblerCodeRefBase {
m_codePtr.dumpWithName("CodeRef", out);
}

static ptrdiff_t offsetOfCodePtr() { return OBJECT_OFFSETOF(MacroAssemblerCodeRef, m_codePtr); }

private:
template<PtrTag otherTag>
MacroAssemblerCodeRef(const MacroAssemblerCodeRef<otherTag>& otherCodeRef)
@@ -2610,55 +2610,6 @@ bool CodeBlock::checkIfOptimizationThresholdReached()
return m_jitExecuteCounter.checkIfThresholdCrossedAndSet(this);
}

#if ENABLE(DFG_JIT)
auto CodeBlock::updateOSRExitCounterAndCheckIfNeedToReoptimize(DFG::OSRExitState& exitState) -> OptimizeAction
{
DFG::OSRExitBase& exit = exitState.exit;
if (!exitKindMayJettison(exit.m_kind)) {
// FIXME: We may want to notice that we're frequently exiting
// at an op_catch that we didn't compile an entrypoint for, and
// then trigger a reoptimization of this CodeBlock:
// https://bugs.webkit.org/show_bug.cgi?id=175842
return OptimizeAction::None;
}

exit.m_count++;
m_osrExitCounter++;

CodeBlock* baselineCodeBlock = exitState.baselineCodeBlock;
ASSERT(baselineCodeBlock == baselineAlternative());
if (UNLIKELY(baselineCodeBlock->jitExecuteCounter().hasCrossedThreshold()))
return OptimizeAction::ReoptimizeNow;

// We want to figure out if there's a possibility that we're in a loop. For the outermost
// code block in the inline stack, we handle this appropriately by having the loop OSR trigger
// check the exit count of the replacement of the CodeBlock from which we are OSRing. The
// problem is the inlined functions, which might also have loops, but whose baseline versions
// don't know where to look for the exit count. Figure out if those loops are severe enough
// that we had tried to OSR enter. If so, then we should use the loop reoptimization trigger.
// Otherwise, we should use the normal reoptimization trigger.

bool didTryToEnterInLoop = false;
for (InlineCallFrame* inlineCallFrame = exit.m_codeOrigin.inlineCallFrame(); inlineCallFrame; inlineCallFrame = inlineCallFrame->directCaller.inlineCallFrame()) {
if (inlineCallFrame->baselineCodeBlock->ownerExecutable()->didTryToEnterInLoop()) {
didTryToEnterInLoop = true;
break;
}
}

uint32_t exitCountThreshold = didTryToEnterInLoop
? exitCountThresholdForReoptimizationFromLoop()
: exitCountThresholdForReoptimization();

if (m_osrExitCounter > exitCountThreshold)
return OptimizeAction::ReoptimizeNow;

// Too few fails. Adjust the execution counter such that the target is to only optimize after a while.
baselineCodeBlock->m_jitExecuteCounter.setNewThresholdForOSRExit(exitState.activeThreshold, exitState.memoryUsageAdjustedThreshold);
return OptimizeAction::None;
}
#endif

void CodeBlock::optimizeNextInvocation()
{
dataLogLnIf(Options::verboseOSR(), *this, ": Optimizing next invocation.");
@@ -80,7 +80,6 @@ namespace JSC {
#if ENABLE(DFG_JIT)
namespace DFG {
class JITData;
struct OSRExitState;
} // namespace DFG
#endif

@@ -706,11 +705,6 @@ class CodeBlock : public JSCell {

void countOSRExit() { m_osrExitCounter++; }

enum class OptimizeAction { None, ReoptimizeNow };
#if ENABLE(DFG_JIT)
OptimizeAction updateOSRExitCounterAndCheckIfNeedToReoptimize(DFG::OSRExitState&);
#endif

static ptrdiff_t offsetOfOSRExitCounter() { return OBJECT_OFFSETOF(CodeBlock, m_osrExitCounter); }

uint32_t adjustedExitCountThreshold(uint32_t desiredThreshold);
@@ -35,82 +35,52 @@

namespace JSC {

MethodOfGettingAValueProfile MethodOfGettingAValueProfile::fromLazyOperand(
CodeBlock* codeBlock, const LazyOperandValueProfileKey& key)
void MethodOfGettingAValueProfile::emitReportValue(CCallHelpers& jit, CodeBlock* optimizedCodeBlock, JSValueRegs regs, GPRReg tempGPR, TagRegistersMode mode) const
{
MethodOfGettingAValueProfile result;
result.m_kind = LazyOperand;
result.u.lazyOperand.codeBlock = codeBlock;
result.u.lazyOperand.bytecodeOffset = key.bytecodeIndex();
result.u.lazyOperand.operand = key.operand();
return result;
}
if (m_kind == Kind::None)
return;

void MethodOfGettingAValueProfile::emitReportValue(CCallHelpers& jit, JSValueRegs regs, GPRReg tempGPR, TagRegistersMode mode) const
{
CodeBlock* baselineCodeBlock = optimizedCodeBlock->baselineAlternative();
CodeBlock* profiledBlock = baselineCodeBlockForOriginAndBaselineCodeBlock(m_codeOrigin, baselineCodeBlock);
switch (m_kind) {
case None:
case Kind::None:
RELEASE_ASSERT_NOT_REACHED();
return;

case Ready:
jit.storeValue(regs, u.profile->specFailBucket(0));
return;

case LazyOperand: {
LazyOperandValueProfileKey key(u.lazyOperand.bytecodeOffset, u.lazyOperand.operand);
case Kind::LazyOperandValueProfile: {
LazyOperandValueProfileKey key(m_codeOrigin.bytecodeIndex(), Operand::fromBits(m_rawOperand));

ConcurrentJSLocker locker(u.lazyOperand.codeBlock->m_lock);
LazyOperandValueProfile* profile =
u.lazyOperand.codeBlock->lazyOperandValueProfiles(locker).add(locker, key);
ConcurrentJSLocker locker(profiledBlock->m_lock);
LazyOperandValueProfile* profile = profiledBlock->lazyOperandValueProfiles(locker).add(locker, key);
jit.storeValue(regs, profile->specFailBucket(0));
return;
}

case UnaryArithProfileReady: {
u.unaryArithProfile->emitObserveResult(jit, regs, tempGPR, mode);
case Kind::UnaryArithProfile: {
if (UnaryArithProfile* result = profiledBlock->unaryArithProfileForBytecodeIndex(m_codeOrigin.bytecodeIndex()))
result->emitObserveResult(jit, regs, tempGPR, mode);
return;
}

case BinaryArithProfileReady: {
u.binaryArithProfile->emitObserveResult(jit, regs, tempGPR, mode);
case Kind::BinaryArithProfile: {
if (BinaryArithProfile* result = profiledBlock->binaryArithProfileForBytecodeIndex(m_codeOrigin.bytecodeIndex()))
result->emitObserveResult(jit, regs, tempGPR, mode);
return;
}
}

RELEASE_ASSERT_NOT_REACHED();
}

void MethodOfGettingAValueProfile::reportValue(JSValue value)
{
switch (m_kind) {
case None:
return;

case Ready:
*u.profile->specFailBucket(0) = JSValue::encode(value);
return;

case LazyOperand: {
LazyOperandValueProfileKey key(u.lazyOperand.bytecodeOffset, u.lazyOperand.operand);

ConcurrentJSLocker locker(u.lazyOperand.codeBlock->m_lock);
LazyOperandValueProfile* profile =
u.lazyOperand.codeBlock->lazyOperandValueProfiles(locker).add(locker, key);
*profile->specFailBucket(0) = JSValue::encode(value);
case Kind::ArgumentValueProfile: {
auto& valueProfile = profiledBlock->valueProfileForArgument(Operand::fromBits(m_rawOperand).toArgument());
jit.storeValue(regs, valueProfile.specFailBucket(0));
return;
}

case UnaryArithProfileReady: {
u.unaryArithProfile->observeResult(value);
return;
}

case BinaryArithProfileReady: {
u.binaryArithProfile->observeResult(value);
case Kind::BytecodeValueProfile: {
auto& valueProfile = profiledBlock->valueProfileForBytecodeIndex(m_codeOrigin.bytecodeIndex());
jit.storeValue(regs, valueProfile.specFailBucket(0));
return;
}
}

RELEASE_ASSERT_NOT_REACHED();
}

@@ -32,6 +32,7 @@
#if ENABLE(DFG_JIT)

#include "BytecodeIndex.h"
#include "CodeOrigin.h"
#include "GPRInfo.h"
#include "Operands.h"
#include "TagRegistersMode.h"
@@ -42,76 +43,74 @@ class UnaryArithProfile;
class BinaryArithProfile;
class CCallHelpers;
class CodeBlock;
class LazyOperandValueProfileKey;
struct ValueProfile;

class MethodOfGettingAValueProfile {
public:
MethodOfGettingAValueProfile()
: m_kind(None)
MethodOfGettingAValueProfile() = default;

static MethodOfGettingAValueProfile unaryArithProfile(CodeOrigin codeOrigin)
{
MethodOfGettingAValueProfile result;
result.m_kind = Kind::UnaryArithProfile;
result.m_codeOrigin = codeOrigin;
return result;
}
MethodOfGettingAValueProfile(ValueProfile* profile)

static MethodOfGettingAValueProfile binaryArithProfile(CodeOrigin codeOrigin)
{
if (profile) {
m_kind = Ready;
u.profile = profile;
} else
m_kind = None;
MethodOfGettingAValueProfile result;
result.m_kind = Kind::BinaryArithProfile;
result.m_codeOrigin = codeOrigin;
return result;
}

MethodOfGettingAValueProfile(UnaryArithProfile* profile)
static MethodOfGettingAValueProfile argumentValueProfile(CodeOrigin codeOrigin, Operand operand)
{
if (profile) {
m_kind = UnaryArithProfileReady;
u.unaryArithProfile = profile;
} else
m_kind = None;
MethodOfGettingAValueProfile result;
result.m_kind = Kind::ArgumentValueProfile;
result.m_codeOrigin = codeOrigin;
result.m_rawOperand = operand.asBits();
return result;
}

MethodOfGettingAValueProfile(BinaryArithProfile* profile)
static MethodOfGettingAValueProfile bytecodeValueProfile(CodeOrigin codeOrigin)
{
if (profile) {
m_kind = BinaryArithProfileReady;
u.binaryArithProfile = profile;
} else
m_kind = None;
MethodOfGettingAValueProfile result;
result.m_kind = Kind::BytecodeValueProfile;
result.m_codeOrigin = codeOrigin;
return result;
}

static MethodOfGettingAValueProfile fromLazyOperand(
CodeBlock*, const LazyOperandValueProfileKey&);

explicit operator bool() const { return m_kind != None; }

static MethodOfGettingAValueProfile lazyOperandValueProfile(CodeOrigin codeOrigin, Operand operand)
{
MethodOfGettingAValueProfile result;
result.m_kind = Kind::LazyOperandValueProfile;
result.m_codeOrigin = codeOrigin;
result.m_rawOperand = operand.asBits();
return result;
}

explicit operator bool() const { return m_kind != Kind::None; }

// The temporary register is only needed on 64-bits builds (for testing BigInt32).
void emitReportValue(CCallHelpers&, JSValueRegs, GPRReg tempGPR, TagRegistersMode = HaveTagRegisters) const;
void reportValue(JSValue);
void emitReportValue(CCallHelpers&, CodeBlock* optimizedCodeBlock, JSValueRegs, GPRReg tempGPR, TagRegistersMode = HaveTagRegisters) const;

private:
enum Kind {
enum class Kind : uint8_t {
None,
Ready,
UnaryArithProfileReady,
BinaryArithProfileReady,
LazyOperand
UnaryArithProfile,
BinaryArithProfile,
BytecodeValueProfile,
ArgumentValueProfile,
LazyOperandValueProfile,
};

Kind m_kind;
union Data {
Data()
: profile(nullptr)
{ }

ValueProfile* profile;
UnaryArithProfile* unaryArithProfile;
BinaryArithProfile* binaryArithProfile;
struct {
CodeBlock* codeBlock;
BytecodeIndex bytecodeOffset;
Operand operand;
} lazyOperand;
} u;
static constexpr unsigned bitsOfKind = 3;
static_assert(static_cast<unsigned>(Kind::LazyOperandValueProfile) <= ((1U << bitsOfKind) - 1));

CodeOrigin m_codeOrigin;
uint64_t m_rawOperand : Operand::maxBits { 0 };
Kind m_kind : bitsOfKind { Kind::None };
};

} // namespace JSC

0 comments on commit 6eaf4a5

Please sign in to comment.