Skip to content
Permalink
Browse files
Add support for using Probe DFG OSR Exit behind a runtime flag.
https://bugs.webkit.org/show_bug.cgi?id=177844
<rdar://problem/34801425>

Reviewed by Saam Barati.

Source/JavaScriptCore:

This is based on the code originally posted in https://bugs.webkit.org/show_bug.cgi?id=175144
(in r221774 and r221832) with some optimizations and bug fixes added.  The probe
based DFG OSR Exit is only enabled if Options::useProbeOSRExit() is true.  We're
landing this behind an option switch to make it easier to tune performance using
the probe based OSR exit.

* JavaScriptCore.xcodeproj/project.pbxproj:
* assembler/MacroAssembler.cpp:
(JSC::stdFunctionCallback):
* assembler/MacroAssemblerPrinter.cpp:
(JSC::Printer::printCallback):
* assembler/ProbeContext.cpp:
(JSC::Probe::executeProbe):
(JSC::Probe::flushDirtyStackPages):
* assembler/ProbeContext.h:
(JSC::Probe::Context::Context):
(JSC::Probe::Context::arg):
* assembler/ProbeFrame.h: Added.
(JSC::Probe::Frame::Frame):
(JSC::Probe::Frame::argument):
(JSC::Probe::Frame::operand):
(JSC::Probe::Frame::setArgument):
(JSC::Probe::Frame::setOperand):
(JSC::Probe::Frame::get):
(JSC::Probe::Frame::set):
* assembler/ProbeStack.cpp:
(JSC::Probe::Page::lowWatermarkFromVisitingDirtyChunks):
(JSC::Probe::Stack::Stack):
(JSC::Probe::Stack::lowWatermarkFromVisitingDirtyPages):
* assembler/ProbeStack.h:
(JSC::Probe::Stack::Stack):
(JSC::Probe::Stack::lowWatermark):
(JSC::Probe::Stack::set):
(JSC::Probe::Stack::savedStackPointer const):
(JSC::Probe::Stack::setSavedStackPointer):
(JSC::Probe::Stack::newStackPointer const): Deleted.
(JSC::Probe::Stack::setNewStackPointer): Deleted.
* bytecode/ArrayProfile.h:
(JSC::ArrayProfile::observeArrayMode):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::updateOSRExitCounterAndCheckIfNeedToReoptimize):
* bytecode/CodeBlock.h:
(JSC::CodeBlock::addressOfOSRExitCounter): Deleted.
* bytecode/ExecutionCounter.h:
(JSC::ExecutionCounter::hasCrossedThreshold const):
(JSC::ExecutionCounter::setNewThresholdForOSRExit):
* bytecode/MethodOfGettingAValueProfile.cpp:
(JSC::MethodOfGettingAValueProfile::reportValue):
* bytecode/MethodOfGettingAValueProfile.h:
* dfg/DFGDriver.cpp:
(JSC::DFG::compileImpl):
* dfg/DFGJITCompiler.cpp:
(JSC::DFG::JITCompiler::linkOSRExits):
(JSC::DFG::JITCompiler::link):
* dfg/DFGOSRExit.cpp:
(JSC::DFG::jsValueFor):
(JSC::DFG::restoreCalleeSavesFor):
(JSC::DFG::saveCalleeSavesFor):
(JSC::DFG::restoreCalleeSavesFromVMEntryFrameCalleeSavesBuffer):
(JSC::DFG::copyCalleeSavesToVMEntryFrameCalleeSavesBuffer):
(JSC::DFG::saveOrCopyCalleeSavesFor):
(JSC::DFG::createDirectArgumentsDuringExit):
(JSC::DFG::createClonedArgumentsDuringExit):
(JSC::DFG::emitRestoreArguments):
(JSC::DFG::OSRExit::executeOSRExit):
(JSC::DFG::reifyInlinedCallFrames):
(JSC::DFG::adjustAndJumpToTarget):
(JSC::DFG::printOSRExit):
* dfg/DFGOSRExit.h:
(JSC::DFG::OSRExitState::OSRExitState):
* dfg/DFGThunks.cpp:
(JSC::DFG::osrExitThunkGenerator):
* dfg/DFGThunks.h:
* dfg/DFGVariableEventStream.cpp:
(JSC::DFG::tryToSetConstantRecovery):
(JSC::DFG::VariableEventStream::reconstruct const):
(JSC::DFG::VariableEventStream::tryToSetConstantRecovery const): Deleted.
* dfg/DFGVariableEventStream.h:
* profiler/ProfilerOSRExit.h:
(JSC::Profiler::OSRExit::incCount):
* runtime/JSCJSValue.h:
* runtime/JSCJSValueInlines.h:
* runtime/Options.h:

Tools:

Enable --useProbeOSrExit=true for dfg-eager and ftl-no-cjit-validate-sampling-profiler
test configurations.

* Scripts/run-jsc-stress-tests:



Canonical link: https://commits.webkit.org/194150@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@222871 268f45cc-cd09-0410-ab3c-d52691b4dbfc
  • Loading branch information
Mark Lam committed Oct 4, 2017
1 parent 8f6165d commit b6678ad5edefcb26d70afd6b01c94484e3c94f1c
Showing 29 changed files with 1,382 additions and 70 deletions.
@@ -1,3 +1,95 @@
2017-10-04 Mark Lam <mark.lam@apple.com>

Add support for using Probe DFG OSR Exit behind a runtime flag.
https://bugs.webkit.org/show_bug.cgi?id=177844
<rdar://problem/34801425>

Reviewed by Saam Barati.

This is based on the code originally posted in https://bugs.webkit.org/show_bug.cgi?id=175144
(in r221774 and r221832) with some optimizations and bug fixes added. The probe
based DFG OSR Exit is only enabled if Options::useProbeOSRExit() is true. We're
landing this behind an option switch to make it easier to tune performance using
the probe based OSR exit.

* JavaScriptCore.xcodeproj/project.pbxproj:
* assembler/MacroAssembler.cpp:
(JSC::stdFunctionCallback):
* assembler/MacroAssemblerPrinter.cpp:
(JSC::Printer::printCallback):
* assembler/ProbeContext.cpp:
(JSC::Probe::executeProbe):
(JSC::Probe::flushDirtyStackPages):
* assembler/ProbeContext.h:
(JSC::Probe::Context::Context):
(JSC::Probe::Context::arg):
* assembler/ProbeFrame.h: Added.
(JSC::Probe::Frame::Frame):
(JSC::Probe::Frame::argument):
(JSC::Probe::Frame::operand):
(JSC::Probe::Frame::setArgument):
(JSC::Probe::Frame::setOperand):
(JSC::Probe::Frame::get):
(JSC::Probe::Frame::set):
* assembler/ProbeStack.cpp:
(JSC::Probe::Page::lowWatermarkFromVisitingDirtyChunks):
(JSC::Probe::Stack::Stack):
(JSC::Probe::Stack::lowWatermarkFromVisitingDirtyPages):
* assembler/ProbeStack.h:
(JSC::Probe::Stack::Stack):
(JSC::Probe::Stack::lowWatermark):
(JSC::Probe::Stack::set):
(JSC::Probe::Stack::savedStackPointer const):
(JSC::Probe::Stack::setSavedStackPointer):
(JSC::Probe::Stack::newStackPointer const): Deleted.
(JSC::Probe::Stack::setNewStackPointer): Deleted.
* bytecode/ArrayProfile.h:
(JSC::ArrayProfile::observeArrayMode):
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::updateOSRExitCounterAndCheckIfNeedToReoptimize):
* bytecode/CodeBlock.h:
(JSC::CodeBlock::addressOfOSRExitCounter): Deleted.
* bytecode/ExecutionCounter.h:
(JSC::ExecutionCounter::hasCrossedThreshold const):
(JSC::ExecutionCounter::setNewThresholdForOSRExit):
* bytecode/MethodOfGettingAValueProfile.cpp:
(JSC::MethodOfGettingAValueProfile::reportValue):
* bytecode/MethodOfGettingAValueProfile.h:
* dfg/DFGDriver.cpp:
(JSC::DFG::compileImpl):
* dfg/DFGJITCompiler.cpp:
(JSC::DFG::JITCompiler::linkOSRExits):
(JSC::DFG::JITCompiler::link):
* dfg/DFGOSRExit.cpp:
(JSC::DFG::jsValueFor):
(JSC::DFG::restoreCalleeSavesFor):
(JSC::DFG::saveCalleeSavesFor):
(JSC::DFG::restoreCalleeSavesFromVMEntryFrameCalleeSavesBuffer):
(JSC::DFG::copyCalleeSavesToVMEntryFrameCalleeSavesBuffer):
(JSC::DFG::saveOrCopyCalleeSavesFor):
(JSC::DFG::createDirectArgumentsDuringExit):
(JSC::DFG::createClonedArgumentsDuringExit):
(JSC::DFG::emitRestoreArguments):
(JSC::DFG::OSRExit::executeOSRExit):
(JSC::DFG::reifyInlinedCallFrames):
(JSC::DFG::adjustAndJumpToTarget):
(JSC::DFG::printOSRExit):
* dfg/DFGOSRExit.h:
(JSC::DFG::OSRExitState::OSRExitState):
* dfg/DFGThunks.cpp:
(JSC::DFG::osrExitThunkGenerator):
* dfg/DFGThunks.h:
* dfg/DFGVariableEventStream.cpp:
(JSC::DFG::tryToSetConstantRecovery):
(JSC::DFG::VariableEventStream::reconstruct const):
(JSC::DFG::VariableEventStream::tryToSetConstantRecovery const): Deleted.
* dfg/DFGVariableEventStream.h:
* profiler/ProfilerOSRExit.h:
(JSC::Profiler::OSRExit::incCount):
* runtime/JSCJSValue.h:
* runtime/JSCJSValueInlines.h:
* runtime/Options.h:

2017-10-04 Ryan Haddad <ryanhaddad@apple.com>

Unreviewed, rolling out r222840.
@@ -4596,6 +4596,7 @@
FEA0C4001CDD7D0E00481991 /* FunctionWhitelist.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = FunctionWhitelist.cpp; sourceTree = "<group>"; };
FEA0C4011CDD7D0E00481991 /* FunctionWhitelist.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = FunctionWhitelist.h; sourceTree = "<group>"; };
FEB137561BB11EEE00CD5100 /* MacroAssemblerARM64.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = MacroAssemblerARM64.cpp; sourceTree = "<group>"; };
FEB41CCB1F73284200C5481E /* ProbeFrame.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ProbeFrame.h; sourceTree = "<group>"; };
FEB51F6A1A97B688001F921C /* Regress141809.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = Regress141809.h; path = API/tests/Regress141809.h; sourceTree = "<group>"; };
FEB51F6B1A97B688001F921C /* Regress141809.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; name = Regress141809.mm; path = API/tests/Regress141809.mm; sourceTree = "<group>"; };
FEB58C12187B8B160098EF0B /* ErrorHandlingScope.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ErrorHandlingScope.cpp; sourceTree = "<group>"; };
@@ -7295,6 +7296,7 @@
FE63DD531EA9B60E00103A69 /* Printer.h */,
FE10AAF31F46826D009DEDC5 /* ProbeContext.cpp */,
FE10AAED1F44D946009DEDC5 /* ProbeContext.h */,
FEB41CCB1F73284200C5481E /* ProbeFrame.h */,
FE10AAE91F44D510009DEDC5 /* ProbeStack.cpp */,
FE10AAEA1F44D512009DEDC5 /* ProbeStack.h */,
FE533CA01F217C310016A1FE /* testmasm.cpp */,
@@ -38,7 +38,7 @@ const double MacroAssembler::twoToThe32 = (double)0x100000000ull;
#if ENABLE(MASM_PROBE)
static void stdFunctionCallback(Probe::Context& context)
{
auto func = static_cast<const std::function<void(Probe::Context&)>*>(context.arg);
auto func = context.arg<const std::function<void(Probe::Context&)>*>();
(*func)(context);
}

@@ -175,7 +175,7 @@ void printMemory(PrintStream& out, Context& context)
void printCallback(Probe::Context& probeContext)
{
auto& out = WTF::dataFile();
PrintRecordList& list = *reinterpret_cast<PrintRecordList*>(probeContext.arg);
PrintRecordList& list = *probeContext.arg<PrintRecordList*>();
for (size_t i = 0; i < list.size(); i++) {
auto& record = list[i];
Context context(probeContext, record.data);
@@ -52,8 +52,9 @@ void executeProbe(State* state)
#endif

if (context.hasWritesToFlush()) {
context.stack().setNewStackPointer(state->cpu.sp());
state->cpu.sp() = std::min(context.stack().lowWatermark(), state->cpu.sp());
context.stack().setSavedStackPointer(state->cpu.sp());
void* lowWatermark = context.stack().lowWatermark(state->cpu.sp());
state->cpu.sp() = std::min(lowWatermark, state->cpu.sp());

state->initializeStackFunction = flushDirtyStackPages;
state->initializeStackArg = context.releaseStack();
@@ -64,7 +65,7 @@ static void flushDirtyStackPages(State* state)
{
std::unique_ptr<Stack> stack(reinterpret_cast<Probe::Stack*>(state->initializeStackArg));
stack->flushWrites();
state->cpu.sp() = stack->newStackPointer();
state->cpu.sp() = stack->savedStackPointer();
}

// Not for general use. This should only be for writing tests.
@@ -191,11 +191,13 @@ class Context {
using FPRegisterID = MacroAssembler::FPRegisterID;

Context(State* state)
: m_state(state)
, arg(state->arg)
, cpu(state->cpu)
: cpu(state->cpu)
, m_state(state)
{ }

template<typename T>
T arg() { return reinterpret_cast<T>(m_state->arg); }

uintptr_t& gpr(RegisterID id) { return cpu.gpr(id); }
uintptr_t& spr(SPRegisterID id) { return cpu.spr(id); }
double& fpr(FPRegisterID id) { return cpu.fpr(id); }
@@ -224,13 +226,10 @@ class Context {
bool hasWritesToFlush() { return m_stack.hasWritesToFlush(); }
Stack* releaseStack() { return new Stack(WTFMove(m_stack)); }

private:
State* m_state;
public:
void* arg;
CPUState& cpu;

private:
State* m_state;
Stack m_stack;

friend JS_EXPORT_PRIVATE void* probeStateForContext(Context&); // Not for general use. This should only be for writing tests.
@@ -0,0 +1,94 @@
/*
* Copyright (C) 2017 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/

#pragma once

#if ENABLE(MASM_PROBE)

#include "CallFrame.h"
#include "ProbeStack.h"

namespace JSC {
namespace Probe {

class Frame {
public:
Frame(void* frameBase, Stack& stack)
: m_frameBase { reinterpret_cast<uint8_t*>(frameBase) }
, m_stack { stack }
{ }

template<typename T = JSValue>
T argument(int argument)
{
return get<T>(CallFrame::argumentOffset(argument) * sizeof(Register));
}
template<typename T = JSValue>
T operand(int operand)
{
return get<T>(static_cast<VirtualRegister>(operand).offset() * sizeof(Register));
}
template<typename T = JSValue>
T operand(int operand, ptrdiff_t offset)
{
return get<T>(static_cast<VirtualRegister>(operand).offset() * sizeof(Register) + offset);
}

template<typename T>
void setArgument(int argument, T value)
{
return set<T>(CallFrame::argumentOffset(argument) * sizeof(Register), value);
}
template<typename T>
void setOperand(int operand, T value)
{
set<T>(static_cast<VirtualRegister>(operand).offset() * sizeof(Register), value);
}
template<typename T>
void setOperand(int operand, ptrdiff_t offset, T value)
{
set<T>(static_cast<VirtualRegister>(operand).offset() * sizeof(Register) + offset, value);
}

template<typename T = JSValue>
T get(ptrdiff_t offset)
{
return m_stack.get<T>(m_frameBase + offset);
}
template<typename T>
void set(ptrdiff_t offset, T value)
{
m_stack.set<T>(m_frameBase + offset, value);
}

private:
uint8_t* m_frameBase;
Stack& m_stack;
};

} // namespace Probe
} // namespace JSC

#endif // ENABLE(MASM_PROBE)
@@ -34,6 +34,8 @@
namespace JSC {
namespace Probe {

static void* const maxLowWatermark = reinterpret_cast<void*>(std::numeric_limits<uintptr_t>::max());

#if ASAN_ENABLED
// FIXME: we should consider using the copy function for both ASan and non-ASan builds.
// https://bugs.webkit.org/show_bug.cgi?id=176961
@@ -49,7 +51,7 @@ static void copyStackPage(void* dst, void* src, size_t size)
*dstPointer++ = *srcPointer++;
}
#else
#define copyStackPage(dst, src, size) std::memcpy(dst, src, size);
#define copyStackPage(dst, src, size) std::memcpy(dst, src, size)
#endif

Page::Page(void* baseAddress)
@@ -84,12 +86,24 @@ void Page::flushWrites()
m_dirtyBits = 0;
}

void* Page::lowWatermarkFromVisitingDirtyChunks()
{
uint64_t dirtyBits = m_dirtyBits;
size_t offset = 0;
while (dirtyBits) {
if (dirtyBits & 1)
return reinterpret_cast<uint8_t*>(m_baseLogicalAddress) + offset;
dirtyBits = dirtyBits >> 1;
offset += s_chunkSize;
}
return maxLowWatermark;
}

Stack::Stack(Stack&& other)
: m_newStackPointer(other.m_newStackPointer)
, m_lowWatermark(other.m_lowWatermark)
, m_stackBounds(WTFMove(other.m_stackBounds))
: m_stackBounds(WTFMove(other.m_stackBounds))
, m_pages(WTFMove(other.m_pages))
{
m_savedStackPointer = other.m_savedStackPointer;
#if !ASSERT_DISABLED
other.m_isValid = false;
#endif
@@ -128,6 +142,18 @@ Page* Stack::ensurePageFor(void* address)
return m_lastAccessedPage;
}

void* Stack::lowWatermarkFromVisitingDirtyPages()
{
void* low = maxLowWatermark;
for (auto it = m_pages.begin(); it != m_pages.end(); ++it) {
Page& page = *it->value;
if (!page.hasWritesToFlush() || low < page.baseAddress())
continue;
low = std::min(low, page.lowWatermarkFromVisitingDirtyChunks());
}
return low;
}

} // namespace Probe
} // namespace JSC

0 comments on commit b6678ad

Please sign in to comment.