Skip to content

Fix segfault in AyonApi::serialCorePost on failed requests - #53

Open
isohedronpipeline wants to merge 5 commits into
ynput:developfrom
isohedronpipeline:bugfix/52-serialcorepost-null-deref
Open

Fix segfault in AyonApi::serialCorePost on failed requests#53
isohedronpipeline wants to merge 5 commits into
ynput:developfrom
isohedronpipeline:bugfix/52-serialcorepost-null-deref

Conversation

@isohedronpipeline

Copy link
Copy Markdown

Fix segfault in AyonApi::serialCorePost on failed requests

Fixes #52

Summary

AyonApi::serialCorePost dereferences the httplib::Result returned by
m_ayonServer->Post(...) without checking whether the request actually
succeeded:

response = m_ayonServer->Post(endPoint, headers, Payload, "application/json");
responseStatus = response->status;   // <-- crashes if response is null

cpp-httplib does not throw a C++ exception when a request fails to
connect or complete (connection refused, TLS failure, timeout, etc.) - it
returns a null/empty Result. The existing catch (const httplib::Error&)
block a few lines down never fires for this case, so any failed POST
segfaults immediately on a null-pointer dereference.

The sibling GET-based code path (used by getSiteRoots()) already
guards against exactly this with an explicit null check and a clean
"response is null" log line. serialCorePost is missing the
equivalent guard.

Impact

Reproduced as a 100%-reproducible Houdini crash: any transient
connectivity issue between the client and the AYON server during a
POST-based resolvePath() call hard-crashes the host DCC (confirmed in
Houdini 21) instead of failing gracefully like the GET path does. Since
the failure only needs to happen once during any serialCorePost call,
this affects any studio hitting a network blip, VPN hiccup, server
restart, or misconfiguration during a resolve - not something specific to
one environment.

What this PR changes

1. Null-check before dereferencing the response (the core fix):

response = m_ayonServer->Post(endPoint, headers, Payload, "application/json");
if (!response) {
    auto err = response.error();
    m_log->warn("AyonApi::serialCorePost response is null: {}", httplib::to_string(err));
    ...
    retries++;
    std::this_thread::sleep_for(std::chrono::milliseconds(m_retryWait));
    continue;
}
responseStatus = response->status;

2. Smarter handling specifically for httplib::Error::SSLServerVerification

While diagnosing this, we found cpp-httplib's SSLClient::load_certs()
has its own latent issue: it wraps CA-cert loading in a std::call_once,
but the bool ret it returns is a fresh local variable on every call - so
if certificate loading genuinely fails once, that failure is silently
reported as success on every subsequent call for that client object's
entire lifetime (since the call_once body never runs again). Combined
with AyonApi holding one long-lived, keep-alive httplib::Client for
the whole session, a single bad first attempt can permanently poison
verification for that object.

Rather than patch vendored cpp-httplib, we detect this specific error
and validate whether retrying is actually worthwhile before doing so:

if (err == httplib::Error::SSLServerVerification) {
    if (m_caCertPath.empty() || !std::filesystem::exists(m_caCertPath) ||
        !std::filesystem::is_regular_file(m_caCertPath)) {
        m_log->error("...cert path is invalid: '{}' - not retrying", m_caCertPath);
        return "";   // fail fast - retrying can't help a broken config
    }
    // cert path looks fine - rebuild the client for a genuine fresh attempt
    m_ayonServer = std::make_unique<httplib::Client>(m_serverUrl);
    m_ayonServer->set_keep_alive(true);
    m_ayonServer->set_ca_cert_path(m_caCertPath.c_str());
    m_ayonServer->enable_server_certificate_verification(true);
}

This avoids two failure modes: wasting the full retry budget when the
cert configuration is genuinely broken (fails fast with one clear log
line instead), and getting permanently stuck retrying against an already
provably-poisoned client object when the cert config is actually fine (a
fresh client gets a genuine second chance).

3. New m_caCertPath member

Needed to support the rebuild above - the winning cert path (from
whichever of the constructor/setSSL() branches determined it) is now
recorded on the instance instead of being used once and discarded.

4. CMakeLists.txt default fix

option(USE_OPENSSL3 "Build against OpenSSL 3.x" OFF) - this vendored
cpp-httplib hard-errors at compile time on any OpenSSL version below
3.0.0, so OFF cannot currently produce a working build under any
configuration. Flipped the default to ON to match reality; anyone
building this repo standalone (via AyonBuild.py, not through
ayon-usd-resolver's build_resolver.py, which already passes
-DUSE_OPENSSL3=ON explicitly) previously hit a confusing compile error
with no indication the fix was a one-line CMake flag.

Testing

  • Built standalone via AyonBuild.py --runStageGRP CleanBuild - compiles
    cleanly with all changes.
  • Built the full ayon-usd-resolver plugin against Houdini 21 with this
    patched ayon-cpp-api as its submodule and verified in a live Houdini
    session: the previously 100%-reproducible crash no longer occurs, and a
    deliberately-triggered SSLServerVerification failure now retries
    cleanly instead of crashing.
  • Verified the retry/rebuild logic's decision correctly distinguishes a
    genuinely invalid cert path (fails fast) from a valid one (retries)
    using targeted manual tests against both cases.

tadeas-hejnic and others added 5 commits March 16, 2026 16:14
…onflict

Release/resolve develop main conflict
serialCorePost dereferences the httplib::Result from Post() without
checking whether the request actually succeeded. cpp-httplib returns
a null Result rather than throwing when a request fails to connect
or complete, so any failed POST segfaults on a null-pointer read
instead of failing gracefully like the sibling GET path does.

Fixes ynput#52
@tadeas-hejnic
tadeas-hejnic changed the base branch from main to develop July 30, 2026 15:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] serialCorePost segfaults on a failed request instead of failing gracefully

2 participants