Error handling docs, switch to APIException, and add function for more fun stuff

Author: LunarWatcher

docs/Error-handling.md

@@ -0,0 +1,55 @@
+# Error handling
+
+For the most part, the library tries to deal with most of the error handling, as well as backoff.
+
+It's worth noting that not all errors are handled at this time, but that the amount of handled issues will grow as the library does.
+
+The `APIConfig` Struct contains two config variables for error handling; `autoHandleBackoff` and `autoHandleDowntime`. If these are set to false, stackapi.cpp throws when these types of errors are encountered.
+
+If they're set to true, the library sleeps for a few minutes, and then re-tries the last invoked request. **This means the main thread will be blocked** if either of these options are true, and the corresponding type of error occurs. If you run the API calls on the main thread, alongside a user interface, this means the UI will be blocked, and you'll likely want to set these options to false. In turn, you need to handle errors manually.
+
+## Automatic error handling
+
+If you don't disable config variables, many errors are automatically handled. At the time of writing, this is with the exception of specific errors returned by the API, caused by incorrect API calls (mainly programmer errors), as these primarily occur during development. Dealing with these kinds of errors are documented in the manual error handling section.
+
+For automatic error handling, there's naturally not much to do. Don't set the config variables to false, and let the library do its thing.
+
+However, in some cases, you may still want to do extra handling of these errors. For this, there's a configurable callback; `APIConfig::errorCallback`.
+
+This callback is invoked for several reasons:
+
+1. Proper errors; non-200 status codes (Cloudflare blocks the request, backoff violations, downtime, internet outage, etc.)
+2. Other waits. At the time of writing, this only applies to respecting backoff before calling an API endpoint. These aren't hard errors, but if you're developing a user-oriented program, this can be used to let the user know they need to wait before the request can be made. 
+3. Service restauration after the previous points; used to signal that whatever was previously reported has been resolved.
+
+\#3 is a very specific use-case for particularly bots that keep track of whether or not a new error has happened to avoid double-reporting errors. It can also be used to do event-driven updates for the user to avoid having handling on every request made. However, the exact handling is up to each consumer application.
+
+## Manual error handling
+
+If automatic error handling is either disabled or unavailable for a given type of error, you'll get exceptions. For the most part, you'll get an `stackapi::APIException`[^1], which contains a few fields that can be used for manual error handling:
+* message: A custom message supplied by stackchat
+* errorMessage: the full response from the API endpoint. May or may not be JSON, may or may not say something useful
+* statusCode: Either the HTTP status code, a modified HTTP status code (HTTP 400 becomes HTTP 500 if the respons isn't JSON; this is the only modified HTTP status code), or `error_id` parsed from the JSON response. 
+* isHTTPStatus: Whether or not the statusCode is an HTTP status code, or an error_id corresponding to one of the API error codes.
+
+See the [/errors](https://api.stackexchange.com/docs/errors) endpoint for documentation on values for `error_id`. Note that if you're using automatic error handling, 500, 502, and 503 are automatically handled, and don't require any custom logic. 
+
+The use of exceptions does require the use of try-catches in the code, but it also (in my biased opinion) simplifies certain parts of the handling. For example, if you're using an endpoint that requires an access_token, and you want to handle access token errors, you can use:
+```cpp
+try {
+    // Note that the NullStruct used here is purely because I don't care enough to look up the return value of this endpoint
+    // In real code, you probably want to check the return value
+    api.post<stackapi::NullStruct>("questions/1234/upvote");
+} catch (APIException e) {
+    if (!e.isHTTPStatus && e.statusCode >= 401 && e.statusCode <= 403) {
+        // Notify the user that their token is bad
+        // Note that you can parse e.errorMessage as JSON to get error_message from it, and append it to the message
+
+    } else {
+        // Unhandled; either handle it or just throw, or ignore it if you prefer
+        throw;
+    }
+}
+```
+
+[^1]: At the time of writing, this is the only exception thrown. Some parts of the library throw std::runtime_error, but these are not in response to API calls, and are more often than not programmer errors.

docs/Getting-started.md

@@ -54,3 +54,4 @@ Due to SSL being used, Linux users have to install libssl-dev (Debian and deriva
 ## After installing
 
 You should now have access to the library. See the [general use document](General-use.md) for getting started with the code, or take a look at the demos.
+

docs/README.md

@@ -5,3 +5,4 @@ Because GitHub, this is a pseudo-chapter list:
 1. [Getting started](Getting-started.md)
 2. [General use](General-use.md)
 3. [Auth](Auth.md)
+4. [Error handling](Error-handling.md)

src/stackapi/StackAPI.cpp

@@ -1,4 +1,5 @@
 #include "StackAPI.hpp"
+#include "stackapi/errors/APIException.hpp"
 
 #include <stdexcept>
 
@@ -18,6 +19,7 @@ nlohmann::json StackAPI::postRaw(const std::string& dest,
     if (dryRun) {
         return {};
     }
+    bool faulted = false;
     do {
         if (opt.autoHandleBackoff.value_or(conf.autoHandleBackoff)) {
             checkBackoff();
@@ -44,8 +46,12 @@ nlohmann::json StackAPI::postRaw(const std::string& dest,
         auto res = sess.Post(url, body, conf.userAgent);
 
         if (auto jsonOpt = checkErrors(res, opt)) {
+            if (faulted) {
+                conf.reportError(FaultType::RESTORED, "");
+            }
             return *jsonOpt;
         }
+        faulted = true;
 
     } while (true);
 }
@@ -54,6 +60,8 @@ nlohmann::json StackAPI::getRaw(const std::string &dest,
                                 const std::map<std::string, std::string>& extraParams,
                                 const APIConfigOpt& opt) {
 
+    bool faulted = false;
+
     do {
 
         if (opt.autoHandleBackoff.value_or(conf.autoHandleBackoff)) {
@@ -81,8 +89,12 @@ nlohmann::json StackAPI::getRaw(const std::string &dest,
         auto res = sess.Get(url, body, conf.userAgent);
 
         if (auto jsonOpt = checkErrors(res, opt)) {
+            if (faulted) {
+                conf.reportError(FaultType::RESTORED, "");
+            }
             return *jsonOpt;
         }
+        faulted = true;
     } while (true);
 
 }
@@ -100,6 +112,7 @@ void StackAPI::checkBackoff() {
 
         auto wait = backoff.secs - consumed + 2 * conf.backoffStrictnessMultiplier;
         if (wait > 0) {
+            this->conf.reportError(FaultType::RATE_LIMIT, "Backoff instructed; waiting " + std::to_string(wait) + " seconds before sending request...");
             std::this_thread::sleep_for(std::chrono::seconds(wait));
         }
 
@@ -109,9 +122,11 @@ void StackAPI::checkBackoff() {
 
 std::optional<nlohmann::json> StackAPI::checkErrors(cpr::Response& res, const APIConfigOpt& opt) {
     auto status_code = res.status_code;
+    bool isHTTPStatus = true;
     if (status_code == 400) { 
         try {
             status_code = nlohmann::json::parse(res.text).at("error_id");
+            isHTTPStatus = false;
         } catch (...) {
             // "json" is HTML
             status_code = 500;
@@ -122,34 +137,38 @@ std::optional<nlohmann::json> StackAPI::checkErrors(cpr::Response& res, const AP
     case 0: {
         if (opt.autoHandleDowntime.value_or(conf.autoHandleDowntime)) {
             spdlog::warn("Host disconnected from the internet. Sleeping 5 minutes...");
+            conf.reportError(FaultType::DOWNTIME, "Host seems to be disconnected from the internet. Retrying in 5 minutes");
             std::this_thread::sleep_for(std::chrono::minutes(5));
         } else {
-            throw std::runtime_error("Connection failed or internet dead (probably the latter): " + res.text + "; " + res.error.message);
+            throw APIException("Connection failed, or your internet connection is down", res.text, res.url.str(), res.status_code, isHTTPStatus);
         }
     } break;
     case 429:
         // Cloudflare bullshit
         if (opt.autoHandleBackoff.value_or(conf.autoHandleBackoff)) {
             spdlog::warn("Cloudflare says no. Sleeping for 5 minutes...");
+            conf.reportError(FaultType::CLOUDFLARE, "Cloudflare has blocked your IP. Retrying in 5 minutes.");
             std::this_thread::sleep_for(std::chrono::minutes(5));
         } else {
-            throw std::runtime_error("Cloudflare backoff");
+            throw APIException("Cloudflare blocked the request", res.text, res.url.str(), res.status_code, isHTTPStatus);
         }
     case 500:
     case 503: {
         if (opt.autoHandleDowntime.value_or(conf.autoHandleDowntime)) {
             spdlog::warn("Stack is down. Sleeping 5 minutes...");
+            conf.reportError(FaultType::DOWNTIME, "Stack's API appears to be down. Retrying in 5 minutes");
             std::this_thread::sleep_for(std::chrono::minutes(5));
         } else {
-            throw std::runtime_error("Stack's servers are down: " + res.text + "; " + res.error.message);
+            throw APIException("Stack's servers appear to be down", res.text, res.url.str(), res.status_code, isHTTPStatus);
         }
     } break;
     case 502: {
         if (opt.autoHandleBackoff.value_or(conf.autoHandleBackoff)) {
             spdlog::warn("Backoff violated. Sleeping for 5 minutes...");
+            conf.reportError(FaultType::RATE_LIMIT, "Backoff violated - sleeping before retrying.");
             std::this_thread::sleep_for(std::chrono::minutes(5));
         } else {
-            throw std::runtime_error("Backoff violated. " + res.text + "; " + res.error.message);
+            throw APIException("Backoff violated", res.text, res.url.str(), res.status_code, isHTTPStatus);
         }
     } break;
     case 200: {
@@ -158,7 +177,7 @@ std::optional<nlohmann::json> StackAPI::checkErrors(cpr::Response& res, const AP
         return json;
     } break;
     default:
-        throw std::runtime_error("Unhandled status code: " + std::to_string(res.status_code) + "; " + res.text + "; " + res.error.message + " on URL " + res.url.str());
+        throw APIException("Likely API error; see errorMessage and statusCode for the type", res.text, res.url.str(), res.status_code, isHTTPStatus);
     }
     return {};
 }

src/stackapi/StackAPI.hpp

@@ -16,6 +16,18 @@
 
 namespace stackapi {
 
+enum class FaultType {
+    DOWNTIME,
+    CLOUDFLARE,
+    RATE_LIMIT,
+    OTHER,
+    /**
+     * Special code that doesn't denote a fault, but notes that whatever fault was previously reported has been fixed.
+     * Note that if FaultType == RESTORED, then message == "", so it shouldn't be displayed directly.
+     */
+    RESTORED
+};
+
 struct APIConfig {
     std::string apiKey = "";
     std::string auth = "";
@@ -53,7 +65,43 @@ struct APIConfig {
 
     cpr::UserAgent userAgent = "StackAPIUnannouncedUser/git";
 
+    /**
+     * Special callback invoked when dealing with backoff or downtime. 
+     * Note that this is only invoked when autoHandleBackoff and/or autoHandleDowntime is true.
+     * If, for example, automatically handling backoff, but autoHandleBackoff is false, an
+     * exception is thrown instead. This callback is meant to be coupled with the built-in
+     * handling systems.
+     *
+     * Note that FaultType doesn't 1:1 correspond with the different config options. Notably,
+     * Cloudflare is treated as backoff, while RATE_LIMIT just refers to the standard backoff.
+     * The categories are meant to be used to separate issues you do or don't care about handling.
+     *
+     * This function is primarily aimed at two things:
+     * 1. Supplying fault notification systems with information; for the most part, this applies to
+     *    CLOUDFLARE, as cloudflare blocking is downtime, while rate limit-based backoff is more expected.
+     *
+     *    This is primarily aimed at bot and bot management, and to make it easier for me to know when to
+     *    poke SE with another complaint about Cloudflare being dumb again
+     * 2. Supplying information to an end-user in, for example, a desktop application, where the auto-handling
+     *    is still desirable behaviour, but where the user also should be notified
+     *
+     * Note that this function is not required. If you don't care about handling any of these, set the value to
+     * nullptr. 
+     *
+     * Also note that not everything reported is an error. This function is also invoked when doing preventative
+     * backoff waiting, and not only when SE blocks the request. This is also reported under FaultType::RATE_LIMIT,
+     * with a message explaining the problem. I have not decided if it makes sense to separate these or not.
+     */
+    std::function<void(FaultType, const std::string& message)> errorCallback = nullptr;
+
     Backoff backoff = { 0, std::chrono::system_clock::now() };
+
+    void reportError(FaultType type, const std::string& message) const {
+        if (errorCallback) {
+            errorCallback(type, message);
+        }
+    }
+
 };
 
 struct APIConfigOpt {
@@ -67,6 +115,7 @@ struct APIConfigOpt {
     std::optional<bool> autoHandleBackoff;
     std::optional<bool> autoHandleDowntime;
     std::optional<bool> treat404AsDowntime;
+
 };
 
 class StackAPI {

src/stackapi/errors/APIException.hpp

@@ -0,0 +1,36 @@
+#pragma once
+
+#include "spdlog/spdlog.h"
+#include <exception>
+#include <string>
+
+#include <stackapi/data/structs/Types.hpp>
+
+namespace stackapi {
+
+class APIException : public std::exception {
+public:
+    std::string message;
+    /**
+     * message usually contains API-provided messages, while errorMessage contains a message straight from the API.
+     *
+     * Note that this may or may not be useful. Not all faults return JSON
+     */
+    std::string errorMessage;
+    std::string url;
+    API_INT statusCode;
+    bool isHTTPStatus;
+
+    APIException(const std::string& message, const std::string& errorMessage, const std::string& url, API_INT statusCode,
+                 bool isHTTPStatus)
+        : message(message), url(url), errorMessage(errorMessage), statusCode(statusCode), isHTTPStatus(isHTTPStatus) {
+        // TODO: Does this make sense?
+        spdlog::error("{}: {} (status code: {} on invoked URL {})", message, errorMessage, statusCode, url);        
+    }
+
+    char* what() {
+        return message.data();
+    }
+};
+
+}