Fix #5018 - C++ CLI: Calling a non existing method in a measure ends up with a crash and stack trace #5023

jmarrec · 2023-11-10T11:58:18Z

Pull request overview

Fixes C++ CLI: Calling a non existing method in a measure ends up with a crash and stack trace #5018
@kbenne saved the day by figuring out we could use the SWIG %feature("director:except")
I just implemented the error messages

TODO: Does this stuff need to be moved at a (much) higher level, so it applies to every SwigDirector we have, not just in Measure.i. Maybe in utilities/core?
- I think it isn't needed, because the only ruby/python stuff we call directly from C++ is the measure methods, but I could be wrong.
- @kbenne thoughts please

Pull Request Author

Model API Changes / Additions
Any new or modified fields have been implemented in the EnergyPlus ForwardTranslator (and ReverseTranslator as appropriate)
Model API methods are tested (in src/model/test)
EnergyPlus ForwardTranslator Tests (in src/energyplus/Test)
If a new object or method, added a test in NREL/OpenStudio-resources: Add Link
If needed, added VersionTranslation rules for the objects (src/osversion/VersionTranslator.cpp)
Verified that C# bindings built fine on Windows, partial classes used as needed, etc.
All new and existing tests passes
If methods have been deprecated, update rest of code to use the new methods

Labels:

If change to an IDD file, add the label IDDChange
If breaking existing API, add the label APIChange
If deemed ready, add label Pull Request - Ready for CI so that CI builds your PR

Review Checklist

This will not be exhaustively relevant to every PR.

Perform a Code Review on GitHub
Code Style, strip trailing whitespace, etc.
All related changes have been implemented: model changes, model tests, FT changes, FT tests, VersionTranslation, OS App
Labeling is ok
If defect, verify by running develop branch and reproducing defect, then running PR and reproducing fix
If feature, test running new feature, try creative ways to break it
CI status: all green or justified

This shows how to turn errors arising from Swig director calls into normal C++ exceptions. ref #5018

jmarrec · 2023-11-10T12:04:55Z

src/measure/Measure.i

 #if defined SWIGRUBY
+  %feature("director:except") {
+    // Mimic openstudio::evalString for a possible approach to reporting more info or a callstack.


For ruby, mimic the https://github.com/NREL/OpenStudio/blob/c0cbe73b51f34c459d67a17da36739795bbf25fc2/ruby/interpreter/RubyEval.hpp#L18

jmarrec · 2023-11-10T12:05:22Z

src/measure/Measure.i

+    // I just **cannot** figure out a way to get the error location in C, without calling $@.to_s. Seems nothing is available here
+    int locationError;
+    VALUE locval = rb_eval_string_protect("$@.to_s", &locationError);
+    std::string loc;
+    if (locationError == 0) {
+      loc = StringValuePtr(locval);
+    } else {
+      loc = "Failed to get VM location";
+    }


This bit seems redundant with the backtrace block below, you get the same info twice.

jmarrec · 2023-11-10T12:08:00Z

src/measure/Measure.i

+#if defined SWIGPYTHON
+  %feature("director:except") {
+
+    if ($error != nullptr) {
+
+      PyObject *exc_type = nullptr;
+      PyObject *exc_value = nullptr;
+      PyObject *exc_tb = nullptr;
+      PyErr_Fetch(&exc_type, &exc_value, &exc_tb);
+      PyErr_NormalizeException(&exc_type, &exc_value, &exc_tb);
+      PyObject *str_exc_value = PyObject_Repr(exc_value); // Now a unicode object
+      PyObject *pyStr2 = PyUnicode_AsEncodedString(str_exc_value, "utf-8", "Error ~");
+      Py_DECREF(str_exc_value);
+      char *strExcValue = PyBytes_AsString(pyStr2); // NOLINT(hicpp-signed-bitwise)
+      Py_DECREF(pyStr2);
+
+      std::string err_msg = "In method '$symname': `" + std::string{strExcValue} + "`";
+
+      // See if we can get a full traceback.
+      // Calls into python, and does the same as capturing the exception in `e`
+      // then `print(traceback.format_exception(e.type, e.value, e.tb))`
+      PyObject *pModuleName = PyUnicode_DecodeFSDefault("traceback");
+      PyObject *pyth_module = PyImport_Import(pModuleName);
+      Py_DECREF(pModuleName);
+
+      if (pyth_module == nullptr) {
+          err_msg += "\nCannot find 'traceback' module, this should not happen";
+          throw Swig::DirectorMethodException(err_msg.c_str());
+      }
+
+      PyObject *pyth_func = PyObject_GetAttrString(pyth_module, "format_exception");
+      Py_DECREF(pyth_module); // PyImport_Import returns a new reference, decrement it
+
+      if (pyth_func || PyCallable_Check(pyth_func)) {
+
+          PyObject *pyth_val = PyObject_CallFunction(pyth_func, "OOO", exc_type, exc_value, exc_tb);
+
+          // traceback.format_exception returns a list, so iterate on that
+          if (!pyth_val || !PyList_Check(pyth_val)) { // NOLINT(hicpp-signed-bitwise)
+              err_msg += "\nIn reportPythonError(), traceback.format_exception did not return a list.";
+          } else {
+            unsigned long numVals = PyList_Size(pyth_val);
+            if (numVals == 0) {
+                err_msg += "\nNo traceback available";
+            } else {
+              err_msg += "\nPython traceback follows:\n```";
+
+              for (unsigned long itemNum = 0; itemNum < numVals; itemNum++) {
+                  PyObject *item = PyList_GetItem(pyth_val, itemNum);
+                  if (PyUnicode_Check(item)) { // NOLINT(hicpp-signed-bitwise) -- something inside Python code causes warning
+                      std::string traceback_line = PyUnicode_AsUTF8(item);
+                      if (!traceback_line.empty() && traceback_line[traceback_line.length() - 1] == '\n') {
+                          traceback_line.erase(traceback_line.length() - 1);
+                      }
+                      err_msg += "\n" + traceback_line;
+                  }
+                  // PyList_GetItem returns a borrowed reference, do not decrement
+              }
+
+              err_msg += "\n```";
+            }
+          }
+
+          // PyList_Size returns a borrowed reference, do not decrement
+          Py_DECREF(pyth_val); // PyObject_CallFunction returns new reference, decrement
+      }
+      Py_DECREF(pyth_func); // PyObject_GetAttrString returns a new reference, decrement it
+      throw Swig::DirectorMethodException(err_msg.c_str());
+    }
+  }
+#endif


For python, I repurposed what Edwin and I wrote for EnergyPlus's PluginInstance::reportPythonError (cf NREL/EnergyPlus#8181 for where I implemented the traceback bit)

jmarrec · 2023-11-10T12:08:45Z

python/engine/test/WrongMethodMeasure/measure.py

+class WrongMethodMeasure(openstudio.measure.ModelMeasure):
+    def name(self):
+        return "Wrong Method Measure"
+
+    def modeler_description(self):
+        return "The arguments method calls a non existing method on the model passed as argument"
+
+    def arguments(self, model: typing.Optional[openstudio.model.Model] = None):
+        model.nonExistingMethod()
+        args = openstudio.measure.OSArgumentVector()


Added a test for both python and ruby that calls a wrong method on model passed as argument in arguments()

jmarrec · 2023-11-10T12:09:19Z

python/engine/test/BadMeasure/measure.py

+class BadMeasure(openstudio.measure.ModelMeasure):
+    def name(self):
+        return "Bad Measure"
+
+    def modeler_description(self):
+        return "The arguments method calls another_method which does a raise ValueError"
+
+    def another_method(self):
+        raise ValueError("oops")
+
+    def arguments(self, model: typing.Optional[openstudio.model.Model] = None):
+        self.another_method()


Both Python and Ruby have this failed measure, that calls another method that throws, so we can check if the traceback is good

jmarrec · 2023-11-10T12:10:34Z

ruby/engine/test/RubyEngine_GTest.cpp

+  // initialize for each test
+  virtual void SetUp() override {
+    std::vector<std::string> args;
+    thisEngine = std::make_unique<openstudio::ScriptEngineInstance>("rubyengine", args);
+
+    (*thisEngine)->registerType<openstudio::measure::OSMeasure*>("openstudio::measure::OSMeasure *");
+    (*thisEngine)->registerType<openstudio::measure::ModelMeasure*>("openstudio::measure::ModelMeasure *");
+    (*thisEngine)->exec("OpenStudio::init_rest_of_openstudio()");
+  }
+  // tear down after each test
+  virtual void TearDown() override {
+    // Want to ensure the engine is reset regardless of the test outcome, or it may throw a segfault
+    thisEngine->reset();
+  }
+
+  std::unique_ptr<openstudio::ScriptEngineInstance> thisEngine;


FYI @kbenne I slightly changed the test you wrote so that the fixture will always reset the engine, regardless of whether the test passed or failed.

jmarrec · 2023-11-10T12:11:19Z

ruby/engine/test/RubyEngine_GTest.cpp

+  std::string expected_exception = fmt::format(R"(SWIG director method error. RuntimeError: oops
+["/Users/julien/Software/Others/OpenStudio2/ruby/engine/test/BadMeasure/measure.rb:12:in `another_method'", "/Users/julien/Software/Others/OpenStudio2/ruby/engine/test/BadMeasure/measure.rb:16:in `arguments'"]
+
+Traceback:
+{0}:12:in `another_method'
+{0}:16:in `arguments')",
+                                               scriptPath.generic_string());
+
+  openstudio::model::Model model;
+  try {
+    measurePtr->arguments(model);
+    ASSERT_FALSE(true) << "Expected measure arguments(model) to throw";
+  } catch (std::exception& e) {
+    std::string error = e.what();
+    EXPECT_EQ(expected_exception, error);


Ruby traceback example

SWIG director method error. RuntimeError: oops ["/Users/julien/Software/Others/OpenStudio2/ruby/engine/test/BadMeasure/measure.rb:12:in `another_method'", "/Users/julien/Software/Others/OpenStudio2/ruby/engine/test/BadMeasure/measure.rb:16:in `arguments'"] Traceback: /Users/julien/Software/Others/OpenStudio2/ruby/engine/test/BadMeasure/measure.rb:12:in `another_method' /Users/julien/Software/Others/OpenStudio2/ruby/engine/test/BadMeasure/measure.rb:16:in `arguments'

Hence why I just said the "$@.to_s" in EvalString (and now measure.i) is redundant if the traceback is available.

jmarrec · 2023-11-10T12:12:44Z

ruby/engine/test/RubyEngine_GTest.cpp

+  std::string expected_exception =
+    fmt::format(R"(SWIG director method error. NoMethodError: undefined method `nonExistingMethod' for #<OpenStudio::Model::Model:ADDRESS>
+["/Users/julien/Software/Others/OpenStudio2/ruby/engine/test/WrongMethodMeasure/measure.rb:12:in `arguments'"]
+
+Traceback:
+{0}:12:in `arguments')",
+                scriptPath.generic_string());
+
+  openstudio::model::Model model;
+  try {
+    measurePtr->arguments(model);
+    ASSERT_FALSE(true) << "Expected measure arguments(model) to throw";
+  } catch (std::exception& e) {
+    std::string error = e.what();
+    EXPECT_EQ(expected_exception, stripAddressFromErrorMessage(error));


I gotta strip the "0x00283498743eff" address here because it naturally changes everry time

jmarrec · 2023-11-10T12:13:17Z

python/engine/test/PythonEngine_GTest.cpp

+  std::string expected_exception = fmt::format(R"(SWIG director method error. In method 'arguments': `ValueError('oops')`
+Python traceback follows:
+```
+Traceback (most recent call last):
+  File "{0}", line 17, in arguments
+    self.another_method()
+  File "{0}", line 14, in another_method
+    raise ValueError("oops")
+ValueError: oops
+```)",


Python tracebacks!

RuntimeError: SWIG director method error. In method 'arguments': `ValueError('oops')` Python traceback follows: ` ` ` Traceback (most recent call last): File "/Users/julien/Software/Others/OpenStudio2/python/engine/test/BadMeasure/./measure.py", line 14, in arguments self.another_method() File "/Users/julien/Software/Others/OpenStudio2/python/engine/test/BadMeasure/./measure.py", line 11, in another_method raise ValueError("oops") ValueError: oops ` ` `

jmarrec · 2023-11-10T12:17:42Z

python/engine/test/PythonEngine_GTest.cpp

+  std::string expected_exception =
+    fmt::format(R"(SWIG director method error. In method 'arguments': `AttributeError("'Model' object has no attribute 'nonExistingMethod'")`
+Python traceback follows:
+```
+Traceback (most recent call last):
+  File "{}", line 14, in arguments
+    model.nonExistingMethod()
+AttributeError: 'Model' object has no attribute 'nonExistingMethod'
+```)",
+                scriptPath.generic_string());


WrongMethodMeasure for python

The fallback is needed for Stack Too deep

…rting in that case: show 8 early lines + 12 final

warning C4244: ‘initializing’: conversion from ‘Py_ssize_t’ to ‘unsigned long’,

kbenne

Thanks for putting this togethor @jmarrec !

kbenne · 2023-11-10T14:46:35Z

src/measure/Measure.i

@@ -96,6 +96,71 @@
 };

 #if defined SWIGRUBY
+  %feature("director:except") {


I also contemplated moving this up to a higher level. I would be fine with that, but I opted to keep it "local" to where the directors are defined. I thought there was a bit of logic in that. Either way is really ok.

ci-commercialbuildings · 2023-11-10T14:58:11Z

CI Results for ded922f:

Ubuntu 20.04:
Ubuntu 22.04:
- OpenStudio-3.7.0-rc2+ded922f7a5-Ubuntu-22.04-x86_64.deb
- OpenStudio-3.7.0-rc2+ded922f7a5-Ubuntu-22.04-x86_64.tar.gz
Darwin:
- OpenStudio-3.7.0-rc2+ded922f7a5-Darwin-x86_64.dmg
- OpenStudio-3.7.0-rc2+ded922f7a5-Darwin-x86_64.tar.gz
Windows:
- OpenStudio-3.7.0-rc2+ded922f7a5-Windows.exe
- OpenStudio-3.7.0-rc2+ded922f7a5-Windows.tar.gz
Code Coverage:

kbenne and others added 4 commits November 9, 2023 23:56

Demo Measure errors as C++ exceptions

05e5996

This shows how to turn errors arising from Swig director calls into normal C++ exceptions. ref #5018

Add director:except with full traceback for Python + add GTest

4382c88

Improve testing for RubyEngine.

0e6c845

Implement a nice traceback for Ruby Director method exception

d725a87

jmarrec assigned kbenne and jmarrec Nov 10, 2023

Cleanup Measure.i from leftovers

21fff07

jmarrec commented Nov 10, 2023

View reviewed changes

jmarrec added component - CLI Pull Request - Ready for CI This pull request if finalized and is ready for continuous integration verification prior to merge. labs component - Measures labels Nov 10, 2023

jmarrec added 5 commits November 10, 2023 13:33

Cleanup unused bits even more

a9a795c

Don't use "$@.to_s" if we manage to produce a backtrace.

3875f88

The fallback is needed for Stack Too deep

Add a test for SystemStackLevel too deep in ruby + cleanup error repo…

f35e8aa

…rting in that case: show 8 early lines + 12 final

Align ruby and python tracebacks

6dc9b65

MSVC complains Py_ssize_t -> long, just use Py_ssize_t

ded922f

warning C4244: ‘initializing’: conversion from ‘Py_ssize_t’ to ‘unsigned long’,

kbenne approved these changes Nov 10, 2023

View reviewed changes

kbenne merged commit 9f14715 into develop Nov 10, 2023
5 of 6 checks passed

kbenne deleted the 5018-measure-errors branch November 10, 2023 15:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix #5018 - C++ CLI: Calling a non existing method in a measure ends up with a crash and stack trace #5023

Fix #5018 - C++ CLI: Calling a non existing method in a measure ends up with a crash and stack trace #5023

jmarrec commented Nov 10, 2023 •

edited

Loading

jmarrec Nov 10, 2023

jmarrec Nov 10, 2023

jmarrec Nov 10, 2023

jmarrec Nov 10, 2023

jmarrec Nov 10, 2023

jmarrec Nov 10, 2023

jmarrec Nov 10, 2023

jmarrec Nov 10, 2023

jmarrec Nov 10, 2023

jmarrec Nov 10, 2023

jmarrec Nov 10, 2023

kbenne left a comment

kbenne Nov 10, 2023

ci-commercialbuildings commented Nov 10, 2023 •

edited

Loading

Fix #5018 - C++ CLI: Calling a non existing method in a measure ends up with a crash and stack trace #5023

Fix #5018 - C++ CLI: Calling a non existing method in a measure ends up with a crash and stack trace #5023

Conversation

jmarrec commented Nov 10, 2023 • edited Loading

Pull request overview

Pull Request Author

Review Checklist

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kbenne left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ci-commercialbuildings commented Nov 10, 2023 • edited Loading

jmarrec commented Nov 10, 2023 •

edited

Loading

ci-commercialbuildings commented Nov 10, 2023 •

edited

Loading