System error code handling and utiltity function cleanup #238

saschazelzer · 2017-09-03T13:41:47Z

This PR addresses issues #191 and #220 and also contains further cleanup.

The first commit contains the actual fixes:

Correctly dinstinguish between C runtime library errors and Win32 API errors
Set errno to 0 before calling C runtime library functions
Use thread-safe strerrror variants if available

It also improves code sharing between the legacy and the new GTest test executable:

Pull out TestUtils.cpp for re-use in legacy and GTest tests
TestUtils.cpp changes:
- Use temporary directory instead of "ProgramData" path on Windows
- Added cross-platform mkdtemp and mkstemp functions for creating unique files and directories (originated from public domain and contributed by me with slight changes to another project a couple of years ago; now just copied here)
- Added helper functions for creating temporary files and directories

The commit also adds unit tests for our internal filesystem related functions:

Create tests for cppmicroservices::fs functions. The tests are compiled for static builds only to avoid exporting internal functions.
Make the gtest-based test executable a proper bundle, in order to compile TestUtils.cpp.

For the above changes, I had to duplicate a little more code from the internal file-system code. Seeing that we then had almost the same content duplicated in the framework and the unit test code I wanted to get rid of that and make it reusable.

This is what the second commit contains:

Create an object library called "util" that contains framework independent utility code chared between bundles and tests. This includes file-system related functions, error handling routines, string manipulation routines.
Add the object library to all bundles and the two test drivers by default

Please note that virtually no new code was written, except for disentangling our C / Win32 error handling, using strerror_r etc. where appropriate, and writing new unit tests (GTest based).

Fixes #191 and #220.

codecov-io · 2017-09-03T13:41:51Z

Codecov Report

Merging #238 into development will increase coverage by 5.15%.
The diff coverage is 78.92%.

@@               Coverage Diff               @@
##           development     #238      +/-   ##
===============================================
+ Coverage        77.58%   82.73%   +5.15%     
===============================================
  Files              117      188      +71     
  Lines             8011    13298    +5287     
===============================================
+ Hits              6215    11002    +4787     
- Misses            1796     2296     +500

Impacted Files	Coverage Δ
framework/test/driver/ZipFile.h	`94.11% <ø> (ø)`
framework/src/service/ServiceObjects.cpp	`67.7% <0%> (ø)`	⬆️
framework/src/bundle/BundleThread.cpp	`75.16% <0%> (+1.32%)`	⬆️
framework/src/service/ServiceListeners.cpp	`98.63% <100%> (ø)`	⬆️
framework/test/bundles/libC1/TestBundleC1.cpp	`88.29% <100%> (ø)`
framework/src/bundle/BundleResourceContainer.cpp	`95.41% <100%> (ø)`	⬆️
...work/test/driver/BundleRegistryPerformanceTest.cpp	`93.15% <100%> (ø)`
framework/test/driver/ResourceCompilerTest.cpp	`97.68% <100%> (ø)`
framework/src/bundle/BundleUtils.cpp	`100% <100%> (+13.04%)`	⬆️
framework/test/driver/FrameworkTest.cpp	`95.62% <100%> (ø)`
... and 101 more

jeffdiclemente · 2017-09-06T10:18:30Z

cmake/usCTestScript_travis.cmake

-set(CTEST_BUILD_FLAGS "-j")
+if(NOT ${CMAKE_CXX_COMPILER_ID} STREQUAL "GNU")
+  # gcc in combination with gcov seems to consume a lot of memory
+  # and my lead to OOM error on Travis containers. Hence we compile


typo: "my" -> "may"

jeffdiclemente · 2017-09-28T17:36:12Z

framework/src/bundle/CoreBundleContext.cpp

+      bundleRegistry.Install(execPath, systemBundle.get());
+    }
+  }
+  catch (const std::exception& e)


the behavior to users will change since exceptions are now suppressed from failing to install embedded bundles. is this the right thing to do?

Good point. That was not intended.

jeffdiclemente · 2017-09-28T18:23:57Z

util/include/cppmicroservices/util/FileSystem.h

+extern const char DIR_SEP;
+
+// Get the path of the calling executable.
+// Throws std::runtime_error is the path cannot be


typo: "is" -> "if"

jeffdiclemente · 2017-09-28T18:28:16Z

util/src/Error.cpp

+
+#else
+  #ifndef WIN32_LEAN_AND_MEAN
+    #define WIN32_LEAN_AND_MEAN


this #ifndef/#define WIN32_LEAN_AND_MEAN is in multiple places. it would be good to put it in a central location. perhaps in GlobalConfig.h?

I think we need #126 resolved and have an internal global .h file (which is not included in public .h files) where we can place such defines.

that sounds good to me.

jeffdiclemente · 2017-09-28T18:36:07Z

util/src/Error.cpp

+
+std::string GetLastWin32ErrorStr()
+{
+#ifdef US_PLATFORM_WINDOWS


if this function is only intended to be called on Windows, I think it should only be visible to the compiler when building on Windows. perhaps move the #ifdef outside of the function body?
This has an added benefit of failing at compile time if this function is called from non-Windows code.

I will look at that.

jeffdiclemente · 2017-09-28T18:40:26Z

util/src/Error.cpp

+  // This is the XSI strerror_r version
+  if (strerror_r(errno, errorString, sizeof errorString))
+  {
+    return "Unknown error";


strerror_r will return a error number indicating why it failed. It would be useful to return the error instead of "unknown".

We could return errno as a number in the returned string (by saving it before calling strerror_r). I think returning the error code from a failing strerror_r call would be misleading, because it will mask the original error for which we wanted to retrieve the textual description.

Sorry, I meant returning a string indicating why strerror_r failed.
According to the documentation for strerror_r it only returns a failure if errno is invalid or the string buffer isn't big enough.

karthikreddy09 · 2017-10-03T19:39:42Z

util/src/Error.cpp

+}
+
+US_MSVC_PUSH_DISABLE_WARNING(4715) // 'function' : not all control paths return a value
+std::string GetExceptionStr(const std::exception_ptr& exc)


The warning push pop can be removed with the following block.

std::string GetExceptionStr(const std::exception_ptr& exc) { std::string exceptionStr; try { if(exc) { std::rethrow_exception(exc); } } catch (const std::exception& e) { exceptionStr = e.what(); } catch (...) { exceptionStr= "unknown"; } return exceptionStr; }

karthikreddy09 · 2017-10-03T19:41:59Z

util/src/FileSystem.cpp

+  return std::string();
+}
+
+static const std::string s_CurrWorkingDir = InitCurrentWorkingDirectory();


Why not make this magic static inside the function?

This code was just moved from a different file and not changed. We had a discussion about the actual code in the original PR. Basically, we need to call InitCurrentWorkingDirectory at a time when there is only the main thread running (during static init).

@karthikreddy09 take a look at PR #216 for more information.

Ah ... now I understand. Thanks for pointing to the previous discussion.

…rror handling.

jeffdiclemente · 2017-10-04T19:21:47Z

framework/src/bundle/CoreBundleContext.cpp

  }
  catch (const std::exception& e)
  {
    DIAG_LOG(*sink) << e.what();
+    // Let the exeception propagate all the way up to the


typo: "exeception"

ksubramz · 2017-10-09T16:31:42Z

framework/test/util/TestUtils.cpp

+}
+
+// A cross-platform version of the POSIX mkdtemp function
+char* mkdtemps_compat(char *tmpl, int suffixlen)


It looks like there's a lot of common code between mkdtemps_compat and mkstemps_compat that can be factored out to remove the duplication?

Mostly for the same reason as above. Laziness, working on bugs or features, afraid of introducing regressions, ...

ksubramz · 2017-10-09T16:53:18Z

framework/test/util/TestUtils.cpp

+
+static const char validLetters[] = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";
+
+// A cross-platform version of the mkstemps function


Sascha, curious about why we need to be compatible with mkstemp (or mkdtemp), AFAIU, these are only called by our tests and if we drop the compatibility criterion, we could avoid passing "char *" and avoid using strlen() etc.

i.e. I think it's better to operate on std::string's instead of (less safe) C-style strings, use std::string methods and throw an exception from this function itself in-case of any error.

The answer is a historical one. The code originates from the public domain on the Internet. As such it was C code, written to be compatible with mkstemp and friends by the original author. I only did very small changes initially (many years ago) and used the code as is in another project. It did its job well enough and I thought I would spend my time elsewhere instead of making it prettier for CppMicroServices.

Feel free to send a PR for C++ification or other improvements, but I think there is not much gain unless there is a bug of course.

Thanks. Sounds good.

ksubramz · 2017-10-10T23:47:20Z

framework/test/util/TestUtils.cpp

+
+static const char validLetters[] = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";
+
+// A cross-platform version of the mkstemps function


Thanks. Sounds good.

saschazelzer added 2 commits August 29, 2017 08:02

Cleanup internal system error handling and add file-system tests.

aa698c6

Fixes #191 and #220.

Share code (filesystem, error handling, etc.) between bundles and tests.

7394b3d

saschazelzer requested a review from a team September 3, 2017 13:47

saschazelzer self-assigned this Sep 3, 2017

Try to avoid OOM error on Travis with gcc.

bd0070d

jeffdiclemente reviewed Sep 28, 2017

View reviewed changes

karthikreddy09 reviewed Oct 3, 2017

View reviewed changes

Fix typos, do not swallow exceptions on bundle install, and improve e…

06bb197

…rror handling.

jeffdiclemente approved these changes Oct 4, 2017

View reviewed changes

Fix typo and old-style cast.

4fb25d9

ksubramz reviewed Oct 9, 2017

View reviewed changes

ksubramz approved these changes Oct 10, 2017

View reviewed changes

saschazelzer force-pushed the system-error-code-handling-cleanup branch from e6caf9b to 4fb25d9 Compare October 11, 2017 05:55

saschazelzer merged commit 4fb25d9 into development Oct 11, 2017

saschazelzer deleted the system-error-code-handling-cleanup branch October 29, 2017 15:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

System error code handling and utiltity function cleanup #238

System error code handling and utiltity function cleanup #238

saschazelzer commented Sep 3, 2017

codecov-io commented Sep 3, 2017 •

edited

Loading

jeffdiclemente Sep 6, 2017

jeffdiclemente Sep 28, 2017

saschazelzer Sep 29, 2017

jeffdiclemente Sep 28, 2017

jeffdiclemente Sep 28, 2017

saschazelzer Oct 4, 2017

jeffdiclemente Oct 4, 2017

jeffdiclemente Sep 28, 2017

saschazelzer Sep 29, 2017

jeffdiclemente Sep 28, 2017

saschazelzer Sep 29, 2017

jeffdiclemente Oct 2, 2017

karthikreddy09 Oct 3, 2017

karthikreddy09 Oct 3, 2017

saschazelzer Oct 4, 2017

jeffdiclemente Oct 4, 2017

karthikreddy09 Oct 4, 2017

jeffdiclemente Oct 4, 2017

ksubramz Oct 9, 2017

saschazelzer Oct 10, 2017

ksubramz Oct 9, 2017 •

edited

Loading

saschazelzer Oct 10, 2017 •

edited

Loading

ksubramz Oct 10, 2017

ksubramz Oct 10, 2017


		static const char validLetters[] = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";

		// A cross-platform version of the mkstemps function

System error code handling and utiltity function cleanup #238

System error code handling and utiltity function cleanup #238

Conversation

saschazelzer commented Sep 3, 2017

codecov-io commented Sep 3, 2017 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ksubramz Oct 9, 2017 • edited Loading

Choose a reason for hiding this comment

saschazelzer Oct 10, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-io commented Sep 3, 2017 •

edited

Loading

ksubramz Oct 9, 2017 •

edited

Loading

saschazelzer Oct 10, 2017 •

edited

Loading