Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Adds OCUnit parsing for more reliable exit codes #43

Open
wants to merge 2 commits into from

1 participant

@cooksey

We're trying to use ios-sim to run tests in the iOS simulator on our continuous integration server. We ran into a problem where ios-sim always returned a failing exit code because it wasn't expecting the simulated app to exit after the tests ran and treated that as an error. These changes add a new "--unit-testing" parameter that turns on parsing of the simulator's output looking for OCUnit's end of a test suite message to determine whether the tests passed or failed. If any tests failed, then we return an error code on exit. If all tests pass, we return 0 on exit.

Sending a pull request in case other users would find this functionality helpful. This is my first pull request, so please let me know if it's not appropriate for the project or if I should set it up differently.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Mar 21, 2013
  1. Matches existing code style

    Greg Cooksey authored
This page is out of date. Refresh to see the latest.
View
2  Source/iPhoneSimulator.h
@@ -22,6 +22,8 @@
BOOL alreadyPrintedData;
BOOL retinaDevice;
BOOL tallDevice;
+ BOOL parseTestOutput;
+ BOOL testsFailed;
}
- (void)runWithArgc:(int)argc argv:(char **)argv;
View
42 Source/iPhoneSimulator.m
@@ -39,6 +39,7 @@ - (void) printUsage {
fprintf(stderr, " --exit Exit after startup\n");
fprintf(stderr, " --debug Attach LLDB to the application on startup\n");
fprintf(stderr, " --use-gdb Use GDB instead of LLDB. (Requires --debug)\n");
+ fprintf(stderr, " --unit-testing Interpret OCUnit test output to determine exit code\n");
fprintf(stderr, " --sdk <sdkversion> The iOS SDK version to run the application on (defaults to the latest)\n");
fprintf(stderr, " --family <device family> The device type that should be simulated (defaults to `iphone')\n");
fprintf(stderr, " --retina Start a retina device\n");
@@ -80,6 +81,16 @@ - (void)session:(DTiPhoneSimulatorSession *)session didEndWithError:(NSError *)e
}
if (error != nil) {
+ if (parseTestOutput && [error.domain isEqualToString:@"DTiPhoneSimulatorErrorDomain"] && error.code == 1) {
+ // This just means that the app exited. That's normal for testing. Base our return code
+ // on whether the tests passed or failed.
+ if (testsFailed) {
+ exit(EXIT_FAILURE);
+ } else {
+ exit(EXIT_SUCCESS);
+ }
+ }
+
exit(EXIT_FAILURE);
}
@@ -123,16 +134,35 @@ - (void)stdioDataIsAvailable:(NSNotification *)notification {
NSString *str = [[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding];
if (!alreadyPrintedData) {
if ([str length] == 0) {
+ [str release];
return;
} else {
alreadyPrintedData = YES;
}
}
- if ([notification object] == stdoutFileHandle) {
- printf("%s", [str UTF8String]);
- } else {
- nsprintf(@"%@", str);
- }
+
+ if (parseTestOutput && !testsFailed) { // Don't need to check for failure if we've already failed
+ // Search for test failure information
+ NSString *rpTestResultsLine = @"(^Executed .*)";
+ NSString *rpNoFailures = @" 0 failures";
+#warning NSRegularExpression is only available in OS X 10.7+
+ NSError *error = nil;
+ NSRegularExpression *rxTestResultsLine = [NSRegularExpression regularExpressionWithPattern:rpTestResultsLine options:NSRegularExpressionAnchorsMatchLines error:&error];
+ [rxTestResultsLine enumerateMatchesInString:str options:0 range:NSMakeRange(0, [str length]) usingBlock:^(NSTextCheckingResult *result, NSMatchingFlags flags, BOOL *stop) {
+ if ([[str substringWithRange:[result range]] rangeOfString:rpNoFailures].location == NSNotFound) {
+ testsFailed = YES;
+ *stop = YES;
+ }
+ }];
+ }
+
+ if ([notification object] == stdoutFileHandle) {
+ printf("%s", [str UTF8String]);
+
+ } else {
+ nsprintf(@"%@", str);
+ }
+ [str release];
}
@@ -340,6 +370,8 @@ - (void)runWithArgc:(int)argc argv:(char **)argv {
shouldStartDebugger = YES;
} else if (strcmp(argv[i], "--use-gdb") == 0) {
useGDB = YES;
+ } else if (strcmp(argv[i], "--unit-testing") == 0) {
+ parseTestOutput = YES;
}
else if (strcmp(argv[i], "--sdk") == 0) {
i++;
View
1  Source/main.m
@@ -22,6 +22,7 @@ int main (int argc, char *argv[]) {
/* Run the loop to handle added input sources, if any */
[[NSRunLoop mainRunLoop] run];
+ [sim release];
[pool release];
return 0;
}
Something went wrong with that request. Please try again.