Permalink
Browse files

Fixed comments parsing for non-trivial sources. Closes #43.

Considering this example (provided by "PrimaryFeather", appledoc would fail assigning the comment to the class:

	/** Class description */
	#ifdef __IPHONE_4_0
	@interface SPBitmapFont : NSObject <NSXMLParserDelegate>
	#else
	@interface SPBitmapFont : NSObject
	#endif

With this update, parses is able to attach the comment although with a twist by always assuming the first declaration as the valid one and ignoring all others. In other words: when using such solutions, make sure the first @interface (or @protocol) is the one you'd like to see in the generated documentation. For above example this means that documented class will list NSXMLParserDelegate as adopted protocol.

Note that to properly handle this the solution was to persist any encountered comments during parsing and only update them when new comments are detected. However this lead to situations that any found comment would be used for all subsequent objects in the same file. Therefore I had to add manual resetting of comments after assigning them to an object.

*Important:* Although all unit tests pass and generated documentation looks ok, this might require additional tweaking. Let's see if some more bug reports come...
  • Loading branch information...
1 parent 9ff8322 commit 4262bb89029fa3575b4b02ca0de72ed661cf4c5b @tomaz committed Jan 6, 2011
View
2 Model/GBMethodsProvider.h
@@ -63,7 +63,7 @@
/** Registers a new section if the given name is valid section name.
- The method validates the name string to have at least one char in it. If so, it sends the receiver `registerSectionWithName:` message, passing it the given name and returns generated `GBMethodSectionData` object.. If the name is `nil` or empty string, no section is registered and `nil` is returned. This is provided only to simplify client code - i.e. no need for testing in each place where section should be registered, while on the other hand, validation tests are nicely encapsulated within the class itself, so no functionality is exposed.
+ The method validates the name string to have at least one char in it and section name is different from last registered section name. If so, it sends the receiver `registerSectionWithName:` message, passing it the given name and returns generated `GBMethodSectionData` object.. If the name is `nil` or empty string, no section is registered and `nil` is returned. This is provided only to simplify client code - i.e. no need for testing in each place where section should be registered, while on the other hand, validation tests are nicely encapsulated within the class itself, so no functionality is exposed.
@param name The name of the section.
@return Returns created `GBMethodSectionData` object or `nil` if name is not valid.
View
1 Model/GBMethodsProvider.m
@@ -55,6 +55,7 @@ - (GBMethodSectionData *)registerSectionIfNameIsValid:(NSString *)string {
if (!string) return nil;
string = [string stringByTrimmingCharactersInSet:[NSCharacterSet whitespaceAndNewlineCharacterSet]];
if ([string length] == 0) return nil;
+ if ([_sections count] > 0 && [[[_sections lastObject] sectionName] isEqualToString:string]) return nil;
return [self registerSectionWithName:string];
}
View
7 Parsing/GBObjectiveCParser.m
@@ -422,7 +422,11 @@ - (BOOL)matchMethodDataForProvider:(GBMethodsProvider *)provider from:(NSString
__block GBSourceInfo *filedata = nil;
GBMethodType methodType = [start isEqualToString:@"-"] ? GBMethodTypeInstance : GBMethodTypeClass;
[self.tokenizer consumeFrom:start to:end usingBlock:^(PKToken *token, BOOL *consume, BOOL *stop) {
- if (!filedata) filedata = [self.tokenizer sourceInfoForToken:token];
+ // Prepare source information and reset comments; we alreay read the values so as long as we have found a method, we should reset the comments to prepare ground for next methods. This is needed due to the way this method works - it actually ends by jumping to the first token after the given end symbol, which effectively positions tokenizer to the first token of the following method. Therefore it already consumes any comment preceeding the method. So we can't reset AFTER finished parsing, but rather before! Note that we should only do it once...
+ if (!filedata) {
+ filedata = [self.tokenizer sourceInfoForToken:token];
+ [self.tokenizer resetComments];
+ }
// Get result types.
NSMutableArray *methodResult = [NSMutableArray array];
@@ -517,6 +521,7 @@ - (BOOL)matchMethodDataForProvider:(GBMethodsProvider *)provider from:(NSString
- (void)registerLastCommentToObject:(GBModelBase *)object {
[object setComment:[self.tokenizer lastComment]];
+ [self.tokenizer resetComments];
}
- (void)registerSourceInfoFromCurrentTokenToObject:(GBModelBase *)object {
View
17 Parsing/GBTokenizer.h
@@ -27,7 +27,9 @@
This example simply iterates over all tokens and prints each one to the log. If you want to parse a block of input with known start and/or end token, you can use one of the block consuming methods instead. Note that you still need to provide the name of the file as this is used for creating `GBSourceInfo` objects for parsed objects!
- To make comments parsing simpler, `GBTokenizer` automatically enables comment reporting to the underlying `PKTokenizer`, however to prevent higher level parsers dealing with complexity of comments, any lookahead and consume method doesn't report them. Instead these methods skip all comment tokens, however they do make them accessible through properties, so if the client wants to check whether there's any comment associated with current token, it can simply ask by sending `lastCommentString`. Additionally, the client can also get the value of a comment just before the last one by sending `previousCommentString` - this can be used to get any method section comments which aren't associated with any element. If there is no "stand-alone" comment before the last one, `previousCommentString` returns `nil`. Both value are automatically cleared when another non-comment token is consumed, so make sure to read it before consuming any further token! `GBTokenizer` goes even further when dealing with comments - it automatically groups single line comments into a single comment group and removes all prefixes and suffixes.
+ To make comments parsing simpler, `GBTokenizer` automatically enables comment reporting to the underlying `PKTokenizer`, however to prevent higher level parsers dealing with complexity of comments, any lookahead and consume method doesn't report them. Instead these methods skip all comment tokens, however they do make them accessible through properties, so if the client wants to check whether there's any comment associated with current token, it can simply ask by sending `lastCommentString`. Additionally, the client can also get the value of a comment just before the last one by sending `previousCommentString` - this can be used to get any method section comments which aren't associated with any element. If there is no "stand-alone" comment before the last one, `previousCommentString` returns `nil`. `GBTokenizer` goes even further when dealing with comments - it automatically groups single line comments into a single comment group and removes all prefixes and suffixes.
+
+ @warning *Note:* Both comment values are persistent until a new comment is found! At that time, previous comment contains the value of last comment and the new comment is stored as last comment. This allows us parsing through complex code (like `#ifdef` / `#elif` / `#else` blocks etc.) without fear of loosing any comment information. It does require manual resetting of comments whenever the comment is actually attached to an object. Resetting is performed by sending `resetComments` message to the receiver.
*/
@interface GBTokenizer : NSObject
@@ -143,12 +145,22 @@
/// @name Comments handling
///---------------------------------------------------------------------------------------
+/** Resets `lastComment` and `previousComment` values.
+
+ This message should be sent whenever a comment is "attached" to an object. As comments are persistent, failing to reset would lead to using the same comment for next object as well!
+
+ @see lastComment
+ @see previousComment
+ */
+- (void)resetComments;
+
/** Returns the last comment or `nil` if comment is not available.
The returned `[GBComment stringValue]` contains the whole last comment string, without prefixes or suffixes. To optimize things a bit, the actual comment string value is prepared on the fly, as you send the message, so it's only handled if needed. As creating comment string adds some computing overhead, you should cache returned value if possible.
If there's no comment available for current token, `nil` is returned.
-
+
+ @see resetComments
@see previousComment
*/
@property (readonly) GBComment *lastComment;
@@ -159,6 +171,7 @@
The returned `[GBComment stringValue]` contains the whole previous comment string, without prefixes or suffixes. To optimize things a bit, the actual comment string value is prepared on the fly, as you send the message, so it's only handled if needed. As creating comment string adds some computing overhead, you should cache returned value if possible.
+ @see resetComments
@see lastComment
*/
@property (readonly) GBComment *previousComment;
View
7 Parsing/GBTokenizer.m
@@ -147,8 +147,6 @@ - (GBSourceInfo *)sourceInfoForToken:(PKToken *)token {
- (BOOL)consumeComments {
// This method checks if current token is a comment and consumes all comments until non-comment token is detected or EOF reached. The result of the method is that current index is positioned on the first non-comment token. If current token is not comment, the method doesn't do anything, but simply returns NO to indicate it didn't find a comment and therefore it didn't move current token. This is also where we do initial comments handling such as removing starting and ending chars etc.
- [self.previousCommentBuilder setString:@""];
- [self.lastCommentBuilder setString:@""];
self.previousCommentSourceInfo = nil;
self.lastCommentSourceInfo = nil;
if ([self eof]) return NO;
@@ -247,6 +245,11 @@ - (NSString *)commentValueFromString:(NSString *)value {
return result;
}
+- (void)resetComments {
+ [self.lastCommentBuilder setString:@"" ];
+ [self.previousCommentBuilder setString:@"" ];
+}
+
- (GBComment *)lastComment {
if ([self.lastCommentBuilder length] == 0) return nil;
NSString *value = [self commentValueFromString:self.lastCommentBuilder];
View
8 Testing/GBMethodsProviderTesting.m
@@ -235,6 +235,14 @@ - (void)testRegisterSectionIfNameIsValid_shouldRejectNilWhitespaceOnlyOrEmptyStr
assertThat([provider registerSectionIfNameIsValid:@""], is(nil));
}
+- (void)testRegisterSectionIfNameIsValid_shouldRejectStringIfEqualsToLastRegisteredSectionName {
+ // setup
+ GBMethodsProvider *provider = [[GBMethodsProvider alloc] initWithParentObject:self];
+ [provider registerSectionWithName:@"name"];
+ // execute & verify
+ assertThat([provider registerSectionIfNameIsValid:@"name"], is(nil));
+}
+
- (void)testUnregisterEmptySections_shouldRemoveAllEmptySections {
// setup
GBMethodsProvider *provider = [[GBMethodsProvider alloc] initWithParentObject:self];
View
32 Testing/GBObjectiveCParser-CategoryParsingTesting.m
@@ -244,6 +244,38 @@ - (void)testParseObjectsFromString_shouldRegisterExtensionDefinitionCommentSourc
assertThatInteger(category.comment.sourceInfo.lineNumber, equalToInteger(5));
}
+- (void)testParseObjectsFromString_shouldRegisterCategoryDefinitionCommentForComplexDeclarations {
+ // setup
+ GBObjectiveCParser *parser = [GBObjectiveCParser parserWithSettingsProvider:[GBTestObjectsRegistry mockSettingsProvider]];
+ GBStore *store = [[GBStore alloc] init];
+ // execute
+ [parser parseObjectsFromString:
+ @"/** Comment */\n"
+ @"#ifdef SOMETHING\n"
+ @"@interface MyClass (MyCategory)\n"
+ @"#else\n"
+ @"@interface MyClass (MyCategory1)\n"
+ @"#endif\n"
+ @"@end" sourceFile:@"filename.h" toStore:store];
+ // verify
+ GBCategoryData *category = [store.categories anyObject];
+ assertThat(category.nameOfClass, is(@"MyClass"));
+ assertThat(category.nameOfCategory, is(@"MyCategory"));
+ assertThat(category.comment.stringValue, is(@"Comment"));
+}
+
+- (void)testParseObjectsFromString_shouldProperlyResetComments {
+ // setup
+ GBObjectiveCParser *parser = [GBObjectiveCParser parserWithSettingsProvider:[GBTestObjectsRegistry mockSettingsProvider]];
+ GBStore *store = [[GBStore alloc] init];
+ // execute
+ [parser parseObjectsFromString:@"/** Comment */ @interface MyClass(MyCategory) -(void)method; @end" sourceFile:@"filename.h" toStore:store];
+ // verify
+ GBCategoryData *category = [store.categories anyObject];
+ GBMethodData *method = [category.methods.methods lastObject];
+ assertThat(method.comment, is(nil));
+}
+
#pragma mark Category definition components parsing testing
- (void)testParseObjectsFromString_shouldRegisterCategoryAdoptedProtocols {
View
32 Testing/GBObjectiveCParser-ClassParsingTesting.m
@@ -189,6 +189,38 @@ - (void)testParseObjectsFromString_shouldRegisterClassDeclarationCommentProperSo
assertThatInteger(class.comment.sourceInfo.lineNumber, equalToInteger(5));
}
+- (void)testParseObjectsFromString_shouldRegisterClassDefinitionCommentForComplexDeclarations {
+ // setup
+ GBObjectiveCParser *parser = [GBObjectiveCParser parserWithSettingsProvider:[GBTestObjectsRegistry mockSettingsProvider]];
+ GBStore *store = [[GBStore alloc] init];
+ // execute
+ [parser parseObjectsFromString:
+ @"/** Comment */\n"
+ @"#ifdef SOMETHING\n"
+ @"@interface MyClass : SuperClass\n"
+ @"#else\n"
+ @"@interface MyClass\n"
+ @"#endif\n"
+ @"@end" sourceFile:@"filename.h" toStore:store];
+ // verify
+ GBClassData *class = [[store classes] anyObject];
+ assertThat(class.nameOfClass, is(@"MyClass"));
+ assertThat(class.nameOfSuperclass, is(@"SuperClass"));
+ assertThat(class.comment.stringValue, is(@"Comment"));
+}
+
+- (void)testParseObjectsFromString_shouldProperlyResetComments {
+ // setup
+ GBObjectiveCParser *parser = [GBObjectiveCParser parserWithSettingsProvider:[GBTestObjectsRegistry mockSettingsProvider]];
+ GBStore *store = [[GBStore alloc] init];
+ // execute
+ [parser parseObjectsFromString:@"/** Comment */ @interface MyClass -(void)method; @end" sourceFile:@"filename.h" toStore:store];
+ // verify
+ GBClassData *class = [[store classes] anyObject];
+ GBMethodData *method = [class.methods.methods lastObject];
+ assertThat(method.comment, is(nil));
+}
+
#pragma mark Class definition components parsing testing
- (void)testParseObjectsFromString_shouldRegisterAdoptedProtocols {
View
13 Testing/GBObjectiveCParser-MethodsParsingTesting.m
@@ -463,6 +463,19 @@ - (void)testParseObjectsFromString_shouldRegisterMethodDefinitionComment {
assertThat([[(GBModelBase *)[methods objectAtIndex:1] comment] stringValue], is(@"Comment2"));
}
+- (void)testParseObjectsFromString_shouldProperlyResetComments {
+ // setup
+ GBObjectiveCParser *parser = [GBObjectiveCParser parserWithSettingsProvider:[GBTestObjectsRegistry mockSettingsProvider]];
+ GBStore *store = [[GBStore alloc] init];
+ // execute
+ [parser parseObjectsFromString:@"@interface MyClass /** Comment1 */ -(id)method1; +(void)method2; @end" sourceFile:@"filename.h" toStore:store];
+ // verify
+ GBClassData *class = [[store classes] anyObject];
+ NSArray *methods = [[class methods] methods];
+ assertThat([[(GBModelBase *)[methods objectAtIndex:0] comment] stringValue], is(@"Comment1"));
+ assertThat([(GBModelBase *)[methods objectAtIndex:1] comment], is(nil));
+}
+
- (void)testParseObjectsFromString_shouldRegisterMethodDeclarationComment {
// setup
GBObjectiveCParser *parser = [GBObjectiveCParser parserWithSettingsProvider:[GBTestObjectsRegistry mockSettingsProvider]];
View
33 Testing/GBObjectiveCParser-ProtocolParsingTesting.m
@@ -68,7 +68,7 @@ - (void)testParseObjectsFromString_shouldRegisterAllProtocolDefinitions {
assertThat([[protocols objectAtIndex:1] nameOfProtocol], is(@"MyProtocol2"));
}
-#pragma mark Class comments parsing testing
+#pragma mark Protocol comments parsing testing
- (void)testParseObjectsFromString_shouldRegisterProtocolDefinitionComment {
// setup
@@ -93,6 +93,37 @@ - (void)testParseObjectsFromString_shouldRegisterProtocolDefinitionCommentSource
assertThatInteger(protocol.comment.sourceInfo.lineNumber, equalToInteger(5));
}
+- (void)testParseObjectsFromString_shouldRegisterProtocolDefinitionCommentForComplexDeclarations {
+ // setup
+ GBObjectiveCParser *parser = [GBObjectiveCParser parserWithSettingsProvider:[GBTestObjectsRegistry mockSettingsProvider]];
+ GBStore *store = [[GBStore alloc] init];
+ // execute
+ [parser parseObjectsFromString:
+ @"/** Comment */\n"
+ @"#ifdef SOMETHING\n"
+ @"@protocol MyProtocol\n"
+ @"#else\n"
+ @"@protocol MyProtocol1\n"
+ @"#endif\n"
+ @"@end" sourceFile:@"filename.h" toStore:store];
+ // verify
+ GBProtocolData *protocol = [store.protocols anyObject];
+ assertThat(protocol.nameOfProtocol, is(@"MyProtocol"));
+ assertThat(protocol.comment.stringValue, is(@"Comment"));
+}
+
+- (void)testParseObjectsFromString_shouldProperlyResetComments {
+ // setup
+ GBObjectiveCParser *parser = [GBObjectiveCParser parserWithSettingsProvider:[GBTestObjectsRegistry mockSettingsProvider]];
+ GBStore *store = [[GBStore alloc] init];
+ // execute
+ [parser parseObjectsFromString:@"/** Comment */ @protocol MyProtocol -(void)method; @end" sourceFile:@"filename.h" toStore:store];
+ // verify
+ GBProtocolData *protocol = [store.protocols anyObject];
+ GBMethodData *method = [protocol.methods.methods lastObject];
+ assertThat(method.comment, is(nil));
+}
+
#pragma mark Protocol components parsing testing
- (void)testParseObjectsFromString_shouldRegisterAdoptedProtocols {
View
18 Testing/GBTokenizerTesting.m
@@ -165,11 +165,11 @@ - (void)testConsume_shouldSetPreviousComment {
assertThat([tokenizer.previousComment stringValue], is(@"first\nfirst1"));
assertThat([tokenizer.lastComment stringValue], is(@"second"));
[tokenizer consume:1];
- assertThat([tokenizer.previousComment stringValue], is(nil));
+ assertThat([tokenizer.previousComment stringValue], is(@"second"));
assertThat([tokenizer.lastComment stringValue], is(@"third"));
[tokenizer consume:1];
- assertThat([tokenizer.previousComment stringValue], is(nil));
- assertThat([tokenizer.lastComment stringValue], is(nil));
+ assertThat([tokenizer.previousComment stringValue], is(@"second"));
+ assertThat([tokenizer.lastComment stringValue], is(@"third"));
}
- (void)testConsume_shouldSetProperCommentWhenConsumingMultipleTokens {
@@ -349,6 +349,18 @@ - (void)testLastCommentString_shouldDetectPreviousAndLastCommentSourceInformatio
assertThatInteger([tokenizer.lastComment.sourceInfo lineNumber], equalToInteger(3));
}
+#pragma mark Miscellaneous methods
+
+- (void)testResetComments_shouldResetCommentValues {
+ // setup - remember that initializer already moves to first non-comment token!
+ GBTokenizer *tokenizer = [GBTokenizer tokenizerWithSource:[PKTokenizer tokenizerWithString:@"/** comment1 */ /** comment2 */ ONE"] filename:@"file"];
+ // execute
+ [tokenizer resetComments];
+ // verify
+ assertThat(tokenizer.lastComment, is(nil));
+ assertThat(tokenizer.previousComment, is(nil));
+}
+
#pragma mark Creation methods
- (PKTokenizer *)defaultTokenizer {

0 comments on commit 4262bb8

Please sign in to comment.