Add Documented Command #4

Open
daschl opened this Issue Jun 15, 2012 · 9 comments

Comments

Projects
None yet
4 participants
@daschl
Contributor

daschl commented Jun 15, 2012

This command should

  • provide a list of unfinished or undocumented classes/methods
  • provide statistics for the above mentioned metrics

@ghost ghost assigned daschl Jun 15, 2012

@blainesch

This comment has been minimized.

Show comment Hide comment
@blainesch

blainesch Dec 17, 2012

Member

How detailed should the information be?

With the new parser linking the docs to a class/method would be easy.
It would be amazing if we could parse the docblocks and say the first @param does not match the first parameter in the method or something similar. The names wouldn't be hard to match...
We could also "require" things like @param and @return on methods.

Anything I'm missing?

Member

blainesch commented Dec 17, 2012

How detailed should the information be?

With the new parser linking the docs to a class/method would be easy.
It would be amazing if we could parse the docblocks and say the first @param does not match the first parameter in the method or something similar. The names wouldn't be hard to match...
We could also "require" things like @param and @return on methods.

Anything I'm missing?

@blainesch

This comment has been minimized.

Show comment Hide comment
@blainesch

blainesch Dec 17, 2012

Member

After some thought I think this should ONLY check if docs exist, possible give it a rating based on the quantity of the docs.. such as short description being 25 points, long description is 25 points, and the presence of docblocks is 50 points and 75 points is required to pass as "good"?

The other aspects such as matching @param to real parameters, requiring specific tags and possibly ordering of tags... should probably be in the docblock syntax rule.

Member

blainesch commented Dec 17, 2012

After some thought I think this should ONLY check if docs exist, possible give it a rating based on the quantity of the docs.. such as short description being 25 points, long description is 25 points, and the presence of docblocks is 50 points and 75 points is required to pass as "good"?

The other aspects such as matching @param to real parameters, requiring specific tags and possibly ordering of tags... should probably be in the docblock syntax rule.

@nateabele

This comment has been minimized.

Show comment Hide comment
@nateabele

nateabele Dec 18, 2012

Owner

I approach it in two ways. The first is accurateness and completeness. These are all direct and easy-to-measure things, that basically boil down to making sure all methods and properties are docblocked (along with the class itself), all method parameters have matching @param tags, etc.

The second is overall quality, which gets to be a bit more subjective and less absolute, but generally, as the size and complexity of a class increases, the number of @link tags, @see cross-references, lengthier text descriptions, etc. should also increase. Definitely open to ideas here.

Owner

nateabele commented Dec 18, 2012

I approach it in two ways. The first is accurateness and completeness. These are all direct and easy-to-measure things, that basically boil down to making sure all methods and properties are docblocked (along with the class itself), all method parameters have matching @param tags, etc.

The second is overall quality, which gets to be a bit more subjective and less absolute, but generally, as the size and complexity of a class increases, the number of @link tags, @see cross-references, lengthier text descriptions, etc. should also increase. Definitely open to ideas here.

@blainesch

This comment has been minimized.

Show comment Hide comment
@blainesch

blainesch Dec 18, 2012

Member

Completeness: In the Rule configuration (#2) we could define items such as "required short description", "required @param", "required @return", "forbidden @var". The rule can throw a violation if this doesn't match correctly.

Accurate: In the Rule we can define standards for how a "short description", "long description", and "@param" should look. This only cares if the existing comment has the correct format, possible code matching (like @param and @return), and also validates the tags are in the correct order.

Do you think it would be over the top to validate @link by possibly curling the url to check if it exists? It would probably throw a warning instead of a violation since they could have downtime.

Quality: The doc command can have a conversion rate (complexity * x = threshold) and based on what it finds it can add them up. In the config we can set "short description" to 10 points, @param to 4 points, other tags to 2 points.

Because the doc command is separate from the syntax command, the doc command could run the syntax Rule against the current code and if a violation is thrown because of a specific docblock it could also deduct points on it as well.

Member

blainesch commented Dec 18, 2012

Completeness: In the Rule configuration (#2) we could define items such as "required short description", "required @param", "required @return", "forbidden @var". The rule can throw a violation if this doesn't match correctly.

Accurate: In the Rule we can define standards for how a "short description", "long description", and "@param" should look. This only cares if the existing comment has the correct format, possible code matching (like @param and @return), and also validates the tags are in the correct order.

Do you think it would be over the top to validate @link by possibly curling the url to check if it exists? It would probably throw a warning instead of a violation since they could have downtime.

Quality: The doc command can have a conversion rate (complexity * x = threshold) and based on what it finds it can add them up. In the config we can set "short description" to 10 points, @param to 4 points, other tags to 2 points.

Because the doc command is separate from the syntax command, the doc command could run the syntax Rule against the current code and if a violation is thrown because of a specific docblock it could also deduct points on it as well.

@nateabele

This comment has been minimized.

Show comment Hide comment
@nateabele

nateabele Dec 18, 2012

Owner

Completeness: In the Rule configuration (#2) we could define items such as "required short description", "required @param", "required @return", "forbidden @var". The rule can throw a violation if this doesn't match correctly.

Yeah, that sounds about right. The only real nuance here is checking each @param tag to ensure the name and type matches the actual parameter name and any type-casting info in the method def. Oftentimes what happens is people update the defs and forget to update the doc to match.

Accurate: In the Rule we can define standards for how a "short description", "long description", and "@param" should look. This only cares if the existing comment has the correct format, possible code matching (like @param and @return), and also validates the tags are in the correct order.

Yup. Actually, the @param thing above might belong here.... not sure.

Do you think it would be over the top to validate @link by possibly curling the url to check if it exists? It would probably throw a warning instead of a violation since they could have downtime.

Hmmm... for now let's not, for a few different reasons. For one, that's generally the kind of thing you'd only validate once. Plus, you could always have a separate tool that crawls a codebase for @link tags and checks for dead URLs, since that'd be more of a longer-term problem not directly associated with code maintenance (which actually sounds pretty handy).

Quality: The doc command can have a conversion rate (complexity * x = threshold) and based on what it finds it can add them up. In the config we can set "short description" to 10 points, @param to 4 points, other tags to 2 points.

Yeah, I was thinking something along those lines. Measuring things like # of methods, # of properties, total cyclomatic complexity, etc., assigning some multipliers to each metric and adding them up on the one side, then similarly scoring the docs on the other side and generating a ratio of some sort.

Owner

nateabele commented Dec 18, 2012

Completeness: In the Rule configuration (#2) we could define items such as "required short description", "required @param", "required @return", "forbidden @var". The rule can throw a violation if this doesn't match correctly.

Yeah, that sounds about right. The only real nuance here is checking each @param tag to ensure the name and type matches the actual parameter name and any type-casting info in the method def. Oftentimes what happens is people update the defs and forget to update the doc to match.

Accurate: In the Rule we can define standards for how a "short description", "long description", and "@param" should look. This only cares if the existing comment has the correct format, possible code matching (like @param and @return), and also validates the tags are in the correct order.

Yup. Actually, the @param thing above might belong here.... not sure.

Do you think it would be over the top to validate @link by possibly curling the url to check if it exists? It would probably throw a warning instead of a violation since they could have downtime.

Hmmm... for now let's not, for a few different reasons. For one, that's generally the kind of thing you'd only validate once. Plus, you could always have a separate tool that crawls a codebase for @link tags and checks for dead URLs, since that'd be more of a longer-term problem not directly associated with code maintenance (which actually sounds pretty handy).

Quality: The doc command can have a conversion rate (complexity * x = threshold) and based on what it finds it can add them up. In the config we can set "short description" to 10 points, @param to 4 points, other tags to 2 points.

Yeah, I was thinking something along those lines. Measuring things like # of methods, # of properties, total cyclomatic complexity, etc., assigning some multipliers to each metric and adding them up on the one side, then similarly scoring the docs on the other side and generating a ratio of some sort.

@blainesch

This comment has been minimized.

Show comment Hide comment
@blainesch

blainesch Jan 3, 2013

Member

@nateabele Why is documented necessary? If they are running their library against the syntax portion of this, it'll throw violations whenever something isn't documented correctly. The documented command will pass when you are mostly documented over the given threshold, even if you have violations. So a library that has no violations will always pass documented.

I guess I'm trying to say that if you are using the syntax tool correctly, you won't need a documented command.

Member

blainesch commented Jan 3, 2013

@nateabele Why is documented necessary? If they are running their library against the syntax portion of this, it'll throw violations whenever something isn't documented correctly. The documented command will pass when you are mostly documented over the given threshold, even if you have violations. So a library that has no violations will always pass documented.

I guess I'm trying to say that if you are using the syntax tool correctly, you won't need a documented command.

@d1rk

This comment has been minimized.

Show comment Hide comment
@d1rk

d1rk Jan 4, 2013

Member

It may be faster, if i just want to know, if all my docblocks are in place...

Member

d1rk commented Jan 4, 2013

It may be faster, if i just want to know, if all my docblocks are in place...

@nateabele

This comment has been minimized.

Show comment Hide comment
@nateabele

nateabele Jan 4, 2013

Owner

@blainesch The idea behind the syntax command is to check the correctness of the structure of the code, which partially extends to docblocks, whereas the documented command checks the accuracy and overall quality/completeness of the documentation. To me, these are two completely different ideas.

Owner

nateabele commented Jan 4, 2013

@blainesch The idea behind the syntax command is to check the correctness of the structure of the code, which partially extends to docblocks, whereas the documented command checks the accuracy and overall quality/completeness of the documentation. To me, these are two completely different ideas.

@d1rk

This comment has been minimized.

Show comment Hide comment
@d1rk

d1rk Oct 20, 2013

Member

I would like to know how the status of this is. I agree to @nateabele last comment. In particular, i see the reason to run a li3_quality documented against a library to see, if it fulfills a certain threshold of quality, more like an information of how careful the developer was. While li3_quality syntax allows me to see, if i can continue the automated integration or if i should abort.

Member

d1rk commented Oct 20, 2013

I would like to know how the status of this is. I agree to @nateabele last comment. In particular, i see the reason to run a li3_quality documented against a library to see, if it fulfills a certain threshold of quality, more like an information of how careful the developer was. While li3_quality syntax allows me to see, if i can continue the automated integration or if i should abort.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment