Skip nil tokens in RDoc::TokenStream#tokens_to_s.

[calleluks]

For reasons I don't know, token_stream sometimes contain nil elements.
This is handled in RDoc::TokenStream.to_html but not in
RDoc::TokenStream#tokens_to_s. This adds handling to
RDoc::TokenStream#tokens_to_s.

[drbrain]

Do you have any source files this occurs for?

[drbrain]

… I'll commit this, but if you have such a file maybe I can find a better fix for it.

[calleluks]

Currently I do not. I will try to reproduce the errors I was seeing and give you one.

[calleluks]

@drbrain I've been able to figure out what is making it fail. It is a migration template inside of thoughtbot/clearance that mixes ruby and erb inside a .rb source file. I've been able to reduce the source to the following minimum failing example:

def a_method
<% whatever -%>
end

One interesting point is that if the dash at the end of the erb tag is removed, it no longer fails:

def a_method
<% whatever %>
end