Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Diagnostics: Add support for finding hidden tokens (comments) #579

Merged
merged 2 commits into from Dec 21, 2023

Conversation

KevinEady
Copy link
Contributor

@KevinEady KevinEady commented Dec 20, 2023

Comments are stored as hidden tokens on the parser's token stream. In order to find the hidden tokens before a given token, access to the token stream is needed. The SourceFile is what keeps the parser, and the compilations SourceFiles are held by the BuilderWorkspace in its source_files member.

The BuilderWorkspace was a heap variable inside CompilerWorkspaceBuilder::build but has now been moved to be a class member of the "root" CompilerWorkspace, so applications can grab the SourceFile for a given compiled file and call sf->get_hidden_tokens_before() like so:

void HoverBuilder::append_comment( const SourceLocation& source_location, std::string& result )
{
  std::string comment;
  const auto& pathname = source_location.source_file_identifier->pathname;
  auto itr = workspace.builder_workspace.source_files.find( pathname );
  auto tokens = workspace.source->get_all_tokens();
  if ( itr != workspace.builder_workspace.source_files.end() )
  {
    auto sf = itr->second;
    auto hidden_tokens = sf->get_hidden_tokens_before( source_location.range.start );

    for ( auto const* token : hidden_tokens )
    {
      auto token_text = Pol::Clib::strtrim( token->getText() );
      if ( token_text.length() == 0 )
      {
        continue;
      }

      comment += "\n" + token_text;
    }
  }
  if ( !comment.empty() )
  {
    result += "\n" + comment;
  }
}

@coveralls
Copy link
Collaborator

coveralls commented Dec 20, 2023

Coverage Status

coverage: 46.275% (+0.003%) from 46.272%
when pulling 6d396ce on ecompile-diagnostics-hidden-tokens
into e34eaee on ecompile-diagnostics.

- Name formal parameters to differentiate between source locations in `VarStatement`.
@KevinEady KevinEady merged commit f58cda6 into ecompile-diagnostics Dec 21, 2023
68 checks passed
@KevinEady KevinEady deleted the ecompile-diagnostics-hidden-tokens branch December 21, 2023 16:08
KevinEady added a commit that referenced this pull request Dec 29, 2023
* Add support for hidden tokens

* Address Discord review comments
- Name formal parameters to differentiate between source locations in `VarStatement`.
KevinEady added a commit that referenced this pull request Jan 29, 2024
* Add support for hidden tokens

* Address Discord review comments
- Name formal parameters to differentiate between source locations in `VarStatement`.
KevinEady added a commit that referenced this pull request Jan 29, 2024
* Add support for hidden tokens

* Address Discord review comments
- Name formal parameters to differentiate between source locations in `VarStatement`.
KevinEady added a commit that referenced this pull request Jan 29, 2024
* Add support for hidden tokens

* Address Discord review comments
- Name formal parameters to differentiate between source locations in `VarStatement`.
KevinEady added a commit that referenced this pull request Jan 29, 2024
* Add support for hidden tokens

* Address Discord review comments
- Name formal parameters to differentiate between source locations in `VarStatement`.
KevinEady added a commit that referenced this pull request Feb 3, 2024
* Add support for hidden tokens

* Address Discord review comments
- Name formal parameters to differentiate between source locations in `VarStatement`.
KevinEady added a commit that referenced this pull request Feb 4, 2024
* Add support for hidden tokens

* Address Discord review comments
- Name formal parameters to differentiate between source locations in `VarStatement`.
KevinEady added a commit that referenced this pull request Feb 6, 2024
* Add support for hidden tokens

* Address Discord review comments
- Name formal parameters to differentiate between source locations in `VarStatement`.
KevinEady added a commit that referenced this pull request Feb 26, 2024
* Add support for hidden tokens

* Address Discord review comments
- Name formal parameters to differentiate between source locations in `VarStatement`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants