Failure when parsing a lambda in a parameter list

[kpeaton]

Here are the steps to repeat this using the Docker image supplied here:

Run the image:

docker run -it registry.gitlab.com/gitlab-org/security-products/analyzers/flawfinder:2 /bin/sh

Enter the following in the container:

apk add git
git clone https://github.com/microsoft/vcpkg.git
export SECURE_LOG_LEVEL=debug
cd vcpkg
../analyzer run

The following (truncated) output is generated:

[INFO] [Flawfinder] [2020-12-02T21:35:02Z] > GitLab Flawfinder analyzer v2.12.0
[INFO] [Flawfinder] [2020-12-02T21:35:02Z] > Detecting project
[INFO] [Flawfinder] [2020-12-02T21:35:02Z] > Found project in /vcpkg/ports/alac-decoder
[INFO] [Flawfinder] [2020-12-02T21:35:02Z] > Running analyzer
Error: File ended while in string.
Error: File ended while in string.
Error: File ended while in string.
Error: File ended while in string.
Error: File ended while in string.
Error: File ended while in string.
Error: File ended while in string.
Error: File ended while in string.
Error: File ended while in string.
[DEBU] [Flawfinder] [2020-12-02T21:35:17Z] > /usr/local/bin/flawfinder -m 1 --csv .
File,Line,Column,Level,Category,Name,Warning,Suggestion,Note,CWEs,Context,Fingerprint
Parsing failed to find end of parameter list; semicolon terminated it in (
            lhs.begin(), lhs.end(), rhs.begin(), rhs.end(), [](const std::string& lhs, const std::string& rhs) {
                return Strings::trim(StringView(lhs)) == Strings::trim(StringView(rhs
Parsing failed to find end of parameter list; semicolon terminated it in (lhs.feature_paragraphs.begin(),
                          lhs.feature_paragraphs.end(),
                          rhs.feature_paragraphs.begin(),
                          rhs.feature_paragraphs.end(
...
(list of hits)
...
[INFO] [Flawfinder] [2020-12-02T21:35:17Z] > Creating report
[FATA] [Flawfinder] [2020-12-02T21:35:17Z] > record on line 2: wrong number of fields

The report generation fails due to the two parsing errors. The second one can be found starting on line 63 in vcpkg/toolsrc/src/vcpkg/sourceparagraph.cpp:

return std::equal(lhs.feature_paragraphs.begin(),
                  lhs.feature_paragraphs.end(),
                  rhs.feature_paragraphs.begin(),
                  rhs.feature_paragraphs.end(),
                  [](const std::unique_ptr<FeatureParagraph>& lhs,
                     const std::unique_ptr<FeatureParagraph>& rhs) { return *lhs == *rhs; });

The error results from the parsing being terminated by the ; within the body of the lambda. I believe a potential solution is to update extract_c_parameters to track the current curly brace level and ignore the occurrences of ; at a level of 1 or greater. I'll test this and submit a pull request if successful.

-Ken

[david-a-wheeler]

Thanks for the report. DOUBLE thanks for telling me how to reproduce it & giving me very clear information on the problem :-).

Your proposed solution sounds exactly right. If you can't get a working pull request, let me know.

[david-a-wheeler]

Hopefully we've resolved this in the development branch. Agree?

[kpeaton]

Yup! That should do it.

Glad someone else was tracking it. I patched it locally but was distracted by other work before I could get it into a pull request.

The only difference I had was an extra error check after the decrement:

if curlylevel < 0:
    internal_warn(
        "Parsing failed to find end of parameter list; "
        "unbalanced brace in %s" % text[pos:pos + 200])
    return parameters