Fixes #624 Checks max_coverage_drop on failed tests

[thomas07vt]

Fixes #624

Just taking out the max coverage drop check outside the success if statement. I can't think of a reason to leave it in the success if statement, but of course I am not too familiar with this codebase :) . Can you think of a reason why we would want to write to .last_run.json on failing tests?

The only thing that I wasn't sure of is, if a test is failing, maybe we want to leave the pre-existing non-zero exit status rather than writing an exit status of 3.

[PragTob]

Hi, first thanks for your contribution! I'll take a look, but first it seems that the build is failing because rubocop inspects some vendored gems... grml.

[thomas07vt]

Yeah I checked the .rubocop.yml file and its using an exclude path that seems like it should work 'vendor/bundle/**/*' so I am not sure what's going on...

[PragTob]

and that failure is a bug in rubocop... PRing then workaround fixing. Then reviewing here hopefully :)

[PragTob]

👋

So, thanks for the PR.

I'm unsure about changing the behavior to always check for coverage drop, especially when tests fail etc. we shouldn't do it imo.

So, imo we should probably just write the last_run json when it was successful - i.e. do the write within the bigger if. If I understand the bug correctly this should also solve it, right? :)

[PragTob]

Argh... I guess all that at_exit behaviour needs to find its way into some other place and refactored, not here, I'm even scared to touch it :D

[PragTob]

Nope, this one we sitll need

[thomas07vt]

If I understand the bug correctly this should also solve it, right? :)

yeah that should work too. I can update my PR to add that change.

[PragTob]

That'd be great!

[thomas07vt]

@PragTob would you prefer the duplicate write call as I have it in the PR, or a duplicate check on ```@exit_status`` like this:

    if @exit_status == SimpleCov::ExitCodes::SUCCESS # No other errors
      if covered_percent < SimpleCov.minimum_coverage # rubocop:disable Metrics/BlockNesting
        $stderr.printf("Coverage (%.2f%%) is below the expected minimum coverage (%.2f%%).\n", covered_percent, SimpleCov.minimum_coverage)
        @exit_status = SimpleCov::ExitCodes::MINIMUM_COVERAGE
      elsif covered_percentages.any? { |p| p < SimpleCov.minimum_coverage_by_file } # rubocop:disable Metrics/BlockNesting
        $stderr.printf("File (%s) is only (%.2f%%) covered. This is below the expected minimum coverage per file of (%.2f%%).\n", SimpleCov.result.least_covered_file, covered_percentages.min, SimpleCov.minimum_coverage_by_file)
        @exit_status = SimpleCov::ExitCodes::MINIMUM_COVERAGE
      elsif (last_run = SimpleCov::LastRun.read) # rubocop:disable Metrics/BlockNesting
        coverage_diff = last_run["result"]["covered_percent"] - covered_percent
        if coverage_diff > SimpleCov.maximum_coverage_drop # rubocop:disable Metrics/BlockNesting
          $stderr.printf("Coverage has dropped by %.2f%% since the last time (maximum allowed: %.2f%%).\n", coverage_diff, SimpleCov.maximum_coverage_drop)
          @exit_status = SimpleCov::ExitCodes::MAXIMUM_COVERAGE_DROP
        end
      end
    end

    if @exit_status == SimpleCov::ExitCodes::SUCCESS
      SimpleCov::LastRun.write(:result => {:covered_percent => covered_percent})
    end

I kinda like the duplicate @exit_status check better, but I'll defer to you :)

[PragTob]

I think I'd just move the write tot he bottom of the bigger if and keep the exit_status check as before I think. But this whole code is ripe for some proper refactoring though :)

[thomas07vt]

@PragTob How about this?

[PragTob]

Should be good enough, thanks! :)