Simplecov cannot generate a new coverage report because it does not skip writing already existing asset files. #741
Comments
|
Hi there @chase-stevens, thanks for the report 💚 The interesting question for me would be why it doesn't have access to these files any more? Seems like there might be something wrong there? The reason it writes already existing asset files again is that the gem version and hence the files might have changed. Sure, we could include some hash in the file names or even the version names but that'd require a build process etc... as the time to write the files again isn't too big we just write them new on each run. I'm also not sure if it'd totally solve your problem. My fear is that if the access rights for the asset files aren't given then simplecov probably also doesn't have the rights to override So - I don't think there's much we can do here and it seems more a problem with your file permissions. If you think we can help please let us know :) |
|
@chase-stevens 👋 hi there, any update on this and the questions from your side? :) |
|
Closing due to missing feedback |
|
When using Nix all the gems are stored read-only in As you can see I would propose that the fix would be to chmod the files, e.g. 644? |
|
@felixscheinost what is "Nix" ? I'm not sure whether this is another problem we should fix or that users should fix... like we already integrate with so much. Us messing with chmoding prodocues potential further issues (what if we can't chmod...) so I'm almost inclined to say that this would be user level to chmod them themselves. |
|
Sorry, forgot to explain what Nix is - seems like I am too much in my own filter bubble :) Nix is a build tool and package manager that focuses on purity and reproducibility. I'll try to find a user level solution and will get back to you later. |
|
@felixscheinost have you found a user level solution yet? While I generally would like to have things working everywhere, doing this would introduce some overhead. Of course, if we switched the asset compilation on simplecov-html to something more modern, have it easy to attach a hash to the file name then it'd be easy but right now I think there are other issues deserving our attention. |
|
I'm a NixOS user too, and I've worked around this by adding the following to require 'simplecov'
Pathname.new(__FILE__).join('..','..','coverage').tap do |cov|
# clear old coverage results
FileUtils.rm_rf cov if cov.exist?
end
SimpleCov.start('rails')But if simplecov skipped rewriting the assets, that would be great! |
|
I was just thinking, we might introduce asset inlining which would also solve this problem - right? |
|
Would it hurt anything if simplecov did @emptyflask 's workaround? rm the asset tree if it exists, immediately before writing new ones? |
|
@slinkp maybe 🤷 Couldn't do it like this because it deletes the whole path. People might (accidentally) throw files in there and we can't just delete them all. Could we delete all the files we're gonna copy over? Probably. Next thing I know maybe for some reason under some OS we don't have the right to create files there... and then that case starts failing all of a sudden. So, I'm not very inclined to include it into default. Asset inlining and/or having hashes build style at the end of the assets would be the proper solutions I think. For the time being people can implement this workaround. Reopening to know/remember it'd be nice to fix these. |
|
Hey, thanks to feedback here I thought about it some more and theoretically I guess we could:
As there seem to be a bit more people running into this than I expected I might try to provide a fix with higher priority, but no promises time is a bit volatile these days :) |
|
I ran into this yesterday (by way of bashcov, which I naively thought would be trivial to package 😆 for an experiment I'm doing on fuzzing shell scripts...). I took 3 swings at local workarounds with varying success; outlining them in case it helps anyone.
|
|
@abathur regarding 3. - interesting, I'd hope that bashcov would allow for this to be set/not use the html formatter if they don't need it. |
|
@PragTob Ok. I assumed it was just the internal state file (and that for some reason specifying the formatter did disable the default, but not produce an intentional output) but good to confirm. Here's my patched bashcov just in case the context helps (or in case it's dead-obvious what I'm doing wrong, but don't feel like you need to debug it, especially if you think this is a bashcov problem). $ cat /nix/store/ykfcfk0airk1l3ipsxgs872scdddshjv-ruby2.6.6-bashcov-1.8.2/lib/ruby/gems/2.6.0/gems/bashcov-1.8.2/bin/bashcov
Wed Jul 29 2020 10:09:40 -->
#!/nix/store/34x64aql2bmlffjf6jzqrbk05h9y5nxr-ruby-2.6.6/bin/ruby
# frozen_string_literal: true
lib = File.expand_path("../../lib", __FILE__)
$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
require "bashcov"
Bashcov.parse_options! ARGV
runner = Bashcov::Runner.new Bashcov.command
status = runner.run
coverage = runner.result
require "simplecov"
SimpleCov.start
SimpleCov.command_name Bashcov.command_name
SimpleCov.root Bashcov.root_directory
SimpleCov.coverage_dir ENV.fetch('BASHCOV_COVERAGE_DIR', 'coverage44')
SimpleCov.formatter SimpleCov::Formatter::SimpleFormatter
result = SimpleCov::Result.new(coverage)
if SimpleCov.use_merging
SimpleCov::ResultMerger.store_result(result)
result = SimpleCov::ResultMerger.merged_result
end
SimpleCov.at_exit do
puts "Run completed using #{Bashcov.fullname}"
result.format!
end
exit status.exitstatus |
by running `FileUtils#cp_r` with the `:remove_destination` option
enabled. This makes asset copying more robust in the face of
restrictive permissions on asset file destination paths. This should be
safe and backward-compatible given that the destination files (or their
contents) will be fully replaced anyway.
Example case:
irb(main):001:0> FileUtils.touch('foo')
=> ["foo"]
irb(main):002:0> FileUtils.touch('bar')
=> ["bar"]
irb(main):003:0> FileUtils.chmod(0o444, 'bar')
=> ["bar"]
irb(main):004:0> FileUtils.cp_r('foo', 'bar')
Errno::EACCES: Permission denied @ rb_sysopen - bar
from /nix/store/2kw126cy93rix4pmh9lcl120njnb6r7r-ruby-3.1.4/lib/ruby/3.1.0/fileutils.rb:1395:in `initialize'
irb(main):005:0> FileUtils.cp_r('foo', 'bar', remove_destination: true)
=> nil
Addresses simplecov-ruby/simplecov#741.
|
Opened a PR to fix this by enabling Also, it appears that |
|
@abathur -- probably much too late for this to be of any real help, but: rather than patching Bashcov, you could define a Here's a sample require 'pathname'
require 'simplecov'
require 'simplecov-console'
module SilenceFormatting
def silent
return unless block_given?
stdout, $stdout = $stdout, StringIO.new
yield
ensure
$stdout = stdout
end
def format(result)
silent { super }
end
end
class SilentHTMLFormatter < SimpleCov::Formatter::HTMLFormatter
include SilenceFormatting
end
class SilentConsoleFormatter < SimpleCov::Formatter::Console
include SilenceFormatting
end
def formatters(silent = false)
[].tap do |f|
if silent
f << SilentHTMLFormatter << SilentConsoleFormatter
else
f << SimpleCov::Formatter::HTMLFormatter << SimpleCov::Formatter::Console
end
begin
require 'coveralls'
f << Coveralls::SimpleCov::Formatter
rescue LoadError # rubocop:disable Lint/HandleExceptions
end
end
end
def multi_formatter(silent = false)
SimpleCov::Formatter::MultiFormatter.new(formatters(silent))
end
SimpleCov.profiles.define 'rbexec_base' do
load_profile 'bundler_filter'
add_group 'Sources', %w[bin lib]
add_group 'Tests', 'spec'
add_filter 'tmp*'
add_filter '/spec/fixtures/'
Pathname.new(__FILE__).parent.tap do |r|
add_filter do |src|
Pathname.new(src.filename).relative_path_from(r).each_filename.any? { |e| e.start_with? '.' }
end
root r
coverage_dir r + 'coverage'
end
end
SimpleCov.profiles.define 'rbexec_rspec' do
load_profile 'rbexec_base'
formatter multi_formatter
end
SimpleCov.profiles.define 'rbexec_bashcov' do
load_profile 'rbexec_base'
command_name ENV.fetch('BASHCOV_COMMAND_NAME', $0)
formatter multi_formatter(true)
Coveralls::Output.silent = true if defined? Coveralls::Output
end
SimpleCov.load_profile 'rbexec_bashcov' if ENV.key? 'BASHCOV_COMMAND_NAME' |
|
Also, Bashcov has Nix support in the works. |
Hi all,
I was able to generate a coverage/index.html the first time I ran rspec with simplecov. Each subsequent time, I have encountered an error where, when attempting to create the
coverage/index.htmlfile, simplecov gets an error that it is not allowed to open an asset file.1: from /nix/store/fnmw699zb2pk8gn38ic99vx9ax53n9g6-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1292:in `open' /nix/store/fnmw699zb2pk8gn38ic99vx9ax53n9g6-ruby-2.5.5/lib/ruby/2.5.0/fileutils.rb:1292:in `initialize': Permission denied @ rb_sysopen - /home/chase/eic/coverage/assets/0.10.2/smoothness/images/ui-bg_glass_65_ffffff_1x400.png (Errno::EACCES)What we believe to be happening is that simplecov is attempting to copy files from the gem kept in our nix/store directory and write them to the coverage/assets directory in the application. The first time writing was successful; however, simplecov does not appear to have the necessary logic to skip writing files that already exist.
Full trace below:
I am using RSpec for tests and I run my tests using the following command
nix-shell --run "rspec"I am requiring and staring SimpleCov in my spec_helper.rb as seen below. Additionally, I have been able to successfully run simplecov once before, so I do not think this is the issue.
Software Versions
Please let me know if you have any questions or if I can provide any additional information and I'll be happy to help.
Best,
Chase
The text was updated successfully, but these errors were encountered: