Flutter test coverage will not report untested files

[100cm]

I run the flutter test --coverage as follows

and here are my project's files;

the generated Icov.info looks like this:

SF:lib/components/button.component.dart
DA:12,1
DA:29,1
DA:45,1
DA:47,1
DA:48,1
DA:49,1
DA:50,1
DA:51,2
DA:52,1
DA:53,1
DA:56,2
DA:57,1
DA:59,2
DA:60,1
DA:61,1
DA:62,1
DA:63,3
DA:64,2
DA:65,1
DA:66,1
DA:67,1
DA:70,1
DA:71,2
DA:72,1
LF:24
LH:24
end_of_record
SF:lib/components/button.css.dart
DA:4,1
DA:6,2
DA:7,2
DA:8,1
DA:10,1
DA:16,3
DA:53,1
DA:73,0
LF:8
LH:7
end_of_record
SF:lib/environment.dart
DA:6,1
DA:8,2
DA:9,1
DA:11,1
DA:13,2
DA:14,2
DA:15,2
DA:16,1
LF:8
LH:8
end_of_record

why coverage didn't include the untested files?

i know how to show coverage by tools like genHtml locv-sumary

I just want to know how to generate the Locv.info that includes untested files in my projects

[100cm]

i searched the same question on stack overflow:

https://stackoverflow.com/questions/54602840/how-can-i-generate-test-coverage-of-untested-files-on-my-flutter-tests

but none answer it.

[BigBl4ckW0lf]

I have found a temporary workaround for this issue.

If you create a "fileLoader_test.dart" file like this:

// ignore_for_file: unused_import
import 'package:my_package/FileA.dart';
import 'package:my_package/folder_one/lib.dart';
import 'package:my_package/folder_two/lib.dart';
void main() { }

then "flutter test --coverage" will recognize all files specified in the "lib.dart" files.

[100cm]

yep @BigBl4ckW0lf I got this, but i must import all files in my project.

[jonahwilliams]

If a file is not reachable by the entrypoint that executes coverage, then because the VM has never loaded the library it doesn't return any coverage information for us.

The tester process could walk the filesystem and insert the remaining file URIs into the coverage report - it's not clear that would always be meaningful information or the right thing to do.

[dangilbert]

I'm using @BigBl4ckW0lf's idea but didn't want to manually add them in case I forget. I've got a CI step which creates the file before running the coverage using this script

#!/bin/sh
file=test/coverage_helper_test.dart
echo "// Helper file to make coverage work for all dart files\n" > $file
echo "// ignore_for_file: unused_import" >> $file
find lib -name '*.dart' | cut -c4- | awk -v package=$1 '{printf "import '\''package:%s%s'\'';\n", package, $1}' >> $file
echo "\nvoid main(){}" >> $file

Just call it with your package name as an argument and it'll generate a test including all dart files in the lib folder

[SBNTT]

Here is a version which exclude .g.dart files

#!/bin/sh
file=test/coverage_helper_test.dart
echo "// Helper file to make coverage work for all dart files\n" > $file
echo "// ignore_for_file: unused_import" >> $file
find lib -not -name '*.g.dart' -and -name '*.dart' | cut -c4- | awk -v package=$1 '{printf "import '\''package:%s%s'\'';\n", package, $1}' >> $file
echo "void main(){}" >> $file

[mbilawisdom]

Here is a version which exclude .g.dart files

#!/bin/sh
file=test/coverage_helper_test.dart
echo "// Helper file to make coverage work for all dart files\n" > $file
echo "// ignore_for_file: unused_import" >> $file
find lib -not -name '*.g.dart' -and -name '*.dart' | cut -c4- | awk -v package=$1 '{printf "import '\''package:%s%s'\'';\n", package, $1}' >> $file
echo "void main(){}" >> $file

were do I have to add thuis

[vrnvorona]

any way to avoid part libraries? Or just ignore the warning?

[Kiruel]

Hi,
If it helps I make my own script for that:

#!/bin/sh
file=test/coverage_helper_test.dart
printf "// Helper file to make coverage work for all dart files\n" > $file
printf "// **************************************************************************\n" >> $file
printf "// Because of this: https://github.com/flutter/flutter/issues/27997#issue-410722816\n" >> $file
printf "// DO NOT EDIT THIS FILE USE: sh scripts/import_files_coverage.sh YOUR_PACKAGE_NAME\n" >> $file
printf "// **************************************************************************\n" >> $file
printf "\n" >> $file
printf "// ignore_for_file: unused_import\n" >> $file
find lib -type f \( -iname "*.dart" ! -iname "*.g.dart" ! -iname "*.freezed.dart" ! -iname "generated_plugin_registrant.dart" \) | cut -c4- | awk -v package="$1" '{printf "import '\''package:%s%s'\'';\n", package, $1}' >> $file
printf "\nvoid main(){}" >> $file

Don't forget to change YOUR_PACKAGE_NAME by your package.

And you can also ignore this file on your .gitignore.

/test/coverage_helper_test.dart

[saurabh-sablok]

any way to avoid part libraries? Or just ignore the warning?

@vrnvorona do you found any way to avoid part libraries from coverage_helper_test.dart file?

[vrnvorona]

any way to avoid part libraries? Or just ignore the warning?

@vrnvorona do you found any way to avoid part libraries from coverage_helper_test.dart file?

Didn't try anything yet sadly, so no.

[danhunsaker]

any way to avoid part libraries? Or just ignore the warning?

@vrnvorona do you found any way to avoid part libraries from coverage_helper_test.dart file?

Add a -not -iname '*.part.dart' to the find command.

[humblerookie]

Its been quite some time and yet there seems to be little progress here, @jonahwilliams anything particular that's blocking this issue? Is it slated to be fixed in the next dart release?

[pdblasi]

To add to the workarounds previously provided, I wrote one up in dart that also gets rid of files with a part of directive. Since I've already got dart anywhere I need to run it, it was easier to write and use a dart script.

Just replace the packageName value with your actual package name and run with dart run in the root of your repo/package. You can also add in additional checks to the where clause around line 15 if you need to customize the files that are included.

import 'dart:io';

const packageName = 'YOUR_PACKAGE_NAME_HERE';

void main() async {
  final cwd = Directory.current.uri;
  final libDir = Directory.fromUri(cwd.resolve('lib'));
  final buffer = StringBuffer();

  var files = libDir
      .listSync(recursive: true)
      .whereType<File>()
      .where((file) =>
          file.path.endsWith('.dart') &&
          !file.path.contains('.freezed.') &&
          !file.path.contains('.g.') &&
          !file.path.endsWith('generated_plugin_registrant.dart') &&
          !file.readAsLinesSync().any((line) => line.startsWith('part of')))
      .toList();

  buffer.writeln('// ignore_for_file: unused_import');
  buffer.writeln();

  for (var file in files) {
    final fileLibPath =
        file.uri.toFilePath().substring(libDir.uri.toFilePath().length);
    buffer.writeln('import \'package:$packageName/$fileLibPath\';');
  }

  buffer.writeln();
  buffer.writeln('void main() {}');
  buffer.writeln();

  final output =
      File(cwd.resolve('test/coverage_helper_test.dart').toFilePath());
  await output.writeAsString(buffer.toString());
}

[kbublitz]

Just came across this issue as well and I found there is also a flutter package that does the exact same thing:
full_coverage
It only mentions flutter files but I see no reason why it shouldn't work for dart projects that are not flutter apps.

We added it as a dev dependency to our pubspec.yaml and just call flutter pub run full_coverage before running flutter test

[PiotrFLEURY]

Hi everyone

I used a simple workaround on my side.
Just make a fake test file and include the main.dart in it.
Every dart fil will be reachable by the coverage.

In the test dir create a fake_test.dart (name it as you want)

// ignore: unused_import
import 'package:example/main.dart';

void main() {
  // Fake test in order to make each file reachable by the coverage
}

Every file used by the app will be scanned by the coverage tool.

🎉 Enjoy 🎉

[akiller]

To add to the workarounds previously provided, I wrote one up in dart that also gets rid of files with a part of directive. Since I've already got dart anywhere I need to run it, it was easier to write and use a dart script.

Just replace the packageName value with your actual package name and run with dart run in the root of your repo/package. You can also add in additional checks to the where clause around line 15 if you need to customize the files that are included.

Thanks for this.

Just a note, if you're using Windows (boo hiss, I know!) you need to replace \ with / for the generated imports, e.g.:

final fileLibPath = file.uri
        .toFilePath()
        .substring(libDir.uri.toFilePath().length)
        .replaceAll('\\', '/'); 

[jayudey-wf]

I've used the approach suggested above in the comments and it's worked great for most of my use cases, however I was wondering if there was a way to perform this action (include untested files in the coverage calculation) when the package uses the build_runner package for generating files?

[PiotrFLEURY]

I think the better approach is to really test the main.

By default any new Flutter app already includes a Widget test for the counter app.
Most of us just deleted it once we change the main.dart content. In my opinion this is our first mistake.

The workaround I suggested can provide metrics about uncovered sources but a cleaner way to do should be to create a simple app should start Widget test like Spring boot does when you generate a new backend from start.spring.io

I am wondering if we should include this test by default in the flutter create template.