--exit-code-threshold does not cause a non-zero exit code

[Jordan-Nelson]

Issue

Running pana --exit-code-threshold 0 . does not cause a non-zero exit even when the package is missing points.

Reproduction steps

1. Clone a repository that is missing pub points. I am using https://pub.dev/packages/amplify_authenticator, which is scoring 110/120.
2. Run pana --exit-code-threshold 0 .
3. Observe that the output indicates that the package is missing points, but the process completes normally.

Output

INFO Running /Users/nejrd/development/flutter/bin/cache/dart-sdk/bin/dart --version...
INFO Running flutter --no-version-check --version --machine...
WARNING pana might update or modify files in ..
Analysis will begin in 15 seconds, hit CTRL+C to abort it.
To remove this message, use --no-warning.
INFO Running git rev-parse --show-toplevel...
INFO Running flutter --no-version-check packages pub upgrade --verbose...
SEVERE Problem with dart pub upgrade
Bad state: For amplify_analytics_plugin_interface, the parsed version null did not match the locked version 0.4.0.
#0 _validateLockedVersions. (package:pana/src/pkg_resolution.dart:221:11)
#1 SplayTreeMap.forEach (dart:collection/splay_tree.dart:604:8)
#2 _validateLockedVersions (package:pana/src/pkg_resolution.dart:217:12)
#3 createPkgResolution (package:pana/src/pkg_resolution.dart:203:5)
#4 PackageContext.resolveDependencies (package:pana/src/package_context.dart:67:26)

#5 PackageAnalyzer._inspect (package:pana/src/package_analyzer.dart:138:27)

#6 PackageAnalyzer.inspectDir. (package:pana/src/package_analyzer.dart:91:14)

#7 withTempDir (package:pana/src/utils.dart:233:12)

#8 main (file:///Users/nejrd/.pub-cache/hosted/pub.dartlang.org/pana-0.21.7/bin/pana.dart:172:19)

INFO Analyzing package...
INFO Running /Users/nejrd/development/flutter/bin/cache/dart-sdk/bin/dart analyze --format machine lib...
INFO Running flutter --no-version-check format --output=none --set-exit-if-changed /private/var/folders/3n/3plhqvn96jq6thcpd50d2pgh0000gr/T/pana_0Ya6tg/packages/amplify_authenticator/lib...
INFO Running /Users/nejrd/development/flutter/bin/cache/dart-sdk/bin/dart pub get ....
INFO Running /Users/nejrd/development/flutter/bin/cache/dart-sdk/bin/dart pub outdated --json --up-to-date --no-dev-dependencies --no-dependency-overrides...

✓ Follow Dart file conventions (20 / 20)

[*] 10/10 points: Provide a valid pubspec.yaml

[*] 5/5 points: Provide a valid README.md

[*] 5/5 points: Provide a valid CHANGELOG.md

✓ Provide documentation (10 / 10)

[*] 10/10 points: Package has an example

• Found example at: example/lib/main.dart

✓ Platform Support (20 / 20)

[*] 20/20 points: Supports 2 of 6 possible platforms (iOS, Android, Web, Windows, MacOS, Linux)

• ✓ Android
• ✓ iOS

These platforms are not supported:

Package does not support platform `Windows`.

Because:

• package:amplify_authenticator/amplify_authenticator.dart that imports:
• package:amplify_authenticator/src/widgets/form_field.dart that imports:
• package:amplify_authenticator/src/widgets/form.dart that imports:
• package:amplify_flutter/amplify_flutter.dart that declares support for platforms: Android, iOS.

Package does not support platform `Linux`.

Because:

• package:amplify_authenticator/amplify_authenticator.dart that imports:
• package:amplify_authenticator/src/widgets/form_field.dart that imports:
• package:amplify_authenticator/src/widgets/form.dart that imports:
• package:amplify_flutter/amplify_flutter.dart that declares support for platforms: Android, iOS.

Package does not support platform `macOS`.

Because:

• package:amplify_authenticator/amplify_authenticator.dart that imports:
• package:amplify_authenticator/src/widgets/form_field.dart that imports:
• package:amplify_authenticator/src/widgets/form.dart that imports:
• package:amplify_flutter/amplify_flutter.dart that declares support for platforms: Android, iOS.

Package does not support platform `Web`.

Because:

• package:amplify_authenticator/amplify_authenticator.dart that imports:
• package:amplify_authenticator/src/widgets/form_field.dart that imports:
• package:amplify_authenticator/src/widgets/form.dart that imports:
• package:amplify_flutter/amplify_flutter.dart that declares support for platforms: Android, iOS.

✗ Pass static analysis (20 / 30)

[~] 20/30 points: code has no errors, warnings, lints, or formatting issues

INFO: Don't import implementation files from another package.

lib/src/services/amplify_auth_service.dart:21:8

   ╷
21 │ import 'package:collection/src/iterable_extensions.dart';
   │        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   ╵

To reproduce make sure you are using the lints_core and run flutter analyze lib/src/services/amplify_auth_service.dart

✓ Support up-to-date dependencies (20 / 20)

[*] 10/10 points: All of the package dependencies are supported in the latest version

Package	Constraint	Compatible	Latest
amplify_auth_cognito	>=0.3.0 <0.5.0	0.4.0	0.4.0
amplify_auth_plugin_interface	>=0.3.0 <0.5.0	0.4.0	0.4.0
amplify_core	>=0.3.0 <0.5.0	0.4.0	0.4.0
amplify_flutter	>=0.3.0 <0.5.0	0.4.0	0.4.0
collection	^1.15.0	1.15.0	1.15.0
flutter	flutter	0.0.0	0.0.0
flutter_localizations	flutter	0.0.0	0.0.0
intl	^0.17.0	0.17.0	0.17.0

Transitive dependencies

Package	Constraint	Compatible	Latest
amplify_analytics_plugin_interface	-	0.4.0	0.4.0
amplify_api_plugin_interface	-	0.4.0	0.4.0
amplify_datastore_plugin_interface	-	0.4.0	0.4.0
amplify_storage_plugin_interface	-	0.4.0	0.4.0
characters	-	1.2.0	1.2.0
clock	-	1.1.0	1.1.0
crypto	-	3.0.1	3.0.1
date_time_format	-	2.0.1	2.0.1
json_annotation	-	4.4.0	4.4.0
material_color_utilities	-	0.1.3	0.1.4
meta	-	1.7.0	1.7.0
plugin_platform_interface	-	2.1.2	2.1.2
sky_engine	-	0.0.99	0.0.99
typed_data	-	1.3.0	1.3.0
uuid	-	3.0.6	3.0.6
vector_math	-	2.1.1	2.1.2

To reproduce run dart pub outdated --no-dev-dependencies --up-to-date --no-dependency-overrides.

[*] 10/10 points: Package supports latest stable Dart and Flutter SDKs

✓ Support sound null safety (20 / 20)

[*] 20/20 points: Package and dependencies are fully migrated to null safety!

Points: 110/120.

Am I misunderstanding the intention behind --exit-code-threshold?

[Jordan-Nelson]

Never mind. There is a non 0 exit code. I had assumed the exit code was 0 since the log output is identical to the output with --exit-code-threshold omitted. It might make sense differentiate the log output for a non-zero exit (example: --set-exit-if-changed for flutter format).

I am going to close this issue though since it is working as expected.

[mrverdant13]

Maybe related #1025