Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test_runner overwrites pubspec.yaml (was: How do I run the tests?) #18

Closed
sethladd opened this issue May 28, 2015 · 19 comments
Closed

test_runner overwrites pubspec.yaml (was: How do I run the tests?) #18

sethladd opened this issue May 28, 2015 · 19 comments

Comments

@sethladd
Copy link
Contributor

Should I just run the the individual files under test/ ? If there's a document that tells me how to test my patch, let me know and I'll give it a shot.

Thanks!

@mkustermann
Copy link
Member

Something like this should do it:

pub-dartlang-dart/ $ cd app
pub-dartlang-dart/app $ pub get
pub-dartlang-dart/app $ pub global activate test_runner
pub-dartlang-dart/app $ pub global run test_runner

@sethladd
Copy link
Contributor Author

Thanks, will try it out. Looks like I'm missing something?

~/Code/pub-dartlang-dart/app[case-insensitive-changelog*] $ pub global run test_runner

Checking Dart SDK binaries...
Dart SDK binaries OK.

Looking for Dart project in "./"...
Found project "foobar_pkg".

Looking for test suites...
Found 4 test suites (4 Standalone VM, 0 Dartium).  

Running all tests...
Test suite failed: configuration_test.dart
Detailed results of test suite configuration_test.dart:
┌──────────────────────────────────────────────────────
│ 
│ Unhandled exception:
│ Uncaught Error: Load Error: FileSystemException: Cannot open file, path = '/Users/sethladd/Code/pub-dartlang-dart/app/test/__test_runner/packages/unittest/vm_config.dart' (OS Error: No such file or directory, errno = 2)
│ Stack Trace:
│ #0      _asyncLoadErrorCallback (dart:_builtin:268)
│ #1      _asyncLoadError (dart:_builtin:287)
│ #2      _loadDataAsyncLoadPort.<anonymous closure> (dart:_builtin:304)
│ #3      _RootZone.runUnary (dart:async/zone.dart:1155)
│ #4      _Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:494)
│ #5      _Future._propagateToListeners (dart:async/future_impl.dart:577)
│ #6      _Future._complete (dart:async/future_impl.dart:358)
│ #7      _cancelAndValue (dart:async/stream_pipe.dart:62)
│ #8      Stream.first.<anonymous closure> (dart:async/stream.dart:911)
│ #9      _RootZone.runUnaryGuarded (dart:async/zone.dart:1093)
│ #10     _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:341)
│ #11     _BufferingStreamSubscription._add (dart:async/stream_impl.dart:270)
│ #12     _StreamController&&_SyncStreamControllerDispatch._sendData (dart:async/stream_controller.dart:698)
│ #13     _StreamController._add (dart:async/stream_controller.dart:570)
│ #14     _StreamController.add (dart:async/stream_controller.dart:516)
│ #15     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:142)
│ 
│ #0      _rootHandleUncaughtError.<anonymous closure> (dart:async/zone.dart:886)
│ #1      _asyncRunCallbackLoop (dart:async/schedule_microtask.dart:41)
│ #2      _asyncRunCallback (dart:async/schedule_microtask.dart:48)
│ #3      _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:96)
│ #4      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:143)
└──────────────────────────────────────────────────────
Test suite failed: handlers_test.dart
Detailed results of test suite handlers_test.dart:
┌─────────────────────────────────────────────────
│ 
│ Unhandled exception:
│ Uncaught Error: Load Error: FileSystemException: Cannot open file, path = '/Users/sethladd/Code/pub-dartlang-dart/app/test/__test_runner/packages/unittest/vm_config.dart' (OS Error: No such file or directory, errno = 2)
│ Stack Trace:
│ #0      _asyncLoadErrorCallback (dart:_builtin:268)
│ #1      _asyncLoadError (dart:_builtin:287)
│ #2      _loadDataAsyncLoadPort.<anonymous closure> (dart:_builtin:304)
│ #3      _RootZone.runUnary (dart:async/zone.dart:1155)
│ #4      _Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:494)
│ #5      _Future._propagateToListeners (dart:async/future_impl.dart:577)
│ #6      _Future._complete (dart:async/future_impl.dart:358)
│ #7      _cancelAndValue (dart:async/stream_pipe.dart:62)
│ #8      Stream.first.<anonymous closure> (dart:async/stream.dart:911)
│ #9      _RootZone.runUnaryGuarded (dart:async/zone.dart:1093)
│ #10     _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:341)
│ #11     _BufferingStreamSubscription._add (dart:async/stream_impl.dart:270)
│ #12     _StreamController&&_SyncStreamControllerDispatch._sendData (dart:async/stream_controller.dart:698)
│ #13     _StreamController._add (dart:async/stream_controller.dart:570)
│ #14     _StreamController.add (dart:async/stream_controller.dart:516)
│ #15     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:142)
│ 
│ #0      _rootHandleUncaughtError.<anonymous closure> (dart:async/zone.dart:886)
│ #1      _asyncRunCallbackLoop (dart:async/schedule_microtask.dart:41)
│ #2      _asyncRunCallback (dart:async/schedule_microtask.dart:48)
│ #3      _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:96)
│ #4      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:143)
└─────────────────────────────────────────────────
Test suite failed: tarball_storage_namer_test.dart
Detailed results of test suite tarball_storage_namer_test.dart:
┌──────────────────────────────────────────────────────────────
│ 
│ Unhandled exception:
│ Uncaught Error: Load Error: FileSystemException: Cannot open file, path = '/Users/sethladd/Code/pub-dartlang-dart/app/test/__test_runner/packages/unittest/vm_config.dart' (OS Error: No such file or directory, errno = 2)
│ Stack Trace:
│ #0      _asyncLoadErrorCallback (dart:_builtin:268)
│ #1      _asyncLoadError (dart:_builtin:287)
│ #2      _loadDataAsyncLoadPort.<anonymous closure> (dart:_builtin:304)
│ #3      _RootZone.runUnary (dart:async/zone.dart:1155)
│ #4      _Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:494)
│ #5      _Future._propagateToListeners (dart:async/future_impl.dart:577)
│ #6      _Future._complete (dart:async/future_impl.dart:358)
│ #7      _cancelAndValue (dart:async/stream_pipe.dart:62)
│ #8      Stream.first.<anonymous closure> (dart:async/stream.dart:911)
│ #9      _RootZone.runUnaryGuarded (dart:async/zone.dart:1093)
│ #10     _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:341)
│ #11     _BufferingStreamSubscription._add (dart:async/stream_impl.dart:270)
│ #12     _StreamController&&_SyncStreamControllerDispatch._sendData (dart:async/stream_controller.dart:698)
│ #13     _StreamController._add (dart:async/stream_controller.dart:570)
│ #14     _StreamController.add (dart:async/stream_controller.dart:516)
│ #15     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:142)
│ 
│ #0      _rootHandleUncaughtError.<anonymous closure> (dart:async/zone.dart:886)
│ #1      _asyncRunCallbackLoop (dart:async/schedule_microtask.dart:41)
│ #2      _asyncRunCallback (dart:async/schedule_microtask.dart:48)
│ #3      _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:96)
│ #4      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:143)
└──────────────────────────────────────────────────────────────
Test suite failed: backend_test.dart
Detailed results of test suite backend_test.dart:
┌────────────────────────────────────────────────
│ 
│ Unhandled exception:
│ Uncaught Error: Load Error: FileSystemException: Cannot open file, path = '/Users/sethladd/Code/pub-dartlang-dart/app/test/__test_runner/packages/unittest/vm_config.dart' (OS Error: No such file or directory, errno = 2)
│ Stack Trace:
│ #0      _asyncLoadErrorCallback (dart:_builtin:268)
│ #1      _asyncLoadError (dart:_builtin:287)
│ #2      _loadDataAsyncLoadPort.<anonymous closure> (dart:_builtin:304)
│ #3      _RootZone.runUnary (dart:async/zone.dart:1155)
│ #4      _Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:494)
│ #5      _Future._propagateToListeners (dart:async/future_impl.dart:577)
│ #6      _Future._complete (dart:async/future_impl.dart:358)
│ #7      _cancelAndValue (dart:async/stream_pipe.dart:62)
│ #8      Stream.first.<anonymous closure> (dart:async/stream.dart:911)
│ #9      _RootZone.runUnaryGuarded (dart:async/zone.dart:1093)
│ #10     _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:341)
│ #11     _BufferingStreamSubscription._add (dart:async/stream_impl.dart:270)
│ #12     _StreamController&&_SyncStreamControllerDispatch._sendData (dart:async/stream_controller.dart:698)
│ #13     _StreamController._add (dart:async/stream_controller.dart:570)
│ #14     _StreamController.add (dart:async/stream_controller.dart:516)
│ #15     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:142)
│ 
│ #0      _rootHandleUncaughtError.<anonymous closure> (dart:async/zone.dart:886)
│ #1      _asyncRunCallbackLoop (dart:async/schedule_microtask.dart:41)
│ #2      _asyncRunCallback (dart:async/schedule_microtask.dart:48)
│ #3      _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:96)
│ #4      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:143)
└────────────────────────────────────────────────

Summary: ALL 4 TEST SUITE(S) FAILED.

@sethladd
Copy link
Contributor Author

Here's my pubspec.lock:

~/Code/pub-dartlang-dart/app[case-insensitive-changelog*] $ cat pubspec.lock
# Generated by pub
# See http://pub.dartlang.org/doc/glossary.html#lockfile
packages:
  collection:
    description: collection
    source: hosted
    version: "1.1.1"
  crypto:
    description: crypto
    source: hosted
    version: "0.9.0"
  gcloud:
    description: gcloud
    source: hosted
    version: "0.1.4+1"
  googleapis:
    description: googleapis
    source: hosted
    version: "0.5.5"
  googleapis_beta:
    description: googleapis_beta
    source: hosted
    version: "0.9.0"
  http:
    description: http
    source: hosted
    version: "0.11.2"
  http_parser:
    description: http_parser
    source: hosted
    version: "0.0.2+6"
  path:
    description: path
    source: hosted
    version: "1.3.5"
  source_span:
    description: source_span
    source: hosted
    version: "1.1.2"
  stack_trace:
    description: stack_trace
    source: hosted
    version: "1.3.2"
  string_scanner:
    description: string_scanner
    source: hosted
    version: "0.1.3+1"

Does that match what you have?

@zoechi
Copy link

zoechi commented May 28, 2015

This is missing a dependency to unittest
(Cannot open file, path = '/Users/sethladd/Code/pub-dartlang-dart/app/test/__test_runner/packages/unittest/vm_config.dart').
The pubspec.lock doesn't contain unittest

@sethladd
Copy link
Contributor Author

interesting. thanks for the analysis. I'll add it to the dev_dependencies.

Hm, looks like my .lock file was messed up. Reverted! :)

@sethladd
Copy link
Contributor Author

Oh, whoa! Looks like pub global run test_runner overwrites the pubspec.yaml file in app/. To repro:

  • clone the repo
  • cd app/
  • pub global run test_runner
  • git status
~/Code/pub-dartlang-dart/app[master*] $ git status
On branch master
Your branch is up-to-date with 'origin/master'.
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

    modified:   pubspec.yaml

no changes added to commit (use "git add" and/or "git commit -a")
~/Code/pub-dartlang-dart/app[master*] $ git diff
diff --git a/app/pubspec.yaml b/app/pubspec.yaml
index 06bfc51..9709d69 100644
--- a/app/pubspec.yaml
+++ b/app/pubspec.yaml
@@ -1,26 +1,7 @@
-name: pub_dartlang_org
-version: 0.1.0
-author: Dart Team
-description: The pub.dartlang.org website.
-dependencies:
-  appengine: '>=0.3.0 <0.4.0'
-  gcloud: '>=0.1.4+1 <0.2.0'
-  googleapis: '>=0.5.4 <0.6.0'
-  googleapis_auth: '>=0.2.2 <0.2.3'
-  http: '>=0.11.0 <0.12.0'
-  intl: '>=0.11.0 <0.12.0'
-  logging: '>=0.9.3 <1.0.0'
-  markdown: '>=0.7.0 <0.8.0'
-  mustache: '>=0.2.3 <0.3.0'
-  pub_server: ">=0.1.0 <0.2.0"
-  shelf: '>=0.5.6 <0.6.0'
-  uuid: ">=0.5.0 <0.5.1"
-
-dev_dependencies:
-  unittest: ">=0.11.0 <0.12.0"
+name: foobar_pkg
+version: 0.1.1
+author: Hans Juergen <hans@juergen.com>
+homepage: http://hans.juergen.com

-dependency_overrides:
-  markdown:
-    path: ../pkg/markdown
-  appengine:
-    path: ../pkg/appengine
+dependencies:
+  gcloud: any

@sethladd sethladd changed the title How do I run the tests? test_runner overwrites pubspec.yaml (was: How do I run the tests?) May 28, 2015
@sethladd sethladd added bug and removed question labels May 28, 2015
@zoechi
Copy link

zoechi commented May 28, 2015

Is the test_runner still maintained? Last update 02/2015. I'd suggest to port the tests to test. Shouldn't be too hard. Usually changing the import from unittest to test and removing the lines related to xxx_config is enough, at least for console tests.

@zoechi
Copy link

zoechi commented May 28, 2015

test_runner might look into subdirectories and run tests for all found project directories. pub run test doesn't do this (only current project).

@mkustermann
Copy link
Member

There should be no confusion about the contents of pubspec.lock because it's checked into the repository.

How would test_runner look into other package's tests, the symlinks are only pointing to lib folders? Even recursively following down will not discover more tests.

test_runner works fine for me on linux

~/repositories/pub-dartlang-dart/app % test_runner
Checking Dart SDK binaries...
Dart SDK binaries OK.

Looking for Dart project in "./"...
Found project "pub_dartlang_org".

Looking for test suites...
Found 7 test suites (7 Standalone VM, 0 Dartium).  

Running all tests...
Test suite passed: tarball_storage_namer_test.dart
Test suite passed: configuration_test.dart
Test suite passed: handlers_test.dart
Test suite passed: backend_test.dart

Summary: ALL 7 TEST SUITE(S) PASSED.

@sethladd
Copy link
Contributor Author

I will try test_runner instead of pub global run test_runner and let you know what happens. I don't have a clue as to why pub global run test_runner is overwriting my pubspec.yaml file. (does that happen for you?)

@sethladd
Copy link
Contributor Author

test_runner, or the tests that it is running, are still overwriting my pubspec.yaml file.

~/Code/pub-dartlang-dart/app[master] $ test_runner

Checking Dart SDK binaries...
Dart SDK binaries OK.

Looking for Dart project in "./"...
Found project "pub_dartlang_org".

Looking for test suites...
Found 4 test suites (4 Standalone VM, 0 Dartium).  

Running all tests...
Test suite passed: configuration_test.dart
Test suite passed: tarball_storage_namer_test.dart
Test suite passed: handlers_test.dart
Test suite failed: backend_test.dart
Detailed results of test suite backend_test.dart:
┌────────────────────────────────────────────────
│ PASS: backend Backend.latestPackages one package
│ PASS: backend Backend.latestPackages empty
│ PASS: backend Backend.latestPackageVersions one package
│ PASS: backend Backend.latestPackageVersions empty
│ PASS: backend Backend.lookupPackage exists
│ PASS: backend Backend.lookupPackage does not exist
│ PASS: backend Backend.lookupPackageVersion exists
│ PASS: backend Backend.lookupPackageVersion does not exist
│ PASS: backend Backend.lookupLatestVersions one version
│ PASS: backend Backend.lookupLatestVersions empty
│ PASS: backend Backend.versionsOfPackage one version
│ PASS: backend Backend.versionsOfPackage empty
│ PASS: backend Backend.downloadUrl
│ PASS: backend.repository GCloudRepository.addUploader not logged in
│ PASS: backend.repository GCloudRepository.addUploader not authorized
│ PASS: backend.repository GCloudRepository.addUploader package does not exist
│ PASS: backend.repository GCloudRepository.addUploader already exists
│ PASS: backend.repository GCloudRepository.addUploader successful
│ PASS: backend.repository GCloudRepository.removeUploader not logged in
│ PASS: backend.repository GCloudRepository.removeUploader not authorized
│ PASS: backend.repository GCloudRepository.removeUploader package does not exist
│ PASS: backend.repository GCloudRepository.removeUploader cannot remove last uploader
│ PASS: backend.repository GCloudRepository.removeUploader cannot remove non-existent uploader
│ PASS: backend.repository GCloudRepository.removeUploader successful
│ PASS: backend.repository GCloudRepository.downloadUrl successful
│ PASS: backend.repository GCloudRepository.download successful
│ PASS: backend.repository GCloudRepository.lookupVersion not found
│ PASS: backend.repository GCloudRepository.lookupVersion successful
│ PASS: backend.repository GCloudRepository.versions not found
│ PASS: backend.repository GCloudRepository.versions found
│ PASS: backend.repository uploading GCloudRepository.startAsyncUpload no active user
│ PASS: backend.repository uploading GCloudRepository.startAsyncUpload successful
│ PASS: backend.repository uploading GCloudRepository.finishAsyncUpload upload-too-big
│ ERROR: backend.repository uploading GCloudRepository.finishAsyncUpload successful
│   Test failed: Caught Failed to read tarball contents.
│   package:pub_dartlang_org/utils.dart 32:29  readTarballFile.<async>
│ PASS: backend.repository uploading GCloudRepository.upload not logged in
│ FAIL: backend.repository uploading GCloudRepository.upload not authorized
│   Expected: true
│     Actual: <false>
│   
│   package:unittest/src/matcher/expect.dart 95:20      expect
│   test/backend_test.dart 706:15                       main.<fn>.<fn>.<fn>.<fn>.<async>.<fn>.<async>.<fn>
│   package:unittest/src/expected_function.dart 146:52  ExpectedFunction._max2
│ FAIL: backend.repository uploading GCloudRepository.upload versions already exist
│   Expected: true
│     Actual: <false>
│   
│   package:unittest/src/matcher/expect.dart 95:20      expect
│   test/backend_test.dart 727:15                       main.<fn>.<fn>.<fn>.<fn>.<async>.<fn>.<async>.<fn>
│   package:unittest/src/expected_function.dart 146:52  ExpectedFunction._max2
│ PASS: backend.repository uploading GCloudRepository.upload upload-too-big
│ ERROR: backend.repository uploading GCloudRepository.upload successful
│   Test failed: Caught Failed to read tarball contents.
│   package:pub_dartlang_org/utils.dart 32:29  readTarballFile.<async>
│ 
│ 35 PASSED, 2 FAILED, 2 ERRORS
└────────────────────────────────────────────────

Summary: 1 TEST SUITE(S) FAILED. 3 TEST SUITE(S) PASSED.

~/Code/pub-dartlang-dart/app[master*] $ git status
On branch master
Your branch is up-to-date with 'origin/master'.
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

    modified:   pubspec.yaml

no changes added to commit (use "git add" and/or "git commit -a")
~/Code/pub-dartlang-dart/app[master*] $ 

and

~/Code/pub-dartlang-dart/app[master*] $ git diff pubspec.yaml 
diff --git a/app/pubspec.yaml b/app/pubspec.yaml
index 06bfc51..9709d69 100644
--- a/app/pubspec.yaml
+++ b/app/pubspec.yaml
@@ -1,26 +1,7 @@
-name: pub_dartlang_org
-version: 0.1.0
-author: Dart Team
-description: The pub.dartlang.org website.
-dependencies:
-  appengine: '>=0.3.0 <0.4.0'
-  gcloud: '>=0.1.4+1 <0.2.0'
-  googleapis: '>=0.5.4 <0.6.0'
-  googleapis_auth: '>=0.2.2 <0.2.3'
-  http: '>=0.11.0 <0.12.0'
-  intl: '>=0.11.0 <0.12.0'
-  logging: '>=0.9.3 <1.0.0'
-  markdown: '>=0.7.0 <0.8.0'
-  mustache: '>=0.2.3 <0.3.0'
-  pub_server: ">=0.1.0 <0.2.0"
-  shelf: '>=0.5.6 <0.6.0'
-  uuid: ">=0.5.0 <0.5.1"
-
-dev_dependencies:
-  unittest: ">=0.11.0 <0.12.0"
+name: foobar_pkg
+version: 0.1.1
+author: Hans Juergen <hans@juergen.com>
+homepage: http://hans.juergen.com

-dependency_overrides:
-  markdown:
-    path: ../pkg/markdown
-  appengine:
-    path: ../pkg/appengine
+dependencies:
+  gcloud: any
~/Code/pub-dartlang-dart/app[master*] $ 

@sethladd
Copy link
Contributor Author

Sorry to be a pain here, but I really appreciate your help. I wanted to submit a patch for the server, but I didn't want to do that until I could pass the tests.

@mkustermann could you try a fresh checkout in a fresh directory, to see what happens?

  • mkdir ~/tmp/pubserver
  • cd ~/tmp/pubserver
  • git clone git@github.com:dart-lang/pub-dartlang-dart.git
  • cd pub-dartlang-dart
  • cd app
  • test_runner

@mkustermann
Copy link
Member

On my linux machine:

kustermann@mkustermann /tmp % export PUB_CACHE=/tmp/new-pub-cache
kustermann@mkustermann /tmp % git clone git@github.com:dart-lang/pub-dartlang-dart.git
Cloning into 'pub-dartlang-dart'...
Enter passphrase for key '/usr/local/google/home/kustermann/.ssh/id_rsa': 
remote: Counting objects: 944, done.
remote: Total 944 (delta 0), reused 0 (delta 0), pack-reused 944
Receiving objects: 100% (944/944), 587.45 KiB | 1.01 MiB/s, done.
Resolving deltas: 100% (482/482), done.
Checking connectivity... done.
kustermann@mkustermann /tmp % cd pub-dartlang-dart 
kustermann@mkustermann /tmp/pub-dartlang-dart % cd app
kustermann@mkustermann /tmp/pub-dartlang-dart/app % pub --version
Pub 1.10.0
kustermann@mkustermann /tmp/pub-dartlang-dart/app % pub global activate test_runner
Resolving dependencies... (11.0s) 
+ ansicolor 0.0.9
+ args 0.12.2+6 (0.13.1 available)
+ collection 1.1.1
+ coverage 0.5.0 (0.6.5 available)
+ crypto 0.9.0
+ http 0.11.2
+ http_parser 0.0.2+6
+ logging 0.11.1
+ matcher 0.12.0+1
+ mockable_filesystem 0.0.3
+ path 1.3.5
+ pool 1.0.2
+ quiver 0.21.4
+ source_span 1.1.2
+ stack_trace 1.3.2
+ string_scanner 0.1.3+1
+ supports_color 0.1.1
+ test_runner 0.2.16+1
+ unittest 0.11.6+1 (0.12.2 available)
+ unscripted 0.5.2+1 (0.6.1+1 available)
+ yaml 2.1.2
Downloading test_runner 0.2.16+1...
Downloading ansicolor 0.0.9...
Downloading coverage 0.5.0...
Downloading pool 1.0.2...
Downloading unittest 0.11.6+1...
Downloading unscripted 0.5.2+1...
Downloading mockable_filesystem 0.0.3...
Downloading supports_color 0.1.1...
Downloading quiver 0.21.4...
Downloading logging 0.11.1...
Downloading matcher 0.12.0+1...
Precompiling executables... 
Loading source assets... 
Precompiled test_runner:test_runner.
Precompiled test_runner:run_tests.
Activated test_runner 0.2.16+1.
Installed executables run_tests and test_runner.
kustermann@mkustermann /tmp/pub-dartlang-dart/app % pub global run test_runner     

Checking Dart SDK binaries...
Dart SDK binaries OK.

Looking for Dart project in "./"...
Found project "pub_dartlang_org".

Looking for test suites...
Found 4 test suites (4 Standalone VM, 0 Dartium).  

Running all tests...
Test suite passed: tarball_storage_namer_test.dart
Test suite passed: configuration_test.dart
Test suite passed: handlers_test.dart
Test suite passed: backend_test.dart

Summary: ALL 4 TEST SUITE(S) PASSED.

kustermann@mkustermann /tmp/pub-dartlang-dart/app % git status
# On branch master
# Your branch is up-to-date with 'origin/master'.
#
nothing to commit, working directory clean

@sethladd
Copy link
Contributor Author

Thanks!

Hm, maybe the tests aren't cleaning up, because I'm getting an auth failure, so my directory is left in a broken state.

│ ERROR: backend.repository uploading GCloudRepository.finishAsyncUpload successful
│   Test failed: Caught Failed to read tarball contents.
│   package:pub_dartlang_org/utils.dart 32:29  readTarballFile.<async>
│ PASS: backend.repository uploading GCloudRepository.upload not logged in
│ FAIL: backend.repository uploading GCloudRepository.upload not authorized

Could I get permissions so my tests would pass? Or, can we run the tests in a way that don't require permissions (so that anyone can run the tests) ?

@mkustermann
Copy link
Member

These tests are only unit tests, there are no permissions necessary AFAIK.

I think I know what the issue could be. So the line in question is app/lib/utils.dart :

 29 
 30 Future<String> readTarballFile(String path, String name) async {
 31   var result = await Process.run('tar', ['-xzf', path, name, '-O']);
 32   if (result.exitCode != 0) throw 'Failed to read tarball contents.';
 33 
 34   return result.stdout;
 35 }

Maybe you either have no tar installed or it behaves differently on macos (e.g. doesn't support -xzf or `-O')?

Could you modify this code and add print(result.stdout); print(result.stderr) ?

@sethladd
Copy link
Contributor Author

Thanks! Looks like -O isn't a thing.

Detailed results of test suite backend_test.dart:
┌────────────────────────────────────────────────
│ tar: -O: Not found in archive
│ tar: Error exit delayed from previous errors.
│ 
│ 
│ tar: -O: Not found in archive
│ tar: Error exit delayed from previous errors.
│ 
│ 
│ tar: -O: Not found in archive
│ tar: Error exit delayed from previous errors.
│ 
│ 
│ tar: -O: Not found in archive
│ tar: Error exit delayed from previous errors.
│ 

@sethladd
Copy link
Contributor Author

still confused why my pubspec.yaml is rewritten during the tests... maybe that's by design, and would be reset if all the tests pass?

I see, the test/utils.dart is writing to app/pubspec.yaml. Maybe we should write to a different file during the tests?

@mkustermann
Copy link
Member

I see, the test/utils.dart is writing to app/pubspec.yaml. Maybe we should write to a different file during the tests?

There is no such thing in test/utils.dart -- though there is something like that in test/backend_test_utils.dart: Our tests are creating a temporary directory, and create a pubspec.yaml file inside there:

See here:

Future withTestPackage(Future func(List<int> tarball)) {
  return withTempDirectory((String tmp) async {
    var readme = new File('$tmp/README.md');
    var changelog = new File('$tmp/CHANGELOG.md');
    var pubspec = new File('$tmp/pubspec.yaml');

    await readme.writeAsString(TestPackageReadme);
    await changelog.writeAsString(TestPackageChangelog);
    await pubspec.writeAsString(TestPackagePubspec);
    ...
  });

I debugged the issue now and the reason is the following. The tests create a temporary directory fill it with all necessary files, create a tarball and then call backend.upload(). This in turn will download the tarball to disc and extract files (such as pubspec.yaml, README.md, ...). The corresponding code is here and the readTarballFile function is defined here as follows:

Future<String> readTarballFile(String path, String name) async {
  var result = await Process.run('tar', ['-xzf', path, name, '-O']);
  if (result.exitCode != 0) throw 'Failed to read tarball contents.';

  return result.stdout;
}

Please notice the -O option which tells tar to write it's files to stdout instead of writing it to disk. Turns out, that on linux, this option works even at the end of the argument (as many other commands), whereas for the macos tar, that seems to not work. I've moved the -O flag now to the beginning which should work on linux and mac (see cf3c946).

Hope that ends this very long discussion :)

@sethladd
Copy link
Contributor Author

sethladd commented Jun 1, 2015

Thanks! Will try it out.

~/Code/pub-dartlang-dart/app[master] $ test_runner

Checking Dart SDK binaries...
Dart SDK binaries OK.

Looking for Dart project in "./"...
Found project "pub_dartlang_org".

Looking for test suites...
Found 4 test suites (4 Standalone VM, 0 Dartium).  

Running all tests...
Test suite passed: tarball_storage_namer_test.dart
Test suite passed: configuration_test.dart
Test suite passed: handlers_test.dart
Test suite passed: backend_test.dart

Summary: ALL 4 TEST SUITE(S) PASSED.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants