Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Driver test to confirm Firestore client behaviors #39

Merged
merged 18 commits into from
Mar 16, 2020
Merged

Driver test to confirm Firestore client behaviors #39

Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
7181855
Updating dependencies
suztomo Mar 10, 2020
18ef242
Updating Podfile
suztomo Mar 10, 2020
5b2861d
Updating iOS files
suztomo Mar 10, 2020
a6e745a
Excluded test that fails in emulator
suztomo Mar 10, 2020
df1df60
Driver test to compare Firestore behaviors
suztomo Mar 10, 2020
aefc710
Revert unnecessary change
suztomo Mar 10, 2020
2c12b61
format
suztomo Mar 10, 2020
a8b60a3
Added timestamp type check
suztomo Mar 10, 2020
8643583
Added iPhone simulator
suztomo Mar 10, 2020
fc30b6b
Adding README
suztomo Mar 10, 2020
680cfd3
Note for Android port
suztomo Mar 10, 2020
c17f805
More test cases
suztomo Mar 10, 2020
1214278
Simplified test cases
suztomo Mar 11, 2020
78d87d3
Merge remote-tracking branch 'origin/master' into i17
suztomo Mar 13, 2020
8b72fd4
Merge remote-tracking branch 'origin/master' into i17
suztomo Mar 13, 2020
95f587e
updating comment
suztomo Mar 13, 2020
afd996d
Removed commented line
suztomo Mar 15, 2020
85a07f3
Updating document
suztomo Mar 16, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,8 @@ doc/api/
.flutter-plugins
.flutter-plugins-dependencies
example/android/.project
example/android/.settings/
example/android/.settings/

# These files are marked not to check in version control
example/ios/Flutter/flutter_export_environment.sh
example/ios/Flutter/Generated.xcconfig
63 changes: 62 additions & 1 deletion example/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,67 @@
Demonstrates how to use the cloud_firestore_mocks plugin.

The example project comes from
https://github.com/FirebaseExtended/flutterfire/tree/master/packages/cloud_firestore/example,
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This URL has changed.

https://github.com/FirebaseExtended/flutterfire/tree/master/packages/cloud_firestore/cloud_firestore/example,
to which I've implemented two unit tests. See
https://github.com/atn832/cloud_firestore_mocks/blob/master/example/test/widget_test.dart.


# Driver Test: test_driver/cloud_firestore_behaviors

The `test_driver/cloud_firestore_behaviors` driver test ensures the behavior of
cloud_firestore_mocks follows the real Firestore client.

It runs the same set of assertions for the following three `Firestore` instances:

- cloud_firestore backed by Cloud Firestore (project ID: flutter-firestore)
- cloud_firestore backed by Firestore emulator
- cloud_firestore_mocks

## Start iOS Simulator

Start iOS simulator. Driver tests require a simulator device to run.

## Setup Firestore Emulator

If you don't have `firebase` command, install [Firebase Cli](https://firebase.google.com/docs/cli#install-cli-mac-linux):

```
curl -sL https://firebase.tools | bash
...
~/Documents/cloud_firestore_mocks $ which firebase
/usr/local/bin/firebase
```
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also had to set up the emulator:

firebase setup:emulators:firestore

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And also run firebase init using some project of mine so that it could initialize a firebase.json file.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added firebase setup:emulators:firestore, and clarified that the test does not expect firebase.json.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My bad, we don't actually need to run setup:emulators:firestore with the latest version! I can remove this comment after the merge if you want.


Run Firestore emulator:

```
~/Documents/cloud_firestore_mocks $ firebase emulators:start --only firestore
...
i firestore: For testing set FIRESTORE_EMULATOR_HOST=localhost:8080
...
```

`test_driver/cloud_firestore_behaviors` assumes the emulator listen on port
8080 (default) on localhost. This works for iOS simulator running in the same
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you require any special config for the iOS simulator? The tests for the emulator failed on my machine. I get this error:

flutter: 00:00 +5 -5: Firestore behavior comparison: Nested objects update (Firestore Emulator) [E]
flutter:   PlatformException(Error 7, FIRFirestoreErrorDomain, 
  false for 'create' @ L16)
flutter:   package:flutter/src/services/message_codecs.dart 569:7        StandardMethodCodec.decodeEnvelope
  package:flutter/src/services/platform_channel.dart 156:18     MethodChannel._invokeMethod
  ===== asynchronous gap ===========================
  dart:async/zone.dart 1064:19                                  _CustomZone.registerBinaryCallback
  dart:async-patch/async_patch.dart 84:23                       _asyncErrorWrapperHelper
  package:test_api/src/backend/invoker.dart                     Invoker.waitForOutstandingCallbacks.<fn>
  dart:async/zone.dart 1126:13                                  _rootRun
  dart:async/zone.dart 1023:19                                  _CustomZone.run
  dart:async/zone.dart 1518:10                                  _runZoned
  dart:async/zone.dart 1465:12                                  runZoned
  package:test_api/src/backend/invoker.dart 239:5               Invoker.waitForOutstandingCallbacks

Copy link
Collaborator Author

@suztomo suztomo Mar 16, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@atn832 I didn't setup anything special for my iOS simulator or Firestore emulator (or firebase.json).

My environment

  • Simulator version: Version 11.3.1 (SimulatorApp-912.5.1 SimulatorKit-570.3 CoreSimulator-681.17.2)
  • My Mac OS Catalina: 10.15.3
  • XCode 11.3.1
  • firebase command 7.15.0 (latest)

Firestore emulator

In your emulator startup output, I don't see "Could not find config (firebase.json) so using defaults.". Aren't you using some firebase.json that defines security rules?

Mine:

~ $ firebase emulators:start --only firestore                        
⚠  Could not find config (firebase.json) so using defaults.
i  emulators: Starting emulators: firestore
⚠  firestore: Did not find a Cloud Firestore rules file specified in a firebase.json config file.
⚠  firestore: The emulator will default to allowing all reads and writes. Learn more about this option: https://firebase.google.com/docs/emulator-suite/install_and_configure#security_rules_configuration.
i  firestore: Serving ALL traffic (including WebChannel) on http://localhost:8080
⚠  firestore: Support for WebChannel on a separate port (8081) is DEPRECATED and will go away soon. Please use port above instead.
i  firestore: firestore emulator logging to firestore-debug.log
✔  firestore: firestore emulator started at http://localhost:8080
i  firestore: For testing set FIRESTORE_EMULATOR_HOST=localhost:8080
✔  emulators: All emulators started, it is now safe to connect.

Would you try running the emulator without firebase.json?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for sharing the details of your environment. I had 7.0.0 installed. I must have had installed it long ago, and running the install script didn't actually update it. With 7.15.0, it works much better!

Version 7.0.0 required a firebase.json or it would fail (Error: Not in a Firebase app directory (could not locate firebase.json)). It also would fail if I hadn't pre-downloaded the emulator. But the new version downloads the newer emulator automatically when first run.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beautiful!

flutter: 00:00 +0: Firestore behavior comparison: Document creation by add (Cloud Firestore)
VMServiceFlutterDriver: Connected to Flutter application.
        path: satisfied (Path is satisfied), interface: en1
flutter: 00:01 +1: Firestore behavior comparison: Document creation by add (Firestore Emulator)
        path: satisfied (Path is satisfied), interface: en1
flutter: 00:01 +2: Firestore behavior comparison: Document creation by add (cloud_firestore_mocks)
flutter: 00:01 +3: Firestore behavior comparison: Document creation by setData (Cloud Firestore)
flutter: 00:02 +4: Firestore behavior comparison: Document creation by setData (Firestore Emulator)
flutter: 00:02 +5: Firestore behavior comparison: Document creation by setData (cloud_firestore_mocks)
flutter: 00:02 +6: Firestore behavior comparison: Timestamp field (Cloud Firestore)
flutter: 00:03 +7: Firestore behavior comparison: Timestamp field (Firestore Emulator)
flutter: 00:03 +8: Firestore behavior comparison: Timestamp field (cloud_firestore_mocks)
flutter: 00:03 +9: Firestore behavior comparison: Unsaved documens (Cloud Firestore)
flutter: 00:04 +10: Firestore behavior comparison: Unsaved documens (Firestore Emulator)
flutter: 00:04 +11: Firestore behavior comparison: Unsaved documens (cloud_firestore_mocks)
flutter: 00:04 +12: Firestore behavior comparison: Nested objects creation with updateData (Cloud Firestore)
flutter: 00:05 +13: Firestore behavior comparison: Nested objects creation with updateData (Firestore Emulator)
flutter: 00:05 +14: Firestore behavior comparison: Nested objects creation with updateData (cloud_firestore_mocks)
flutter: 00:05 +15: Firestore behavior comparison: Nested objects update (Cloud Firestore)
flutter: 00:06 +16: Firestore behavior comparison: Nested objects update (Firestore Emulator)
flutter: 00:06 +17: Firestore behavior comparison: Nested objects update (cloud_firestore_mocks)
flutter: 00:06 +18: (tearDownAll)
flutter: 00:06 +19: All tests passed!
Stopping application instance.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Glad to hear it worked.

Mac. (This port setting may need to be changed for Android simulator.)

## Run Driver Test

Open another terminal while keeping the emulator running.
Run the following command in the "example" directory.

```
~/Documents/cloud_firestore_mocks/example $ flutter drive --target=test_driver/cloud_firestore_behaviors.dart
...
flutter: 00:01 +3: Firestore behavior comparison: Unsaved documens (Cloud Firestore)
flutter: 00:01 +4: Firestore behavior comparison: Unsaved documens (Firestore Emulator)
flutter: 00:01 +5: Firestore behavior comparison: Unsaved documens (cloud_firestore_mocks)
flutter: 00:01 +6: (tearDownAll)
flutter: 00:01 +7: All tests passed!
Stopping application instance.
```

After waiting for few minutes (around 10 minutes for the first invocation),
"All tests passed!" indicates the driver test passed.
This means that the behaviors of the three `Firestore` instances are the same
for the test cases.
9 changes: 0 additions & 9 deletions example/ios/Flutter/Generated.xcconfig
View file
Open in desktop

This file was deleted.

10 changes: 0 additions & 10 deletions example/ios/Flutter/flutter_export_environment.sh
View file
Open in desktop

This file was deleted.

83 changes: 49 additions & 34 deletions example/ios/Podfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,49 +15,64 @@ def parse_KV_file(file, separator='=')
if !File.exists? file_abs_path
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Flutter command said the Podfile was out-of-date and suggested regenerating; I didn't manually modify this.

return [];
end
pods_ary = []
generated_key_values = {}
skip_line_start_symbols = ["#", "/"]
File.foreach(file_abs_path) { |line|
next if skip_line_start_symbols.any? { |symbol| line =~ /^\s*#{symbol}/ }
plugin = line.split(pattern=separator)
if plugin.length == 2
podname = plugin[0].strip()
path = plugin[1].strip()
podpath = File.expand_path("#{path}", file_abs_path)
pods_ary.push({:name => podname, :path => podpath});
else
puts "Invalid plugin specification: #{line}"
end
}
return pods_ary
File.foreach(file_abs_path) do |line|
next if skip_line_start_symbols.any? { |symbol| line =~ /^\s*#{symbol}/ }
plugin = line.split(pattern=separator)
if plugin.length == 2
podname = plugin[0].strip()
path = plugin[1].strip()
podpath = File.expand_path("#{path}", file_abs_path)
generated_key_values[podname] = podpath
else
puts "Invalid plugin specification: #{line}"
end
end
generated_key_values
end

target 'Runner' do
# Prepare symlinks folder. We use symlinks to avoid having Podfile.lock
# referring to absolute paths on developers' machines.
system('rm -rf .symlinks')
system('mkdir -p .symlinks/plugins')
# Flutter Pod

# Flutter Pods
generated_xcode_build_settings = parse_KV_file('./Flutter/Generated.xcconfig')
if generated_xcode_build_settings.empty?
puts "Generated.xcconfig must exist. If you're running pod install manually, make sure flutter pub get is executed first."
end
generated_xcode_build_settings.map { |p|
if p[:name] == 'FLUTTER_FRAMEWORK_DIR'
symlink = File.join('.symlinks', 'flutter')
File.symlink(File.dirname(p[:path]), symlink)
pod 'Flutter', :path => File.join(symlink, File.basename(p[:path]))
copied_flutter_dir = File.join(__dir__, 'Flutter')
copied_framework_path = File.join(copied_flutter_dir, 'Flutter.framework')
copied_podspec_path = File.join(copied_flutter_dir, 'Flutter.podspec')
unless File.exist?(copied_framework_path) && File.exist?(copied_podspec_path)
# Copy Flutter.framework and Flutter.podspec to Flutter/ to have something to link against if the xcode backend script has not run yet.
# That script will copy the correct debug/profile/release version of the framework based on the currently selected Xcode configuration.
# CocoaPods will not embed the framework on pod install (before any build phases can generate) if the dylib does not exist.

generated_xcode_build_settings_path = File.join(copied_flutter_dir, 'Generated.xcconfig')
unless File.exist?(generated_xcode_build_settings_path)
raise "Generated.xcconfig must exist. If you're running pod install manually, make sure flutter pub get is executed first"
end
}
generated_xcode_build_settings = parse_KV_file(generated_xcode_build_settings_path)
cached_framework_dir = generated_xcode_build_settings['FLUTTER_FRAMEWORK_DIR'];

unless File.exist?(copied_framework_path)
FileUtils.cp_r(File.join(cached_framework_dir, 'Flutter.framework'), copied_flutter_dir)
end
unless File.exist?(copied_podspec_path)
FileUtils.cp(File.join(cached_framework_dir, 'Flutter.podspec'), copied_flutter_dir)
end
end

# Keep pod path relative so it can be checked into Podfile.lock.
pod 'Flutter', :path => 'Flutter'

# Plugin Pods

# Prepare symlinks folder. We use symlinks to avoid having Podfile.lock
# referring to absolute paths on developers' machines.
system('rm -rf .symlinks')
system('mkdir -p .symlinks/plugins')
plugin_pods = parse_KV_file('../.flutter-plugins')
plugin_pods.map { |p|
symlink = File.join('.symlinks', 'plugins', p[:name])
File.symlink(p[:path], symlink)
pod p[:name], :path => File.join(symlink, 'ios')
}
plugin_pods.each do |name, path|
symlink = File.join('.symlinks', 'plugins', name)
File.symlink(path, symlink)
pod name, :path => File.join(symlink, 'ios')
end
end

# Prevent Cocoapods from embedding a second Flutter framework and causing an error with the new Xcode build system.
Expand Down
14 changes: 6 additions & 8 deletions example/ios/Runner.xcodeproj/project.pbxproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@
3B80C3931E831B6300D905FE /* App.framework */ = {isa = PBXFileReference; lastKnownFileType = wrapper.framework; name = App.framework; path = Flutter/App.framework; sourceTree = "<group>"; };
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Flutter updated this file; I didn't manually modify this.

5C6F5A6F1EC3CCCC008D64B5 /* GeneratedPluginRegistrant.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = GeneratedPluginRegistrant.h; sourceTree = "<group>"; };
5C6F5A701EC3CCCC008D64B5 /* GeneratedPluginRegistrant.m */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.objc; path = GeneratedPluginRegistrant.m; sourceTree = "<group>"; };
64D7CC71DA6013C628EE84F1 /* Pods-Runner.debug.xcconfig */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = text.xcconfig; name = "Pods-Runner.debug.xcconfig"; path = "Pods/Target Support Files/Pods-Runner/Pods-Runner.debug.xcconfig"; sourceTree = "<group>"; };
7A1ECC901E8EDB6900309407 /* GoogleService-Info.plist */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.plist.xml; path = "GoogleService-Info.plist"; sourceTree = "<group>"; };
7AFA3C8E1D35360C0083082E /* Release.xcconfig */ = {isa = PBXFileReference; lastKnownFileType = text.xcconfig; name = Release.xcconfig; path = Flutter/Release.xcconfig; sourceTree = "<group>"; };
7AFFD8ED1D35381100E5BB4D /* AppDelegate.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = AppDelegate.h; sourceTree = "<group>"; };
Expand All @@ -57,6 +58,7 @@
97C146FD1CF9000F007C117D /* Assets.xcassets */ = {isa = PBXFileReference; lastKnownFileType = folder.assetcatalog; path = Assets.xcassets; sourceTree = "<group>"; };
97C147001CF9000F007C117D /* Base */ = {isa = PBXFileReference; lastKnownFileType = file.storyboard; name = Base; path = Base.lproj/LaunchScreen.storyboard; sourceTree = "<group>"; };
97C147021CF9000F007C117D /* Info.plist */ = {isa = PBXFileReference; lastKnownFileType = text.plist.xml; path = Info.plist; sourceTree = "<group>"; };
CDA54DDE72ECF4F155A1262A /* Pods-Runner.release.xcconfig */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = text.xcconfig; name = "Pods-Runner.release.xcconfig"; path = "Pods/Target Support Files/Pods-Runner/Pods-Runner.release.xcconfig"; sourceTree = "<group>"; };
E13AAF33B0B411D7B2D38642 /* libPods-Runner.a */ = {isa = PBXFileReference; explicitFileType = archive.ar; includeInIndex = 0; path = "libPods-Runner.a"; sourceTree = BUILT_PRODUCTS_DIR; };
/* End PBXFileReference section */

Expand All @@ -77,6 +79,8 @@
840012C8B5EDBCF56B0E4AC1 /* Pods */ = {
isa = PBXGroup;
children = (
64D7CC71DA6013C628EE84F1 /* Pods-Runner.debug.xcconfig */,
CDA54DDE72ECF4F155A1262A /* Pods-Runner.release.xcconfig */,
);
name = Pods;
sourceTree = "<group>";
Expand Down Expand Up @@ -242,16 +246,13 @@
files = (
);
inputPaths = (
"${SRCROOT}/Pods/Target Support Files/Pods-Runner/Pods-Runner-resources.sh",
"${PODS_CONFIGURATION_BUILD_DIR}/gRPC/gRPCCertificates.bundle",
);
name = "[CP] Copy Pods Resources";
outputPaths = (
"${TARGET_BUILD_DIR}/${UNLOCALIZED_RESOURCES_FOLDER_PATH}/gRPCCertificates.bundle",
);
runOnlyForDeploymentPostprocessing = 0;
shellPath = /bin/sh;
shellScript = "\"${SRCROOT}/Pods/Target Support Files/Pods-Runner/Pods-Runner-resources.sh\"\n";
shellScript = "\"${PODS_ROOT}/Target Support Files/Pods-Runner/Pods-Runner-resources.sh\"\n";
showEnvVarsInLog = 0;
};
95BB15E9E1769C0D146AA592 /* [CP] Embed Pods Frameworks */ = {
Expand All @@ -260,16 +261,13 @@
files = (
);
inputPaths = (
"${SRCROOT}/Pods/Target Support Files/Pods-Runner/Pods-Runner-frameworks.sh",
"${PODS_ROOT}/../.symlinks/flutter/ios-release/Flutter.framework",
);
name = "[CP] Embed Pods Frameworks";
outputPaths = (
"${TARGET_BUILD_DIR}/${FRAMEWORKS_FOLDER_PATH}/Flutter.framework",
);
runOnlyForDeploymentPostprocessing = 0;
shellPath = /bin/sh;
shellScript = "\"${SRCROOT}/Pods/Target Support Files/Pods-Runner/Pods-Runner-frameworks.sh\"\n";
shellScript = "\"${PODS_ROOT}/Target Support Files/Pods-Runner/Pods-Runner-frameworks.sh\"\n";
showEnvVarsInLog = 0;
};
9740EEB61CF901F6004384FC /* Run Script */ = {
Expand Down
4 changes: 2 additions & 2 deletions example/pubspec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ description: Demonstrates how to use the cloud_firestore_mocks plugin.
dependencies:
flutter:
sdk: flutter
cloud_firestore: ^0.13.1+1
firebase_core: "^0.4.0"
cloud_firestore: ^0.13.4
firebase_core: "^0.4.4"

dev_dependencies:
flutter_driver:
Expand Down
Loading