Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't read file in integration test android #94197

Open
ductranit opened this issue Nov 25, 2021 · 18 comments
Open

Can't read file in integration test android #94197

ductranit opened this issue Nov 25, 2021 · 18 comments
Labels
f: integration_test The flutter/packages/integration_test plugin found in release: 2.6 Found to occur in 2.6 found in release: 2.8 Found to occur in 2.8 framework flutter/packages/flutter repository. See also f: labels. has reproducible steps The issue has been confirmed reproducible and is ready to work on P3 Issues that are less important to the Flutter project platform-android Android applications specifically team-android Owned by Android platform team triaged-android Triaged by Android platform team

Comments

@ductranit
Copy link

ductranit commented Nov 25, 2021

I had the same problem with this issue
So when I run integration test, the Directory.current is / - this is the root dir on my mac device and it's readonly.
I was trying to read json file & custom font in the test, so I have to pass the real path on my mac like:

setUpAll(() async {
    var currentDir = Directory.current.path;  // return "/"
    rootPath ='/Users/admin/Documents/flutter/my_app';
   final fontPath = '$rootPath/assets/fonts/custom_font.ttf';
   loader.addFont(File(fontPath)
          .readAsBytes()
          .then((bytes) => ByteData.view(Uint8List.fromList(bytes).buffer)));
   loader.load()
  // load json...
  var json = await File('$rootPath/assets/data.json'). readAsString();
  });

If I run command flutter test integration_test on ios, it works fine, I can load custom font, json & can also take screenshot with this path.
But when I run on android device, it throw file not found error with the same path when I load font, json

FileSystemException: Cannot open file, path = '/Users/admin/Documents/flutter/my_app/assets/fonts/custom_font.ttf' (OS Error: No such file or directory, errno = 2)
  dart:io                                                                                                                _File.readAsBytes
  Users/admin/Documents/flutter/my_app/integration_test/app_test.dart 49:11                                        main.<fn>
  Users/admin/Documents/flutter/my_app/integration_test/app_test.dart 42:12                                        main.<fn>
  ===== asynchronous gap ===========================
  package:stream_channel                                                                                                 _GuaranteeSink.add
  var/folders/11/wq1vn8yd2739xsmmzbq3m06w0000gn/T/flutter_tools.6Thidy/flutter_test_listener.DYB5ji/listener.dart 53:22  main.<fn>
  var/folders/11/wq1vn8yd2739xsmmzbq3m06w0000gn/T/flutter_tools.6Thidy/flutter_test_listener.DYB5ji/listener.dart 52:20  main.<fn>

So my questions are:

  • Why do integration test has different directory path with unit test according to this issue? The unit flutter has the current project dir by default, but integration test has / as root
  • What is the official way to read/write data in integration test? why I can't read file when run on android?
flutter doctor -v
[✓] Flutter (Channel master, 2.6.0-12.0.pre.826, on Mac OS X 10.15.7 19H1519 darwin-x64, locale en-US)
    • Flutter version 2.6.0-12.0.pre.826 at /Users/admin/Documents/Soft/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 6c1455a23a (25 hours ago), 2021-11-23 21:13:05 -0500
    • Engine revision 539239e256
    • Dart version 2.16.0 (build 2.16.0-42.0.dev)
    • DevTools version 2.8.0

[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    • Android SDK at /Users/admin/Library/Android/sdk
    • Platform android-31, build-tools 31.0.0
    • ANDROID_HOME = /Users/admin/Library/Android/sdk
    • ANDROID_SDK_ROOT = /Users/admin/Library/Android/sdk
    • Java binary at: /Applications/Android Studio 4.2 Preview.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7281165)
    • All Android licenses accepted.

[!] Xcode - develop for iOS and macOS (Xcode 12.4)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    ! Flutter recommends a minimum Xcode version of 13.
      Download the latest version or update via the Mac App Store.
    • CocoaPods version 1.11.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2020.3)
    • Android Studio at /Applications/Android Studio 4.2 Preview.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7281165)

[✓] VS Code (version 1.58.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.24.0

[✓] VS Code (version 1.63.0-insider)
    • VS Code at /Applications/Visual Studio Code - Insiders.app/Contents
    • Flutter extension version 3.28.0

[✓] Connected device (2 available)
    • macOS (desktop) • macos  • darwin-x64     • Mac OS X 10.15.7 19H1519 darwin-x64
    • Chrome (web)    • chrome • web-javascript • Google Chrome 96.0.4664.55
@darshankawar darshankawar added the in triage Presently being triaged by the triage team label Nov 29, 2021
@darshankawar
Copy link
Member

@ductranit
Since you are running the integration test on a device, can you try to bundle them in as an asset and then try to access it using rootBundle ?

@darshankawar darshankawar added the waiting for customer response The Flutter team cannot make further progress on this issue until the original reporter responds label Nov 29, 2021
@ductranit
Copy link
Author

@ductranit Since you are running the integration test on a device, can you try to bundle them in as an asset and then try to access it using rootBundle ?

I didn't try it yet, but I need to access the system path to load custom fonts (for golden test), load some test configs, take some screenshots & save to files. How can bundle support for that?
I noticed that my test code works when I run integration test on ios, but not for android.

@github-actions github-actions bot removed the waiting for customer response The Flutter team cannot make further progress on this issue until the original reporter responds label Nov 29, 2021
@darshankawar
Copy link
Member

How can bundle support for that?

Since you are saving to the files, and the same file if you add it to pubspec.yaml under assets and refer it to something like await rootBundle.loadString('assets/my_text.txt');

@darshankawar darshankawar added the waiting for customer response The Flutter team cannot make further progress on this issue until the original reporter responds label Nov 30, 2021
@ductranit
Copy link
Author

@darshankawar So I tried to read by rootBundle & it works

final bytes = await rootBundle.load('assets/data.json');

But as I said, the problem is that I need to write screenshot to file which bundle can't help.
I tried to save screenshot with the path like:
screen.png
/Users/admin/Documents/flutter/my_app/assets/screen.png
assets/screen.png
and none of them can work

await binding.convertFlutterSurfaceToImage();
await tester.pumpAndSettle();
final data = await binding.takeScreenshot(name);
 final path = 'screen.png';  // or `/Users/admin/Documents/flutter/my_app/screen.png` or `assets/screen.png`
final imageFile = File(path);
await imageFile.create(recursive: true);
await imageFile.writeAsBytesSync(data);

It throws exception when write file:

══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following FileSystemException was thrown running a test:
Cannot create file, path = 'screen.png' (OS Error: Read-only file system, errno = 30)

When the exception was thrown, this was the stack:
#0      _File.create.<anonymous closure> (dart:io/file_impl.dart:255:9)
<asynchronous suspension>
<asynchronous suspension>
(elided 5 frames from dart:async and package:stack_trace)

The test description was:
  Test app flow
════════════════════════════════════════════════════════════════════════════════════════════════════
══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following message was thrown:
Multiple exceptions (5) were detected during the running of the current test, and at least one was
unexpected.

P/s: I had to change FlutterFragmentActivity to FlutterActivity to take screenshot because of this issue, but it will have a trouble because other plugins require fragment activity

@github-actions github-actions bot removed the waiting for customer response The Flutter team cannot make further progress on this issue until the original reporter responds label Dec 2, 2021
@darshankawar
Copy link
Member

@ductranit
Thanks for the update. Since it's throwing The following FileSystemException was thrown running a test: Cannot create file, path = 'screen.png' (OS Error: Read-only file system, errno = 30), can you try to pass full path ? ie instead of relative path, try to pass absolute path.

@darshankawar darshankawar added the waiting for customer response The Flutter team cannot make further progress on this issue until the original reporter responds label Dec 3, 2021
@ductranit
Copy link
Author

ductranit commented Dec 3, 2021

As I said above, all kinds of path have exception.
This is the full path /Users/admin/Documents/flutter/my_app/assets/screen.png when I create file
await imageFile.create();

══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following FileSystemException was thrown running a test:
Cannot create file, path =
'/Users/admin/Documents/flutter/my_app/assets/screen.png' (OS Error: No such
file or directory, errno = 2)

When the exception was thrown, this was the stack:
#0      _File.create.<anonymous closure> (dart:io/file_impl.dart:255:9)
<asynchronous suspension>
<asynchronous suspension>
(elided 5 frames from dart:async and package:stack_trace)

The test description was:
  Test app flow
════════════════════════════════════════════════════════════════════════════════════════════════════
07:12 +0 -1: Test app flow [E]                                                                                                      
  Test failed. See exception logs above.
  The test description was: Test app flow

If I tried with await imageFile.create(recursive: true);, then it throws this exception

══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following FileSystemException was thrown running a test:
Creation failed, path = '/Users' (OS Error: Read-only file system, errno = 30)

When the exception was thrown, this was the stack:
#0      _Directory.create.<anonymous closure> (dart:io/directory_impl.dart:117:11)
<asynchronous suspension>
<asynchronous suspension>
(elided 5 frames from dart:async and package:stack_trace)

The test description was:
  Test app flow
════════════════════════════════════════════════════════════════════════════════════════════════════
06:18 +0 -1: Test app flow [E]                                                                                                      
  Test failed. See exception logs above.
  The test description was: Test app flow

The problem is that the root path of integration test is / - this is the root dir on my mac device and it's readonly

@github-actions github-actions bot removed the waiting for customer response The Flutter team cannot make further progress on this issue until the original reporter responds label Dec 3, 2021
@darshankawar
Copy link
Member

Thanks for the update. I see what you are saying. Can you provide us a minimal code sample, ie, main.dart and integration_test -> app_test.dart file that we can directly use and verify it further ?

@darshankawar darshankawar added the waiting for customer response The Flutter team cannot make further progress on this issue until the original reporter responds label Dec 3, 2021
@ductranit
Copy link
Author

ductranit commented Dec 11, 2021

@darshankawar Here is it ui_test.zip
This sample is just created by flutter create on latest master
I run the test by flutter test integration_test

@github-actions github-actions bot removed the waiting for customer response The Flutter team cannot make further progress on this issue until the original reporter responds label Dec 11, 2021
@darshankawar
Copy link
Member

Thanks for the info. I verified this on Android and get same behavior / exception on latest stable and master. Keeping it open for further insights from the team.

app_test.dart

import 'dart:io';

import 'package:flutter_test/flutter_test.dart';
import 'package:integration_test/integration_test.dart';
import 'package:ui_test/main.dart' as app;

void main() {
  setUpAll(() async {});
  final binding = IntegrationTestWidgetsFlutterBinding.ensureInitialized()
      as IntegrationTestWidgetsFlutterBinding;
  testWidgets('Test app flow', (WidgetTester tester) async {
    app.main();
    await tester.pumpAndSettle();
    expect(find.byType(app.MyHomePage), findsOneWidget);
    await _takeScreenshot(tester, binding, 'Home', app.MyHomePage);
  });
}

Future<void> _takeScreenshot(
    WidgetTester tester,
    IntegrationTestWidgetsFlutterBinding binding,
    String name,
    Type type) async {
  await binding.convertFlutterSurfaceToImage();
  await tester.pumpAndSettle();
  final data = await binding.takeScreenshot(name);

  // root dir
  const rootDir = '/Users/admin/Documents/flutter/ui_test';
  final imageFile = File('$rootDir/assets/$name.png');
  print('take screenshot ${imageFile.path} with size ${data.length}');
  if (!imageFile.existsSync()) {
    await imageFile.create(recursive: true);
  }

  await imageFile.writeAsBytes(data);
}

main.dart

import 'package:flutter/material.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({Key? key}) : super(key: key);

  // This widget is the root of your application.
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        // This is the theme of your application.
        //
        // Try running your application with "flutter run". You'll see the
        // application has a blue toolbar. Then, without quitting the app, try
        // changing the primarySwatch below to Colors.green and then invoke
        // "hot reload" (press "r" in the console where you ran "flutter run",
        // or simply save your changes to "hot reload" in a Flutter IDE).
        // Notice that the counter didn't reset back to zero; the application
        // is not restarted.
        primarySwatch: Colors.blue,
      ),
      home: const MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  const MyHomePage({Key? key, required this.title}) : super(key: key);

  // This widget is the home page of your application. It is stateful, meaning
  // that it has a State object (defined below) that contains fields that affect
  // how it looks.

  // This class is the configuration for the state. It holds the values (in this
  // case the title) provided by the parent (in this case the App widget) and
  // used by the build method of the State. Fields in a Widget subclass are
  // always marked "final".

  final String title;

  @override
  State<MyHomePage> createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  int _counter = 0;

  void _incrementCounter() {
    setState(() {
      // This call to setState tells the Flutter framework that something has
      // changed in this State, which causes it to rerun the build method below
      // so that the display can reflect the updated values. If we changed
      // _counter without calling setState(), then the build method would not be
      // called again, and so nothing would appear to happen.
      _counter++;
    });
  }

  @override
  Widget build(BuildContext context) {
    // This method is rerun every time setState is called, for instance as done
    // by the _incrementCounter method above.
    //
    // The Flutter framework has been optimized to make rerunning build methods
    // fast, so that you can just rebuild anything that needs updating rather
    // than having to individually change instances of widgets.
    return Scaffold(
      appBar: AppBar(
        // Here we take the value from the MyHomePage object that was created by
        // the App.build method, and use it to set our appbar title.
        title: Text(widget.title),
      ),
      body: Center(
        // Center is a layout widget. It takes a single child and positions it
        // in the middle of the parent.
        child: Column(
          // Column is also a layout widget. It takes a list of children and
          // arranges them vertically. By default, it sizes itself to fit its
          // children horizontally, and tries to be as tall as its parent.
          //
          // Invoke "debug painting" (press "p" in the console, choose the
          // "Toggle Debug Paint" action from the Flutter Inspector in Android
          // Studio, or the "Toggle Debug Paint" command in Visual Studio Code)
          // to see the wireframe for each widget.
          //
          // Column has various properties to control how it sizes itself and
          // how it positions its children. Here we use mainAxisAlignment to
          // center the children vertically; the main axis here is the vertical
          // axis because Columns are vertical (the cross axis would be
          // horizontal).
          mainAxisAlignment: MainAxisAlignment.center,
          children: <Widget>[
            const Text(
              'You have pushed the button this many times:',
            ),
            Text(
              '$_counter',
              style: Theme.of(context).textTheme.headline4,
            ),
          ],
        ),
      ),
      floatingActionButton: FloatingActionButton(
        onPressed: _incrementCounter,
        tooltip: 'Increment',
        child: const Icon(Icons.add),
      ), // This trailing comma makes auto-formatting nicer for build methods.
    );
  }
}

flutter doctor -v

[√] Flutter (Channel master, 2.6.0-12.0.pre.975, on Microsoft Windows [Version 10.0.19042.1348], locale en-US)
    • Flutter version 2.6.0-12.0.pre.975 at D:\FlutterSDK\flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 9b1bbdbd16 (2 hours ago), 2021-12-09 00:04:07 -0500
    • Engine revision 30b0105c72
    • Dart version 2.16.0 (build 2.16.0-85.0.dev)
    • DevTools version 2.9.1

[√] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    • Android SDK at C:\Users\Win\AppData\Local\Android\sdk
    • Platform android-31, build-tools 31.0.0
    • Java binary at: C:\Program Files\Android\Android Studio\jre\bin\java
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189)
    • All Android licenses accepted.

[√] Chrome - develop for the web
    • Chrome at C:\Program Files (x86)\Google\Chrome\Application\chrome.exe

[√] Visual Studio - develop for Windows (Visual Studio Community 2019 16.11.2)
    • Visual Studio at C:\Program Files (x86)\Microsoft Visual Studio\2019\Community
    • Visual Studio Community 2019 version 16.11.31624.102
    • Windows 10 SDK version 10.0.19041.0

[√] Android Studio (version 2020.3)
    • Android Studio at C:\Program Files\Android\Android Studio
    • Flutter plugin can be installed from:
       https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
       https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189)

[√] VS Code (version 1.60.1)
    • VS Code at C:\Users\Win\AppData\Local\Programs\Microsoft VS Code
    • Flutter extension can be installed from:
       https://marketplace.visualstudio.com/items?itemName=Dart-Code.flutter

[√] Connected device (4 available)
    • SM A260G (mobile) • 5200763ebcfa861f • android-arm    • Android 8.1.0 (API 27)
    • Windows (desktop) • windows          • windows-x64    • Microsoft Windows [Version 10.0.19042.1348]
    • Chrome (web)      • chrome           • web-javascript • Google Chrome 96.0.4664.45
    • Edge (web)        • edge             • web-javascript • Microsoft Edge 96.0.1054.43

• No issues found!

[√] Flutter (Channel stable, 2.8.0, on Microsoft Windows [Version 10.0.19042.1348], locale en-US)
    • Flutter version 2.8.0 at D:\FlutterSDK\flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision cf44000065 (8 hours ago), 2021-12-08 14:06:50 -0800
    • Engine revision 40a99c5951
    • Dart version 2.15.0

[√] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    • Android SDK at C:\Users\Win\AppData\Local\Android\sdk
    • Platform android-31, build-tools 31.0.0
    • Java binary at: C:\Program Files\Android\Android Studio\jre\bin\java
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189)
    • All Android licenses accepted.

[√] Chrome - develop for the web
    • Chrome at C:\Program Files (x86)\Google\Chrome\Application\chrome.exe

[√] Visual Studio - develop for Windows (Visual Studio Community 2019 16.11.2)
    • Visual Studio at C:\Program Files (x86)\Microsoft Visual Studio\2019\Community
    • Visual Studio Community 2019 version 16.11.31624.102
    • Windows 10 SDK version 10.0.19041.0

[√] Android Studio (version 2020.3)
    • Android Studio at C:\Program Files\Android\Android Studio
    • Flutter plugin can be installed from:
       https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
       https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189)

[√] VS Code (version 1.61.0)
    • VS Code at C:\Users\Win\AppData\Local\Programs\Microsoft VS Code
    • Flutter extension version 3.27.0

[√] Connected device (4 available)
    • SM A260G (mobile) • 5200763ebcfa861f • android-arm    • Android 8.1.0 (API 27)
    • Windows (desktop) • windows          • windows-x64    • Microsoft Windows [Version 10.0.19042.1348]
    • Chrome (web)      • chrome           • web-javascript • Google Chrome 96.0.4664.45
    • Edge (web)        • edge             • web-javascript • Microsoft Edge 96.0.1054.43

• No issues found!

/cc @jiahaog

@darshankawar darshankawar added found in release: 2.6 Found to occur in 2.6 found in release: 2.8 Found to occur in 2.8 framework flutter/packages/flutter repository. See also f: labels. has reproducible steps The issue has been confirmed reproducible and is ready to work on f: integration_test The flutter/packages/integration_test plugin platform-android Android applications specifically and removed in triage Presently being triaged by the triage team labels Dec 13, 2021
@jiahaog
Copy link
Member

jiahaog commented Dec 14, 2021

Are you running the iOS integration test on a simulator? I'm not sure if that will work on a physical device.

This is probably happening because of the different filesystems used by the device. On iOS simulators, the filesystem of the simulator is the same as the host. Similarly for unit tests, the test run within the flutter tester device which runs on the host and has access to the same host filesystem. This is not the case on iOS physical devices or Android emulators or Android physical devices, which do not share the filesystem of the host.

@ductranit
Copy link
Author

ductranit commented Dec 14, 2021

@jiahaog yes, I run it on ios simulator. I just try on ios device and got the same android issue.
But if the device file system is different with the host, then how can I write file into it during the test?
As you can see above, I was trying to use relative path like screen.png or assets/screen.png but none of them can work.

@goderbauer goderbauer added the P6 label Feb 16, 2022
@00SunnyDay00
Copy link
Contributor

any update? same problem~

@00SunnyDay00
Copy link
Contributor

actually, it seems like golden test doesn't work in integration test

@lexxxel
Copy link

lexxxel commented May 12, 2022

actually, it seems like golden test doesn't work in integration test

I stumbled across the same issue. Any solutions?

@ductranit
Copy link
Author

@lexxxel @00SunnyDay00 I have to put the golden images into flutter assets (define in pubspec.yaml), then copy those golden files into internal storage of simulator/device when starting app and compare goldens

@lexxxel
Copy link

lexxxel commented May 13, 2022

@ductranit so you can't use flutter test integration_test --update-golden to automatically create/update those screenshots AND you have to have a separate pubspec.yaml for integration tests (because nobody wants to deploy this screenshots to production)?
If so, that makes golden integration tests unpractical.

@ductranit
Copy link
Author

@lexxxel actually I did a trick for --update-golden : I upload the new golden into our server, and I have a script to download latest golden before running integration tests.
And yes for a separate pubspec.yaml.
I run tests & deploy build from CI system, and this issue stops me to do normally.
I use separate script for deploy & test app to generate pubspec.yaml & goldens.
I know this is an ugly hack,but there is no better way unless flutter dev supports for it.

@lexxxel
Copy link

lexxxel commented May 13, 2022

@ductranit I might found a 'better' solution. Requirement: a emulator with root access. I do use (minimal example):

Directory tempDir = await getTemporaryDirectory();
String tempPath = tempDir.path;
await expectLater(introText, matchesGoldenFile(tempPath + "/QrScannerPageModel.verify.png"));

to push the .png into a folder that is predictable and adb pull/push within the ci pipeline to move the files onto the emulator

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
f: integration_test The flutter/packages/integration_test plugin found in release: 2.6 Found to occur in 2.6 found in release: 2.8 Found to occur in 2.8 framework flutter/packages/flutter repository. See also f: labels. has reproducible steps The issue has been confirmed reproducible and is ready to work on P3 Issues that are less important to the Flutter project platform-android Android applications specifically team-android Owned by Android platform team triaged-android Triaged by Android platform team
Projects
None yet
Development

No branches or pull requests

7 participants