Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add offline mode data manager #2322

Closed

Conversation

AshAman999
Copy link
Member

@AshAman999 AshAman999 commented Jun 19, 2022

What

  • Added menu for an easier experience for offline mode
  • Added method to update the local with fresh results from server
  • Added method to clear the local DB
  • Added view to show stats about local DB size and products available for offline scanning

NOTE:

Translations left to be done later

Screenshot

Light Mode Dark Mode
image image

Fixes bug(s)

Part of

@AshAman999 AshAman999 requested a review from a team as a code owner June 19, 2022 15:50
Copy link
Member

@M123-dev M123-dev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great start @AshAman999 I added some comments. Mostly about in code comments

packages/smooth_app/lib/database/dao_product.dart Outdated Show resolved Hide resolved
packages/smooth_app/lib/database/dao_product.dart Outdated Show resolved Hide resolved
packages/smooth_app/lib/database/dao_product.dart Outdated Show resolved Hide resolved
packages/smooth_app/lib/database/dao_product.dart Outdated Show resolved Hide resolved
packages/smooth_app/lib/database/dao_product.dart Outdated Show resolved Hide resolved
packages/smooth_app/lib/database/dao_product_list.dart Outdated Show resolved Hide resolved
packages/smooth_app/lib/database/dao_string_list.dart Outdated Show resolved Hide resolved
packages/smooth_app/lib/pages/offline_data.dart Outdated Show resolved Hide resolved
packages/smooth_app/lib/pages/offline_data.dart Outdated Show resolved Hide resolved
@AshAman999 AshAman999 requested a review from teolemon June 19, 2022 17:41
Copy link
Contributor

@monsieurtanuki monsieurtanuki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @AshAman999!
I've seen many things that puzzled me, among them:

  • in DaoProduct you've put some code regarding access to the server - in a DAO we don't give a damn about the rest of the world, we just read and write elements. If you need an access to the server to populate your table, do it elsewhere
  • it looks like you generously delete all the database, including the history and the scan history. I'm not familiar with the concepts of offline mode for smoothie, but that does not seem appropriate.

And I've written some other comments too...

packages/smooth_app/lib/database/dao_product.dart Outdated Show resolved Hide resolved
packages/smooth_app/lib/database/dao_product.dart Outdated Show resolved Hide resolved
packages/smooth_app/lib/database/dao_string_list.dart Outdated Show resolved Hide resolved
packages/smooth_app/lib/database/dao_string_list.dart Outdated Show resolved Hide resolved
packages/smooth_app/lib/pages/offline_data.dart Outdated Show resolved Hide resolved
packages/smooth_app/lib/pages/offline_data.dart Outdated Show resolved Hide resolved
packages/smooth_app/lib/pages/offline_data.dart Outdated Show resolved Hide resolved
packages/smooth_app/lib/pages/offline_data.dart Outdated Show resolved Hide resolved
packages/smooth_app/lib/pages/offline_data.dart Outdated Show resolved Hide resolved
@AshAman999
Copy link
Member Author

Hello @monsieurtanuki
Thank you for your detailed review, really appreciate that, and did the changes as requested by you.

  • Yeah deleted the user list on purpose, In case I don't delete that, The user searches for something he already has searched and it shows no product available that was fetched from history, so to remove the confusion did the changes.

  • Talking about Dao got your concerns and moved the query to another file.

  • Getting confused about putting this in transaction, counting on you for some help here

    await daoProduct.clearAll();
     await daoProductList.clearAll();
  • Also please have another look at _refreshLocalDatabase in the offline_data_page.dart

@codecov-commenter
Copy link

codecov-commenter commented Jun 20, 2022

Codecov Report

Merging #2322 (e182bd6) into develop (2ea0da3) will decrease coverage by 1.75%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           develop   #2322      +/-   ##
==========================================
- Coverage     8.86%   7.10%   -1.76%     
==========================================
  Files          161     220      +59     
  Lines         6623   10762    +4139     
==========================================
+ Hits           587     765     +178     
- Misses        6036    9997    +3961     
Impacted Files Coverage Δ
...kages/smooth_app/lib/widgets/attribute_button.dart 0.00% <0.00%> (-92.00%) ⬇️
...s/smooth_app/lib/data_models/user_preferences.dart 8.73% <0.00%> (-23.57%) ⬇️
packages/smooth_app/lib/themes/smooth_theme.dart 60.00% <0.00%> (-22.98%) ⬇️
...p/lib/generic_lib/dialogs/smooth_alert_dialog.dart 15.11% <0.00%> (-19.10%) ⬇️
...mooth_app/lib/data_models/product_preferences.dart 21.68% <0.00%> (-9.75%) ⬇️
packages/smooth_app/lib/main.dart 14.65% <0.00%> (-3.24%) ⬇️
.../smooth_app/lib/pages/onboarding/welcome_page.dart 0.00% <0.00%> (-3.13%) ⬇️
.../smooth_app/lib/pages/onboarding/scan_example.dart 0.00% <0.00%> (-2.28%) ⬇️
...p/lib/pages/onboarding/consent_analytics_page.dart 0.00% <0.00%> (-1.57%) ⬇️
...ackages/smooth_app/lib/pages/scan/scan_header.dart 3.33% <0.00%> (-1.43%) ⬇️
... and 238 more

Help us with your feedback. Take ten seconds to tell us how you rate us.

@teolemon teolemon modified the milestone: V1.1 Jun 20, 2022
Copy link
Contributor

@monsieurtanuki monsieurtanuki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @AshAman999 for your changes!

  • Yeah deleted the user list on purpose, In case I don't delete that, The user searches for something he already has searched and it shows no product available that was fetched from history, so to remove the confusion did the changes.

I don't understand what you're up to. So far, the core of your code is about refreshing the products that are already there, am I right? Nothing to do with user search.
If you want to impact user search, you have to code inside the user search. For instance, if you would like the user to type the start of a barcode and automatically suggest all matching barcodes, that's something you'll have to do in the search itself. And you don't need to erase all the database or even refresh the products for that btw.
Therefore, here's my question: What is your quest?

  • Getting confused about putting this in transaction, counting on you for some help here

My bad: the notion of transaction is a SQL notion. That would have made sense if both DaoProduct and DaoProductList were in SQL, but that is not the case.

packages/smooth_app/lib/pages/offline_data_page.dart Outdated Show resolved Hide resolved
packages/smooth_app/lib/pages/offline_data_page.dart Outdated Show resolved Hide resolved
packages/smooth_app/lib/pages/offline_data_page.dart Outdated Show resolved Hide resolved
packages/smooth_app/lib/pages/offline_data_page.dart Outdated Show resolved Hide resolved
packages/smooth_app/lib/pages/offline_data_page.dart Outdated Show resolved Hide resolved
packages/smooth_app/lib/database/dao_product.dart Outdated Show resolved Hide resolved
packages/smooth_app/lib/pages/offline_data_page.dart Outdated Show resolved Hide resolved
packages/smooth_app/lib/pages/offline_data_page.dart Outdated Show resolved Hide resolved
packages/smooth_app/lib/pages/offline_data_page.dart Outdated Show resolved Hide resolved
@AshAman999
Copy link
Member Author

I don't understand what you're up to. So far, the core of your code is about refreshing the products that are already there, am I right? Nothing to do with user search.

As an added benefit I wanted to let the users free their memory from the app in case of necessity, maybe I have an old Samsung or just that I don't want so much cache
Currently, if I want to delete the local data it's not an easy task as a user without reinstalling the app.

And you don't need to erase all the database or even refresh the products for that btw.
Therefore, here's my question: What is your quest?

Suppose I mostly use the offline scanned items for my scans, So whenever I visit my grocery store there is poor reception of network so I in advance cache some items about different products and also some products that I mostly scan. The next time I visit stores I want to be sure that the product attributes have not changed. So to refresh all the items in the local cache there is this button so that if I refresh the cache after a month I can be sure that the products and the date is reliable again now

Found the idea here, somewhere on google docs presented to me , and that's my goal here, so taking one step at a time

Freshness
We should invalidate the cache after a given amount of time, or at least display a warning that the data might not be up-to-date anymore, especially if we move beyond a basic version with just the scores.
We should display when products are taken from offline database.
We should also use offline if the net is not available, like Google Translator.
In terms of marketing "offline by default for everybody" is nice, especially if it's highlighted during the onboarding process. By default, a minimalistic version would be loaded, and people would be able to go for the full thing if they so wish.
All products viewed would be stored for reference.

@monsieurtanuki
Copy link
Contributor

As an added benefit I wanted to let the users free their memory from the app in case of necessity, maybe I have an old Samsung or just that I don't want so much cache
Currently, if I want to delete the local data it's not an easy task as a user without reinstalling the app.

Yes you can: just go to your app manager, click on your app and say "delete the app data".
Btw with an old Samsung you cannot use the app altogether, for https compatibility reasons.
And deleting data is not the core of your PR, let's stay focused.

Suppose I mostly use the offline scanned items for my scans, So whenever I visit my grocery store there is poor reception of network so I in advance cache some items about different products and also some products that I mostly scan.

No you can't: you've just erased your scan product list.

The next time I visit stores I want to be sure that the product attributes have not changed. So to refresh all the items in the local cache there is this button so that if I refresh the cache after a month I can be sure that the products and the date is reliable again now

So this has nothing to do with offline cache of 5K products, right? It's just about refreshing all the products, like in #1651.

My suggestions:

  • if you really want to include your "delete all" feature, please do it in dev mode for the moment.
  • in the rest of the app, forget about deleting products. There's absolutely no added value in that. Just upsert/replace them.
  • in the rest of the app, forget about deleting product lists. If really needed, let the user delete them but one by one.
  • if you refresh the products, don't forget to refresh the current product lists (cf. The app should offer to reload all products on language change #1651 (comment))

@AshAman999
Copy link
Member Author

Thanks again @monsieurtanuki

Sorry for all the confusion that you went through, I guess the implications of why I went ahead to add a menu is something that might be confusing here 😕

No you can't: you've just erased your scan product list.

Meant to say, the user just refreshed the product not deleted them.

if you really want to include your "delete all" feature, please do it in dev mode for the moment

It's currently in dev mode only

in the rest of the app, forget about deleting product lists. If really needed, let the user delete them one by one

Actually a really great idea, I can include that feature in the next upcoming PRs, I just wanted to get some work already done here before the first evaluation 😅

if you refresh the products, don't forget to refresh the current product lists (cf. #1651 (comment))

It does actually refresh the product list as well, just wasn't able to update the timestamp for the product list, I had added a todo there and had thought maybe you could have helped me there,
Currently when I click the update now button everything get's updated including the product in the product list, what I wanted was to replace the time stamp on the product list on every entry with the timestamp when the refresh now was called.

For time being if it's about the implication we are confused about, I can discuss it with Pierre in the next smoothie meeting, and rest thanks again for correcting my code to be more efficient, I hope at least the code part is now in good condition to maybe merge if needed

@teolemon teolemon mentioned this pull request Jun 30, 2022
9 tasks
@teolemon teolemon changed the title feat: new menu for offline mode feat: Add offline mode data manager Jun 30, 2022
@teolemon teolemon added this to the Offline Scan milestone Jun 30, 2022
return rowsDeleted;
}

Future<int> getLength() async {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A getter would be better here

packages/smooth_app/lib/database/dao_product_list.dart Outdated Show resolved Hide resolved
packages/smooth_app/lib/database/local_database.dart Outdated Show resolved Hide resolved

@override
Widget build(BuildContext context) {
final LocalDatabase localDatabase = context.watch<LocalDatabase>();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please split this Widget into multiple ones.
This one is way to bigger for a Stateful one

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Crafting the constant widgets to be seperate

@teolemon
Copy link
Member

teolemon commented Jul 8, 2022

There's a merge conflict to solve @AshAman999 to keep the PR mergeable

@AshAman999
Copy link
Member Author

There's a merge conflict to solve @AshAman999 to keep the PR mergeable

There's are some more changes to do, and this section will make more sense in the app after we start storing big number of data converting to draft for now :)

@AshAman999 AshAman999 marked this pull request as draft July 8, 2022 14:59
@@ -73,6 +73,18 @@ class DaoProduct extends AbstractSqlDao
return result;
}

// Returns the no of rows deleted/effected from the table
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Returns the no of rows deleted/effected from the table
// Returns the number of rows deleted/effected from the table

}
}

class OfflineDataDetials extends StatelessWidget {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small typo in the name

}
}

class AppBar extends StatelessWidget {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this Widget public?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • the name is confusing with the official AppBar. Please use another one

) ??
false;
if (confirmed) {
final double size = await localDatabase.getSizeinMb();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in -> In

@@ -276,6 +280,53 @@ class DaoProductList extends AbstractDao {
return result;
}

/// Returns the number of items cleared from the box
Future<int> clearAll() async {
final int clearedItems = await _getBox().clear();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should use a single line instruction here

@@ -43,4 +43,7 @@ class DaoStringList extends AbstractDao {
}
return false;
}

// Returns the number of items removed from the box
Future<int> removeAll() async => _getBox().clear();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Async is useless here

databasesRootPath = await getDatabasesPath();
}
final String databasePath = join(databasesRootPath, 'smoothie.db');
final String databasePath = await _getDatabasePath();
final Database database = await openDatabase(
databasePath,
version: 2,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you make this a constant

//Keeping at a cap of 500 for better performance
final List<Product> productList = <Product>[];
List<String> chunks = <String>[];
for (int i = 0; i < barcodes.length; i += 500) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is 500? Please may it a constant

@AshAman999 AshAman999 closed this Sep 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

Design a menu in the settings to let user delete the scan history saved in db
6 participants