Import System Unreliable

[hactar]

Do you want to request a feature, report a bug, or ask a question about wit?

Bug

Issue

When reimporting a large wit app, the import is incomplete and the result unusable. This happens even if you do not modify the zip. There are multiple issues on GitHub that have been raised (#645, #504, #417, #381, #358, #316, #344, etc) , this issue describes a concrete, reproducible example of failure.

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem.

(tested with both safari and firefox)

1. Export the app under https://wit.ai/sindre/WienBot-PROD
2. Without modifying the zip, reimport the app
3. Export the new app
4. compare the two zip files

What is the expected behaviour?

The exported zip files should match, apart from the app.json file which contains the new app name.

Actual behaviour

After a two minutes the import times out.

Probably because of this, the actual import:

• Is missing ~7000 lines in the expressions.json file: 64% (!) of the expressions were lost.
• Has ~20 entity files missing a few expressions, possibly due to the loss in the expression file?
• Has search strategies for the entities set up incorrectly, the initial app only has "keywords" entities, the reimported version has most entities set to "free-text, keywords" (default value?), with only very few entities correctly set to "keywords" only.
• Some stories do not validate, probably because they are missing examples from the expressions file and because the search strategies are set up incorrectly.

You can see my reimport here: https://wit.ai/hactar/WienBotFiltered

Workaround

Possible workaround might be using the new /samples api, but as the import leaves you in a "half done" state, I'm not sure if calling /samples will fix all issues, and it definitely won't fix the "free-text, keywords" issue.

--
For proper in-production use, we need a reliable way of backing up and restoring data - I'm currently stuck as I have no way to "fork" my bot and merge it with a different one. If you look at the number of people reporting issues with import, this is really causing problems for us.

Maybe you could increase the timeout during import as a workaround?

Thanks.

[klemensz]

Any news? I just made en export of an app with 18 entities, and re-imported it to a new app. Now there are 38 entities, most of them don't have a name, expressions are missing. Unusable...

[patapizza]

@klemensz We're working on improving the import reliability. This is likely going to still take a few more months. Thank you for your patience.

[klemensz]

OK, we need it this week. Thanks anyway.

[tchapi]

Any update on this ? Thanks !

[stopachka]

Hey team, this is coming closer and closer. If you see more specific issues, that are consistently reproducible, please open a task for them, and reference this issue .

We will aim to fix with an architectural change, but in the meantime I want to make sure there are no reproducible errors that we are letting slide

[Barbog]

Closing due to no movement on the issue. Please re-open or file a new task should the issue be persisting.