Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(admin): create bot from template is fast again #11218

Merged
merged 1 commit into from
Jan 14, 2022
Merged

fix(admin): create bot from template is fast again #11218

merged 1 commit into from
Jan 14, 2022

Conversation

EFF
Copy link
Member

@EFF EFF commented Jan 14, 2022 •
edited

Description

For some reason creating a bot from a template was taking ages. Imn short for each file that we write we ensured that the bot wasn't locked. In the case of creating a chatbot, it is by design not locked, for sure. Simply ignoring the locking status drastically speed the bot creation up.

Here are screenshots of time spent on bot creation request.

Before: (6.84s)
Screen Shot 2022-01-14 at 4 23 03 PM

After (410ms)
Screen Shot 2022-01-14 at 4 21 28 PM

Type of change

  • Bug fix

@EFF EFF requested a review from laurentlp January 14, 2022 21:30
laurentlp
laurentlp approved these changes Jan 14, 2022
View reviewed changes
Copy link
Contributor

@laurentlp laurentlp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotta go fast 💨!!!!

@allardy
Copy link
Member

allardy commented Jan 14, 2022 •
edited

Hmm I don't really understand how "ignoreLock" can have this impact; this has been there for ages. I noticed a bigger delay ever since we added the "lockfile" in the DiskStorageDriver (it writes a lockfile everytime before creating each files, then delete the lock)

image

The "ignoreLocked" was only done once and it only checked if the bot config existed and was locked; it was executed once then uploaded all the files

@EFF
Copy link
Member Author

EFF commented Jan 14, 2022

@allardy Don't know what to say. Just look at the numbers. 30 times faster. I could have gone deeper and fix the underlying issue but there's no point at this time as this fixes it and the lock check is totally useless when creating a bot

@allardy
Copy link
Member

allardy commented Jan 14, 2022

@EFF Just digged a bit deeper.. Since we read the config file each time we upsert a file, it locks, read, then release the lock... I guess we might bring it down to 50ms if we ignored the "lockfile" when creating the bot. Anyway, it make sense. GJ

@EFF EFF merged commit 3ed45c1 into master Jan 14, 2022
@EFF EFF deleted the f_fix-slow-bot-create branch January 14, 2022 22:15
@github-actions github-actions bot mentioned this pull request Jan 14, 2022
Closed
@EFF EFF mentioned this pull request Jan 17, 2022
Merged
samuelmasse added a commit that referenced this pull request Jan 20, 2022
@samuelmasse @laurentlp
@EFF @dependabot @StephanieTremblay @allardy
chore: merge master into next (#11249)
b1c2d9d 
* chore(ci_cd): use gh-actions v1 (#11211)

* chore(messaging): add a migration in the core to migrate messaging database (#5723)

* create migration

* runMessagingMigration

* update version

* bump messaging

* bump versions

* 0.0.1 migration

* bot token migration

* fix

* fix

* fix(misunderstood): strip slots for amend screen (#11214)

* chore(deps): bump follow-redirects in /modules/.nlu-testing (#11212)

Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.13.1 to 1.14.7.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](follow-redirects/follow-redirects@v1.13.1...v1.14.7)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump follow-redirects from 1.13.1 to 1.14.7 in /modules/nlu (#11209)

Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.13.1 to 1.14.7.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](follow-redirects/follow-redirects@v1.13.1...v1.14.7)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump follow-redirects in /modules/channel-web (#11208)

Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.13.1 to 1.14.7.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](follow-redirects/follow-redirects@v1.13.1...v1.14.7)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update README.md (#5746)

The links in the chat 3rd party Oauth and inter-bot communication weren't updated. So I updated them.

* chore: update some libs to cover some vulns (#11215)

* fix(misunderstood): strip slots for amend screen

* chore: update multiple dependencies

* updated translation checker script

* updated translation checker script

* fix(admin): create bot from template is fast again (#11218)

* chore(ci_cd): added action to create release pull request (#11210)

* chore(ci_cd): added action to create release pull request

* use options instead of values

* delete release version script

* remove update pkg version script

* create-release -> create-release-pr

* chore(ci_cd): added action to create pre-release (#11216)

* chore(ci_cd): added action to create pre-release

* added missing extension

* chore(dev): bump studio version (#11220)

* chore(dev): bump studio version

* chore(dev): update studio version

* chore(dev): v12.26.9 (#11223)

* v12.26.9

* remove dead code

* update package.json in /build

* chore(e2e): improve e2e tests (#5519)

* chore(e2e): improve e2e tests setup

* run all e2e tests

* test

* improved jest environment

* test

* should fix unit tests

* test

* bold tests

* moved util into utils folder + added new line at end of files

* small refactor

* small refactor

* small improvements

* chore(e2e): admin tests are now working and up to date (#5772)

* chore(e2e): admin tests are now working

* fix logging of req & res

* fix unit tests

* fix collaborator admin e2e tests

* refactor + fix logging after tests ended issue

* should fix admin e2e auth tests

* small refactor

* small refactor

* chore(e2e): studio tests are now working (#5774)

* chore(e2e): studio tests are now working

* should fix nlu e2e tests

* small refactor

* chore(e2e): modules tests are now working (#11205)

* chore(e2e): modules tests are now working

* waitFor -> waitForTimeout

* fix types resolution with jest-circus

* remove unnecessary code

* bump studio nlu test timeout + added readme

* typos

* fix failing and flaky tests

* make sure host is down before checking it is up when restarting server

* temp test - run e2e tests on all os

* should fix test failing on windows

* should fix tests on mac

* might fix macos tests

* small fix

* temporarily comment macos e2e tests

* chore(dev): cleanup unused modules (#11226)

* chore(dev): cron to publish next binaries (#11236)

* chore(dev): cron to publish next binaries

* remove dead step and checkout next branch

* use ref not path

* chore(dev): use single quote in workflow (#11237)

* chore(dev): cron to publish next binaries

* remove dead step and checkout next branch

* use ref not path

* chore(dev): use single quote

* inputs under inputs field

* use format syntax

* chore(ci_cd): create draft release instead of pre release (#11224)

* chore(messaging): use new http client (#11217)

* use new client

* fix

* port

* fix

* fix

* fix

* new sync routes

* admin key

* bump

* fix

* fix

* bump

* move resolution

* chore(deps): update dependencies (#11244)

* chore(deps): update dependencies

* bump trim-off-newlines

* fix

Co-authored-by: Laurent Leclerc Poulin <laurentlp@users.noreply.github.com>
Co-authored-by: Francois-Xavier P. Darveau <955524+EFF@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: StephanieTremblay <94993374+StephanieTremblay@users.noreply.github.com>
Co-authored-by: Yann Allard <42552874+allardy@users.noreply.github.com>
Co-authored-by: Francois-Xavier P. Darveau <frxpoulin@gmail.com>
samuelmasse added a commit that referenced this pull request Jan 20, 2022
@samuelmasse @laurentlp
@EFF @dependabot @StephanieTremblay @allardy
chore(runtime): fix messaging (#11255)
6eb8f63 
* chore(ci_cd): use gh-actions v1 (#11211)

* chore(messaging): add a migration in the core to migrate messaging database (#5723)

* create migration

* runMessagingMigration

* update version

* bump messaging

* bump versions

* 0.0.1 migration

* bot token migration

* fix

* fix

* fix(misunderstood): strip slots for amend screen (#11214)

* chore(deps): bump follow-redirects in /modules/.nlu-testing (#11212)

Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.13.1 to 1.14.7.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](follow-redirects/follow-redirects@v1.13.1...v1.14.7)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump follow-redirects from 1.13.1 to 1.14.7 in /modules/nlu (#11209)

Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.13.1 to 1.14.7.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](follow-redirects/follow-redirects@v1.13.1...v1.14.7)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump follow-redirects in /modules/channel-web (#11208)

Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.13.1 to 1.14.7.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](follow-redirects/follow-redirects@v1.13.1...v1.14.7)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update README.md (#5746)

The links in the chat 3rd party Oauth and inter-bot communication weren't updated. So I updated them.

* chore: update some libs to cover some vulns (#11215)

* fix(misunderstood): strip slots for amend screen

* chore: update multiple dependencies

* updated translation checker script

* updated translation checker script

* fix(admin): create bot from template is fast again (#11218)

* chore(ci_cd): added action to create release pull request (#11210)

* chore(ci_cd): added action to create release pull request

* use options instead of values

* delete release version script

* remove update pkg version script

* create-release -> create-release-pr

* chore(ci_cd): added action to create pre-release (#11216)

* chore(ci_cd): added action to create pre-release

* added missing extension

* chore(dev): bump studio version (#11220)

* chore(dev): bump studio version

* chore(dev): update studio version

* chore(dev): v12.26.9 (#11223)

* v12.26.9

* remove dead code

* update package.json in /build

* chore(e2e): improve e2e tests (#5519)

* chore(e2e): improve e2e tests setup

* run all e2e tests

* test

* improved jest environment

* test

* should fix unit tests

* test

* bold tests

* moved util into utils folder + added new line at end of files

* small refactor

* small refactor

* small improvements

* chore(e2e): admin tests are now working and up to date (#5772)

* chore(e2e): admin tests are now working

* fix logging of req & res

* fix unit tests

* fix collaborator admin e2e tests

* refactor + fix logging after tests ended issue

* should fix admin e2e auth tests

* small refactor

* small refactor

* chore(e2e): studio tests are now working (#5774)

* chore(e2e): studio tests are now working

* should fix nlu e2e tests

* small refactor

* chore(e2e): modules tests are now working (#11205)

* chore(e2e): modules tests are now working

* waitFor -> waitForTimeout

* fix types resolution with jest-circus

* remove unnecessary code

* bump studio nlu test timeout + added readme

* typos

* fix failing and flaky tests

* make sure host is down before checking it is up when restarting server

* temp test - run e2e tests on all os

* should fix test failing on windows

* should fix tests on mac

* might fix macos tests

* small fix

* temporarily comment macos e2e tests

* chore(dev): cleanup unused modules (#11226)

* chore(dev): cron to publish next binaries (#11236)

* chore(dev): cron to publish next binaries

* remove dead step and checkout next branch

* use ref not path

* chore(dev): use single quote in workflow (#11237)

* chore(dev): cron to publish next binaries

* remove dead step and checkout next branch

* use ref not path

* chore(dev): use single quote

* inputs under inputs field

* use format syntax

* chore(ci_cd): create draft release instead of pre release (#11224)

* chore(messaging): use new http client (#11217)

* use new client

* fix

* port

* fix

* fix

* fix

* new sync routes

* admin key

* bump

* fix

* fix

* bump

* move resolution

* chore(deps): update dependencies (#11244)

* chore(deps): update dependencies

* bump trim-off-newlines

* fix

* bump messaging

* fix

* bump

Co-authored-by: Laurent Leclerc Poulin <laurentlp@users.noreply.github.com>
Co-authored-by: Francois-Xavier P. Darveau <955524+EFF@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: StephanieTremblay <94993374+StephanieTremblay@users.noreply.github.com>
Co-authored-by: Yann Allard <42552874+allardy@users.noreply.github.com>
Co-authored-by: Francois-Xavier P. Darveau <frxpoulin@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@laurentlp laurentlp laurentlp approved these changes

@allardy allardy allardy approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@EFF @allardy @laurentlp