Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: move bulk print operation to the background #25358

Merged
merged 10 commits into from
Mar 13, 2024
Merged

feat: move bulk print operation to the background #25358

merged 10 commits into from
Mar 13, 2024

Conversation

akhilnarang
Copy link
Member

@akhilnarang akhilnarang commented Mar 12, 2024 •
edited

People want to print a lot of documents sometimes, that will usually cause a timeout.
This moves bulk printing to the background by default, so that users won't face that issue.
Changed the restriction to 500 documents.

If the user does close the tab, they won't know when the process is done and what the file name / URL is going to be, they will however be able to guess from the list of files ($doctype_uuid() basically), if they really want to.

Eventually can make use of #25138

image
image

@akhilnarang akhilnarang requested a review from a team as a code owner March 12, 2024 10:09
@akhilnarang akhilnarang requested review from ankush and removed request for a team March 12, 2024 10:09
@github-actions github-actions bot added the add-test-cases Add test case to validate fix or enhancement label Mar 12, 2024
@akhilnarang akhilnarang force-pushed the bulk-print-background branch 4 times, most recently from 4bacee2 to 1085f6a Compare March 13, 2024 06:36
ankush
ankush previously requested changes Mar 13, 2024
View reviewed changes
Copy link
Member

@ankush ankush left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you try this with ~500 documents. Just to see if it's reliable and resource usage is constant.

frappe/utils/print_format.py Outdated Show resolved Hide resolved
frappe/public/js/frappe/list/bulk_operations.js Outdated Show resolved Hide resolved
frappe/utils/print_format.py Show resolved Hide resolved
frappe/utils/print_format.py Outdated Show resolved Hide resolved
@akhilnarang akhilnarang requested a review from ankush March 13, 2024 09:05
@akhilnarang akhilnarang added backport version-14-hotfix backport to version 14 backport version-15-hotfix Backport the PR to v15 and removed add-test-cases Add test case to validate fix or enhancement labels Mar 13, 2024
@ankush ankush dismissed their stale review March 13, 2024 09:12

completed

@akhilnarang akhilnarang force-pushed the bulk-print-background branch 2 times, most recently from 23e406f to 9892a83 Compare March 13, 2024 09:42
@akhilnarang akhilnarang requested a review from ankush March 13, 2024 09:44
@akhilnarang akhilnarang force-pushed the bulk-print-background branch 2 times, most recently from 52c03c0 to 27e9133 Compare March 13, 2024 10:22
@akhilnarang
feat: move bulk print operation to the background
bf6cb1a 
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
@akhilnarang
fix: open PDF in new tab
5d0db0c 
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
@akhilnarang
fix: update message
9bf22b7 
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
@akhilnarang
fix: add back a limitation to number of the documents
1caae03 
Don't allow printing more than 500 documents

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
@akhilnarang
fix: let backend generate task ID
5a6bff9 
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
@akhilnarang
fix: make filename more user-friendly
508e4d9 
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
@akhilnarang
refactor(bulk_print): choose queue dynamically
5e37ac7 
Update docstrings and type hints a bit

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
@akhilnarang
refactor: add in a new endpoint for background printing
0ec3e4a 
Let the original one stay as-is for backward compatibility

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
@akhilnarang
fix: unsubscribe from task after completion
6a5af14 
Also update event name to be more logical

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
@akhilnarang
refactor: make download button a primary action, update text
5f087ed 
Signed-off-by: Akhil Narang <me@akhilnarang.dev>
@akhilnarang akhilnarang merged commit 6c11dae into frappe:develop Mar 13, 2024
20 of 23 checks passed
This was referenced Mar 13, 2024
Merged
Merged
akhilnarang added a commit that referenced this pull request Mar 13, 2024
@mergify @akhilnarang
feat: move bulk print operation to the background (backport #25358) (#…
4508239 
…25396)

* feat: move bulk print operation to the background

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
(cherry picked from commit bf6cb1a)

# Conflicts:
#	frappe/utils/print_format.py

* fix: open PDF in new tab

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
(cherry picked from commit 5d0db0c)

* fix: update message

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
(cherry picked from commit 9bf22b7)

* fix: add back a limitation to number of the documents

Don't allow printing more than 500 documents

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
(cherry picked from commit 1caae03)

* fix: let backend generate task ID

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
(cherry picked from commit 5a6bff9)

* fix: make filename more user-friendly

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
(cherry picked from commit 508e4d9)

* refactor(bulk_print): choose queue dynamically

Update docstrings and type hints a bit

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
(cherry picked from commit 5e37ac7)

* refactor: add in a new endpoint for background printing

Let the original one stay as-is for backward compatibility

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
(cherry picked from commit 0ec3e4a)

# Conflicts:
#	frappe/utils/print_format.py

* fix: unsubscribe from task after completion

Also update event name to be more logical

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
(cherry picked from commit 6a5af14)

* refactor: make download button a primary action, update text

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
(cherry picked from commit 5f087ed)

* chore: fix conflicts

Signed-off-by: Akhil Narang <me@akhilnarang.dev>

---------

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
Co-authored-by: Akhil Narang <me@akhilnarang.dev>
akhilnarang added a commit that referenced this pull request Mar 13, 2024
@mergify @akhilnarang
feat: move bulk print operation to the background (backport #25358) (#…
513487e 
…25397)

* feat: move bulk print operation to the background

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
(cherry picked from commit bf6cb1a)

# Conflicts:
#	frappe/utils/print_format.py

* fix: open PDF in new tab

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
(cherry picked from commit 5d0db0c)

* fix: update message

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
(cherry picked from commit 9bf22b7)

* fix: add back a limitation to number of the documents

Don't allow printing more than 500 documents

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
(cherry picked from commit 1caae03)

* fix: let backend generate task ID

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
(cherry picked from commit 5a6bff9)

* fix: make filename more user-friendly

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
(cherry picked from commit 508e4d9)

* refactor(bulk_print): choose queue dynamically

Update docstrings and type hints a bit

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
(cherry picked from commit 5e37ac7)

* refactor: add in a new endpoint for background printing

Let the original one stay as-is for backward compatibility

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
(cherry picked from commit 0ec3e4a)

* fix: unsubscribe from task after completion

Also update event name to be more logical

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
(cherry picked from commit 6a5af14)

* refactor: make download button a primary action, update text

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
(cherry picked from commit 5f087ed)

* chore: fix conflicts

Signed-off-by: Akhil Narang <me@akhilnarang.dev>

---------

Signed-off-by: Akhil Narang <me@akhilnarang.dev>
Co-authored-by: Akhil Narang <me@akhilnarang.dev>
akhilnarang added a commit to akhilnarang/frappe that referenced this pull request Mar 14, 2024
@akhilnarang
Revert "feat: move bulk print operation to the background (backport f…
af20ca6 
…rappe#25358) (frappe#25396)"

Some `frappe.realtime` APIs are different now, will figure it out later if required on v14

This reverts commit 4508239.
akhilnarang added a commit that referenced this pull request Mar 14, 2024
@akhilnarang
Revert "feat: move bulk print operation to the background (backport #…
490f031 
…25358) (#25396)" (#25400)

Some `frappe.realtime` APIs are different now, will figure it out later if required on v14

This reverts commit 4508239.
frappe-pr-bot pushed a commit that referenced this pull request Mar 19, 2024
@frappe-bot
chore(release): Bumped to Version 15.18.0
cb7d23b 
# [15.18.0](v15.17.3...v15.18.0) (2024-03-19)

### Bug Fixes

* add params in make_request arguments ([#25418](#25418)) ([18cf09b](18cf09b))
* allow transitioning to long text (backport [#25419](#25419)) ([#25475](#25475)) ([dbb326e](dbb326e))
* avoid closing filter popover on any date picker interactions ([cd98a21](cd98a21))
* Avoid setting filter on now/today button ([592e6f9](592e6f9))
* better filename for prepared report ([8eb1b24](8eb1b24))
* child table rating fields ([#25433](#25433)) ([#25436](#25436)) ([6f242ca](6f242ca))
* consider all datepicker elements ([#25426](#25426)) ([361fbfe](361fbfe))
* **ControlText:** Preserve indentation when read only ([#25262](#25262)) ([#25469](#25469)) ([c783f22](c783f22))
* debounce filter refresh ([9b75187](9b75187))
* default filter setup on todo list ([#25455](#25455)) ([#25464](#25464)) ([21e1280](21e1280))
* dont add trailing decimal separator ([#25389](#25389)) ([#25391](#25391)) ([760d9b2](760d9b2))
* dont add useless distinct clause ([8891e8e](8891e8e))
* dont render very large reports, offer export instead ([3f0760c](3f0760c))
* dont show tooltip if already limited ([#25361](#25361)) ([eb8b9f4](eb8b9f4))
* escape text types before setting disp area ([#25520](#25520)) ([#25523](#25523)) ([bc83b24](bc83b24))
* escape value in multiselect pill ([#25516](#25516)) ([#25518](#25518)) ([236a3b5](236a3b5))
* exclude irrelevant regional links ([#25510](#25510)) ([#25512](#25512)) ([9fe8b6a](9fe8b6a))
* filters can use more than 140 chars ([2d2b4a9](2d2b4a9))
* handle distinct for fieldname ([#25511](#25511)) ([#25515](#25515)) ([9168b0f](9168b0f))
* handle invalid fetch from split ([fa131ca](fa131ca))
* handle meta.fields being undefined ([#25539](#25539)) ([01cca38](01cca38))
* handle parent rename in child workspace ([ef3d068](ef3d068))
* hide datepicker after picking date ([d91a2b8](d91a2b8))
* **json_handler:** handle `uuid.UUID` ([ebaaa6b](ebaaa6b)), closes [#25242](#25242)
* Log ipython commands (backport [#25364](#25364)) ([#25368](#25368)) ([a2ebb4e](a2ebb4e))
* log mariadb console usage ([9797457](9797457))
* log psql console usage as well (backport [#25417](#25417)) ([#25423](#25423)) ([f0fa5d1](f0fa5d1))
* prevent TypeError if doc not defined ([#25540](#25540)) ([6b23504](6b23504))
* **QuickEntry:** Don't allow Tab Breaks ([#24950](#24950)) ([#25468](#25468)) ([a31943e](a31943e))
* **recorder:** handle frappe.db.sql(run=0) ([#25450](#25450)) ([#25454](#25454)) ([d130777](d130777))
* Respect document language in communication ([#25402](#25402)) ([#25403](#25403)) ([0d09b51](0d09b51))
* show attachments on notifications too ([#25443](#25443)) ([#25445](#25445)) ([371fa09](371fa09))
* skip setting of contact full name if its too long ([#25509](#25509)) ([d5685d7](d5685d7))
* skip virtual doctype in tags ([f18ac26](f18ac26))
* Skip virtual doctype rename for dynamic links ([#25479](#25479)) ([#25481](#25481)) ([3353488](3353488))
* Skip virtual doctypes while renaming ([#25473](#25473)) ([#25478](#25478)) ([aaacc51](aaacc51))
* support child tables in count with limit ([d8a797b](d8a797b))
* traceback colours ([#25356](#25356)) ([b6d2785](b6d2785))
* **UX:** let user see actual count on click ([5c1bca8](5c1bca8))
* validate homepage paths (backport [#25409](#25409)) ([#25412](#25412)) ([0aeaeeb](0aeaeeb))
* we want `DF.Literal[None]`, not `DF.LiteralNone` ([#25394](#25394)) ([f008eeb](f008eeb))

### Features

* connect to redis sentinel for redis cache ([#25398](#25398)) ([#25446](#25446)) ([498b1a4](498b1a4))
* debug stuck process by sending SIGUSR1 (backport [#25502](#25502)) ([#25527](#25527)) ([6c8cc5c](6c8cc5c))
* move bulk print operation to the background (backport [#25358](#25358)) ([#25397](#25397)) ([513487e](513487e))
* Store printed PDF attachments on communication ([#25439](#25439)) ([#25444](#25444)) ([bd15abc](bd15abc))
* support countig till a limit ([615800f](615800f))

### Performance Improvements

* Avoid ordering in count query ([#25451](#25451)) ([#25452](#25452)) ([07aadd3](07aadd3))
* compact prepared report files ([3adbb35](3adbb35))
* don't extract backups whenever possible ([#25350](#25350)) ([9c6fb8a](9c6fb8a)), closes [/github.com//pull/24898#discussion_r1521076386](https://github.com//github.com/frappe/frappe/pull/24898/issues/discussion_r1521076386)
* reduce import overhead on background jobs (backport [#25459](#25459)) ([#25460](#25460)) ([eab5b8d](eab5b8d))
* show estimated count on list view ([7ee9719](7ee9719))
frappe-pr-bot pushed a commit that referenced this pull request Mar 19, 2024
@frappe-bot
chore(release): Bumped to Version 14.68.0
4ea9c50 
# [14.68.0](v14.67.1...v14.68.0) (2024-03-19)

### Bug Fixes

* allow exporting large reports that can't be rendered (backport [#25395](#25395)) ([#25404](#25404)) ([b3f4d5d](b3f4d5d))
* allow transitioning to long text (backport [#25419](#25419)) ([#25474](#25474)) ([60d8c1c](60d8c1c))
* avoid closing filter popover on any date picker interactions ([0784e61](0784e61))
* Avoid setting filter on now/today button ([83766c0](83766c0))
* child table rating fields ([#25433](#25433)) ([#25435](#25435)) ([57cf161](57cf161))
* consider all datepicker elements ([#25426](#25426)) ([ad79301](ad79301))
* debounce filter refresh ([7f65916](7f65916))
* don't allow setting an invalid rating ([#22633](#22633)) ([#25432](#25432)) ([502b907](502b907))
* dont add trailing decimal separator ([#25389](#25389)) ([#25390](#25390)) ([08e8d8c](08e8d8c))
* escape text types before setting disp area ([#25520](#25520)) ([#25522](#25522)) ([e446770](e446770))
* escape value in multiselect pill ([#25516](#25516)) ([#25517](#25517)) ([dbcf783](dbcf783))
* filters can use more than 140 chars ([d04bff9](d04bff9))
* handle distinct for fieldname (backport [#25511](#25511)) ([#25514](#25514)) ([3344df0](3344df0))
* handle parent rename in child workspace ([c3d9e1f](c3d9e1f))
* hide datepicker after picking date ([39796ea](39796ea))
* Log ipython commands (backport [#25364](#25364)) ([#25367](#25367)) ([997628c](997628c))
* log mariadb console usage ([6f4981c](6f4981c))
* show attachments on notifications too ([#25443](#25443)) ([#25448](#25448)) ([3251586](3251586))
* skip virtual doctype in tags ([b2ae437](b2ae437))
* Skip virtual doctype rename for dynamic links ([#25479](#25479)) ([#25480](#25480)) ([f368236](f368236))
* Skip virtual doctypes while renaming ([#25473](#25473)) ([#25477](#25477)) ([c423223](c423223))
* validate homepage paths (backport [#25409](#25409)) ([#25411](#25411)) ([ee1c8e3](ee1c8e3))

### Features

* connect to redis sentinel for redis cache (backport [#25398](#25398)) ([#25449](#25449)) ([6ce4036](6ce4036))
* debug stuck process by sending SIGUSR1 (backport [#25502](#25502)) ([#25526](#25526)) ([95d1224](95d1224))
* move bulk print operation to the background (backport [#25358](#25358)) ([#25396](#25396)) ([4508239](4508239))
* Store printed PDF attachments on communication (backport [#25439](#25439)) ([#25447](#25447)) ([9b7a02f](9b7a02f))

### Performance Improvements

* cap max count on list views (backport [#25348](#25348)) ([#25463](#25463)) ([3bc5d8a](3bc5d8a)), closes [#25361](#25361)
* compact prepared report files ([0aa3c8e](0aa3c8e))

### Reverts

* Revert "feat: move bulk print operation to the background (backport #25358) (#25396)" (#25400) ([490f031](490f031)), closes [#25358](#25358) [#25396](#25396) [#25400](#25400)
@akashkrishna619 akashkrishna619 mentioned this pull request Mar 25, 2024
Merged
This was referenced Mar 26, 2024
Closed
Merged
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 28, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@ankush ankush Awaiting requested review from ankush

Assignees
No one assigned
Labels
backport version-14-hotfix backport to version 14 backport version-15-hotfix Backport the PR to v15
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@akhilnarang @ankush