Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimized Favicon downloading #3751

Merged
merged 1 commit into from
Aug 13, 2023

Conversation

BlackDex
Copy link
Collaborator

@BlackDex BlackDex commented Aug 4, 2023

Some optimizations in regards to downloading Favicon's.

I also encounterd some issues with accessing some sites where the connection got dropped or closed early. This seems a reqwest/hyper thingy, hyperium/hyper#2136. This is now also fixed.

General:

  • Decreased struct size (reduces memory usage)
  • Decreased memory allocations (faster HTML parsing)
  • Optimized tokenizer a bit more to only emit tags when all attributes are there and are valid.

reqwest/hyper connection issue:
The following changes helped solve the connection issues to some sites. The endresult is that some icons are now able to be downloaded always instead of sometimes.

  • Enabled some extra reqwest features, deflate and native-tls-alpn (Which do not bring in any extra crates since other crates already enabled them, but they were not active for Vaultwarden it self)
  • Configured reqwest to have a max amount of idle pool connections per host
  • Configured reqwest to timeout the idle connections in 10 seconds

src/api/icons.rs Outdated Show resolved Hide resolved
src/api/icons.rs Outdated Show resolved Hide resolved
src/api/icons.rs Outdated Show resolved Hide resolved
src/api/icons.rs Outdated Show resolved Hide resolved
Some optimizations in regards to downloading Favicon's.

I also encounterd some issues with accessing some sites where the
connection got dropped or closed early. This seems a reqwest/hyper
thingy, hyperium/hyper#2136. This is now also
fixed.

General:

- Decreased struct size
- Decreased memory allocations
- Optimized tokenizer a bit more to only emit tags when all attributes are there and are valid.

reqwest/hyper connection issue:
The following changes helped solve the connection issues to some sites.
The endresult is that some icons are now able to be downloaded always instead of sometimes.

- Enabled some extra reqwest features, `deflate` and `native-tls-alpn`
  (Which do not bring in any extra crates since other crates already enabled them, but they were not active for Vaultwarden it self)
- Configured reqwest to have a max amount of idle pool connections per host
- Configured reqwest to timeout the idle connections in 10 seconds
@BlackDex BlackDex merged commit 3d2df6c into dani-garcia:main Aug 13, 2023
3 checks passed
@BlackDex BlackDex deleted the optimize-icon-fetching branch August 13, 2023 17:31
arthurgeek pushed a commit to arthurgeek/vaultwarden-fly-template that referenced this pull request Sep 2, 2023
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [vaultwarden/server](https://togithub.com/dani-garcia/vaultwarden) |
stage | patch | `1.29.1-alpine` -> `1.29.2-alpine` |

---

### Release Notes

<details>
<summary>dani-garcia/vaultwarden (vaultwarden/server)</summary>

###
[`v1.29.2`](https://togithub.com/dani-garcia/vaultwarden/releases/tag/1.29.2)

[Compare
Source](https://togithub.com/dani-garcia/vaultwarden/compare/1.29.1...1.29.2)

Minor release to fix an issue forcing user to set amaster password when
logging in even when it's already set

#### What's Changed

- Fix .env.template file by
[@&#8203;BlackDex](https://togithub.com/BlackDex) in
[dani-garcia/vaultwarden#3734
- Fix UserOrg status during LDAP Import by
[@&#8203;BlackDex](https://togithub.com/BlackDex) in
[dani-garcia/vaultwarden#3740
- Update images to Bookworm and PQ15 and Rust v1.71 by
[@&#8203;BlackDex](https://togithub.com/BlackDex) in
[dani-garcia/vaultwarden#3573
- Implement "login with device" by
[@&#8203;quexten](https://togithub.com/quexten) in
[dani-garcia/vaultwarden#3592
- chore: Bump web vault to v2023.7.1 and bump Rust by
[@&#8203;GeekCornerGH](https://togithub.com/GeekCornerGH) in
[dani-garcia/vaultwarden#3769
- Optimized Favicon downloading by
[@&#8203;BlackDex](https://togithub.com/BlackDex) in
[dani-garcia/vaultwarden#3751
- add UserDecryptionOptions to login response by
[@&#8203;stefan0xC](https://togithub.com/stefan0xC) in
[dani-garcia/vaultwarden#3813
- add new secretsmanager plan for web-v2023.8.x by
[@&#8203;stefan0xC](https://togithub.com/stefan0xC) in
[dani-garcia/vaultwarden#3797
- Allow Authorization header for Web Sockets by
[@&#8203;BlackDex](https://togithub.com/BlackDex) in
[dani-garcia/vaultwarden#3806
- Update admin interface by
[@&#8203;BlackDex](https://togithub.com/BlackDex) in
[dani-garcia/vaultwarden#3730

**Full Changelog**:
dani-garcia/vaultwarden@1.29.1...1.29.2

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "on saturday" (UTC), Automerge - At
any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://developer.mend.io/github/arthurgeek/vaultwarden-fly-template).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNi42OC4xIiwidXBkYXRlZEluVmVyIjoiMzYuNjguMSIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
arthurgeek pushed a commit to arthurgeek/vaultwarden-fly that referenced this pull request Sep 2, 2023
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [vaultwarden/server](https://togithub.com/dani-garcia/vaultwarden) |
stage | patch | `1.29.1-alpine` -> `1.29.2-alpine` |

---

### Release Notes

<details>
<summary>dani-garcia/vaultwarden (vaultwarden/server)</summary>

###
[`v1.29.2`](https://togithub.com/dani-garcia/vaultwarden/releases/tag/1.29.2)

[Compare
Source](https://togithub.com/dani-garcia/vaultwarden/compare/1.29.1...1.29.2)

Minor release to fix an issue forcing user to set amaster password when
logging in even when it's already set

##### What's Changed

- Fix .env.template file by
[@&#8203;BlackDex](https://togithub.com/BlackDex) in
[dani-garcia/vaultwarden#3734
- Fix UserOrg status during LDAP Import by
[@&#8203;BlackDex](https://togithub.com/BlackDex) in
[dani-garcia/vaultwarden#3740
- Update images to Bookworm and PQ15 and Rust v1.71 by
[@&#8203;BlackDex](https://togithub.com/BlackDex) in
[dani-garcia/vaultwarden#3573
- Implement "login with device" by
[@&#8203;quexten](https://togithub.com/quexten) in
[dani-garcia/vaultwarden#3592
- chore: Bump web vault to v2023.7.1 and bump Rust by
[@&#8203;GeekCornerGH](https://togithub.com/GeekCornerGH) in
[dani-garcia/vaultwarden#3769
- Optimized Favicon downloading by
[@&#8203;BlackDex](https://togithub.com/BlackDex) in
[dani-garcia/vaultwarden#3751
- add UserDecryptionOptions to login response by
[@&#8203;stefan0xC](https://togithub.com/stefan0xC) in
[dani-garcia/vaultwarden#3813
- add new secretsmanager plan for web-v2023.8.x by
[@&#8203;stefan0xC](https://togithub.com/stefan0xC) in
[dani-garcia/vaultwarden#3797
- Allow Authorization header for Web Sockets by
[@&#8203;BlackDex](https://togithub.com/BlackDex) in
[dani-garcia/vaultwarden#3806
- Update admin interface by
[@&#8203;BlackDex](https://togithub.com/BlackDex) in
[dani-garcia/vaultwarden#3730

**Full Changelog**:
dani-garcia/vaultwarden@1.29.1...1.29.2

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "on saturday" (UTC), Automerge - At
any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://developer.mend.io/github/arthurgeek/vaultwarden-fly).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNi42OC4xIiwidXBkYXRlZEluVmVyIjoiMzYuNjguMSIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants