Skip to content

refactor: improve BlockingLoreAPI errors and remove its unwraps #54

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

lorenzoberts
Copy link
Contributor

@lorenzoberts lorenzoberts commented Sep 28, 2024

This PR was built over lorenzo/use-mockall-lib (#45) to avoid dealing with conflicts after it's merge, but they have no direct relation so you can ignore 38e6b14

With that being said, this is another PR which does not include any new functionality, it aims to improve general code quality by removing repetitive code and unnecessary unwraps mostly in src/lore_api_client.rs. The improvements are as follows:

  1. Create a centralized error enum (ClientError) for all the client traits (and consequently for the client itself)

I thought it didn't make much sense to have a specific error to each trait, since all of them were focused on handling request errors (expect for EndOfFeed). Besides, by using the thiserror create we can convert errors easily, allowing ClientError to capture all rewquest:Errors and convert it into a ClientError. Also there were also unnecessary unwraps that could be removed by simply using ?.

  1. Centralize the steps of making the HTTP request and getting it's body

It was also another point I thought could be improved, since all trait implementations had the same steps to get a response body for given target url.

  1. Avoid using reqwest::blocking::get directly

According to reqwest::blocking::getdocs, this method should be avoided since it creates a single client for each of the requests. Even though this will probably not have noticeable changes in performance, I thought it was better to make rewquest::blocking::Client an attribute of BlockingAPIClient so that all requests can use the same HTTP client.

Regarding EndOfFeed, I struggled to decide if it made more sense to implement it in LoreSession or in the API client. In the end I decided to keep it where it was.
Lastly, I considered using Arc around BlockingAPIClient, but I realized this only makes a difference in multi-threaded contexts. I don't know if it could help resolve #3, but since I couldn't reproduce this flaky behavior I don't know if it persists.

EDIT: After reading about Rc (the single-threaded equivalent of Arc) and the reqwest client, I found that it can be used to share the same client across structs that need the client. That said, simply cloning the client is also an option since it internally reuses underlying resources, so it doesn't recreate all pools, connections, etc., with every clone. Both solutions are likely better than creating a new client each time, so I will add another commit to implement this. As always, let me know what you think.

Helps #28

…lockingLoreAPIClient

This commit adds the "automock" library to Cargo.toml, which is used to
automatically mock traits and structs, avoiding the need for manual
implementation of each trait function just for testing. Besides that, this
commit also replaces the FakeLoreAPIClient used in src/lore_session/tests.rs
with an "automock" for BlockingLoreAPIClient. This way, each test can have
an explicit expectation for each of the called functions, facilitating
test implementation and improving test quality overall.

Closes: kworkflow#34

Signed-off-by: Lorenzo Bertin Salvador <lorenzobs@usp.br>
This commit adds a centralized client error for BlockingLoreAPICLient.
It improves code quality by removing repetitive handling of request errors
and unecessary unwraps.
The new ClientError derives thiserror::Error, which makes error conversion
easier between structs/functions.

Signed-off-by: Lorenzo Bertin Salvador <lorenzobs@usp.br>
@lorenzoberts lorenzoberts force-pushed the lorenzo/improve-api-client branch from e266b0c to c860cfb Compare September 28, 2024 17:30
We were initializing a new client in every struct we wanted to
use it, but it is cheaper to clone the client since it reuses
underlying resources. This commit adds the BlockingAPIClient
to every struct that needs it and initializes the client only once
(during app startup).

Signed-off-by: Lorenzo Bertin Salvador <lorenzobs@usp.br>
@davidbtadokoro
Copy link
Collaborator

davidbtadokoro commented Oct 5, 2024

Hey @lorenzoberts, thanks for yet another great PR!

At first glance, I thought you've only done some quality work on BlockingLoreAPIClient, but after reading the PR description (awesome, btw) and reviewing it, I feel like this is a major change!

Besides the great points you've brought up, which I agree with entirely, I think this has the potential of solving #3, indeed. This issue is one of the more confusing weak points in patch-hub , and I've always suspected it was due to the go-horse way I've implemented this HTTP client. To be honest, I've implemented this right from the outset of patch-hub when I had little to no knowledge of Rust 🤣

I will test this for real ASAP, and even if it doesn't solve #3, I don't see a reason why not merge. Thanks for the impressive work, and I will hit you back soon.

@davidbtadokoro
Copy link
Collaborator

davidbtadokoro commented Oct 7, 2024

Hey @lorenzoberts, thanks for the wait. The text below isn't about your change, per se; it's just some context about the work I did to try to solve #3 from what you've brought to the table.

My Side Quest

I took more than expected as I was in high hopes of this PR solving #3 collaterally. I had some offline talks with Nelson about the root of the problem, and we were not convinced of any concrete reason. Just to inform you about the matter:

  1. Sometimes, it may take 6+ seconds to get a response from Lore API. I found that in those scenarios, for some reason, connecting to a VPN provider, i.e., not changing a single line of code, only the networking conditions, "solves" the problem.
  2. When the flaky behavior is in effect, the strange thing is that, through a web browser making the same exact underlying request, we don't observe the problem.

Some time ago, I saw that using reqwest::blocking::get was a really bad idea. From what I've explained above (and the convos with Nelson), it was a long shot that avoiding this instantiation could solve the problem, which was one I was on board to take. It goes without saying that this didn't work; otherwise, I wouldn't be writing this huge text 😬

About the PR itself

I've ended up merging it as it was indeed a huge quality improvement to the lore_api_client mod, both in the quality of code and the actual functionality robustness. I must also give kudos for the great commit messages and overall good practices. I basically didn't alter anything, but, as always, edits are annotated in the commit messages.

Nevertheless, thank you so much for the great work! Change merged into the unstable branch 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants