Skip to content

proposal: enhancing staticmap generation with custom tile fetching#15

Open
hiratara wants to merge 5 commits intodanielalvsaaker:masterfrom
hiratara:tile-fetcher-trait
Open

proposal: enhancing staticmap generation with custom tile fetching#15
hiratara wants to merge 5 commits intodanielalvsaaker:masterfrom
hiratara:tile-fetcher-trait

Conversation

@hiratara
Copy link
Copy Markdown
Contributor

@hiratara hiratara commented Feb 4, 2024

Hello,

Hope you're doing well! I've been using your project and found it incredibly useful for my needs. As I often generate static maps and frequently adjust parameters, I've been considering ways to enhance the tile fetching process for efficiency.

I'm utilizing tokio and reqwest for async operations and aiming to use http_cache_reqwest for local caching of map tiles. This approach should minimize redundant fetches, speed up development, and reduce unnecessary requests. Additionally, I'm exploring throttling options to manage API request rates better.

This PR includes:

  • A new TileFetcher trait for customizable tile fetching, adding flexibility.
  • Preserving the original implementation in DefaultTileFetcher, utilizing attohttpc and rayon, to maintain a robust default option. My intention is not to replace but to expand the available choices.

Here's how I'm implementing my custom TileFetcher:

Details
pub struct TokioFetcher {
    client: ClientWithMiddleware,
    runtime: Arc<Runtime>,
}

impl TokioFetcher {
    pub fn new() -> Self {
        let client = ClientBuilder::new(Client::new())
            .with(Cache(HttpCache {
                mode: CacheMode::Default,
                manager: CACacheManager::default(),
                options: HttpCacheOptions::default(),
            }))
            .build();
        let runtime = Arc::new(Runtime::new().unwrap());
        TokioFetcher { client, runtime }
    }
}

impl TileFetcher for TokioFetcher {
    fn fetch(&self, urls: &[&str]) -> Vec<Result<Vec<u8>, Error>> {
        self.runtime.block_on(async {
            let handles = urls
                .into_iter()
                .map(|&url| {
                    let client = self.client.clone();
                    let url = url.to_owned();
                    tokio::spawn(async move {
                        let body = client
                            .get(&url)
                            .header(header::USER_AGENT, "MyCustomUserAgent/1.0")
                            .send()
                            .await
                            .map_err(to_boxed_error)?
                            .bytes()
                            .await
                            .map_err(to_boxed_error)?
                            .to_vec();
                        Ok(body)
                    })
                })
                .collect::<Vec<_>>();

            futures::future::join_all(handles)
                .await
                .into_iter()
                .map(|r| match r {
                    Ok(result) => result,
                    Err(e) => Err(to_boxed_error(e)),
                })
                .zip(urls)
                .map(|(res, url)| {
                    res.map_err(|e| Error::TileError {
                        error: e,
                        url: url.to_string(),
                    })
                })
                .collect::<Vec<_>>()
        })
    }
}

I've crafted this PR to introduce these features, thinking they might benefit others facing similar challenges. Eager to hear your feedback or suggestions for any improvements.

@danielalvsaaker
Copy link
Copy Markdown
Owner

This seems like a great addition! I considered something similar earlier but didn't land on an API I was satisfied with. I considered providing both a sync and async version of a tile provider trait, but it proved to be too clumsy. Instead, I think your sample of a custom Tokio tile provider implementation should be sufficient for such use cases.

Would you be interested in also providing an example in examples/, to illustrate its usage?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants