-
Notifications
You must be signed in to change notification settings - Fork 92
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add DriverManager::get_drivers_for_filename
for the ability to auto detect compatible Driver
s for writing data
#510
Changes from 10 commits
5f44a24
9fb996b
f7d6bcb
3d569e0
0b8d80d
79300cd
563eb97
62f7ccd
3dbc42f
05f46c4
7c90f62
223b846
2feaf29
5c32a2c
ed5a991
45473a6
fc0361a
2c1fc76
773872d
a187d38
f54cd81
326feda
04a549c
98fbc23
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -410,6 +410,90 @@ impl DriverManager { | |
Ok(Driver { c_driver }) | ||
} | ||
|
||
/// Get the Driver based on the file extension from filename | ||
/// | ||
/// Searches for the available extensions in the registered | ||
/// drivers and returns the matches. The determined driver is | ||
/// checked for writing capabilities as | ||
/// [`Dataset::open`](Dataset::open) can already open datasets | ||
/// with just path. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not sure I understand why we limit returned drivers to ones with "write" capabilities... maybe read-only use cases might also want a list of drivers to pick from (e.g. when reading JP2k, there are several options and you might want to inspect the metadata to decide on which to select). There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. So, initially I was thinking that, but then I realized that when we already have a dataset, we can just call The use case we're thinking is to when we need to write a dataset, currently the only way I could find to write to file is to create a file from a driver, which requires you to know the driver in advance. Of course, you can make the user pass the driver name along with the filename and call it a day (that's what I was doing on one of my program), but all users are not likely to know about the driver names. And if we can more or less guess based on the file extension, then we make it convenient and remove any user error (could write a Tiff file in .shp extension file and it'll write it). And since GIS programs let you write to any file, making the program only dependent on the filename makes it easier to integrate with other programs as well. I think renaming the function to mention it's for writing should help with it. |
||
/// | ||
/// See also: [`get_driver_by_name`](Self::get_driver_by_name) | ||
/// | ||
/// # Example | ||
/// | ||
/// ```rust, no_run | ||
/// use gdal::DriverManager; | ||
/// # fn main() -> gdal::errors::Result<()> { | ||
/// let compatible_drivers = | ||
/// DriverManager::get_drivers_for_filename("test.gpkg", true) | ||
/// .iter() | ||
/// .map(|d| d.short_name()) | ||
/// .collect::<Vec<String>>(); | ||
/// println!("{:?}", compatible_drivers); | ||
/// # Ok(()) | ||
/// # } | ||
/// ``` | ||
/// ```text | ||
/// ["GPKG"] | ||
/// ``` | ||
pub fn get_drivers_for_filename(filename: &str, is_vector: bool) -> Vec<Driver> { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why is I guess what I'm wondering here is if we should only be evaluating against There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't know. In principle, the user might have all sorts of extra drivers installed. Matching the GDAL logic seemed like a safe approach. We could ask Even, I guess. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The logic I went by is this: there can be multiple drivers for the same extension, and some could support raster some vector and some both. And this function is (related with other comments) made to know the driver in advance while creating a file as if we have a file already, then we can call It was also in the function Inicola sent me as an example, so it's my understanding of that, not my innovation. |
||
let ext = { | ||
let filename = filename.to_ascii_lowercase(); | ||
let e = match filename.rsplit_once('.') { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If we want to ensure true logical parity with GDAL C behavior, we'd use There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I did consider using the rust's "extension()" method on the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It would be nice to have this work for There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I'd like this because otherwise it's pretty annoying to convert a There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That's exactly I don't want to do it lol. Like for that: I can try, but unless we can use the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
That conflicts with the first sentence :D Users can just call Anyway, I'm trying to do it with
|
||
Some(("", _)) => "", // hidden file no ext | ||
Some((f, "zip")) => { | ||
// zip files could be zipped shp or gpkg | ||
if f.ends_with(".shp") { | ||
"shp.zip" | ||
} else if f.ends_with(".gpkg") { | ||
"gpkg.zip" | ||
} else { | ||
"zip" | ||
} | ||
} | ||
Some((_, e)) => e, // normal file with ext | ||
None => "", | ||
}; | ||
e.to_string() | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not needed, but will change anyway if we go with There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't get it. I did some changes here. But idk. I'm really reluctant to use path, as we could have URLs for PG and other backends. And rust path library is tested for local file paths only (at least from what I can see) URLs could complicate things in the future. |
||
}; | ||
|
||
let mut drivers: Vec<Driver> = Vec::new(); | ||
for i in 0..DriverManager::count() { | ||
Atreyagaurav marked this conversation as resolved.
Show resolved
Hide resolved
|
||
let d = DriverManager::get_driver(i).expect("Index for this loop should be valid"); | ||
let can_create = d.metadata_item("DCAP_CREATE", "").is_some() | ||
|| d.metadata_item("DCAP_CREATECOPY", "").is_some(); | ||
let check_vector = is_vector && d.metadata_item("DCAP_VECTOR", "").is_some(); | ||
let check_raster = !is_vector && d.metadata_item("DCAP_RASTER", "").is_some(); | ||
let check_vector_translate = | ||
is_vector && d.metadata_item("DCAP_VECTOR_TRANSLATE_FROM", "").is_some(); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. See comment above... i.e. should we consider selection and filtering two different operations? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. See this description of what GDAL does when From my 5 minute reading it looks like some of the logic is delegated to https://github.com/OSGeo/gdal/blob/master/gcore/gdaldriver.cpp#L2649C25-L2649C45 My concern here is not wanting to diverge from the accepted GDAL logic. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
The functions I linked to in #506 (comment) is used by the GDAL binaries to pick an output format. If we offer this function, we should match the latter, and use a better name ( There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We do need to change the function name. I was also looking for ideas as the ones I came up got too long. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Ahhh... I thought there was something already in the code, but didn't realize it was C++ only :/ There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yeah, it is for binary apps. Not the part of the |
||
if !((can_create && (check_vector || check_raster)) || (check_vector_translate)) { | ||
continue; | ||
} | ||
|
||
if let Some(e) = &d.metadata_item("DMD_EXTENSION", "") { | ||
if *e == ext { | ||
drivers.push(d); | ||
continue; | ||
} | ||
} | ||
if let Some(e) = d.metadata_item("DMD_EXTENSIONS", "") { | ||
if e.split(' ').collect::<Vec<&str>>().contains(&ext.as_str()) { | ||
drivers.push(d); | ||
continue; | ||
} | ||
} | ||
|
||
if let Some(pre) = d.metadata_item("DMD_CONNECTION_PREFIX", "") { | ||
if filename.starts_with(&pre) { | ||
drivers.push(d); | ||
} | ||
} | ||
} | ||
|
||
drivers | ||
} | ||
|
||
/// Register a driver for use. | ||
/// | ||
/// Wraps [`GDALRegisterDriver()`](https://gdal.org/api/raster_c_api.html#_CPPv418GDALRegisterDriver11GDALDriverH) | ||
|
@@ -455,6 +539,8 @@ impl DriverManager { | |
|
||
#[cfg(test)] | ||
mod tests { | ||
use std::collections::HashSet; | ||
|
||
use super::*; | ||
|
||
#[test] | ||
|
@@ -466,4 +552,46 @@ mod tests { | |
assert!(DriverManager::count() > 0); | ||
assert!(DriverManager::get_driver(0).is_ok()); | ||
} | ||
|
||
#[test] | ||
fn test_driver_extension() { | ||
// convert the driver into short_name for testing purposes | ||
let drivers = |filename, is_vector| { | ||
DriverManager::get_drivers_for_filename(filename, is_vector) | ||
.iter() | ||
.map(|d| d.short_name()) | ||
.collect::<HashSet<String>>() | ||
}; | ||
let gdal_version: i64 = crate::version::version_info("VERSION_NUM").parse().unwrap(); | ||
if DriverManager::get_driver_by_name("ESRI Shapefile").is_ok() { | ||
assert!(drivers("test.shp", true).contains("ESRI Shapefile")); | ||
// `shp.zip` only supported from gdal version 3.1 | ||
// https://gdal.org/drivers/vector/shapefile.html#compressed-files | ||
if gdal_version >= 3010000 { | ||
Atreyagaurav marked this conversation as resolved.
Show resolved
Hide resolved
|
||
assert!(drivers("test.shp.zip", true).contains("ESRI Shapefile")); | ||
} | ||
} | ||
if DriverManager::get_driver_by_name("GPKG").is_ok() { | ||
assert!(drivers("test.gpkg", true).contains("GPKG")); | ||
// `gpkg.zip` only supported from gdal version 3.7 | ||
// https://gdal.org/drivers/vector/gpkg.html#compressed-files | ||
if gdal_version >= 3070000 { | ||
Atreyagaurav marked this conversation as resolved.
Show resolved
Hide resolved
|
||
assert!(drivers("test.gpkg.zip", true).contains("GPKG")); | ||
} | ||
} | ||
if DriverManager::get_driver_by_name("GTiff").is_ok() { | ||
assert!(drivers("test.tiff", false).contains("GTiff")); | ||
} | ||
if DriverManager::get_driver_by_name("netCDF").is_ok() { | ||
assert!(drivers("test.nc", false).contains("netCDF")); | ||
} | ||
if DriverManager::get_driver_by_name("Elasticsearch").is_ok() { | ||
// Elasticsearch short name was ElasticSearch in older versions | ||
if gdal_version >= 3010000 { | ||
assert!(drivers("ES:test", true).contains("Elasticsearch")); | ||
} else { | ||
assert!(drivers("ES:test", true).contains("ElasticSearch")); | ||
} | ||
} | ||
} | ||
metasim marked this conversation as resolved.
Show resolved
Hide resolved
|
||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't forget to add the PR link here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you. Done.