Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sublocation and City fields seem swapped for many locations. #38

Closed
aphalo opened this issue Dec 18, 2022 · 47 comments
Closed

Sublocation and City fields seem swapped for many locations. #38

aphalo opened this issue Dec 18, 2022 · 47 comments
Assignees
Labels
bug Something isn't working

Comments

@aphalo
Copy link

aphalo commented Dec 18, 2022

As mentioned in another issue: Many thanks for developing GeoTagNinja!!

I only used GeoTagNinja on a few ORF files, but I can already report that it works smoothly together with Capture One and IMatch. These two programs displayed the GPS data corectly.

Hower, Sublocation and City seem to be swapped, with Sublocation showing the city name and City showing the neighbourhood name.

The swap happens for most places I checked: Finland, Sweden, Argentina, USA. I see the swap in New York, while at least some places in London seem o.k. (Tested by selecting different locations on the map, and copying the data to an image).

Please, let me know if I can help in some way.

@aphalo aphalo added the bug Something isn't working label Dec 18, 2022
@nemethviktor
Copy link
Owner

Hey - thanks ;) - glad you like it

Specific to the issue. It's annoyingly the API that sends back bad data. They don't have a "city" tag but just something like "adminname1" and "adminname2" which are inputted on a per-item basis by whichever random user that edited the API database - so it's entirely arbitrary if it's correct or not. I can add in a button to swap the two values if the user wants but likely that's as good as I can make it I'm afraid ..

@Clariden
Copy link

The problem is the way GTN uses the Geonames' findNearbyPlaceName API. It uses the toponymName as city name and the adminName2 as sublocation name (with the exception of London where adminName2 is the city name). This is wrong. The adminName2 is something between state and city level (it's variing between the countries) and therefore, it's never a sublocation name. The truth is, with findNearbyPlaceName, you'll find no sublocation names.

If you want to find sublocation names, you have to use the findNearby API. Make sure, the API delivers several locations (with maxRows=<any number > 1>). The toponymName of the first location is your sublocation name. Then you look out for a populated place (fcode=PPL): This is your city name. In some countries, you can find the city name by adminName too. For USA, e.g. it's adminName3, for Germany and France, it's adminName4.

Hope, this helps.

@nemethviktor
Copy link
Owner

@Clariden awesome thanks for that! I'll have a look next week and incorporate it

@Clariden
Copy link

Clariden commented Dec 18, 2022

Hi Viktor,

If you want to stick with findNearbyPlaceName, I'd suggest you to do the following:

In a country where you now which admin level the cities belong to (see below), use the adminNameX as city name. If the toponymName doesn't match the adminNameX, use the toponymName as sublocation name. toponymNames for populated places may be city names or names of some populated entity below city level, but they're never used for something above city level.

In a country where city names are not assigned to a specific admin level, I'd use the toponymName as the city name and leave the sublocation name blank.

Countries where the city names are adminName1:
LIE, SMR, MNE, MKD, MLT, SVN

Countries where the city names are adminName2:
ALA, BRA, COL, CUB, CYP, DNK, FRO, GTM, HND, HRV, ISL, LUX, LVA, NIC, NLD, NOR, PRI, PRT, ROU, SWE

Countries where the city names are adminName3:
AUT, CHE, CHL, CZE, EST, ESP, FIN, GRC, ITA, PAN, PER, POL, SRB, SVK, USA, ZAF

Countries where the city names are adminName4:
BEL, DEU, FRA, GUF, GLP, MTQ

@nemethviktor
Copy link
Owner

Hi @Clariden -- thanks for that one again. Good knowledge of the API there !:)
I'll try with the second one (i.e. sticking to the current logic and amending it with what you described per country) - people can test it out and if they don't quite like I can change it.

@nemethviktor
Copy link
Owner

One more question then if I may.
By default "State/Province" is acquired from adminName1, which, in case of the cities in the relevant array would lead to duplication. For those I may assume adminName2 should be used for State/Province?

@Clariden
Copy link

No. These countries where adminName1 is the city name (Liechtenstein, Macedonia, Malta, Montenegro, San Marino, Slovenia) are little countries. They don't have any states. They may have regions, but Geonames don't seem to have an adminName2 for them. So leave "State/Province" blank.

There's one little problem with adminNames. For some countries, you'll get "Town of Smalltown" or "City of Bigcity" as city names rather than "Smalltown" and "Bigcity". But at least you get a city name and not the name of a city district or an isolated dwelling as with toponymName.

@nemethviktor
Copy link
Owner

Awesome, thank you.
Right, I haven't checked in code yet (will wait for feedback) but can people test drive a dev build @aphalo and anyone else interested.

One comment, I added a line of code to ensure City is not blank (more precisely, if the rules/results would stipulate that Sublocation is not blank but City is blank then those would be swapped. Let me know if this is okay/logical to everyone else too? I'm open to changes....)

@aphalo
Copy link
Author

aphalo commented Dec 18, 2022

@nemethviktor I checked places I am familiar with in Argentina, Chile, Finland and Sweden with the dev build.

Finland seems now to be fine. With Sweden I am not so familiar, but also seems o.k.
In Chile outside of Santiago Metropolitan Area names seem o.k., but in the metropolitan area, the neighbourhoods show as cities.

The situation is similar for Argentina, with names rather o.k. outside the city of Buenos Aires. However, the situation is even more confusing because of the names themselves. The city of Buenos Aires is nowadays officially Ciudad Autónoma de Buenos Aires, at the same administrative level as a province, but a city. Surrounding the city is the Provincia de Buenos Aires, with other cities under its administration, but not the city of Buenos Aires. So instead of moving the sublocation to the city field, at least in these two cases, the correct action would be to copy the province field into the city field.

I haven't checked as it is getting late here. But other cities that are administratively like the two above are Washington DC and Distrito Federal de Méjico. Possibly also Brasilia the capital of Brasil.

In Argentina I also see some inconsistencies in the data near province borders. Like as if the matching is sometimes done to the nearest town or village even across province borders, and then the wrong name for the Province field results. Of course, in addition, the field City in sparsely populated areas gets filled with the name of the nearest village, which can be very small and rather far away. These later problem may be in the data rather than in your code, or just in the name used in the metadata standard for the field.

@nemethviktor
Copy link
Owner

nemethviktor commented Dec 19, 2022

@aphalo -- long post coming up: There is a relatively easy way to dissect the difference between the API and the code's doings -> check the API :)
In particular:

http://api.geonames.org/findNearbyPlaceNameJSON?formatted=true&lat=47.506124&lng=19.144025&style=FULL&username=YOURUSERNAME&password=YOURPASSWORD

replace the lat, long, username and password with the data that you require/use. We are generally looking for the various adminName (1-2-3-4 as per above) and ToponymName values.

I think there are inherent limitations both posed by the API and the generic user's needs. The coordinates above point to a heavily populated part of Budapest Hungary - I grew up there. Depending on the settings the return data could be either that the coordinates belong to "Budapest 16" [technically it will say Budapest XVI, as districts of Budapest use Roman numerals] or a place called Csomor, which is a very small town about 10km off.
While this may be "too much info" but I would like to show the issue here.

  • The coordinates as I said point to Budapest 16, which is incorrect because it should be 14, not 16 - basically I think the API thinks it's closer to 16, which is wrong anyway because it is in 14, and if honest yet another district, 10's borders are a lot closer to those coords than those of 16 - point is the API has its own logic and there's little anyone can do about it/them.
  • 14 has a common name of "Zuglo", this doesn't show reliably in the output. When I searched around, sometimes it came up Zuglo as a sublocation, sometimes Budapest 14, sometimes random stuff.
  • If I'm really nit-picky the particular area is commonly referred to as Rakosfalva. The problem here is that if Budapest is City then what is Zuglo (sub?) and what is Rakosfalva? (sub-sub? no such tag) - basically this is because of logic whereas one of these (Csomor) is a settlement on its own whereas Rakosfalva is a sub-settlement according to the API, and it's technically correct because Csomor is indeed a town in its own right (just literally miles off), whereas Zuglo is a district of Budapest and Rakosfalva is a subdivision of Zuglo.
  • In-between Budapest 14/Zuglo and Csomor there are 4 or 5 various subdistricts and gods only know what other classifications of sub-settlements such as the one described above. For someone that doesn't quite know the precise details to this extent it's easy to miss something is "wrong" - most tourists don't care though.

The API does have a functionality where we can filter results have the cities with at least a population of 1000, 5000, 15000.) This could be a way around certain things but there will always be the odd one user for whom it's important that they took a picture at Lower-Guglinsekeresd (fake town name, kinda means "don't-even-search-it-on-google") rather than the nearest reasonably sized settlement. Again an option could be having a setting button where the user can restrict search hits to the top x-thousand.

As for the comment re: settlements near the borders: I think, there is a setting in the API localCountry , which restricts the search to the country to which the coords belong -> however this is defaulted to true so no need to set manually.

Ultimately what GTN needs to do is to try to serve the majority of the use-cases in a way that can be coded logically. E.g. what's been attempted above where country X has the City value taken from adminNameX is a good way to go around it. Admittedly in the currently published code London is treated specially (as i live here and i know the place - but this "special treatment" will be removed in the upcoming code) but I'd be disinclined to start treating sub-country-levels specially because then people will ask to do the same for more and more places and I don't know which one of those are valid. Some of these may be better taken up with the geoNames maintainer.

Again there could be some logic coded that the user could introduce their own settings to say "if adminNameX contain valueX then have that as City etc" but I think this could become very complicated very quickly both to code on my side and to understand on the user side, if you just think about your Buenos A example(s) or my Budapest example(s). So possibly the cities=10000 optional filter could work better.

@aphalo
Copy link
Author

aphalo commented Dec 19, 2022

@nemethviktor I will check in the evening how the API works. One possible way out would be to keep the City field empty by default if the retrieved data shows it empty and have the user choose in the dialogue box whether to copy Province to City or move Sublocation to City. A more elaborate approach would be for GetTagNinja to save initialization settings and allow the user to save any Province that needs special handling. I agree with you that special handling of individual cases is not the type of thing one would like to hardcode in an application.

@nemethviktor
Copy link
Owner

A more elaborate approach would be for GTN to save initialization settings and allow the user to save any Province that needs special handling

You'll still run the risk of the data being "just different". If you refer to my comment about the whole thing in "Budapest 14" above that's kindof the precise problem. GeoNames relies on people inputting things arbitrarily. They don't (seem to) have quality control. So for the coords in the post above, sublocations of Budapest XIV, Budapest XIV kerület (lit: district), Budapest 14, Budapest 14 kerület, Budapest 14. kerület (note the dot), Budapest Zugló, Zugló and about a half dozen others would technically be valid outputs and I've seen them all. And I'm not even bringing in the umlauts problem here. Or the fact that the coords should belong to (whichever derivative of) Budapest 14, not Budapest 16 as the API pulls it. You've also mentioned Spanish language differences for Buenos A.

This generally harkens back to the part where I wrote "user could introduce their own settings to say "if adminNameX contain valueX then have that as City etc" but I think this could become very complicated very quickly both to code on my side and to understand on the user side" - I think that's ultimately what you're asking for.

@aphalo
Copy link
Author

aphalo commented Dec 19, 2022

@nemethviktor Hi what I meant was to allow the user to disable for specific provinces the code you mention in:

"One comment, I added a line of code to ensure City is not blank (more precisely, if the rules/results would stipulate that Sublocation is not blank but City is blank then those would be swapped. Let me know if this is okay/logical to everyone else too? I'm open to changes....)"

I would rather have the City field remain empty, or with a recognizable placeholder, when the data is missing so that I can write a command line script using ExifTool to do the edits I need. Filling a missing City field with a sublocation would make writing such a script very difficult, so I would like to be able to disable any code that fills empty fields by moving data from other fields. For my purpose a way of manually switching off the filling of City with a sublocation irrespective of the province or location would be enough, if it is a runtime option that I can select.

I'll take a break before checking the geonames.org site, as I just finished working on a manuscript that took since I woke up this morning. I wrote this meanwhile, as I think you interpreted my request as more complex than what I had asked, and in fact what I actually need is even simpler than what I earlier wrote...

@nemethviktor
Copy link
Owner

@aphalo No worries. I'm just poking the code to add in your other request atm (the "favourites") and then off for a social so no rush. :)

Re what you wrote about placeholders. I think we can strike a balance here. Remember for most people placeholders aren't really acceptable as they expect to see either blanks or real values.
With that in mind: I'm easy on reverting the code logic for the city-sublocation swap. In addition, I could add a simple Form where user can apply their own custom value to the empty-only fields of files if they want. Alternatively build in to the Settings as an opt-in option. I'm tempted to think the form is easier because I'd envisage this to be more ad-hoc but from my personal perspective it doesn't make much difference, can code either.

@Clariden
Copy link

@nemethviktor Maybe you could make an option to fill in the city name only if there's an administrative name for the city.

The Budapest problem is caused by the way how Geonames works. In the free edition, every toponym is just a point, not an area with a shape. Geonames looks for the toponym point closest to your coordinates. But populated places aren't points but areas. Finding the nearest point gives you no guarantee that your coordinates are within the area related to this point. So with Geonames you'll never get high precision results. If accuracy is important to you, you'll have to switch to another API, maybe to Nominatim or Google. But they're not 100% accurate either.

@nemethviktor
Copy link
Owner

Thanks for that (yet again ;)) - I think an optional setting to use the cities5000 (etc) might be an easier way around.

As for the alternative APIs I'm hoping to add support for more than just the current one eventually but 1) there hasn't been a request for them so far 2) I don't have access to paid alternatives and not quite keen to get one unless I personally need it (not likely) so unless someone wants it and is willing to temporarily provide me with an account, not likely just yet ;)

@aphalo
Copy link
Author

aphalo commented Dec 19, 2022

Hi, thanks again! I have been playing with geonames.org both using the link you provided and other entry points. I mostly used an R package called 'geonamer' as R is the language I am most familiar with. I need to let my ideas settle in place. So take what I will write below as blockbusting my ideas rather than a suggestion for a solution for the wrong sublocations problem during MANUAL geocoding, but of course not useful when reading geocodes from GPX files.

  1. What about letting the user set the radius used to fetch the nearby sublocations, and offering a list of a few (at most 5?-8? nearest) matching sublocations within the radius for the user to select from?

  2. Adding a third button bellow the map panel that instead of setting directly the data in the image file, opens the dialogue with the data ready to push to the images, but giving the user an opportunity to check and edit it in advance. (Simplest for the user to understand and easiest to implement, I would think).

  3. Combining 1) and 2).

@nemethviktor
Copy link
Owner

I remember geosetter had something like your first suggestion and although the API's maxRows defaults to something other than 1 I don't seem to get more than 1 row back, ever. Perhaps I am missing something obvious but if we did get more than 1 row back that could be presented to the user to choose from.

@Clariden
Copy link

@nemethviktor

I remember geosetter had something like your first suggestion and although the API's maxRows defaults to something other than 1 I don't seem to get more than 1 row back, ever.

You're misssing the radius. http://api.geonames.org/findNearbyPlaceNameJSON?formatted=true&lat=47.506124&lng=19.144025&maxRows=10&radius=10&style=FULL&username=YOURNAME&password=YOURPASSWORD

I think an optional setting to use the cities5000 (etc) might be an easier way around.

I wouldn't do that. It adds more complexity to the code without really solving the problem. In your Budapest example, even cities=cities15000 wouldn't change the result. In addition, this solution is a bit too focused on a specific use case (urban photography in big cities).

@nemethviktor
Copy link
Owner

@Clariden awesome, thanks :) -- how do you know all this stuff? I've kinda read (some) of the documentation on the API but most of this either doesn't come up or I totally missed it.
I'll work something on the multiple rows/radius logic.
I've found that if I default to a number that happens to be too small, there would be no data returned. This makes a certain amount of sense I guess but at the same time...what should then be the default logic? 10 miles? 50 miles? [or just default to 10 and use a Setting in the Settings for the user to do their own bidding I guess.]

Noted re: Cities

@nemethviktor
Copy link
Owner

nemethviktor commented Dec 20, 2022

Meh - I'm still having "London issues" so to say.
With the "original" query -> http://api.geonames.org/findNearbyPlaceNameJSON?formatted=true&lat=51.48925&lng=0.086837&style=FULL&username=XXX

This is Plumsted but for reality's sake London. In this case that would come from adminName2 (so I'm assuming it's an exception to the rules above as the UK isn't an adminName2-country) --> "adminName2": "Greater London"

If I use the extended logic from the other entry point:

http://api.geonames.org/findNearbyJSON?formatted=true&lat=51.48925&lng=0.086837&fcode=PPL&style=FULL&radius=20&username=XXX

The first instance of "populated place" ultimately returns the same logic (where I would either take "toponymName": "Plumstead" or "adminName2": "Greater London"

Suggestions pls? :)

@Clariden
Copy link

In the UK, it's quite complicated with Geonames. For big cities, I think, adminName2 usually will do it. But for towns and villages this won't work. Take Bovingdon (http://api.geonames.org/findNearbyPlaceNameJSON?formatted=true&lat=51.72249669136385&lng=-0.5376024482441658&style=FULL&username=USERNAME) a randomly chosen example.
adminName2 is "Hertfordshire", the county, adminName3 is "Dacorum District" and Bovingdon itself is adminName4. So if you want to get it right, I'm afraid you have to build a huge database with all the specifics of UK and other countries where you can't assign city names to a particular admin level. But that's not feasible. So I think if you want to stick with Geonames API, you have to live with those inaccuracies :/ But then again, I'm no expert at all.
An alternative would be to use the Nominatim API by OpenStreetMap. Nominatim is free but it isn't 100% accurate either and it's quite slow (~1 request per second).

@nemethviktor
Copy link
Owner

Thanks - I might need to introduce some user-level controls after all.

The 1 request per second can be a bit slow. It's not unreasonable to suspect that people could have hundreds of different locations in a folder/session so waiting literal minutes might push their patience a bit ;)

@Clariden
Copy link

@nemethviktor My proposal for finding city names with Geonames is this:

  • Check the country code. If it belongs to a country where we know the admin level of the cities, use the approprate adminName as city name and the toponymName of the nearest populated place as sublocation name.
  • For all other countries, use the toponymName of the nearest populated place as city name by default. But make an option "Use administrative names as city names only" (or something like that). If this option is checked, use the toponymName of the nearest populated place as sublocation name and leave the city name blank.

@nemethviktor
Copy link
Owner

Thanks for that - I'm tempted to think that for the "other" countries there is a capability limitation on how things work. Sticking to the London/Plumstead example if I restrict data to PPL only (http://api.geonames.org/findNearbyPlaceNameJSON?formatted=true&lat=51.48925&lng=0.086837&fcode=PPL&style=FULL&username=XXX) then toponymName still returns Plumstead , and any version of London only comes up in adminName2, which is basically custom/adhoc/arbitrary.

I'll work on some simple-ish logic to allow user to customise some of the findings on their own.

@nemethviktor
Copy link
Owner

nemethviktor commented Dec 20, 2022

Right, can people please have a play with https://drive.google.com/file/d/18iI77SIdrIv-joOtyT0-MzqOVtB5OgM0/view?usp=share_link

Things that are new in here:

  • Settings has option for returning X rows in Y miles radius.
  • Settings has option for replacing blanks with whatever the user wants
  • Not new but just reiterate this is using @Clariden 's logic re: adminName1/2/3/4 countries.
  • I still have the "Greater London" hack in. Eventually it will be removed and replaced w/ something more user-driven but I would like to do a general release before the end of Thursday as there hasn't been one for 3 wks and I'm off on Fri for 2 wks.

cc @aphalo

@aphalo
Copy link
Author

aphalo commented Dec 20, 2022

@nemethviktor

Hi, thanks!

Settings has option for returning X rows in Y miles radius. o.k. The returned list is great! Helps a lot!

Settings has option for replacing blanks with whatever the user wants **not very useful now, because we still have City filled with a neighbourhood, and sublocation is empty.** Is the "fill empty City with sublocation" code still in use? (see next point)

Not new but just reiterate this is using @Clariden 's logic re: adminName1/2/3/4 countries. does not work well for Buenos Aires F. D. (Argentina) or Santiago Metropolitan (Chile). It is dificult to know if the problem is in geonames.org or in the logic in your code. I need to repeat the same searches directly through the API.

I still have the "Greater London" hack in. Eventually it will be removed and replaced w/ something more user-driven but I would like to do a general release before the end of Thursday as there hasn't been one for 3 wks and I'm off on Fri for 2 wks.

If no nearby location is found, the coordinates are silently assigned to the image leaving City, etc. empty. A warning message or the list dialog with a single entry, or even the possibility of expanding the radius would be more user friendly.

In Patagonia I had to use in places a radius of 70 miles (a way of using km would be a nice addition) to get a nearby search hit. This same setting used in more densily populated places, or maybe repeating the same or a closeby place, seems to make searches intermitently fail silently (maybe be the geonames server throtles down). If it is a timeout, a message to the user would help, if a very slow response, maybe some indication that the search is running. I need to play more with the API. I will give you feedback on what might be possible with the API once you are back. I am also interested in place names that are not populated, say mountains, lakes, forests, etc. These would be useful especially in sparsely populated areas. I will explore in the API how to search for them.

@Clariden
Copy link

does not work well for Buenos Aires F. D. (Argentina) or Santiago Metropolitan (Chile).

Geonames doesn't have an admin level for Argentine cities. So it's hard to find the city names programmatically.
In Chile, all communes are on third adminstrative level. So it's easy to get the city name. But Geonames doesn't always find a place which is in the commune you're looking for. An example is San Ramón, Greater Santiago (lat/lon: -33.533333, -70.641667). The nearest place Geonames finds is "Lo Ovalle", 1.93664 km from the point we are looking for. Lo Ovalle belongs to the city La Cisterna (adminName3). This is why you get the wrong city name.

a way of using km would be a nice addition

In Geonames, afaik, the radius is always in kilometers, not in miles.

@nemethviktor
Copy link
Owner

@aphalo - for the sake of simplicity going forward may I ask that you share your adhoc settings (that is lat/long and if relevant radius)

In some order of appearance...

radius in kms

I've checked and apparently it is in KM - my bad.

Is the "fill empty City with sublocation" code still in use?

Not any more.

does not work well for Buenos Aires F. D.

I think Clariden just replied to this one so see their answer pls - however on my side: this is where you need to provide details (lat/long) so I can test. With that in mind and also the above, I did a search for B.A.
image

http://api.geonames.org/findNearbyPlaceNameJSON?formatted=true&lat=-34.603333&lng=-58.381667&style=FULL&username=XXX

As you can see what you'd be after is in adminName1 but because ARG isn't a country where City comes from adminName1 it then gets duly ignored (bcs for the non-classified countries City is now ToponymName.
This is a bit like how Greater London sits in adminName2, even though it shouldn't. I probably won't have time to code the user-driven change-logic before the end of the week but the only way I see around this is that the user themself can make the rule to say if adminName1.Contains("Buenos Aires F.D.") {City = adminName1} (well, in a bit less code-y way but that'd be the outcome)

If no nearby location is found, the coordinates are silently assigned to the image leaving City, etc. empty. A warning message or the list dialog with a single entry, or even the possibility of expanding the radius would be more user friendly.
++ Patagonia

image

possibility of expanding the radius would be more user friendly -> that's doable I guess. I've implemented a logic that if the original query returns 0 rows then extend to 300km

See updated dev version here

searches intermittently fail silent

Can I have a lat/long/radius/maxRows please here so I can debug.

@aphalo
Copy link
Author

aphalo commented Dec 20, 2022

@nemethviktor Hi. Thanks! In your search for Buenos Aires F. D., the city in my view should be Beunos Aires, and what is in city would make more sense as sublocation. However, I should accept that the admin level below Buenos Aires F. D. may well be what is in the field City. So, the logic is indeed tricky. So, I do not think you should spend time in trying to get this right automatically. Yes, letting users add a rule should work nicely. Really the problem is the name of the field in the EXIF rules, most of the Earth's area is outside cities. It took me some time to get familiar with the geonames API, but in fact using admin levels makes much better sense than the fields EXIF uses, so I am now happy to accept something else than a city name in the field City. Up to state/province the mapping is doable cleanly. Sorry that I did not realise this before.

Probably 100 km woud be enough.

Got it! What I was doing is certainly unusual. Sending to coordinates to the image file twice without changing the position of the marker on the map. In this situation the list to choose from is not displayed (the data could be cached and redisplayed). The only reason to do this is if one selects the wrong line and wants to redisplay the list to choose a different line. This happens irrespective of the lat and lng. I see this happening with radius 1 and radius 100, with 8 or 5 shown toponym options.

This took some time because of uninstalling and installing the new version.

Now I should go to sleep...

@nemethviktor
Copy link
Owner

Just on the note of using the same coordinates twice -> that's a feature not a bug. It's entirely possible that user has more than 1 pictures in a folder that, for whatever reason have identical coordinates. In this case there is no point in querying the API multiple times for something we ultimately know. Alas on every instance the API is queried the return data is saved in the memory for the length of the session.
The reason why the location picker doesn't come up is the same. The if the API returns precisely 1 row, the data with the coordinate pair is autosaved. If it returns more than 1 then the user is offered to pick and the choice is saved. Regardless, next time (within a session) the same coordinates are encountered the values are read from the memory instead.

@aphalo
Copy link
Author

aphalo commented Dec 21, 2022

o.k., and you store in memory only the location chosen from the list the first time, rather than the whole list... so if the user happens to choose the wrong row the first time, the user needs to close the session and start a new session to correct the mistake... the approach makes sense except in this exceptional case. Providing a way of forcing a new search when needed would remove this limitation.

@aphalo
Copy link
Author

aphalo commented Dec 21, 2022

Copied from issue #37 as it belongs here.

Thanks! I quickly skimmed the IPTC standard, the IPTC Extenssion seems now preferred for location data. Reading between lines, it seems that sublocation is that level 5 when below a city but possibly at level 4 when outside a city. I guess, then that the correct behaviour is to leave the City field empty.

https://iptc.org/std/photometadata/specification/IPTC-PhotoMetadata#location-structure
https://iptc.org/std/photometadata/specification/IPTC-PhotoMetadata#location-created
https://iptc.org/std/photometadata/specification/IPTC-PhotoMetadata#location-structure

The most informative is the third of the links, but I got there navigating the other two.

@Clariden
Copy link

@aphalo I think, this is maybe a misunderstanding. When the standard says that sublocation is level 4, it's refering to its own hierarchy (level 1 = photoshop:Country, level 2 = photoshop:State, level 3 = photoshop:City, level 4 = Iptc4xmpCore:Location). This has nothing to do with administrative levels in a country. In fact, sublocations can also be parks, churches, monuments and whatever without any administrative relevance. (BTW, in IPTC Extensions, sublocation is level 5, because there's also a property for the world region.)

@nemethviktor
Copy link
Owner

0f986f9
Copied to gdrive too. I'll keep this one open for now for discussion's sake.
@aphalo there's now a "right click" (use it on selected files) and it can clear the cache for those files.

@aphalo
Copy link
Author

aphalo commented Dec 21, 2022

@nemethviktor Many thanks! It works great! As far as I remember Geotagger allowed renaming and deleting favourites. Not important now but will be needed in the long run.

@nemethviktor
Copy link
Owner

I'll add it in at some point in January when I'm back

@aphalo
Copy link
Author

aphalo commented Dec 21, 2022

@Clariden @nemethviktor What I tried to justify above about IPTC is that code that copies Sublocation into an empty City field should not be activated by default or at least there should be a way for the user to deactivate it. So, I think we agree, as you say, sublocations can be names for many different types of features, that I would not like to be silently moved to the City field in the images. The point I am trying to make is that if in the retrieved data the administrative level corresponinding to City is empty, the City field must be set to an empty value in the image rather than try to fill it. Maybe I am misinterpreting what I see happening in some regions and the problem is elsewhere. During the next couple of weeks I will run searches directly in geonames.org in parallel with GeoTagNinja to work out what is going on.

@Clariden
Copy link

sublocations can be names for many different types of features, that I would not like to be silently moved to the City field in the images.

The way GTN currently works, it queries only populated places (cities, towns, villages, or other agglomerations of buildings where people live and work, according to Geonames). So it can't happen that your city name will be a monument or a church. In rural areas, the name of the nearest populated place is usually the name of the city or town. But in urban areas, there are many populated places. So the name of the nearest populated place is probably not the city name. For this reason, I suggested an option "Use administrative names as city names only" (which is disabled by default). If you're geotagging photos from a major city for which Geotag doesn't have an adminNameX, you can enable this option and GTN will leave the city name blank. If your photos are from rural areas, you can leave this option disabled an GTN will use the name of the nearest populated place as city name which is usually fine.

@nemethviktor But I fully respect your decision not to impelement such an option, especially since too many options can confuse the user. Btw many thanks for mentioning me as a contributor!

@nemethviktor
Copy link
Owner

Welcome :) - re contributor mention.

Specific to the 'use administrative names as city names only' -- i may have missed this....where/how do I do that?

@Clariden
Copy link

@nemethviktor I thought of a CheckBox in the Settings. Read it to a bool variable, something like useAdminCityNamesOnly. And then in line 1989 of HelperStatic.Exif.cs: else if (!useAdminCityNamesOnly). Something like that?

@aphalo
Copy link
Author

aphalo commented Dec 21, 2022

Great! Thanks!

@nemethviktor
Copy link
Owner

@nemethviktor I thought of a CheckBox in the Settings. Read it to a bool variable, something like useAdminCityNamesOnly. And then in line 1989 of HelperStatic.Exif.cs: else if (!useAdminCityNamesOnly). Something like that?

Thanks, yes -- but I think my question wasn't clear. What I was asking is more along the lines of what logic would that be (?)

@Clariden
Copy link

@nemethviktor To be honest, I'm not quite sure if I understand your question. Maybe it's because one half of my brain is already on Christmas vacation 😉. But the idea was to make an option which avoids the guesswork when it comes to the city name: If you don't have an adminNameX for the city or don't know which administrative level you have to look at, leave the city name blank and use the toponymName as sublocation name. The term 'sublocation' (in IPTC Core, btw, it's just 'location') is somewhat less strictly defined than the term 'city'. Therefore, it may be more okay to fill the sublocation field with a city name than the other way round...?

@nemethviktor
Copy link
Owner

Haha let's come back to it in January. I'm off from tomorrow anyway till the 1st.
Enjoy holidays and thanks for all the help and feedback everyone.

@nemethviktor
Copy link
Owner

nemethviktor commented Jan 6, 2023

Right there is a new version on the dev link (usual link) it has Custom Rules.
I have only superficially tested it because it's a bit complex and I'd admittedly generally advise against using it as I expect it'd have some performance ramifications and the user needs to understand how the API outputs stuff.
For the "Greater London" issue previously this would be the solution:

image

Also ps.: I'm happy to try to make it more user-friendly but I don't want to make it more complex (such as include AND/OR trees.) -- the relevant bit of code is already a little lengthy.

@nemethviktor
Copy link
Owner

Closing as I think this is fixed. Reopen if needed pls.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants