Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is there anyway to only import citynames/postalcodes for united states? #74

Closed
donnaloia opened this issue Feb 12, 2015 · 25 comments
Closed

Comments

@donnaloia
Copy link

I ask this because I get a memory error during import. Do 'import all' import every city from every country? I was hoping a workaround would be to only import united states information.

The other work arounds to the memory issue seem very complicated.

Please advise.

@coderholic
Copy link
Owner

Yes import all imports data from all countries. There's no a command line option to import data for a single country, but one thing that should work is to import all country first (with import countries), and then delete all countries but the US. If you then import regions and then cities it'll only load those from the US.

@donnaloia
Copy link
Author

Okay thanks. To delete all countries except the US, can I do something like Countries.objects.all().exclude(name='United States").delete() ?

@donnaloia
Copy link
Author

I wanted to double check first, because I'm scared to do the wrong thing and delete everything.

@coderholic
Copy link
Owner

Yes Countries.objects.all().exclude(name='United States").delete(), or the SQL query DELETE FROM cities_country WHERE code != 'US'

After you've run that, and then imported the other data you can see how many cities you have for each country (which should only show data for the US) with the following query:

SELECT country_id, count(1) FROM cities_city GROUP BY country_id

@donnaloia
Copy link
Author

okay thank you so much. I imported regions and cities like you told me. Do I also need to run a seperate import for Postal codes?

@coderholic
Copy link
Owner

Yeah.

@donnaloia
Copy link
Author

Last question, I did 'manage.py sqlclear cities' and then 'manage.py syncdb' to clear the table and start over again. Next I did '/manage.py cities --import=Country' just like before but it just says INFO-file up to date.

For some reason Django thinks this information is already populated in the database even though I dropped the cities table, deleting everything. Any advice on how to proceed?

@coderholic
Copy link
Owner

You need to do --force. I think it's up to date because it sees the
downloaded files.

On 17 February 2015 at 14:09, Benjamin Donnaloia notifications@github.com
wrote:

Last question, I did 'manage.py sqlclear cities' and then 'manage.py
syncdb' to clear the table and start over again. Next I did '/manage.py
cities --import=Country' just like before but it just says INFO-file up to
date.

For some reason Django thinks this information is already populated in the
database even though I dropped the cities table, deleting everything. Any
advice on how to proceed?


Reply to this email directly or view it on GitHub
#74 (comment)
.

@donnaloia
Copy link
Author

I can now see countries has been populated in the database, but when i query PostalCode.objects.all() it returns an empty list... I have used the --force --import=PostalCode and can see the information loading in the console window, it takes about 10-15 minutes, but after, PostalCode.objects.all() still returns an empty list and all of my PostalCode queries that previously worked, now do not work because there is no PostalCode data. Do you have any suggestions? I already did sqlclear cities and tried to rebuild the tables and repopulate database by doing sqlclear and then syncdb.

@coderholic
Copy link
Owner

Did you import regions/cities/districts first?

@donnaloia
Copy link
Author

No, what I have previously done is import country, import city and import postalcode (in this order), and I had no problem. I did the same order again after I did a 'sqlclear cities', and now there is some issues. should I do import in a different order than this?

@coderholic
Copy link
Owner

That order should be fine. Can you confirm that the city import worked, and you now have cities?

@donnaloia
Copy link
Author

Yes, I can confirm city import worked.

@coderholic
Copy link
Owner

And you have a CITIES_POSTAL_CODES setting?

@donnaloia
Copy link
Author

Yes, I do.

@donnaloia
Copy link
Author

It's very strange because when I import PostalCode I can see the activity in the console window, it appears to be loading the information, but still there is nothing, I'm so confused.

@coderholic
Copy link
Owner

What do you see when you do a select count(1) from cities_postalcodes from
dbshell?

On 17 February 2015 at 15:47, Benjamin Donnaloia notifications@github.com
wrote:

It's very strange because when I import PostalCode I can see the activity
in the console window, it appears to be loading the information, but still
there is nothing, I'm so confused.


Reply to this email directly or view it on GitHub
#74 (comment)
.

@donnaloia
Copy link
Author

Nothing happens, it just returns to the next line. This is my first time using dbshell, so maybe I am doing something incorrect. I typed 'select count(1) from cities_postalcodes' and hit return.

@coderholic
Copy link
Owner

you need to add a semicolon on the end, eg:

# select count(1) from cities_postalcode;
count
-------
91307
(1 row)

@donnaloia
Copy link
Author

count


0
(1 row)

@coderholic
Copy link
Owner

Can you do the same for cities and countries?

On 17 February 2015 at 17:14, Benjamin Donnaloia notifications@github.com
wrote:

count

0
(1 row)


Reply to this email directly or view it on GitHub
#74 (comment)
.

@donnaloia
Copy link
Author

count 250 for country. count 140926 for cities.

@donnaloia
Copy link
Author

Should I try 'sqlclear cities' again?

@coderholic
Copy link
Owner

Sure you could give it a try. Might also be worth trying with a completely new DB.

@donnaloia
Copy link
Author

Well, after doing sqlclear and then force import (for the fourth time), I can now see PostalCodes, I'm not sure what the problem was before, but I don't even want to think about it anymore.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants