New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

changing opendatastorage location #19

Open
mpuchowicz opened this Issue Jan 18, 2019 · 4 comments

Comments

Projects
None yet
2 participants
@mpuchowicz
Copy link

mpuchowicz commented Jan 18, 2019

Local storage works fine when using the default location. However when changing location the directory is not created and files are no longer saved locally (either in the desired or default location) nor are any error messages reported. To change location I saved a file opendata.ini with the content:

[Storage]
local_storage_path = P:\mpJupyter

I open python from the Anaconda prompt with:
P:\mpJupyter>jupyter notebook

If I change the path specified in the .ini file to a location that doesn't exit I get an error as might be expected.

Thanks for your your help and work on this library.

mike

@AartGoossens AartGoossens self-assigned this Jan 19, 2019

@AartGoossens

This comment has been minimized.

Copy link
Contributor

AartGoossens commented Jan 21, 2019

For me it works like it should (using pipenv and running pipenv run ipython from the directory where I saved the opendata.ini file). That is not to say that there is no bug, but at least the code works for my setup.

I am not familiar with Anaconda but a quick Google search already returned some seemingly similar issues (e.g. this one). I will try to install Anaconda later and see if I can replicate the issue.
To double check (and for people that have the same issue and come across this issue in the future): Is the opendata.ini file saved in P:\mpJupyter? The opendata.ini files needs to be saved in the same directory as from where Python is running.

Questions for further debugging:

  • What does import os; print(os.getcwd()) return?
  • What does from opendata.conf import settings; print(settings.local_storage) return?

As a workaround you can set settings.local_storage to the desired location after import. Moreover, I think this is a equally valid method for overriding the storage path. I will add it to the documentation (i.e. README.md).

@mpuchowicz

This comment has been minimized.

Copy link
Author

mpuchowicz commented Jan 22, 2019

To confirm, yes
opendata.ini file was saved in P:\mpJupyter

running:
import os; print(os.getcwd())
returns:
P:\mpJupyter

running:
from opendata.conf import settings; print(settings.local_storage)
returns:
C:\Users\mpuchowi\opendatastorage

I am able to get the workaround running:
settings.local_storage = 'P:\mpJupyter\opendatastorage'

and now running:
print(settings.local_storage)
returns:
P:\mpJupyter\opendatastorage

saving an athlete locally now creates the opendatastorage folder in the desired location and metadata and data are stored as expected.

@AartGoossens

This comment has been minimized.

Copy link
Contributor

AartGoossens commented Jan 22, 2019

This is getting interesting: I just installed Anaconda and it works as expected for me (linux).

I am starting to suspect that the things go wrong on this line.

What does import os; os.path.isdir('P:\mpJupyter') return?

@mpuchowicz

This comment has been minimized.

Copy link
Author

mpuchowicz commented Jan 23, 2019

I figured out the issue on my end. When I originally created the opendata.ini I saved it with an incorrect name which I realized right away. However I renamed the file the .ini extension was hidden so I manually typed the .ini extension so that the file actually saved as opendata.ini.ini but appeared as opendata.ini in my folder. So with that error when the jupyter notebook is open in generates the following;

running:
import os; print(os.getcwd())

returns:
P:\mpJupyter

and

running:
import os; os.path.isdir('P:\mpJupyter')

returns:
True

but

running:
from opendata.conf import settings; print(settings.local_storage)

returns:
C:\Users\mpuchowi\opendatastorage

The output is interesting in that it appears that one section of the code was able to parse and fix my error with the ini.ini but another part wasn't able to and reverted to the default location for the opedatastorage folder without generating an error?

Once I fixed the .ini extension issue the only remaining thing is that an opendatastorage folder is not created, instead the data and metadata folders are saved in my mpJupyter folder.

I can fix this by changing the opendata.ini content to;

[Storage]

local_storage_path = P:\mpJupyter\opendatastorage

and adding a folder named opendatastorage in the mpJupyter folder.

For consistency with the default creation of the opendatastorage folder the instructions for overwriting the default could specify that the content of the the opendata.ini file should be

[Storage]
local_storage_path = \opendatastorage

and also create a folder named opendatastorage in the location of the working directory. If the folder is not created prior to importing OpenData an error is generated that the path does not exist.

Hopefully my mistake can help save someone a headache in the future.

mp

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment