Skip to content

Commit

Permalink
Rename my_configuration to mycfg for brevity
Browse files Browse the repository at this point in the history
  • Loading branch information
karlicoss committed Dec 19, 2019
1 parent cd80409 commit 066641a
Show file tree
Hide file tree
Showing 31 changed files with 65 additions and 70 deletions.
2 changes: 1 addition & 1 deletion .circleci/config.yml
Expand Up @@ -15,7 +15,7 @@ jobs:
# - run: python3 -m mypy --namespace-packages my
# - run: python3 -m pylint -E my

# TODO need to keep full my_configuration.py as example? to dummy run CI
# TODO need to keep full mycfg.py as example? to dummy run CI



Expand Down
26 changes: 13 additions & 13 deletions README.org
Expand Up @@ -25,17 +25,17 @@ Short example to give you an idea: "which subreddits I find most interesting?"
| QuantifiedSelf | 28 |

* Setting up
** =my_configuration= package for private paths/repositories (optional)
** =mycfg= package for private paths/repositories (optional)
If you're not planning to use private configuration (some modules don't need it) you can skip straight to the next step. Still, I'd recommend you to read anyway.

First you need to tell the package where to look for your data and external repositories, which is done though a separate (private) package named ~my_configuration~.
First you need to tell the package where to look for your data and external repositories, which is done though a separate (private) package named ~mycfg~.

You can see example in ~my_configuration_template~. You can copy it somewhere else and modify to your needs.
You can see example in ~mycfg_template~. You can copy it somewhere else and modify to your needs.

Some explanations:

#+begin_src bash :exports results :results output
for x in $(find my_configuration_template/ | grep -v -E 'mypy_cache|.git|__pycache__'); do
for x in $(find mycfg_template/ | grep -v -E 'mypy_cache|.git|__pycache__'); do
if [[ -L "$x" ]]; then
echo "l $x -> $(readlink $x)"
elif [[ -d "$x" ]]; then
Expand All @@ -49,9 +49,9 @@ Some explanations:

#+RESULTS:
#+begin_example
d my_configuration_template/
d my_configuration_template/my_configuration
f my_configuration_template/my_configuration/__init__.py
d mycfg_template/
d mycfg_template/mycfg
f mycfg_template/mycfg/__init__.py
---
class paths:
"""
Expand All @@ -60,23 +60,23 @@ f my_configuration_template/my_configuration/__init__.py
class hypexport:
export_dir = '/tmp/my_demo/backups/hypothesis'
---
d my_configuration_template/my_configuration/repos
l my_configuration_template/my_configuration/repos/hypexport -> /tmp/my_demo/hypothesis_repo
d mycfg_template/mycfg/repos
l mycfg_template/mycfg/repos/hypexport -> /tmp/my_demo/hypothesis_repo
#+end_example

As you can see, generally you specify fixed paths (e.g. to backup directory) in ~__init__.py~.
Feel free to add other files as well though to organize better, it's a real python package after all!

Some things (e.g. links to external packages like [[https://github.com/karlicoss/hypexport][hypexport]]) are specified as normal symlinks in ~repos~ directory.
That way you get easy imports (e.g. =import my_configuration.repos.hypexport.model=) and proper IDE integration.
That way you get easy imports (e.g. =import mycfg.repos.hypexport.model=) and proper IDE integration.

# TODO link to post about exports?
** =with_my= helper script
Next, point =with_my= script to your private configuration:

#+begin_src bash
cp with_my.example with_my
vim with_my # specify path to your my_configuration (if you want to use it)
vim with_my # specify path to your mycfg (if you want to use it)
#+end_src

It's also convenient to put =with_my= somewhere in your system path so you can run it from anywhere.
Expand Down Expand Up @@ -113,8 +113,8 @@ or, set up as ~mypy.ini~ file:

#+begin_src
[mypy]
mypy_path=/path/to/my_configuration_dir
mypy_path=/path/to/mycfg_dir
#+end_src


# TODO hmm, if package isn't using my_configuration then we don't really need it?
# TODO hmm, if package isn't using mycfg then we don't really need it?
4 changes: 2 additions & 2 deletions demo.py
Expand Up @@ -35,9 +35,9 @@ def run():
with_my = 'my_repo/with_my'
copy('my_repo/with_my.example', with_my)

my_configuration_root = abspath('my_repo/my_configuration_template')
mycfg_root = abspath('my_repo/mycfg_template')
# edit the config and set path to private configuration
my = Path(with_my).read_text().replace("MY_CONFIGURATION_DIR = ''", "MY_CONFIGURATION_DIR = '{}'".format(my_configuration_root))
my = Path(with_my).read_text().replace("MYCFG_DIR = ''", "MYCFG_DIR = '{}'".format(mycfg_root))
Path(with_my).write_text(my)
#

Expand Down
10 changes: 5 additions & 5 deletions misc.org
Expand Up @@ -13,23 +13,23 @@ Various thoughts on organizing
- not mypy/pylint friendly at all?

- Second alternative:
symlinks in my_configuration and direct import?
symlinks in mycfg and direct import?

+ mypy/pylint friendly
? keeping a symlink to model.py is not much worse than hardcoding path. so it's ok I guess

* Thoughts on organizing imports
- First way:
import my_configuration.hypexport_model as hypexport
works, but my_configuration is scattered across the repository?
import mycfg.hypexport_model as hypexport
works, but mycfg is scattered across the repository?

Second way:
from . import my_configuration?
from . import mycfg?
doesn't seem to work with subpackages?
right, perhaps symlinking is a good idea after all?...

Third way:
import my_configuration.repos.hypexport.model as hypexport
import mycfg.repos.hypexport.model as hypexport
works, but MYPYPATH doesn't seem to be happy...
ok, --namespace-packages solves it..

10 changes: 2 additions & 8 deletions my/__init__.py
@@ -1,13 +1,7 @@
# TODO just eval setup file to populate paths etc?
# TODO note sure if it would

# TODO maybe just import everything?
# TODO how to make it mypy friendly? maybe defensive import? or mypy config? or interface file?

try:
import my_configuration
paths = my_configuration.paths # type: ignore
import mycfg
except ImportError:
import warnings
warnings.warn("my_configuration package isn't found! That might result in issues")
paths = None # type: ignore
warnings.warn("mycfg package isn't found! That might result in issues")
2 changes: 1 addition & 1 deletion my/bluemaestro/__init__.py
Expand Up @@ -14,7 +14,7 @@
# TODO vendorize in my. pkg? It's quite handy...
from kython.klogging import LazyLogger

from my_configuration import paths
from mycfg import paths

logger = LazyLogger('bluemaestro', level=logging.DEBUG)

Expand Down
2 changes: 1 addition & 1 deletion my/body/blood.py
Expand Up @@ -8,7 +8,7 @@
from kython.org import parse_org_date
from kython.kerror import Res, echain

from my_configuration import paths
from mycfg import paths

import pandas as pd # type: ignore

Expand Down
4 changes: 2 additions & 2 deletions my/books/kobo.py
@@ -1,7 +1,7 @@
from typing import Callable, Union, List

from my_configuration import paths
from my_configuration.repos.kobuddy.src.kobuddy import *
from mycfg import paths
from mycfg.repos.kobuddy.src.kobuddy import *

set_databases(paths.kobuddy.export_dir)

Expand Down
2 changes: 1 addition & 1 deletion my/calendar/holidays.py
Expand Up @@ -4,7 +4,7 @@
from typing import Tuple, Iterator, List, Union


from my_configuration.holidays_data import HOLIDAYS_DATA
from mycfg.holidays_data import HOLIDAYS_DATA


# pip3 install workalendar
Expand Down
4 changes: 2 additions & 2 deletions my/coding/github.py
Expand Up @@ -8,8 +8,8 @@

from ..common import get_files

from my_configuration import paths
import my_configuration.repos.ghexport.model as ghexport
from mycfg import paths
import mycfg.repos.ghexport.model as ghexport


def get_logger():
Expand Down
4 changes: 2 additions & 2 deletions my/fbmessenger.py
Expand Up @@ -3,8 +3,8 @@
from tempfile import TemporaryDirectory
from typing import Iterator, Union

import my_configuration.repos.fbmessengerexport.model as messenger
from my_configuration import paths
import mycfg.repos.fbmessengerexport.model as messenger
from mycfg import paths


def _dump_helper(model: messenger.Model, tdir: Path) -> None:
Expand Down
3 changes: 2 additions & 1 deletion my/feedbin.py
@@ -1,7 +1,8 @@
from . import paths
from .common import listify
from ._rss import Subscription

from mycfg import paths

import json
from pathlib import Path
from typing import Dict, List
Expand Down
3 changes: 2 additions & 1 deletion my/feedly.py
@@ -1,7 +1,8 @@
from . import paths
from .common import listify
from ._rss import Subscription

from mycfg import paths

import json
from pathlib import Path
from typing import Dict, List
Expand Down
2 changes: 1 addition & 1 deletion my/foursquare.py
Expand Up @@ -22,7 +22,7 @@ def get_logger():


def _get_exports() -> List[Path]:
from my_configuration import paths
from mycfg import paths
return get_files(paths.foursquare.export_path, '*.json')


Expand Down
10 changes: 5 additions & 5 deletions my/hypothesis.py
Expand Up @@ -6,9 +6,9 @@


try:
# TODO might be worth having a special mode for type checking with and without my_configuration?
# TODO might be worth having a special mode for type checking with and without mycfg?
# TODO could somehow use typing.TYPE_CHECKING for that?
import my_configuration.repos.hypexport.model as hypexport
import mycfg.repos.hypexport.model as hypexport
Highlight = hypexport.Highlight
Model = hypexport.Model
except:
Expand All @@ -26,7 +26,7 @@ def export_path(self) -> Path:
if ep is not None:
return Path(ep)
else:
from my_configuration import paths
from mycfg import paths
return Path(paths.hypothesis.export_path)

@property
Expand All @@ -37,7 +37,7 @@ def hypexport(self):
else:
global Model
global Highlight
import my_configuration.repos.hypexport.model as hypexport
import mycfg.repos.hypexport.model as hypexport
# TODO a bit hacky.. not sure how to make it both mypy and runtime safe..
Model = hypexport.Model
Highlight = hypexport.Highlight
Expand All @@ -52,7 +52,7 @@ def configure(*, export_path: Optional[PathIsh]=None, hypexport_path: Optional[P
hypexport_path_=hypexport_path,
)

# TODO for the purposes of mypy, try importing my_configuration anyway?
# TODO for the purposes of mypy, try importing mycfg anyway?
# return type for this method as well
# TODO check if it works at runtime..
def get_model() -> Model:
Expand Down
6 changes: 3 additions & 3 deletions my/instapaper.py
Expand Up @@ -3,7 +3,7 @@
Set via
- ~configure~ method
- or in ~my_configuration.instpaper.export_path~
- or in ~mycfg.instpaper.export_path~
TODO upload my exporter script to github..
"""
Expand All @@ -29,8 +29,8 @@ def _get_files():
# TODO use helper method from common to get json[s]?
export_path = _export_path
if export_path is None:
# fallback to my_configuration
# TODO import my_configuration?
# fallback to mycfg
# TODO import mycfg?
from . import paths
export_path = paths.instapaper.export_path
return list(sorted(Path(export_path).glob('*.json')))
Expand Down
2 changes: 1 addition & 1 deletion my/jawbone/__init__.py
Expand Up @@ -7,7 +7,7 @@
import logging
import pytz

from my_configuration import paths
from mycfg import paths


BDIR = paths.jawbone.export_dir
Expand Down
2 changes: 1 addition & 1 deletion my/jawbone/plots.py
Expand Up @@ -87,7 +87,7 @@ def iter_useful(data_file: str):

# TODO <<< hmm. these files do contain deep and light sleep??
# also steps stats??
from my_configuration import paths
from mycfg import paths

p = paths.jawbone.export_dir / 'old_csv'
# TODO with_my?
Expand Down
2 changes: 1 addition & 1 deletion my/lastfm/__init__.py
Expand Up @@ -7,7 +7,7 @@

import pytz

from my_configuration import paths
from mycfg import paths

# TODO Json type?
# TODO memoised properties?
Expand Down
3 changes: 1 addition & 2 deletions my/media/imdb.py
Expand Up @@ -6,8 +6,7 @@

from ..common import get_files

# TODO eh. rename to my_cfg? easier to type
from my_configuration import paths
from mycfg import paths

def _get_last():
return max(get_files(paths.imdb.export_path, glob='*.csv'))
Expand Down
2 changes: 1 addition & 1 deletion my/media/youtube.py
Expand Up @@ -8,7 +8,7 @@

from ..common import get_files

from my_configuration import paths
from mycfg import paths


def _get_last_takeout():
Expand Down
4 changes: 2 additions & 2 deletions my/pdfs.py
@@ -1,12 +1,12 @@
#!/usr/bin/env python3
from . import paths
from .common import import_file

from pathlib import Path


# path to pdfannots (https://github.com/0xabu/pdfannots)
pdfannots = import_file(paths.pdfs.pdfannots_py)
import mycfg.repos.pdfannots.pdfannots as pdfannots
from mycfg import paths


from datetime import datetime
Expand Down
2 changes: 1 addition & 1 deletion my/pocket.py
Expand Up @@ -41,7 +41,7 @@ def highlights(self) -> Sequence[Highlight]:
# TODO add tags?


# TODO integrate with my_configuration
# TODO integrate with mycfg
def get_articles(json_path: Path) -> Sequence[Article]:
import json
raw = json.loads(json_path.read_text())['list']
Expand Down
4 changes: 2 additions & 2 deletions my/reddit.py
Expand Up @@ -2,8 +2,8 @@
from pathlib import Path, PosixPath
from typing import List, Sequence, Mapping

from my_configuration import paths
import my_configuration.repos.rexport.model as rexport
from mycfg import paths
import mycfg.repos.rexport.model as rexport


# TODO Move this to kython.kompress?
Expand Down
6 changes: 3 additions & 3 deletions my/rescuetime.py
Expand Up @@ -9,19 +9,19 @@
# TODO get rid of it
from kython import group_by_cmp # type: ignore

from my_configuration import paths
from mycfg import paths


def get_logger():
return logging.getLogger("my.rescuetime")


def _get_exports() -> List[Path]:
from my_configuration import paths
from mycfg import paths
return get_files(paths.rescuetime.export_path, '*.json')


import my_configuration.repos.rescuexport.model as rescuexport
import mycfg.repos.rescuexport.model as rescuexport
Model = rescuexport.Model


Expand Down

0 comments on commit 066641a

Please sign in to comment.