Skip to content

Commit

Permalink
pymake -> pmk
Browse files Browse the repository at this point in the history
  • Loading branch information
dtrckd committed Feb 2, 2018
1 parent 696fb89 commit dcdd0e3
Show file tree
Hide file tree
Showing 19 changed files with 79 additions and 188 deletions.
3 changes: 3 additions & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
include LICENSE README.md requirements.txt Makefile
include pymake/core/*.template
recursive-include repo Makefile requirements.txt pymake.cfg
9 changes: 6 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
.ONESHELL:
SHELL = /bin/bash

PACKAGE := pymake
PACKAGE := pmk
# Assumes Python3
pip3_version := $(shell pip3 --version 2>/dev/null)

Expand All @@ -26,11 +26,14 @@ endif
install_short:
python3 setup.py install --user

push_pip:
#python3 setup.py register -r pypi
python3 setup.py sdist upload -r pypi


uninstall:
pip3 uninstall $(PACKAGE)
rm -vf $(HOME)/.local/bin/pymake
rm -vf $(HOME)/.local/bin/$(PACKAGE)

build:
python3 setup.py build
Expand All @@ -43,7 +46,7 @@ clean: clean_cython
#find -name "*.orig" | xargs rm -fi
-@rm -rf build/
-@rm -rf dist/
-@rm -rf $(PACKAGE).egg-info/
-@rm -rf *.egg-info/

clean_cython:

40 changes: 20 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,26 +72,26 @@ You may also need the following package : `apt-get install poppler-utils`
Then a typical pymake usage :

```bash
pymake run --script fit --path path/to/your/pdfs/ # index your pdf documents, take a coffee
pymake run --script search "your text search request" # show relevant information
pmk run --script fit --path path/to/your/pdfs/ # index your pdf documents, take a coffee
pmk run --script search "your text search request" # show relevant information
```

Or equivalently (aliases):
```bash
pymake -x fit --path path/to/your/pdfs/
pymake -x search "your text search request"
pmk -x fit --path path/to/your/pdfs/
pmk -x search "your text search request"
```

Or show only the first match : `pymake -x search "your text search request" --limit 1`
Or show only the first match : `pmk -x search "your text search request" --limit 1`

To add new models, new scripts, or specs, you need to create it in the dedicated folder following the base class implementations.

Then you can list some informations about pymake objects :

* What experiments are there: `pymake -l spec`
* What models are there: `pymake -l model`
* What scripts are there: `pymake -l script`
* Show signatures of methods in scripts ('ir' script): `pymake -l --script ir`
* What experiments are there: `pmk -l spec`
* What models are there: `pmk -l model`
* What scripts are there: `pmk -l script`
* Show signatures of methods in scripts ('ir' script): `pmk -l --script ir`


## Documentation [](#4)
Expand Down Expand Up @@ -139,29 +139,29 @@ If new models or scripts are added in the project, you'll need to update the pym
List/Search information :

```bash
pymake -l spec # (or just `pymake -l`) show available designs of experimentation
pymake -l model # show available models
pymake -l script # show available scripts
pymake show expe_name # or just pymake expe_name
pmk -l spec # (or just `pymake -l`) show available designs of experimentation
pmk -l model # show available models
pmk -l script # show available scripts
pmk show expe_name # or just pymake expe_name
```

Run experiments :

```bash
pymake run [expe_name] --script script_name [script options...]
pmk run [expe_name] --script script_name [script options...]
# Or shortly (alias):
pymake [expe_name] -x script_name
pmk [expe_name] -x script_name
# Run in parallel:
pymake [expe_name] -x script_name --cores N_CORES
pmk [expe_name] -x script_name --cores N_CORES
```

Show Paths for disks I/O:

pymake path [expe_name] [script options...]
pmk path [expe_name] [script options...]

Show individuals commands for asynchronously purpose (@deprecated) :

pymake cmd [expe_name] [script options...]
pmk cmd [expe_name] [script options...]

##### Designing experiments

Expand Down Expand Up @@ -209,7 +209,7 @@ The third class is the `ExpGroup` which allows to group several design of experi
exp3 = ExpGroup([exp1, exp2])
```

You can then run `pymake -l` to see our design of experiments.
You can then run `pmk -l` to see our design of experiments.

##### Designing Model

Expand All @@ -221,7 +221,7 @@ Basically, A model is a class inside `model/` that have a method `fit`.

A scipt is a piece of code that you execute which is parametrised by a **specification**. More specifically, Scripts are methods of class that inherits a `ExpeFormat` and that lives inside the `script/` folder.

Once you defined some scripts, you'll be able to list them with `pymake -l script`, and to run them, by their name, with `pymake [specification_id] -x script_name`.
Once you defined some scripts, you'll be able to list them with `pmk -l script`, and to run them, by their name, with `pmk [specification_id] -x script_name`.

Then each experiments defined in your design (or _default_expe if no specification_id is given), will go through the script method. Then, a bunch of facilities are living inside the method at runtime :

Expand Down
4 changes: 4 additions & 0 deletions changelog.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@


# 0.37 release
first alpha stable release of pymake.
2 changes: 1 addition & 1 deletion pymake/Todo.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@ Debug

CLI
---
* pmk diff expe1 expe2 # show diff between expe...
* pmk push [spec] [opts] # push expe in spec !!! (Update MAN)
* pmk fetch text/hugh (corpus etc !)

Bash auto-Completion
---------------
Expand Down
6 changes: 3 additions & 3 deletions pymake/core/gramexp.py
Original file line number Diff line number Diff line change
Expand Up @@ -492,7 +492,7 @@ def get_file_format(expe):
def get_parser(description=None, usage=None):
import pymake.core.gram as _gram
import pkg_resources
_version = pkg_resources.get_distribution('pymake').version
_version = pkg_resources.get_distribution('pmk').version
parser = _gram.ExpArgumentParser(description=description, epilog=usage,
formatter_class=argparse.RawDescriptionHelpFormatter)
parser.add_argument('--version', action='version', version='%(prog)s '+_version)
Expand Down Expand Up @@ -960,7 +960,7 @@ def execute_parallel(self):
# Target = subprocess.check_output(['which','pymake']).strip().decode()
#except:
# Target = 'python3 /home/ama/adulac/.local/bin/pymake'
Target = 'pymake'
Target = 'pmk'
basecmd = [Target] + basecmd[1:]

# Create commands indexs
Expand Down Expand Up @@ -1013,7 +1013,7 @@ def execute_parallel_net(self, nhosts=None):
basecmd = sys.argv.copy()
#Target = './zymake.py'
#basecmd = ['python3', Target] + basecmd[1:]
Target = 'pymake'
Target = 'pmk'
basecmd = [Target] + basecmd[1:]
cmdlines = None

Expand Down
5 changes: 4 additions & 1 deletion pymake/parallel.sh
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
#!/bin/bash

#############
### GNU Parallel parameters
#
# @Deprecated script
#
#
JOBS="3"

COMMAND="python3 ./fit.py -w -i 200 --refdir debug"
Expand Down
37 changes: 0 additions & 37 deletions pymake/px.sh

This file was deleted.

17 changes: 0 additions & 17 deletions pymake/pyrallel.sh

This file was deleted.

25 changes: 0 additions & 25 deletions pymake/pysync.sh

This file was deleted.

File renamed without changes.
1 change: 0 additions & 1 deletion pymake/util/stopwords.txt

This file was deleted.

6 changes: 5 additions & 1 deletion repo/ml/Makefile
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
.ONESHELL:
SHELL = /bin/bash

default:
default: install

install:
pip3 install --user -r requirements.txt


fetch_data: net_data text_data lut

Expand Down
33 changes: 0 additions & 33 deletions repo/ml/data/backsync.sh

This file was deleted.

28 changes: 0 additions & 28 deletions repo/ml/data/reports/backsync.sh

This file was deleted.

6 changes: 3 additions & 3 deletions pymake/dot.sh → repo/ml/dot.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ Engine='circo dot fdp neato nop nop1 nop2 osage patchwork sfdp twopi'

for e in $Engine; do
echo "doting $e"
dot -K$e -Tpng graph.dot > ../results/graph_$e.png
dot -K$e -Tpng graph.dot > data/reports/graph_$e.png
done

cp ../results/graph_dot.png .
ristretto ../results/graph_dot.png
cp data/reports/graph_dot.png .
ristretto data/reports/graph_dot.png
1 change: 1 addition & 0 deletions repo/ml/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
sympy
1 change: 0 additions & 1 deletion requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ numpy
scipy
networkx
nltk
sympy
matplotlib
pandas
#plotly ?
Expand Down

0 comments on commit dcdd0e3

Please sign in to comment.