Multi-evaluation Binary Classifier #15
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
these groups go on nodes immediately below Root
since classifications are binary many of the individual nodes actually don't affect training classification and we can remove those resulting in a much smaller easier to analyze tree
mjohnson541
force-pushed
the
covdep_associated
branch
from
c7ade02
to
814815d
Compare
hwpang
requested changes
May 7, 2024
| if node.depth <= depth: | ||
| n = pydot.Node(name=name, label=name, fontname="Helvetica", fontsize="16") | ||
| if images: | ||
| img = node.group.draw("png") |
There was a problem hiding this comment.
Suggested change
| img = node.group.draw("png") | |
| img = node.group.draw("pdf") |
There was a problem hiding this comment.
Better quality of figures when writing paper and making slides. The figures don't have fuzzy edges when you enlarge them. Additionally, overleaf compiles faster if you include all your figures as pdf.
| @@ -1,27 +1,31 @@ | |||
| from IPython.display import Image, display | |||
| import pydot | |||
| import os | |||
| import numpy as np | |||
There was a problem hiding this comment.
Suggested change
| import numpy as np | |
| import numpy as np | |
| from pathlib import Path |
Comment on lines
9
to
11
| graph.set_fontsize("10") | ||
| if not os.path.exists("./tree"): | ||
| os.makedirs("./tree") |
There was a problem hiding this comment.
Can we change all the use of os to pathlib? pathlib is the more modern way since Python 3.4
Suggested change
| graph.set_fontsize("10") | |
| if not os.path.exists("./tree"): | |
| os.makedirs("./tree") | |
| graph.set_fontsize("10") | |
| save_dir = Path("./tree") | |
| save_dir.mkdir(exist_ok=True) |
Comment on lines
+18
to
+20
| with open("./tree/" + node.name + ".png", "wb") as f: | ||
| f.write(img) | ||
| n.set_image(os.path.abspath("./tree/" + node.name + ".png")) |
There was a problem hiding this comment.
Suggested change
| with open("./tree/" + node.name + ".png", "wb") as f: | |
| f.write(img) | |
| n.set_image(os.path.abspath("./tree/" + node.name + ".png")) | |
| node_save_path = (save_dir / node.name + ".pdf").resolve() | |
| with open(node_save_path, "wb") as f: | |
| f.write(img) | |
| n.set_image(node_save_path)) |
Comment on lines
29
to
30
| graph.write_dot("./tree/tree.dot") | ||
| graph.write_png("./tree/tree.png") |
There was a problem hiding this comment.
Suggested change
| graph.write_dot("./tree/tree.dot") | |
| graph.write_png("./tree/tree.png") | |
| graph.write_dot(save_dir / "tree.dot") | |
| graph.write_png(save_dir / "tree.png") |
| sidt_val_values = [self.evaluate(d.mol) for d in self.validation_set] | ||
| true_val_values = [d.value for d in self.validation_set] | ||
|
|
||
| P,N,PP,PN,TP,FN,FP,TN = analyze_binary_classification(sidt_train_values,true_train_values) |
There was a problem hiding this comment.
I'm pretty sure there are metrics functions to compute accuracy, recall, precision, etc. we can just import from sklearn to make the code cleaner. Can you do that?
Suggested change
| P,N,PP,PN,TP,FN,FP,TN = analyze_binary_classification(sidt_train_values,true_train_values) | |
| P, N, PP, PN, TP, FN, FP, TN = analyze_binary_classification(sidt_train_values, true_train_values) |
| } | ||
| self.best_rule_map = {name:self.nodes[name].rule for name in self.best_tree_nodes} | ||
|
|
||
| logging.info("# nodes: {}".format(len(self.nodes))) |
There was a problem hiding this comment.
Suggested change
| logging.info("# nodes: {}".format(len(self.nodes))) | |
| logging.info(f"# nodes: {len(self.nodes)}") |
| 2) merges nodes with their parents if they do not result in different predictions | ||
| """ | ||
|
|
||
| self.datum_truth_map = {datum:[getattr(n,"rule") for n in self.mol_node_maps[datum]["nodes"]] for datum in self.datums} |
There was a problem hiding this comment.
Suggested change
| self.datum_truth_map = {datum:[getattr(n,"rule") for n in self.mol_node_maps[datum]["nodes"]] for datum in self.datums} | |
| self.datum_truth_map = {datum: [getattr(n,"rule") for n in self.mol_node_maps[datum]["nodes"]] for datum in self.datums} |
| """ | ||
|
|
||
| self.datum_truth_map = {datum:[getattr(n,"rule") for n in self.mol_node_maps[datum]["nodes"]] for datum in self.datums} | ||
| self.datum_node_map = {datum:[n for n in self.mol_node_maps[datum]["nodes"]] for datum in self.datums} |
There was a problem hiding this comment.
Suggested change
| self.datum_node_map = {datum:[n for n in self.mol_node_maps[datum]["nodes"]] for datum in self.datums} | |
| self.datum_node_map = {datum: [n for n in self.mol_node_maps[datum]["nodes"]] for datum in self.datums} |
|
|
||
| assert len(new) == 0 | ||
| assert len(comp) == 0 | ||
| pnew = new_class_true/Nnew |
There was a problem hiding this comment.
Suggested change
| pnew = new_class_true/Nnew | |
| pnew = new_class_true / Nnew |
hwpang
requested changes
May 7, 2024
| "source": [ | ||
| "data = []\n", | ||
| "for sm in stable_smiles:\n", | ||
| " data.append(Datum(Molecule().from_smiles(sm),True))\n", |
There was a problem hiding this comment.
Suggested change
| " data.append(Datum(Molecule().from_smiles(sm),True))\n", | |
| " data.append(Datum(Molecule().from_smiles(sm), True))\n", |
Can you run black formatter through the notebook too
|
Can you also add a pytest for this new type of tree? I don't think it's best practice to only rely on notebook test. The |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds an SIDT algorithm for Multi-Evaluation binary classification.
It also adds some smaller improvements:
Allows plotting only to specified depth
Saves rules as well as nodes in postpruning
allows specification of an initial set of splits from the root node
An example notebook for unstable Q.OOH classification is provided.