Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

馃摗 馃搲 Adding Tensorboard Tracker #416

Merged
merged 41 commits into from Jun 2, 2021
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
ba010ce
added tensorboard tracker file and into init
sbonner0 May 3, 2021
6685768
start of docs
sbonner0 May 3, 2021
c4d0f6d
Add tutorial to index
cthoyt May 4, 2021
9c0fafc
Update src/pykeen/trackers/tensorboard.py
sbonner0 May 4, 2021
63eb1a4
Add type annotation in class for instance variable SummaryWriter
cthoyt May 4, 2021
e77570c
Update src/pykeen/trackers/tensorboard.py
sbonner0 May 4, 2021
65c7768
update start_run to reflect new naming
sbonner0 May 6, 2021
dec8030
Update tensorboard.py
cthoyt May 6, 2021
f68330c
expand docs with log dir arg
sbonner0 May 6, 2021
ee4fcab
more docs
sbonner0 May 6, 2021
a4e49c2
Update using_tensorboard.rst
mberr May 7, 2021
0233cb0
Update utils.py
cthoyt May 7, 2021
65fd2c7
Update ablation.py
cthoyt May 7, 2021
18226e0
Merge branch 'master' into feature/tensorboard
cthoyt May 11, 2021
2536233
Trigger CI
PyKEEN-bot May 11, 2021
ebf1633
fixed time string and added docs on starting tensorboard server
sbonner0 May 12, 2021
10b9a55
Revert
cthoyt May 16, 2021
60b1912
Merge branch 'master' into feature/tensorboard
cthoyt May 16, 2021
3406d5b
Update utils.py
cthoyt May 16, 2021
75ed7c3
Update __init__.py
cthoyt May 16, 2021
73ceddf
Remove trailing whitespace from template
cthoyt May 16, 2021
66056d8
Regenerate README
cthoyt May 16, 2021
5f8fdfd
added tensorboard to setup.cfg
sbonner0 May 17, 2021
adb6500
updated the docs with install
sbonner0 May 17, 2021
7ab0a6b
remove redundant use of comment arg
sbonner0 May 17, 2021
33f4f5a
moved writer into init
sbonner0 May 17, 2021
6232bb4
added else to catch edge cases on paths
sbonner0 May 17, 2021
cbfd683
added hpo example to docs
sbonner0 May 26, 2021
f3bb9c4
Merge branch 'master' into feature/tensorboard
cthoyt May 26, 2021
b62fb25
Trigger CI
PyKEEN-bot May 26, 2021
059a7ae
Add tests
cthoyt May 27, 2021
a62dcbc
Update installation extras table
cthoyt May 27, 2021
584b7a6
Minor editing and revising
cthoyt May 27, 2021
fbbf4d0
Update CHANGELOG.rst
cthoyt May 27, 2021
b9f5611
Simplify storage of path
cthoyt May 27, 2021
177f817
Simplify storage and annotation of summary writer
cthoyt May 27, 2021
4573221
Path isn't reused so don't save it
cthoyt May 27, 2021
9f33160
removed unused tag arg
sbonner0 Jun 2, 2021
44033ec
Trigger CI
PyKEEN-bot Jun 2, 2021
1106a00
Merge branch 'master' into feature/tensorboard
cthoyt Jun 2, 2021
2bf60c5
Trigger CI
PyKEEN-bot Jun 2, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/source/tutorial/trackers/index.rst
Expand Up @@ -6,4 +6,5 @@ Tracking Results during Training
using_mlflow
using_neptune
using_wandb
using_tensorboard
using_file
41 changes: 41 additions & 0 deletions docs/source/tutorial/trackers/using_tensorboard.rst
@@ -0,0 +1,41 @@
Using Tensorboard
=========================

`Tensorboard <https://www.tensorflow.org/tensorboard/>`_ (TB) is a service for tracking experimental results during training.
It is part of the Tensorflow project.

Minimal Pipeline Example
---------------------------------
A CSV log file can be generated with the following:

.. code-block:: python

from pykeen.pipeline import pipeline

pipeline_result = pipeline(
model='RotatE',
dataset='Kinships',
result_tracker='tensorboard',
)

It is placed in a subdirectory of :mod:`pystow` default data directory with PyKEEN called ``tensorboard``,
cthoyt marked this conversation as resolved.
Show resolved Hide resolved
which will likely be at ``~/.data/pykeen/logs/tensorboard`` on your system. The file is named based on the
current time if no alternative is provided.

Specifying a Name
-----------------
If you want to specify the name of the log file in the default directory, use the ``experiment_name`` keyword
argument like:

.. code-block:: python

from pykeen.pipeline import pipeline

pipeline_result = pipeline(
model='RotatE',
dataset='Kinships',
result_tracker='tensorboard',
result_tracker_kwargs=dict(
experiment_name='test',
),
)
2 changes: 2 additions & 0 deletions src/pykeen/trackers/__init__.py
Expand Up @@ -7,6 +7,7 @@
from .base import ResultTracker
from .file import CSVResultTracker, FileResultTracker, JSONResultTracker
from .mlflow import MLFlowResultTracker
from .tensorboard import TensorBoardResultTracker
from .neptune import NeptuneResultTracker
from .wandb import WANDBResultTracker

Expand All @@ -20,6 +21,7 @@
'WANDBResultTracker',
'JSONResultTracker',
'CSVResultTracker',
'TensorBoardResultTracker',
# Utilities
'tracker_resolver',
]
Expand Down
78 changes: 78 additions & 0 deletions src/pykeen/trackers/tensorboard.py
@@ -0,0 +1,78 @@
# -*- coding: utf-8 -*-

"""An adapter for TensorBoard."""

import datetime
import pathlib
from typing import Any, Dict, Mapping, Optional, TYPE_CHECKING, Type

from .base import ResultTracker
from ..constants import PYKEEN_LOGS
from ..utils import flatten_dictionary

if TYPE_CHECKING:
import torch.utils.tensorboard

__all__ = [
'TensorBoardResultTracker',
]


class TensorBoardResultTracker(ResultTracker):
"""A tracker for TensorBoard."""

#: The class that's used to instantiate a summarywriter
SummaryWriter: Type['torch.utils.tensorboard.SummaryWriter']
cthoyt marked this conversation as resolved.
Show resolved Hide resolved

def __init__(
self,
experiment_path: Optional[str] = None,
experiment_name: Optional[str] = None,
tags: Optional[Dict[str, Any]] = None,
):
"""
Initialize result tracking via Tensorboard.

:param experiment_path:
The experiment path. A custom path at which the tensorboard logs will be saved.
:param experiment_name:
The name of the experiment, will be used as a sub directory name for the logging. If no default is given,
the current time is used. If set, experiment_path is set, this argument has no effect.
:param tags:
The additional run details which are presented as tags to be logged
"""
from torch.utils.tensorboard import SummaryWriter as _SummaryWriter
self.summary_writer_cls = _SummaryWriter
self.tags = tags
sbonner0 marked this conversation as resolved.
Show resolved Hide resolved

if experiment_path is None:
if experiment_name is None:
experiment_name = datetime.datetime.now().isoformat()
sbonner0 marked this conversation as resolved.
Show resolved Hide resolved
path = PYKEEN_LOGS.joinpath("tensorboard", experiment_name)
elif isinstance(experiment_path, str):
cthoyt marked this conversation as resolved.
Show resolved Hide resolved
path = pathlib.Path(experiment_path)
self.path = path
sbonner0 marked this conversation as resolved.
Show resolved Hide resolved

def start_run(self, run_name: Optional[str] = None) -> None: # noqa: D102
self.writer = self.SummaryWriter(log_dir=self.path, comment=run_name)

def log_metrics(
self,
metrics: Mapping[str, float],
step: Optional[int] = None,
prefix: Optional[str] = None,
) -> None: # noqa: D102
metrics = flatten_dictionary(dictionary=metrics, prefix=prefix)
for key, value in metrics.items():
self.writer.add_scalar(tag=key, scalar_value=value, global_step=step)
self.writer.flush()

def log_params(self, params: Mapping[str, Any], prefix: Optional[str] = None) -> None: # noqa: D102
params = flatten_dictionary(dictionary=params, prefix=prefix)
for key, value in params.items():
self.writer.add_text(tag=str(key), text_string=str(value))
self.writer.flush()

def end_run(self) -> None: # noqa: D102
self.writer.flush()
self.writer.close()