Skip to content
This repository has been archived by the owner on Jan 9, 2024. It is now read-only.

Commit

Permalink
Remove obsolete documentations from Foreshadow. (#212)
Browse files Browse the repository at this point in the history
* Remove obsolete documentations from Foreshadow.

* Remove redundant user guide from Foreshadow.
  • Loading branch information
jzhang-gp committed Mar 20, 2020
1 parent 116c6c5 commit 2f26a16
Show file tree
Hide file tree
Showing 7 changed files with 1 addition and 509 deletions.
56 changes: 0 additions & 56 deletions doc/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,61 +9,5 @@ Foreshadow
:members:
:undoc-members:

dp
------------
.. automodule:: foreshadow.dp
:members:
:undoc-members:


Intents
-------
.. automodule:: foreshadow.intents
:members:
:undoc-members:

Transformers
------------
Internal Transformers
~~~~~~~~~~~~~~~~~~~~~

.. automodule:: foreshadow.transformers.internals
:members:
:undoc-members:

Smart Transformers
~~~~~~~~~~~~~~~~~~
.. automodule:: foreshadow.transformers.smart
:members:
:undoc-members:

Transformer Bases
~~~~~~~~~~~~~~~~~
.. automodule:: foreshadow.transformers.base
:members:
:undoc-members:

Estimators
----------
.. automodule:: foreshadow.estimators
:members:
:undoc-members:

Optimizers
----------
.. automodule:: foreshadow.optimizers
:members:
:undoc-members:

Utils
-----
.. automodule:: foreshadow.utils
:members:
:undoc-members:

Core
----
.. automodule:: foreshadow.core
:members:
:undoc-members:

26 changes: 0 additions & 26 deletions doc/architecture.rst

This file was deleted.

54 changes: 0 additions & 54 deletions doc/contrib.rst

This file was deleted.

37 changes: 0 additions & 37 deletions doc/developers.rst
Original file line number Diff line number Diff line change
Expand Up @@ -186,43 +186,6 @@ Adding transformers is quite simple. Simply write a class with the `fit` `transf
After writing your transformer make sure place it in the internals folder in its own file with the associated tests for the transformer in the mirrored test directory and you are all set. If you want to add an external transformer that is not already supported by foreshadow submit a pull request with the appropriate modification to the `externals.py` file in transformers.


Adding Smart Transformers
-------------------------

Building smart transformers is even easier than build transformers. Simply extend :py:class:`SmartTransformer <foreshadow.transformers.base.SmartTransformer>` and implement the :py:func:`_get_transformer`. Modify the example below to suit your needs.

.. code-block:: python
class CustomTransformerSelector(SmartTransformer):
def _get_transformer(self, X, y=None, **fit_params):
data = X.iloc[:, 0] # get single column to decide upon
# perform some computation to determin the best transformer to choose
return BestTransformer() # return an instance of the selected transformer
Add the smart transformer implementation to the bottom of the `smart.py` file and add the appropriate tests to the mirrored tests folder as well.


Adding Intents
--------------

Intents are where the magic of Foreshadow all comes together. You need to be thoughtful when adding an intent especially with respect to where your intent will slot into the intent tree. This positioning will determine the priority with which the intent is mapped to a column. You will need to subclass your intent off of the parent intent that you determine is the best fit. Intents should be constructed in the form matching :py:class:`BaseIntent <foreshadow.intents.BaseIntent>`.

You will need to set the :py:attr:`dtype <foreshadow.intents.BaseIntent.dtype>`, :py:attr:`children <foreshadow.intents.BaseIntent.children>`, :py:attr:`single_pipeline <foreshadow.intents.BaseIntent.single_pipeline>`, and :py:attr:`multi_pipeline <foreshadow.intents.BaseIntent.multi_pipeline>` class attributes. You will also need to implement the :py:meth:`is_intent <foreshadow.intents.BaseIntent.is_intent>` classmethod. In most cases when adding an intent you can initialize :py:attr:`children <foreshadow.intents.BaseIntent.children>` to an empty list. Set the :py:attr:`dtype <foreshadow.intents.BaseIntent.dtype>` to the most appropriate initial form of that entering your intent.

Use the :py:attr:`single_pipeline <foreshadow.intents.BaseIntent.single_pipeline>` field to determine the transformers that will be applied to a **single** column that is mapped to your intent. Add a **unique** name describing each step that you choose to include in your pipeline. This field is represented as a list of PipelineTemplateEntry objects which are constructed using the following format `PipelineTemplateEntry([unique_name], [class], [can_operate_on_y])` The class name is either a singular transformer class, or a tuple of the form `([cls], {**args})` where args will be passed into the constructor of the transformer. The final boolean determines whether that transformer should be applied when operating on y-variables.

It is important to note the utility of smart transformers here as you can now include branched logic in your pipelines deciding between different individual transformers based on the input data at runtime. The :py:attr:`multi_pipeline <foreshadow.intents.BaseIntent.multi_pipeline>` pipeline should be used to apply transformations to all columns of a specific intent after the single pipelines have been evaluated. The same rules for defining the pipelines themselves apply here as well.

The :py:meth:`is_intent <foreshadow.intents.BaseIntent.is_intent>` classmethod determines whether a specific column maps to an intent. Use this method to apply any heuristics, logic, or methods of determine whether a raw column maps to the intent that you are defining. Below is an example intent definition that you can modify to suit your needs.

The :py:meth:`column_summary <foreshadow.intents.BaseIntent.column_summary>` classmethod is used to generate statistical reports each time an intent operates on a columns allowing a user to examine how effective the intent will be in processing the data. These reports can be accessed by calling the :py:meth:`summarize <foreshadow.preprocessor.summarize>` method after fitting the Foreshadow object.

Make **sure** to go to the parent intent and add your intent class name to the ordered :py:attr:`children <foreshadow.intents.BaseIntent.children>` field in the order of priority among the previously defined intents. The last intent in this list will be the most preferred intent upon evaluation in the case of multiple intents being able to process a column.

Take a look at the :py:class:`NumericIntent <foreshadow.intents.NumericIntent>` implementation for an example of how to implement an intent.


Future Architecture Roadmap
---------------------------

Expand Down
2 changes: 1 addition & 1 deletion doc/faq.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@
Frequently Asked Questions
==========================

Test page
To be added.
32 changes: 0 additions & 32 deletions doc/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,45 +2,13 @@

.. include:: ../README.rst

The User Guide
--------------
.. toctree::
:maxdepth: 2

users

.. toctree::
:maxdepth: 1
:hidden:

faq


The Developer Guide
-------------------
.. toctree::
:maxdepth: 2

developers

.. toctree::
:maxdepth: 1
:hidden:

contrib

API
---
.. toctree::
:maxdepth: 2

api

.. toctree::
:maxdepth: 1

architecture

Changelog
---------
.. toctree::
Expand Down

0 comments on commit 2f26a16

Please sign in to comment.