Initial release

Summary:

add pip package setup:

U you can use pip install -e . for dev and python setup.py sdist bdist_wheel for releasing package

api package

• api.apps: for running server apps (such as fastapi apps)
• api.database: database needed for API endpoints via sqlalchemy
• api.models: payload and response models for all endpoints validated via pydantic

configs package

• configs.core: parsing different configs as json used generally for all internal and 3rdpary modules used
• configs.data: some sample data for usage demo

data package

• data.constant: add constants for domain specific knowledge. Constant lists, dicts, and of course custom Enum is used to properly show const data and domain specific info that need tbe used across entire library if used.
• data.functional: contains some general functions that does nothing in fact! hence can be used every where! Just some annoying things that I dont want to do again.
• data.logic: how to apply different logics to data to extend, summarize and transform, via general or domain specific knowledge
• data.pdf: methods for reading pdf for conversion to tabular data. For now, Adobe XFA pdfs are supported
• data.preprocessor: All sort of transforms that can be used for all sort of data (for now, since not many methods are here). For now, handling files in general is handled.

utils package

• utils.loggers: a general Logger class that redirects all logging into desired stream (internal and 3rdparty) and adds interface for saving artifacts in mlflow artifact directory. Also, has reset methods to use same logger without any more configs to be used for multiple experiments in an isolated manner (each experiment has its own dirs and files numbered)
• utils.visualization: some helpers methods that might use for all visualization. No main plot here, you create main plot first, then call these methods to modify it (such as changing font size for only a single plot)

models package

• models.estimators: hosts wrapper around 3rd party model libs and also all models defined as estimator. note that only model definition should be here and no training or evaluation script.
  • models.estimators.networks: hosts only neural network estimators (since they are large and common and might need many tweaks)
• models.evaluators: hosts config files, evaluating scripts and custom or wrapped 3rd party metrics (this is where metrics and evaluation for models.estimators can be hosted)
  • models.evaluators.metrics: wrapper around 3rd party metrics (sklearn metrics) or custom metrics for all sort of estimators and trainers
• models.preprocessors: hosts preprocessing only needed for direct model usage. for instance, no feature engineering here, only things similar to sklearn transformation (onehot) or categorization that are model specific. Also, user can create wrappers for famous libraries such as PyTorch and sklearn if needed.
  • models.preprocessors.core: contains methods and classes that can be used across all model preprocessors. Given sklearn duck typing is so popular, most of the methods are for that.
  • models.preprocessors.helpers: helpers for all modules of preprocessors. E.g. for now it contains a preview for transformation done in models.preprocessors.core.
• models.trainers: contains configs and scripts for training (similar to models/preprocessors/evaluators.
  • models.trainers.aml_flaml: A wrapper around flaml Auto ML framework. Since flaml is a low code library and contains training script itself, for now, this module has only extensions for built in methods. E.g. flaml uses a wrapper around sklearn models, so there is find_estimator to extract underlying model. similarly, log_mlflow_model exists to log underlying sklearn based model as mlflow model since there is no flavor in mlflow that matches flaml (but sklearn) is supported.
• models.preprocessors.weights: hosts methods for saving, loading, and customizing weights. Note that usually all libs and frameworks have a built in method for saving and loading weights. But if any customization or simplification needed, it should be hosted here.