Module : Libraries (lib)

Introduction

The lib module offers supporting libraries for the functional modules.

Spark

A collection of packages that offer a wrapper for using apache spark functions and features.

execSession

Apache Spark requires a session instance. For example, for creating a spark Dataframe purely for spark data workloads. While sparkRDBM and sparkNoSQL create specific sessions, this package doesn't require all the database connection information.

   from dongcha.modules.lib.spark import execSession
   mySession = execSession.session

Utils

A collection of packages that offer generic and standardized dongcha platform-wide utilities.

Reference

Reference package handles the data workloads, mainly, for referring static (lookup) type data.

   from dongcha.modules.lib.utils import reference as ref
   clsRef = ref.dataWorkLoads(desc=__desc__,)