Feature generate from existing data (#163)

* wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip
databrickslabs · Apr 9, 2023 · 615c56b · 615c56b
1 parent 37f0c31
commit 615c56b
Show file tree

Hide file tree

Showing 11 changed files with 694 additions and 104 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -12,6 +12,8 @@ All notable changes to the Databricks Labs Data Generator will be documented in
 * Additional build ordering enhancements to reduce circumstances where explicit base column must be specified
 
 #### Added
+* Scripting of data generation code from schema (Experimental)
+* Scripting of data generation code from dataframe (Experimental)
 * Added top level `random` attribute to data generator specification constructor
 
 

diff --git a/README.md b/README.md
@@ -51,6 +51,7 @@ used in other computations
 * use of SQL expressions in synthetic data generation
 * plugin mechanism to allow use of 3rd party libraries such as Faker
 * Use within a Databricks Delta Live Tables pipeline as a synthetic data generation source
+* Generate synthetic data generation code from existing schema or data (experimental)
 
 Details of these features can be found in the online documentation  -
  [online documentation](https://databrickslabs.github.io/dbldatagen/public_docs/index.html). 

diff --git a/dbldatagen/__init__.py b/dbldatagen/__init__.py
@@ -27,7 +27,7 @@
 from .datagen_constants import DEFAULT_RANDOM_SEED, RANDOM_SEED_RANDOM, RANDOM_SEED_FIXED, \
                                RANDOM_SEED_HASH_FIELD_NAME, MIN_PYTHON_VERSION, MIN_SPARK_VERSION
 from .utils import ensure, topologicalSort, mkBoundsList, coalesce_values, \
-    deprecated, parse_time_interval, DataGenError, split_list_matching_condition
+    deprecated, parse_time_interval, DataGenError, split_list_matching_condition, strip_margins
 from ._version import __version__
 from .column_generation_spec import ColumnGenerationSpec
 from .column_spec_options import ColumnSpecOptions