Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change default categorical transformer to categorical_fuzzy in HMA1 #214

Closed
csala opened this issue Oct 20, 2020 · 1 comment · Fixed by #259
Closed

Change default categorical transformer to categorical_fuzzy in HMA1 #214

csala opened this issue Oct 20, 2020 · 1 comment · Fixed by #259
Assignees
Labels
internal The issue doesn't change the API or functionality
Milestone

Comments

@csala
Copy link
Contributor

csala commented Oct 20, 2020

The HMA1 algorithm uses the GaussianCopula class, which, by default, uses the one_hot_encoding categorical transformer.

This may provoke high dimensionality issues when modeling tables with lots of columns with several depth levels, as reported in #209

To fix this, the default model_kwargs for the tabular model should set the categorical transformer to categorical_fuzzy, which does not create new columns.

@csala csala added the internal The issue doesn't change the API or functionality label Oct 21, 2020
@csala csala self-assigned this Nov 24, 2020
@csala csala added this to the 0.4.6 milestone Nov 24, 2020
@erenarkangil
Copy link

Hi Carles, thank you for the feedback.

I have a high dimensional data with over 2000 categoric variables, but still have memory problems. I wonder the limit of fuzzy transformer. Here is my code:

from sdv.relational import HMA1

fuzzy = {'categorical_transformer': 'categorical_fuzzy'}
model = HMA1(metadata1,model_kwargs=fuzzy)

Many thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
internal The issue doesn't change the API or functionality
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants