Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How physics based loss function applied to synthetic data generation? #618

Closed
kilickursat opened this issue Oct 29, 2021 · 8 comments
Closed
Labels
feature request Request for a new feature

Comments

@kilickursat
Copy link

kilickursat commented Oct 29, 2021

Problem description

I have soil parameters and they are small in size. I applied CopulaGAN and also GaussianCopula to increase the size of data. I have obtained large size soil parameters. By the way, the dataset is only a continuous variable.
My purpose is to obtain physics consistency while data generation because I would like to show the synthetic data will be generated based on rock mechanics law. Therefore, I would like to add rock mechanics law as a loss function to the Copulas.
But I didn't see loss inside the Copulas class.
Do I need to add rock mechanics law into the constraints?
Or how can I implement this physics law to the data generation?

What I already tried

I have applied GaussianCopula and CopulaGAN so far. The dataset has 7 columns 18 rows (real data). After application, I have obtained 7 columns 700 rows (synthetic data). The dataset is only consisting of continuous variables.

@kilickursat kilickursat added pending review question General question about the software labels Oct 29, 2021
@npatki
Copy link
Contributor

npatki commented Oct 29, 2021

Unfortunately, the SDV does not support modifying the loss function right now, and it's not currently in our roadmap.

Constraints are meant to help ensure that strict rules found in the dataset are followed. Use these for cases when all rows must follow some explicit logic.

@npatki npatki added feature request Request for a new feature and removed question General question about the software labels Oct 29, 2021
@kilickursat
Copy link
Author

Thanks for the reply. So, could you elaborate on the constraints?
I have 7 features and they are related to each other because the real data was obtained from laboratory tests. How can I apply constraints to sustain logic in data generation?
For example,
I have triaxial test results like effective stress, pressure, axial stress, so the effective stress was obtained by axial stress and pressure in the empirical formula. When I try to increase the number of these parameters, how can I use constraints?
How do these rows follow explicit logic?
Thank you.

@npatki
Copy link
Contributor

npatki commented Oct 31, 2021

No problem! If your columns follow a deterministic formula then you can use the ColumnFormula constraint -- for example ColumnC = ColumnA - ColumnB. The SDV will then synthesize ColumnA & ColumnB, and then use the logic to compute ColumnC based on that.

You can apply the ColumnFormula constraint multiple times for different columns, as long as the logic to compute columns is not circular.

@kilickursat
Copy link
Author

Many Thanks, Neha, I will apply and inform the GitHub issue about the outcome.

@kilickursat
Copy link
Author

kilickursat commented Nov 1, 2021

I have taken the following error after ColumnFormula for UniqueConstraints. I have also attached some part of data.

set

issue2

issue

@npatki
Copy link
Contributor

npatki commented Nov 1, 2021

Hello, thanks for the feedback!

Issue #595 has more details about the ConstraintsNotMetError. Could you check there and add your details to that discussion?

@kilickursat
Copy link
Author

Thanks for the reply. I have already checked #595 issue but I could not understand how can I solve this error. I will add my detail in the 595 issues. Thanks for your help.

@npatki
Copy link
Contributor

npatki commented Nov 15, 2021

Hello, I'm closing off this issue since he had further discussion in #595.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request Request for a new feature
Projects
None yet
Development

No branches or pull requests

2 participants