DECI demo simulation error

[saichaitanyamolabanti]

I'm new to DECI and wanted to replicate the exact steps mentioned in "DECI Demo" file.

When i tried to first run this script - research_experiments/DECI/data_generation/csuite/simulate.py
I'm getting an error "ValueError: cannot reindex from a duplicate axis" - the error is coming from the first method call of "two_node_lin" method and iterating over :
Observational
Interventional
Counterfactual -> this is where It got stuck !!

Could you guys, please attach the data in azua/data/ folder, can we remove the data generating step as I spent a lot of time in debugging this thing?

and also, I ran the below command and it's going to the exception block, any ideas on this one ?
try:
from evaluation_pipeline.aml_azua_context import setup_azua_context_in_aml
azua_context = setup_azua_context_in_aml()
except ImportError:
from azua.azua.experiment.azua_context import AzuaContext
azua_context = AzuaContext()

@ae-foster @pawni @themanojkumar @cvarun16 @ChengZhangMSRC @makukl

[saichaitanyamolabanti]

@microsoftopensource @msftgits

[saichaitanyamolabanti]

@ae-foster any help? I just can't run our deci demo script, you can refer to above errors.
Could you please give us data such that we can just load it from and move with other methods ?

[ae-foster]

Hi @saichaitanyamolabanti , apologies that you are having difficulty with the CSuite data generation script. Unfortunately, we were not able to reproduce the problem in our internal repository. We aim to release the latest version of our internal code soon, at which point the problem should be resolved. In the meantime, we are exploring the possibility of releasing the raw data so you can bypass running simulate.py and get on with running some experiments. We will let you know as soon as we find out whether this is possible.

[saichaitanyamolabanti]

Hey @ae-foster I did a couple of edits on the code and got the results, but don't know where I went wrong because of the observations I made, please look into below and let me know any suggestions:

scenario-1: to make simulate.py work
observed, there were problems with pairplots and assert statements atleast in my case, so removed them in the scripts to generate data from simulate.py script
pyro_utils.py file - Line 206:
"""
if discrete:
sns.pairplot(pd.concat(df_list), hue="intervention", plot_kws=dict(), grid_kws=dict(), kind="hist")
else:
sns.pairplot(pd.concat(df_list), hue="intervention", plot_kws=dict(alpha=0.05, size=0.7), grid_kws=dict())
"""
utils file - Line:69 #assert intervention_samples.shape == reference_samples.shape == original_samples.shape
able to generate data, but able to see interventions file in all folders as NaN values, you can see below.

Scenario-2: generating causal graph
even though, intervention dataset was almost Nan filled, I wanna give a try in generating causal graph.
load_data has some problems again with assert statements. so commented -
if is_counterfactual:
assert (conditioning_values.shape[0] == intervention_samples.shape[0]), "Counterfactual data expects the conditioning to be of equivalent shape as the interventional data.
assert (conditioning_mask == 1).all(), "Counterfactual data expects the conditioning to be full."
now, data got loaded and sample folder graphs got generated

Scenario-3: csuite_large_backdoor generated not a DAG
experimented in generating csuite_large_backdoor causal graph but able to see a error message "not a DAG generated"
commented #assert (np.trace(scipy.linalg.expm(adj_mat)) - self.num_nodes) == 0, "Generate non DAG graph" to generate forecfully, got below graph

could you please provide me suggestions on which way I can try better, because there's definitely something wrong. If it's about the dataset, could we please attach a valid dataset to just mitigate all these effects ?

[ae-foster]

Hi @saichaitanyamolabanti - thanks for your patience. We have agreed as a team to release the data into the public domain, which should resolve the issues you're having. We need to go through an internal process to have final agreement on releasing the synthetic data. We will update you when it is released!

[saichaitanyamolabanti]

thanks @ae-foster, i'll wait for you guys responding back on the synthetic data for atleast one use-case. btw, could you please tell the above causal graph is incorrect right? any suggestions on my previous comment ?

[ae-foster]

For the causal DAG generated, it could be caused by incorrect data, or alternatively a problem with optimization. Augmented Lagrangian training can take a long time to converge. Did you run using the same hyperparameter settings that were recommended (including the auglag parameters)?

[ae-foster]

Referring back to the original question, which was about access to the synthetic data, I am happy to let you know that you can now access our "CSuite" synthetic datasets here https://github.com/microsoft/csuite without having to run any data generation code! This repository contains all the documentation about these datasets, how to download and use them, etc.