Silent mode when creating reports.

[alejandrofdez-us]

Problem Description

Creating a batch of reports prints all details in console.

Expected behavior

A silent option when creating the report would be appreciated.

Additional context

I have a huge set of experiments, with lots of synthetic samples. I compute the quality and diagnostic reports for every sample, and every experiment, so the console is continuosly printing all reports summaries, which is not useful. I store these metrics as CSV to make further comparisons between experiments.

[npatki]

Hi @alejandrofdez-us, thanks for filing this feature request. We'll keep it open and provide updates as we solve it.

Workarounds

If you'd like to run your reports in the meantime, there seem to be methods for silencing the statements on your own outside of the SDMetrics library. The relevant portions are the tqdm library and stdout. For example, I see the following Stackoverflow conversations for silencing tqdm and stdout.

Trying these suggestions seemed to correctly silence the output for me (but your mileage may vary).

from tqdm import tqdm
from functools import partialmethod
from sdmetrics import load_demo 
from sdmetrics.reports.single_table import QualityReport
import sys, os

# silence the tqdm and printing functions
tqdm.__init__ = partialmethod(tqdm.__init__, disable=True)
sys.stdout = open(os.devnull, 'w')

# generate the report
real_data, synthetic_data, metadata = load_demo(modality='single_table')
report = QualityReport()
report.generate(real_data, synthetic_data, metadata)

[alejandrofdez-us]

Dear npatki,
thanks for your response. I've already tried this workaround, but it still prints the summary in console. For instance, in your example it prints the following:

Overall Quality Score: 82.84%

Properties:
Column Shapes: 82.78%
Column Pair Trends: 82.9%

Fortunately, at least tqdm is disabled with your proposal, but sdv manages to get the real stdout somehow.

[alejandrofdez-us]

Apart from reporting, I would add that the sampling generation should have a silence mode as well.

[npatki]

Hi @alejandrofdez-us,

thanks for your response. I've already tried this workaround, but it still prints the summary in console.

You're welcome! Based on how exactly you are running the Python script, there may be other solutions to this. For example, here is another option.

Apart from reporting, I would add that the sampling generation should have a silence mode as well.

You are welcome to file a new issue for this in the SDV library. We can use it to separately track and provide updates for it.

[npatki]

Update! This is on the roadmap for the next SDMetrics release.

Both quality and diagnostic reports will include a verbose parameter that will be True by default but can be toggled.

report = QualityReport()
report.generate(real_data, synthetic_data, metadata, verbose=False)

report = DiagnosticReport()
report.generate(real_data, synthetic_data, metadata, verbose=False)