You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Sampling the full number of rows is most efficient, but batch sampling is a useful feature for progress tracking & memory consumption. Let's enable batch sampling for each of our sampling methods (#690, #691, #692).
Expected behavior
In sample():
Add batch_size param (default: same as num_rows to yield only 1 batch)
In sample_conditions() and sample_remaining_columns():
Batch size determined by existing batch_size_per_try param
For all methods:
Add output_file_path: Name of file to write to (default: None)
Show progress bar when sampling
Periodically write to output_file_path. If None, then periodically write to a temp file.
# works for all methods: sample, sample_conditions, sample_remaining_columns# show progress bar while sampling# write to a temp file that we can later delete>>>synthetic_data=model.sample(num_rows=1000)
76%|████████████████████████████ |756/1000 [00:33<00:10, 229.00it/s]
# write to file path while also returning the samples; show progress>>>synthetic_data=model.sample(num_rows=1000, output_file_path="./results/sample.csv")
76%|████████████████████████████ |756/1000 [00:33<00:10, 229.00it/s]
Error States
When the system crashes or the user exits in the middle of sampling.
# works for all methods: sample, sample_conditions, sample_remaining_columns# Partial results available in requested file path>>>synthetic_data=model.sample(output_file_path='./results/synthetic.csv')
76%|████████████████████████████ |756/1000 [00:33<00:10, 229.00it/s]
^CError: Samplingterminated. Partialresultsarestoredin'./results/synthetic.csv'# If no file path, partial results are in a temp file# Temp file will be overwritten on next sample, so tell the user to save it>>>synthetic_data=model.sample()
^CError: Samplingterminated. Partialresultsarestoredinatemporaryfile: '.sample.csv.temp'.
Thisfilewillbeoverriddenthenexttimeyousample. Renamethefileifyouwishtosavetheseresults.
The text was updated successfully, but these errors were encountered:
Problem Description
Sampling the full number of rows is most efficient, but batch sampling is a useful feature for progress tracking & memory consumption. Let's enable batch sampling for each of our sampling methods (#690, #691, #692).
Expected behavior
In
sample()
:batch_size
param (default: same asnum_rows
to yield only 1 batch)In
sample_conditions()
andsample_remaining_columns()
:batch_size_per_try
paramFor all methods:
output_file_path
: Name of file to write to (default:None
)output_file_path
. IfNone
, then periodically write to a temp file.Error States
When the system crashes or the user exits in the middle of sampling.
The text was updated successfully, but these errors were encountered: