Skip to content
No description, website, or topics provided.
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
data
multiple
single
.ruby-version
Gemfile
Gemfile.lock
README.md
base_experiment.rb
common.rb
experiment_runner.rb
generate_file.rb
multiple_experiments.rb
single_experiments.rb

README.md

Experiments for processing big text files with Ruby

This are a set of experiments for testing the performance of approaches for processing big ruby text files.

Read the companion blog post

Single file

Name Time Memory
InMemoryExperiment 1.686s 4125MB
InternalIteratorExperiment 1.219s 0MB
EnumeratorExperiment 2.462s 0MB
ExternalIteratorExperiment 1.279s 0MB

Multiple files

Name Time Memory
InMemoryExperiment 3.824s 175.6MB
EnumeratorExperiment 4.919s 0MB
ExternalIteratorExperiment 3.465s 0MB
ThreadsAndQueuesExperiment 4.936s 0MB

Run the tests

ruby single_experiments.rb # run single-file processing experiments
ruby mutiple_experiments.rb # run multiple-file processing experiments

If you want to generate synthetic text files:

ruby generate_file <file name> <number of lines to generate>
You can’t perform that action at this time.