Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Clone this wiki locally
Random perf numbers (until we have a perfbot)
This test imports one day of MLB pitch data into noms using xml_importer.
Numbers indicate time (as measured by
time) to write to an empty directory vs. one that already contains the necessary tiered directory structure that FileStore uses (but not the files themselves). A "Dumb copy" entails opening every file in a file store, reading every byte, and writing every byte to a new file in another location. This is intended to mimic a best-case noms importer that doesn't need to do any processing at all, but does need to create the same number of files and populate them with the same data.
- Dumb copy: 3.5s/2.4s
- xml_importer: 8.5s/6.2s
We also measured the time to import the same day worth of data to AWSStore. Eep:
- xml_importer: 35m32s
- pitchmap/index: 10m48s