Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
Replace list with generator when iterating headers #4388
Beautiful - thanks!
For what it is worth, I believe the amount of memory can still grow unbounded because of this line:
It should be replaced by something that reads in lines not exceeding 10 Kb or something like that.
In fact - I'd take caching that first 10 Kb and not using the iterator over re-reading from the disk over and over. Caching the headers and using an iterator though is the best combination I suspect so this is still a very solid step in the right direction.