Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix #5: Support for very large revisions and very large files. #9

Merged
merged 54 commits into from
Oct 5, 2015

Commits on Sep 30, 2015

  1. Configuration menu
    Copy the full SHA
    7fc640d View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    1e90fdb View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    bf2bdd1 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    65b5c0d View commit details
    Browse the repository at this point in the history
  5. Incremental change to make the processing of SVN dump files more fine…

    … grained (down from SvnRevision to SvnNode).
    
    * Move TerminatingValidator out of test source into main source, as it might be useful.
    cstroe committed Sep 30, 2015
    Configuration menu
    Copy the full SHA
    5445ed2 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    718f3d9 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    eb2f2cb View commit details
    Browse the repository at this point in the history
  8. SvnDumpFileParser now passes the Nodes on to the Consumer, instead of…

    … having the Consumer do that.
    
    * SvnDumpFileParser will not add nodes to an SvnRevision, as those should be consumed separately.
    * Update SvnDumpWriterImpl and SvnDumpInMemory to work with this new way.
    cstroe committed Sep 30, 2015
    Configuration menu
    Copy the full SHA
    ddf295b View commit details
    Browse the repository at this point in the history
  9. Update PathCollisionTest to test through the SvnDumpFileParser, inste…

    …ad of getting the SvnDump as a whole.
    cstroe committed Sep 30, 2015
    Configuration menu
    Copy the full SHA
    c5fc80b View commit details
    Browse the repository at this point in the history

Commits on Oct 1, 2015

  1. Configuration menu
    Copy the full SHA
    d7cab01 View commit details
    Browse the repository at this point in the history
  2. Update ClearRevisionTest to work with the new parser.

    Cosmin Stroe committed Oct 1, 2015
    Configuration menu
    Copy the full SHA
    aa1d380 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    8a01cfa View commit details
    Browse the repository at this point in the history
  4. Teach SvnDumpConsumers how to form a consumer chain.

    * Previously, this was implemented as a separate class, ConsumerChain.
    * Because of the granularity changes to SvnDumpFileParser to parse SvnNodes separately from their SvnRevision, many SvnDumpConsumers can no longer properly operate.
    * Instead of teaching the ConsumerChain how to represent and handle SvnNode additions/deletions that may come from an SvnDumpMutator, I'm teaching SvnDumpConsumers how to chain to each other.
    * This removes the need for a separate ConsumerChain class.
    * This also places the burder of continuing the consumer chain on each consumer, which is a downside of this approach.
    cstroe committed Oct 1, 2015
    Configuration menu
    Copy the full SHA
    c030ffa View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    d3e897b View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    66015ca View commit details
    Browse the repository at this point in the history
  7. Updated tests for UpdateAuthorForEmptyRevisions and fixed UpdateAutho…

    …rForEmptyRevisions to properly chain.
    cstroe committed Oct 1, 2015
    Configuration menu
    Copy the full SHA
    8e96e14 View commit details
    Browse the repository at this point in the history
  8. Fix NodeAdd.

    cstroe committed Oct 1, 2015
    Configuration menu
    Copy the full SHA
    8f5d084 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    5a8b11f View commit details
    Browse the repository at this point in the history
  10. Fix for NodeHeaderChange.

    cstroe committed Oct 1, 2015
    Configuration menu
    Copy the full SHA
    1730b1c View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    184eaf8 View commit details
    Browse the repository at this point in the history

Commits on Oct 2, 2015

  1. Fix ConsumerChainTest to use correct consume method.

    Cosmin Stroe committed Oct 2, 2015
    Configuration menu
    Copy the full SHA
    846e1b7 View commit details
    Browse the repository at this point in the history
  2. Use the same file for before and after assertions.

    Cosmin Stroe committed Oct 2, 2015
    Configuration menu
    Copy the full SHA
    8cb8c0b View commit details
    Browse the repository at this point in the history
  3. Fix PathChange.

    Cosmin Stroe committed Oct 2, 2015
    Configuration menu
    Copy the full SHA
    758c686 View commit details
    Browse the repository at this point in the history
  4. SvnDumpRevisions should keep the chain going.

    Cosmin Stroe committed Oct 2, 2015
    Configuration menu
    Copy the full SHA
    d395df3 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    c11a0a4 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    793616e View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    5952087 View commit details
    Browse the repository at this point in the history
  8. Remove warnings. Minor refactor of test.

    Cosmin Stroe committed Oct 2, 2015
    Configuration menu
    Copy the full SHA
    116e852 View commit details
    Browse the repository at this point in the history
  9. finish() should keep the chain going. Refactor test.

    Cosmin Stroe committed Oct 2, 2015
    Configuration menu
    Copy the full SHA
    3a32507 View commit details
    Browse the repository at this point in the history
  10. SvnDumpInMemory should keep the chain going.

    Cosmin Stroe committed Oct 2, 2015
    Configuration menu
    Copy the full SHA
    fa35973 View commit details
    Browse the repository at this point in the history
  11. SvnDumpFileParserDoppleganger assumes some of the responsibility enco…

    …ded in SvnDumpFileParserTest.consume
    Cosmin Stroe committed Oct 2, 2015
    Configuration menu
    Copy the full SHA
    cf766f8 View commit details
    Browse the repository at this point in the history
  12. Been meaning to get rid of this for a while.

    Cosmin Stroe committed Oct 2, 2015
    Configuration menu
    Copy the full SHA
    a65f64e View commit details
    Browse the repository at this point in the history
  13. Use the doppelganger instead!

    Cosmin Stroe committed Oct 2, 2015
    Configuration menu
    Copy the full SHA
    a5d872b View commit details
    Browse the repository at this point in the history
  14. Add convenience method.

    Cosmin Stroe committed Oct 2, 2015
    Configuration menu
    Copy the full SHA
    603ac0f View commit details
    Browse the repository at this point in the history
  15. Another convenience method.

    Cosmin Stroe committed Oct 2, 2015
    Configuration menu
    Copy the full SHA
    b7b3799 View commit details
    Browse the repository at this point in the history
  16. Use the doppelganger.

    Cosmin Stroe committed Oct 2, 2015
    Configuration menu
    Copy the full SHA
    b7da701 View commit details
    Browse the repository at this point in the history
  17. Let's parse files while we're at it.

    Cosmin Stroe committed Oct 2, 2015
    Configuration menu
    Copy the full SHA
    5bf9e48 View commit details
    Browse the repository at this point in the history
  18. Use the doppelganger.

    Cosmin Stroe committed Oct 2, 2015
    Configuration menu
    Copy the full SHA
    3235b1f View commit details
    Browse the repository at this point in the history
  19. Use the doppelganger.

    Cosmin Stroe committed Oct 2, 2015
    Configuration menu
    Copy the full SHA
    be0c357 View commit details
    Browse the repository at this point in the history
  20. Use the doppelganger.

    Cosmin Stroe committed Oct 2, 2015
    Configuration menu
    Copy the full SHA
    bdc4965 View commit details
    Browse the repository at this point in the history
  21. Copy constructors for everyone!

    Cosmin Stroe committed Oct 2, 2015
    Configuration menu
    Copy the full SHA
    243d539 View commit details
    Browse the repository at this point in the history
  22. Expose InputStream consume.

    Cosmin Stroe committed Oct 2, 2015
    Configuration menu
    Copy the full SHA
    a7379a0 View commit details
    Browse the repository at this point in the history
  23. Use the doppelganger!

    Cosmin Stroe committed Oct 2, 2015
    Configuration menu
    Copy the full SHA
    7a20dd1 View commit details
    Browse the repository at this point in the history
  24. Assert on the order of consumption.

    Cosmin Stroe committed Oct 2, 2015
    Configuration menu
    Copy the full SHA
    c197c3a View commit details
    Browse the repository at this point in the history
  25. Fix test.

    Cosmin Stroe committed Oct 2, 2015
    Configuration menu
    Copy the full SHA
    99e39b2 View commit details
    Browse the repository at this point in the history
  26. Finish granularity change. We new process SvnNodes one at a time, ind…

    …ependently from other nodes in the revision.
    
    * This means we don't have to hold all the nodes in memory at the same time, just one node.
    * This should fix the "SvnRevision >30GB" requirement.
    Cosmin Stroe committed Oct 2, 2015
    Configuration menu
    Copy the full SHA
    9955bbd View commit details
    Browse the repository at this point in the history
  27. A descriptive to string on SvnRevision.

    Cosmin Stroe committed Oct 2, 2015
    Configuration menu
    Copy the full SHA
    3b2c03a View commit details
    Browse the repository at this point in the history

Commits on Oct 3, 2015

  1. A descriptive to string on SvnDumpPreamble.

    Cosmin Stroe committed Oct 3, 2015
    Configuration menu
    Copy the full SHA
    6116ae3 View commit details
    Browse the repository at this point in the history
  2. Add convenience method for reading from an InputStream.

    Cosmin Stroe committed Oct 3, 2015
    Configuration menu
    Copy the full SHA
    fe70ae2 View commit details
    Browse the repository at this point in the history
  3. Adding a FileContentChunk.

    * We're going to split up the file content of SvnNodes into smaller pieces, so that we don't have to store the entire file content in memory.
    Cosmin Stroe committed Oct 3, 2015
    Configuration menu
    Copy the full SHA
    dd82cc8 View commit details
    Browse the repository at this point in the history
  4. Towards using FileContentChunks.

    Cosmin Stroe committed Oct 3, 2015
    Configuration menu
    Copy the full SHA
    3a2755b View commit details
    Browse the repository at this point in the history

Commits on Oct 5, 2015

  1. Correct assertion message.

    Cosmin Stroe committed Oct 5, 2015
    Configuration menu
    Copy the full SHA
    cc28dce View commit details
    Browse the repository at this point in the history
  2. Parse SvnNode file content of SVN dump using one FileContentChunk.

    * Add ending calls to SvnDumpConsumer: endRevision(...), endNode(...), and endChunks().  Makes SvnDumpWriterImpl code much cleaner and clearer.
    Cosmin Stroe committed Oct 5, 2015
    Configuration menu
    Copy the full SHA
    9f82ff5 View commit details
    Browse the repository at this point in the history
  3. File content is now processed in chunks (4 MB chunks by default). This

    …fixes #5.
    
    * You can change the chunk size by calling setFileContentChunkSize(long) on the SvnDumpFileParser.
    Cosmin Stroe committed Oct 5, 2015
    Configuration menu
    Copy the full SHA
    a2f7163 View commit details
    Browse the repository at this point in the history