Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/10385 loadnexus speed #56

Merged
merged 17 commits into from
Nov 20, 2014
Merged

Commits on Oct 23, 2014

  1. refs #10385. Extract spectrum info to object.

    This is the first step towards reusing the same spectrum information rather than fetching it renewed each time.
    One caveat with this is that you will HAVE to be sure that all workspaces in the group are equivalent in terms of instrument detector mappings etc.
    What i intend to do next is:
    
    1) For workspaces that look to relate to multiperiod group workspaces, default to reuse the spectrum-detectector mappings, once obtained on a lazy basis.
    2) Put a boolean property with value in place (which defaults to true) called "PredictiveInstrumentLoading". If false, we would go back to old behaviour.
    OwenArnold committed Oct 23, 2014
    Configuration menu
    Copy the full SHA
    1978e14 View commit details
    Browse the repository at this point in the history
  2. refs #10385. Naming issue root problem.

    There is a naming issue here, which is actually the root cause of the problem! The query that was being performed retrieved the number of workspace entries, NOT the number of periods.
    Going forward. If the number of periods exactly matches the number of workspace entries, we have a multiperiod group workspace, and can happily apply the optimisations we want.
    
    Next step will be to actually extract the nperiods. To do that we need to try to fetch it out of the logs for The first entry.
    OwenArnold committed Oct 23, 2014
    Configuration menu
    Copy the full SHA
    fdf6281 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    d0ba5d5 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    8a0c599 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    61959e5 View commit details
    Browse the repository at this point in the history

Commits on Oct 29, 2014

  1. Configuration menu
    Copy the full SHA
    9a67731 View commit details
    Browse the repository at this point in the history
  2. refs #10385. Test file is twice as fast now.

    There are lots of TODOs still to resolve. We also need to add tests for this sort of multiperiod loading.
    OwenArnold committed Oct 29, 2014
    Configuration menu
    Copy the full SHA
    ccc1c23 View commit details
    Browse the repository at this point in the history
  3. refs #10385. Add failing test.

    We are not currently handling log extraction properly, which is why the test is failing.
    
    File size is just over the threshold, but shouldn't be a problem. I've tried to trim it down as much as possible.
    OwenArnold committed Oct 29, 2014
    Configuration menu
    Copy the full SHA
    757b004 View commit details
    Browse the repository at this point in the history

Commits on Oct 30, 2014

  1. refs #10385. Remove log values

    This fixes the regression test. We clear the logs off the temporary workspace, that way we don't have to do a similar thing in the loop, and we avoid copying them over each time. This is not a fully formed solution, but it's a start.
    OwenArnold committed Oct 30, 2014
    Configuration menu
    Copy the full SHA
    ab1da07 View commit details
    Browse the repository at this point in the history

Commits on Oct 31, 2014

  1. refs #10385. Reporting added and extra property.

    Also better robustness around workspace type.
    OwenArnold committed Oct 31, 2014
    Configuration menu
    Copy the full SHA
    8e03062 View commit details
    Browse the repository at this point in the history
  2. refs #10385 Refactor new functionality to method.

    Should read better now, although the entire algorithm is still a mess.
    OwenArnold committed Oct 31, 2014
    Configuration menu
    Copy the full SHA
    aba603d View commit details
    Browse the repository at this point in the history
  3. refs #10385. Remove instrument info caching

    This does result in a speed up, but the caching is being bypassed at the moment since our strategy is to clone and overwrite. I've removed the caching code as part of the clean up, but I've kept the SpecInfo type around as it makes the usage clearer, and does not result in a performance penalty.
    OwenArnold committed Oct 31, 2014
    Configuration menu
    Copy the full SHA
    d88bbe4 View commit details
    Browse the repository at this point in the history
  4. refs #10385. Minor changes.

    OwenArnold committed Oct 31, 2014
    Configuration menu
    Copy the full SHA
    5b18d28 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    fb70f73 View commit details
    Browse the repository at this point in the history
  6. refs #10385 Extend tests.

    Extend tests. Do not process spectrum lists.
    OwenArnold committed Oct 31, 2014
    Configuration menu
    Copy the full SHA
    ce10e43 View commit details
    Browse the repository at this point in the history
  7. refs #10385. Fix warnings.

    OwenArnold committed Oct 31, 2014
    Configuration menu
    Copy the full SHA
    e084e3b View commit details
    Browse the repository at this point in the history

Commits on Nov 3, 2014

  1. Configuration menu
    Copy the full SHA
    679c839 View commit details
    Browse the repository at this point in the history