Permalink
Commits on Sep 22, 2016
  1. Merge pull request #467 from alphagov/add-patch-to-write-api

    Add patch to write api
    leenagupte committed on GitHub Sep 22, 2016
Commits on Sep 21, 2016
  1. Add PATCH route to write API

    leelongmore committed Sep 14, 2016
Commits on Sep 14, 2016
  1. Fix replicate-db script

    leelongmore committed Sep 14, 2016
Commits on Jul 12, 2016
  1. Merge pull request #466 from alphagov/data-migration

    Data migration to remove empty summary records
    leelongmore committed on GitHub Jul 12, 2016
  2. Data migration to remove empty summary records

    Empty summary records have been introduced due,
    we think, to an empty row in the TxEx Google
    spreadsheet. These records are causing the
    transform of the summary records into the service
    aggregate records to fail.
    leelongmore committed Jul 12, 2016
Commits on Mar 3, 2016
  1. Merge pull request #465 from alphagov/return-user-friendly-date-error…

    …-messages
    
    Return a more human friendly date error message
    leelongmore committed Mar 3, 2016
  2. Return a more human friendly date error message

    Return a different error message for errors where the date is not a
    date and for where the date is not a valid date.
    leenagupte committed Mar 3, 2016
Commits on Feb 29, 2016
  1. Merge pull request #464 from alphagov/fix-curl-in-readme

    Correct curl command in README
    leelongmore committed Feb 29, 2016
  2. Correct curl command in README

    leenagupte committed with leelongmore Feb 29, 2016
Commits on Feb 10, 2016
  1. Merge pull request #463 from alphagov/add-service-id-to-dataset

    Add service_id to service_aggregates_latest_dataset_values if missing
    leelongmore committed Feb 10, 2016
  2. Add service_id to service_aggregates_latest_dataset_values if missing

    leenagupte committed Feb 10, 2016
Commits on Feb 9, 2016
  1. Merge pull request #462 from alphagov/update-replicate-script

    Update replicate_db script with new puppet repo details
    leelongmore committed Feb 9, 2016
  2. Update replicate_db script with new puppet repo details

    leenagupte committed Feb 9, 2016
Commits on Jan 28, 2016
  1. Merge pull request #461 from alphagov/set-failover

    Set the celery failover strategy to round-robin
    leelongmore committed Jan 28, 2016
  2. Set the celery failover strategy to round-robin

    We made a change to allow the broker url to accept a list of rabbitmq
    nodes. When node 1 was taken down, the celery process did not connect to
    the next node in the list.
    
    The default failover strategy for kombu is round-robin, but we
    believe that celery is overwriting this and setting it to none if the
    failover strategy is not explictly configured in the application
    config.
    leenagupte committed Jan 28, 2016
Commits on Jan 14, 2016
  1. Merge pull request #444 from alphagov/decrease-graphite-disk-usage

    Remove data group and type from statsd timer name
    tombooth committed Jan 14, 2016
Commits on Dec 18, 2015
  1. Merge pull request #460 from alphagov/update-db-location

    Update instructions for the replicate-db script.
    leelongmore committed Dec 18, 2015
  2. Update instructions for the replicate-db script.

    leenagupte committed Dec 18, 2015
Commits on Dec 11, 2015
  1. Merge pull request #459 from alphagov/add-service-id

    Add service_id to datasets of type latest_dataset_value
    leelongmore committed Dec 11, 2015
  2. Add service_id to datasets of type latest_dataset_value

    The import_dashboards.py script in Stagecraft (used for
    importing Transactions Explorer data) expects to
    find a service_id key in the service aggregates dataset.
    This key was missed when this transform was produced and
    is causing import_dashboards.py to fail.
    leenagupte committed Dec 11, 2015
Commits on Dec 9, 2015
  1. Merge pull request #458 from alphagov/change-dev-to-match-other-envs

    Change dev to match other envs
    leelongmore committed Dec 9, 2015
Commits on Dec 7, 2015
  1. Merge pull request #438 from alphagov/get-latest-tx-data

    Get latest tx data
    leelongmore committed Dec 7, 2015
Commits on Dec 2, 2015
  1. Change development config to match other environments

    The spike into Services page data anomalies has uncovered that the
    configuration for transform in the development differs from that in
    preview, staging in production. This means that you can't just start
    backdrop and curl the transform end point update a transform data set.
    
    In all of the other environments both backdrop write and transform use
    user backdrop_write and vhost /backdrop_write for the queue config.
    In development, backdrop write uses user backdrop_write and vhost /backdrop_write
    for the queue config, but transforms uses user transformer and vhost /transformations.
    
    To successfully trigger a transform in development with this set up I had to create
    the transformer user and /transformations vhost manually in rabbitmq.
    
    To make life easier for future developers, the transform config has been changed
    to match backdrop_write. This change won't affect how transform run in the
    other environments.
    leenagupte committed Dec 2, 2015
  2. Update README

    1. Update the database replication instructions
    
    2. Add a note about where to find RabbitMQ logs
    
    Fix formatting to be PEP8 compliant.
    leenagupte committed Dec 2, 2015
  3. Merge pull request #455 from alphagov/encode-unicode-strings-in-record

    Encode unicode strings in dataset
    leelongmore committed Dec 2, 2015
Commits on Dec 1, 2015
  1. Merge pull request #456 from alphagov/update-dev-transform-config

    Update transform config in development
    leelongmore committed Dec 1, 2015
  2. Update transform config in development

    leenagupte committed Dec 1, 2015
Commits on Nov 26, 2015
  1. Encode unicode strings in dataset

    If the dataset pushed to backdrop contains any unicode characters,
    a 500 error is returned to the user.
    
    All of the top-level strings in the dataset have been encoded as utf-8
    before continuing processing and validating of the dataset.
    
    The check is limited to only the top-level strings in a dictionary
    as currently all datasets pushed to backdrop are flat.
    leenagupte committed Nov 12, 2015
Commits on Nov 9, 2015
  1. Revert 6bbf316

    leenagupte committed Nov 9, 2015
Commits on Oct 28, 2015
  1. Merge pull request #450 from alphagov/only-allow-utc-timestamp-in-ids

    Format timestamp before creating id field
    leelongmore committed Oct 28, 2015
  2. Format timestamp before creating id field

    Slight differences in the way the _timestamp cell is formatted (in
    file uploads) results in duplicated data in the dataset.
    
    What’s happening is this: _timestamp is listed as an auto_id for
    the data set. All auto_id fields and values are joined together and
    then base64 encoded and stored in the _id field.
    
    AFTER the _id field has been created for the data record, the
    _timestamp is parsed and converted to utc format. This means that
    the date format of the timestamp can differ from the date format of
    the timestamp in the _id.
    
    The _id is used as a unique key for the data set.
    The same _timestamp with a different date format will generate a
    different _id value and will added as a new row in backdrop.
    
    The fix was just to move converting the _timestamp to utc format
    to BEFORE the base64 encoded _id is generated.
    leenagupte committed Oct 28, 2015
Commits on Oct 13, 2015
  1. Remove statsd timer

    These stats are taking up approximately 80GB of disk space, because
    each data group and type combination will create a new whisper file. We
    dont need to know in this much detail how long the endpoints are taking
    to respond, and we can get the general information from nginx logs
    timmow committed Sep 18, 2015
  2. Merge pull request #446 from alphagov/update-url-for-govuk-dev-vm

    Update stagecraft url to work on gov.uk dev vm
    carolinegreen committed Oct 13, 2015
  3. Update stagecraft url to work on gov.uk dev vm

    The port number needs updating in the feature tests as well to work
    correctly
    timmow committed Oct 9, 2015