Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make Data Count support: backend #4821 #5329

Merged
merged 95 commits into from
Mar 15, 2019
Merged

Make Data Count support: backend #4821 #5329

merged 95 commits into from
Mar 15, 2019

Commits on Nov 19, 2018

  1. stub our docs and API for Make Data Count #4821

    @pdurbin
    pdurbin committed Nov 19, 2018
    Configuration menu
    Copy the full SHA
    4dd10bd View commit details
    Browse the repository at this point in the history

Commits on Nov 29, 2018

  1. clarify that we'll send citations to DataCite #4821

    @pdurbin
    pdurbin committed Nov 29, 2018
    Configuration menu
    Copy the full SHA
    17cbf37 View commit details
    Browse the repository at this point in the history

Commits on Dec 6, 2018

  1. link to HTML version of CoP #4821

    @pdurbin
    pdurbin committed Dec 6, 2018
    Configuration menu
    Copy the full SHA
    5f59a8c View commit details
    Browse the repository at this point in the history

Commits on Dec 7, 2018

  1. add "datasetmetrics" table #4821

    @pdurbin
    pdurbin committed Dec 7, 2018
    Configuration menu
    Copy the full SHA
    e8ac623 View commit details
    Browse the repository at this point in the history

Commits on Dec 10, 2018

  1. stub out counter-processor setup in Vagrant #4821 #5385

    @pdurbin
    pdurbin committed Dec 10, 2018
    Configuration menu
    Copy the full SHA
    02c5538 View commit details
    Browse the repository at this point in the history

Commits on Dec 11, 2018

  1. stand up counter-processor, continued #4821

    @pdurbin
    pdurbin committed Dec 11, 2018
    Configuration menu
    Copy the full SHA
    e835e06 View commit details
    Browse the repository at this point in the history

Commits on Dec 12, 2018

  1. add SUSHI JSON example and stub out parsing of it #4821

    @pdurbin
    pdurbin committed Dec 12, 2018
    Configuration menu
    Copy the full SHA
    9f7c990 View commit details
    Browse the repository at this point in the history

Commits on Dec 21, 2018

  1. add draft architecture drawing #4821

    @pdurbin
    pdurbin committed Dec 21, 2018
    Configuration menu
    Copy the full SHA
    ac5e29d View commit details
    Browse the repository at this point in the history

Commits on Jan 2, 2019

  1. Merge branch 'develop' into 4821-make-data-count

    matthew-a-dunlap committed Jan 2, 2019
    Configuration menu
    Copy the full SHA
    b76d92c View commit details
    Browse the repository at this point in the history

  2. Merge branch '5384-data-usage-logs-MDC' into 4821-make-data-count

    matthew-a-dunlap committed Jan 2, 2019
    Configuration menu
    Copy the full SHA
    8d1c6ab View commit details
    Browse the repository at this point in the history

  3. Temp fix null erroring for MDC #5384 #4821

    matthew-a-dunlap committed Jan 2, 2019
    Configuration menu
    Copy the full SHA
    af728cf View commit details
    Browse the repository at this point in the history

  4. MDC log var collection in construct #5384 #4821

    matthew-a-dunlap committed Jan 2, 2019
    Configuration menu
    Copy the full SHA
    4416b41 View commit details
    Browse the repository at this point in the history

Commits on Jan 4, 2019

  1. MDC log dl with guestbook #4821 #5384 

    Lots of comments in this commit, will clean up soon
    matthew-a-dunlap committed Jan 4, 2019
    Configuration menu
    Copy the full SHA
    dc11ec1 View commit details
    Browse the repository at this point in the history

  2. #4821 parse sample json add dummy data api

    @sekmiller
    sekmiller committed Jan 4, 2019
    Configuration menu
    Copy the full SHA
    e36c0ed View commit details
    Browse the repository at this point in the history

  3. #4821 remove extraneous test code

    @sekmiller
    sekmiller committed Jan 4, 2019
    Configuration menu
    Copy the full SHA
    61f78d7 View commit details
    Browse the repository at this point in the history

  4. #4821 check for existing metrics records

    @sekmiller
    sekmiller committed Jan 4, 2019
    Configuration menu
    Copy the full SHA
    0d38f10 View commit details
    Browse the repository at this point in the history

  5. MDC access api #4821 #5384

    matthew-a-dunlap committed Jan 4, 2019
    Configuration menu
    Copy the full SHA
    c035dbd View commit details
    Browse the repository at this point in the history

  6. Merge branch '4821-make-data-count' of https://github.com/IQSS/dataverse 

     into 4821-make-data-count
    matthew-a-dunlap committed Jan 4, 2019
    Configuration menu
    Copy the full SHA
    85bb860 View commit details
    Browse the repository at this point in the history

  7. MDC File page and correct API #4821 #5384

    matthew-a-dunlap committed Jan 4, 2019
    Configuration menu
    Copy the full SHA
    e41ab08 View commit details
    Browse the repository at this point in the history

  8. MDC daily logs and cleanup #4821 #5384 

    Next step is to pipe these into counter processor and see what happens on the MDC server
    matthew-a-dunlap committed Jan 4, 2019
    Configuration menu
    Copy the full SHA
    c17e3e2 View commit details
    Browse the repository at this point in the history

Commits on Jan 7, 2019

  1. MDC Path to log files #4821 #5384

    matthew-a-dunlap committed Jan 7, 2019
    Configuration menu
    Copy the full SHA
    0bfd936 View commit details
    Browse the repository at this point in the history

  2. change addDummyData to addUsageMetricsFromSushiReport #4821

    @pdurbin
    pdurbin committed Jan 7, 2019
    Configuration menu
    Copy the full SHA
    184580d View commit details
    Browse the repository at this point in the history

  3. MDC log dl reqUrl etc part1 #4821 #5384 

    This is incomplete as most downloads don't log to guestbook until the end via the  DownloadInstanceWriter and it is unclear how to get this info that late in the pipe.
    matthew-a-dunlap committed Jan 7, 2019
    Configuration menu
    Copy the full SHA
    bd7ff84 View commit details
    Browse the repository at this point in the history

  4. Merge branch '4821-make-data-count' of https://github.com/IQSS/dataverse 

     into 4821-make-data-count
    matthew-a-dunlap committed Jan 7, 2019
    Configuration menu
    Copy the full SHA
    b60a679 View commit details
    Browse the repository at this point in the history

Commits on Jan 8, 2019

  1. stub out citation parser #4821

    @pdurbin
    pdurbin committed Jan 8, 2019
    Configuration menu
    Copy the full SHA
    94eccc7 View commit details
    Browse the repository at this point in the history

  2. MDC log dl reqUrl etc part2 #4821 #5384 

    Access api & DownloadInstanceWriter now have the info needed to create an MDC entry. This included adding a new constructor to for MDC logging that takes uriInfo and headers.
    matthew-a-dunlap committed Jan 8, 2019
    Configuration menu
    Copy the full SHA
    51b4ed2 View commit details
    Browse the repository at this point in the history

  3. download citations from DataCite hub #4821

    @pdurbin
    pdurbin committed Jan 8, 2019
    Configuration menu
    Copy the full SHA
    f9b0424 View commit details
    Browse the repository at this point in the history

  4. add note about multiple Glassfish servers, other doc tweaks #4821

    @pdurbin
    pdurbin committed Jan 8, 2019
    Configuration menu
    Copy the full SHA
    3ad3918 View commit details
    Browse the repository at this point in the history

  5. MDC - massaging DV logs into CP #4821 #5384 

    At this point counterprocessor with our custom regex is readable by Counter Processor and generates sushi. Right now it is not reporting ANY unique investigations and almost certainly should. More investigation is needed.
    matthew-a-dunlap committed Jan 8, 2019
    Configuration menu
    Copy the full SHA
    0e5bbd5 View commit details
    Browse the repository at this point in the history

  6. MDC restore historic regex for testing #4821 #5384

    matthew-a-dunlap committed Jan 8, 2019
    Configuration menu
    Copy the full SHA
    9e336ad View commit details
    Browse the repository at this point in the history

Commits on Jan 9, 2019

  1. Merge branch 'develop' into 4821-make-data-count

    matthew-a-dunlap committed Jan 9, 2019
    Configuration menu
    Copy the full SHA
    87b7dbd View commit details
    Browse the repository at this point in the history

  2. MDC use possibly bad publisher info to sumbit report #4821 #5384

    matthew-a-dunlap committed Jan 9, 2019
    Configuration menu
    Copy the full SHA
    69d48c7 View commit details
    Browse the repository at this point in the history

Commits on Jan 10, 2019

  1. #4821 removing citations from metrics table

    @sekmiller
    sekmiller committed Jan 10, 2019
    Configuration menu
    Copy the full SHA
    830dc61 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'develop' into 4821-make-data-count

    @sekmiller
    sekmiller committed Jan 10, 2019
    Configuration menu
    Copy the full SHA
    5c0585b View commit details
    Browse the repository at this point in the history

Commits on Jan 11, 2019

  1. #4821 Add External Citations table

    @sekmiller
    sekmiller committed Jan 11, 2019
    Configuration menu
    Copy the full SHA
    3947167 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'develop' into 4821-make-data-count

    @sekmiller
    sekmiller committed Jan 11, 2019
    Configuration menu
    Copy the full SHA
    5e0e2e6 View commit details
    Browse the repository at this point in the history

Commits on Jan 14, 2019

  1. add new SUSHI file to test with #4821 

    sushi_sample_logs2.json was created by deploying Dataverse (commit
5e0e2e6) to EC2 along with Counter Processor 0.0.1.
    @pdurbin
    pdurbin committed Jan 14, 2019
    Configuration menu
    Copy the full SHA
    004e16a View commit details
    Browse the repository at this point in the history

Commits on Jan 15, 2019

  1. allow year and country to be specified #4821

    @pdurbin
    pdurbin committed Jan 15, 2019
    Configuration menu
    Copy the full SHA
    e162839 View commit details
    Browse the repository at this point in the history

  2. #4821 fix json parser

    @sekmiller
    sekmiller committed Jan 15, 2019
    Configuration menu
    Copy the full SHA
    00d85ff View commit details
    Browse the repository at this point in the history

Commits on Jan 16, 2019

  1. #4821 fix global id parsing

    @sekmiller
    sekmiller committed Jan 16, 2019
    Configuration menu
    Copy the full SHA
    a4e9bf6 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'develop' into 4821-make-data-count #4821

    @pdurbin
    pdurbin committed Jan 16, 2019
    Configuration menu
    Copy the full SHA
    0126de4 View commit details
    Browse the repository at this point in the history

  3. add dev page on Make Data Count #4821

    @pdurbin
    pdurbin committed Jan 16, 2019
    Configuration menu
    Copy the full SHA
    e55574c View commit details
    Browse the repository at this point in the history

Commits on Jan 17, 2019

  1. #4821 fix citation parser

    @sekmiller
    sekmiller committed Jan 17, 2019
    Configuration menu
    Copy the full SHA
    cf3351c View commit details
    Browse the repository at this point in the history

  2. update guide. Counter Processor required #4821

    @pdurbin
    pdurbin committed Jan 17, 2019
    Configuration menu
    Copy the full SHA
    0bd875d View commit details
    Browse the repository at this point in the history

  3. link to MDC scripts, add cd #4821

    @pdurbin
    pdurbin committed Jan 17, 2019
    Configuration menu
    Copy the full SHA
    f28c205 View commit details
    Browse the repository at this point in the history

  4. explain how to install and run Counter Processor #4821 #5440

    @pdurbin
    pdurbin committed Jan 17, 2019
    Configuration menu
    Copy the full SHA
    0b185f0 View commit details
    Browse the repository at this point in the history

Commits on Jan 18, 2019

  1. #4821 Add query to display view/download data

    @sekmiller
    sekmiller committed Jan 18, 2019
    Configuration menu
    Copy the full SHA
    54269d9 View commit details
    Browse the repository at this point in the history

  2. #4821 simplify display query

    @sekmiller
    sekmiller committed Jan 18, 2019
    Configuration menu
    Copy the full SHA
    78b413e View commit details
    Browse the repository at this point in the history

Commits on Jan 25, 2019

  1. (UNRELATED) Added missing export options to docs [#3697] 

    Added DataCite 4 and OAI_ORE to a couple doc pages where they were missing. NOT RELATED TO REST OF BRANCH.
    @dlmurphy
    dlmurphy committed Jan 25, 2019
    Configuration menu
    Copy the full SHA
    c77315e View commit details
    Browse the repository at this point in the history

  2. (UNRELATED) export option doc fix [#3697]

    @dlmurphy
    dlmurphy committed Jan 25, 2019
    Configuration menu
    Copy the full SHA
    d5d5e22 View commit details
    Browse the repository at this point in the history

Commits on Jan 28, 2019

  1. Merge branch 'develop' into 4821-make-data-count

    @sekmiller
    sekmiller committed Jan 28, 2019
    Configuration menu
    Copy the full SHA
    544e05e View commit details
    Browse the repository at this point in the history

  2. stub out Dataset Metrics in User Guide, reorg #4821 

    Also moved user facing API endpoints to API Guide.
    @pdurbin
    pdurbin committed Jan 28, 2019
    Configuration menu
    Copy the full SHA
    916bd87 View commit details
    Browse the repository at this point in the history

  3. Fixing syntax errors [#4821] 

    Fixing some errors that sphinx threw up
    @dlmurphy
    dlmurphy committed Jan 28, 2019
    Configuration menu
    Copy the full SHA
    ec54e0c View commit details
    Browse the repository at this point in the history

  4. Make Data Count doc edits [#4821] 

    Made various edits, syntax changes, clarifications. Commented out all "FIXME"s. Still there, just not visible when the docs are built.
    @dlmurphy
    dlmurphy committed Jan 28, 2019
    Configuration menu
    Copy the full SHA
    e151deb View commit details
    Browse the repository at this point in the history

Commits on Jan 29, 2019

  1. Add note about not fixing publisher/publisher-id #4821 

    The fields are not actually used by DataCite and is not useful to us, so trying to make this work (which seems to require a code change to Counter Processor) is not worth effort.
    matthew-a-dunlap committed Jan 29, 2019
    Configuration menu
    Copy the full SHA
    47f4fda View commit details
    Browse the repository at this point in the history

  2. emphasize that Handle installations can't upload to hub #4821

    @pdurbin
    pdurbin committed Jan 29, 2019
    Configuration menu
    Copy the full SHA
    aacd279 View commit details
    Browse the repository at this point in the history

  3. Working fake publisher/publisher id with new comment #4821

    matthew-a-dunlap committed Jan 29, 2019
    Configuration menu
    Copy the full SHA
    deded79 View commit details
    Browse the repository at this point in the history

Commits on Jan 31, 2019

  1. Test MDC logger #4821

    matthew-a-dunlap committed Jan 31, 2019
    Configuration menu
    Copy the full SHA
    7e59d5d View commit details
    Browse the repository at this point in the history

  2. MDC clean, minor refactor, extra test #4821

    matthew-a-dunlap committed Jan 31, 2019
    Configuration menu
    Copy the full SHA
    76c1181 View commit details
    Browse the repository at this point in the history

  3. add sushi example with "volume" and "machine" #4821 

    We need to upgrade to Counter Processor 0.1 to get "volume".

"machine" can be generated by downloading from wget.
    @pdurbin
    pdurbin committed Jan 31, 2019
    Configuration menu
    Copy the full SHA
    36fa81a View commit details
    Browse the repository at this point in the history

  4. #4821 Add Human vs. Machine metrics

    @sekmiller
    sekmiller committed Jan 31, 2019
    Configuration menu
    Copy the full SHA
    3d7fdcc View commit details
    Browse the repository at this point in the history

Commits on Feb 1, 2019

  1. not ready to process volume yet #4821 

    See CDLUC3/counter-processor#8
    @pdurbin
    pdurbin committed Feb 1, 2019
    Configuration menu
    Copy the full SHA
    8c20419 View commit details
    Browse the repository at this point in the history

Commits on Feb 5, 2019

  1. MDC config tweak and doc edits #4821

    matthew-a-dunlap committed Feb 5, 2019
    Configuration menu
    Copy the full SHA
    1de793a View commit details
    Browse the repository at this point in the history

  2. Document log touching for Make Data Count #4821

    matthew-a-dunlap committed Feb 5, 2019
    Configuration menu
    Copy the full SHA
    73d4388 View commit details
    Browse the repository at this point in the history

Commits on Feb 7, 2019

  1. Merge branch 'develop' into 4821-make-data-count

    @sekmiller
    sekmiller committed Feb 7, 2019
    Configuration menu
    Copy the full SHA
    40a96d2 View commit details
    Browse the repository at this point in the history

Commits on Feb 8, 2019

  1. #4821 add metrics api endpoints

    @sekmiller
    sekmiller committed Feb 8, 2019
    Configuration menu
    Copy the full SHA
    1db160f View commit details
    Browse the repository at this point in the history

Commits on Feb 11, 2019

  1. continue improving guides for Make Data Count #4821

    @pdurbin
    pdurbin committed Feb 11, 2019
    Configuration menu
    Copy the full SHA
    c2b0afd View commit details
    Browse the repository at this point in the history

  2. #4821 fix retrieve metrics api

    @sekmiller
    sekmiller committed Feb 11, 2019
    Configuration menu
    Copy the full SHA
    402513c View commit details
    Browse the repository at this point in the history

  3. only pass persistentId in dev for SUSHI file #4821

    @pdurbin
    pdurbin committed Feb 11, 2019
    Configuration menu
    Copy the full SHA
    d68281b View commit details
    Browse the repository at this point in the history

  4. get APIs working without a date, more docs #4821

    @pdurbin
    pdurbin committed Feb 11, 2019
    Configuration menu
    Copy the full SHA
    96f1fbe View commit details
    Browse the repository at this point in the history

Commits on Feb 12, 2019

  1. expose all available dataset metrics via API #4821

    @pdurbin
    pdurbin committed Feb 12, 2019
    Configuration menu
    Copy the full SHA
    81f8701 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'develop' into 4821-make-data-count #4821

    @pdurbin
    pdurbin committed Feb 12, 2019
    Configuration menu
    Copy the full SHA
    d88c01c View commit details
    Browse the repository at this point in the history

Commits on Feb 14, 2019

  1. Syntax fix (unrelated to issue) 

    Fixed a syntax error that probably came from a different issue.
    @dlmurphy
    dlmurphy committed Feb 14, 2019
    Configuration menu
    Copy the full SHA
    cc3eeba View commit details
    Browse the repository at this point in the history

  2. Typo fix [#4821]

    @dlmurphy
    dlmurphy committed Feb 14, 2019
    Configuration menu
    Copy the full SHA
    c6eab30 View commit details
    Browse the repository at this point in the history

Commits on Feb 20, 2019

  1. Merge branch 'develop' into 4821-make-data-count

    matthew-a-dunlap committed Feb 20, 2019
    Configuration menu
    Copy the full SHA
    5b11ce3 View commit details
    Browse the repository at this point in the history

  2. Guides fixes #4821

    matthew-a-dunlap committed Feb 20, 2019
    Configuration menu
    Copy the full SHA
    78cb180 View commit details
    Browse the repository at this point in the history

Commits on Feb 25, 2019

  1. Merge branch 'develop' into 4821-make-data-count

    @sekmiller
    sekmiller committed Feb 25, 2019
    Configuration menu
    Copy the full SHA
    9fc9b71 View commit details
    Browse the repository at this point in the history

  2. #4821 fix add metrics endpoint for no dataset case

    @sekmiller
    sekmiller committed Feb 25, 2019
    Configuration menu
    Copy the full SHA
    388d69a View commit details
    Browse the repository at this point in the history

  3. #4821 fix doc for add metrics

    @sekmiller
    sekmiller committed Feb 25, 2019
    Configuration menu
    Copy the full SHA
    da71e5c View commit details
    Browse the repository at this point in the history

  4. doc improvements #4821 

    Switch from /home to /usr/local.

Add missing trailing quotes.
    @pdurbin
    pdurbin committed Feb 25, 2019
    Configuration menu
    Copy the full SHA
    44acd97 View commit details
    Browse the repository at this point in the history

  5. have doc examples match yaml config #4821

    @pdurbin
    pdurbin committed Feb 25, 2019
    Configuration menu
    Copy the full SHA
    51cfde8 View commit details
    Browse the repository at this point in the history

Commits on Feb 26, 2019

  1. #4821 fix retrieve citations api

    @sekmiller
    sekmiller committed Feb 26, 2019
    Configuration menu
    Copy the full SHA
    d2cf1a7 View commit details
    Browse the repository at this point in the history

  2. rename to "citationUrl" to allow for future fields #4821

    @pdurbin
    pdurbin committed Feb 26, 2019
    Configuration menu
    Copy the full SHA
    865c99b View commit details
    Browse the repository at this point in the history

  3. more docs and debug output #4821

    @pdurbin
    pdurbin committed Feb 26, 2019
    Configuration menu
    Copy the full SHA
    d97125c View commit details
    Browse the repository at this point in the history

  4. add datasetexternalcitations table to diagram #4821

    @pdurbin
    pdurbin committed Feb 26, 2019
    Configuration menu
    Copy the full SHA
    677bf7a View commit details
    Browse the repository at this point in the history

  5. capture some decisions from tech hours #4821

    @pdurbin
    pdurbin committed Feb 26, 2019
    Configuration menu
    Copy the full SHA
    1b527aa View commit details
    Browse the repository at this point in the history

Commits on Feb 27, 2019

  1. Merge branch 'develop' into 4821-make-data-count

    @sekmiller
    sekmiller committed Feb 27, 2019
    Configuration menu
    Copy the full SHA
    23c4748 View commit details
    Browse the repository at this point in the history

Commits on Feb 28, 2019

  1. Log metadata downloads #4821

    @matthew-a-dunlap
    matthew-a-dunlap committed Feb 28, 2019
    Configuration menu
    Copy the full SHA
    a92ad9d View commit details
    Browse the repository at this point in the history

  2. #4821 add support for non-country metrics

    @sekmiller
    sekmiller committed Feb 28, 2019
    Configuration menu
    Copy the full SHA
    522c26f View commit details
    Browse the repository at this point in the history

  3. MDC export regex match 0 or more #4821

    @matthew-a-dunlap
    matthew-a-dunlap committed Feb 28, 2019
    Configuration menu
    Copy the full SHA
    545512a View commit details
    Browse the repository at this point in the history

  4. Revert "capture some decisions from tech hours #4821 " 

    This reverts commit 1b527aa.
    @pdurbin
    pdurbin committed Feb 28, 2019
    Configuration menu
    Copy the full SHA
    f2d7d54 View commit details
    Browse the repository at this point in the history

Commits on Mar 1, 2019

  1. MDC mult ways to get datasetVersion in guestbook #4821

    @matthew-a-dunlap
    matthew-a-dunlap committed Mar 1, 2019
    Configuration menu
    Copy the full SHA
    278568d View commit details
    Browse the repository at this point in the history

  2. MDC expanded dataset api logging #4821

    @matthew-a-dunlap
    matthew-a-dunlap committed Mar 1, 2019
    Configuration menu
    Copy the full SHA
    6375598 View commit details
    Browse the repository at this point in the history

Commits on Mar 6, 2019

  1. MDC Missed basic dataset json api #4821 

    The fun never ends!
    @matthew-a-dunlap
    matthew-a-dunlap committed Mar 6, 2019
    Configuration menu
    Copy the full SHA
    0b46993 View commit details
    Browse the repository at this point in the history

Commits on Mar 8, 2019

  1. Merge branch 'develop' into 4821-make-data-count

    @sekmiller
    sekmiller committed Mar 8, 2019
    Configuration menu
    Copy the full SHA
    b14c921 View commit details
    Browse the repository at this point in the history