Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Consolidating our KEGG data download #2121

Merged
merged 84 commits into from
Sep 25, 2023
Merged

Consolidating our KEGG data download #2121

merged 84 commits into from
Sep 25, 2023

Commits on Aug 29, 2023

  1. Merge remote-tracking branch 'origin/metabolic-network-storage' into … 

    …kegg_download_consolidation
    @ivagljiva
    ivagljiva committed Aug 29, 2023
    Configuration menu
    Copy the full SHA
    cd9aa86 View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2023

  1. add mode argument for selecting data to download

    @ivagljiva
    ivagljiva committed Aug 30, 2023
    Configuration menu
    Copy the full SHA
    1f67986 View commit details
    Browse the repository at this point in the history

  2. the most simple refactor: conditionally run KEGG setup or refdbs setup

    @ivagljiva
    ivagljiva committed Aug 30, 2023
    Configuration menu
    Copy the full SHA
    6b09101 View commit details
    Browse the repository at this point in the history

  3. a little fix for people whose PROTEIN_DATA dir doesn't exist yet

    @ivagljiva
    ivagljiva committed Aug 30, 2023
    Configuration menu
    Copy the full SHA
    8df990b View commit details
    Browse the repository at this point in the history

Commits on Aug 31, 2023

  1. update description

    @ivagljiva
    ivagljiva committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    2d5181a View commit details
    Browse the repository at this point in the history

  2. update copyright to 2023

    @ivagljiva
    ivagljiva committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    f0b401e View commit details
    Browse the repository at this point in the history

  3. rename setup program

    @ivagljiva
    ivagljiva committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    bd1eef2 View commit details
    Browse the repository at this point in the history

  4. make kofam setup independent of the rest

    @ivagljiva
    ivagljiva committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    f7d5d56 View commit details
    Browse the repository at this point in the history

  5. add kofam mode for downloading

    @ivagljiva
    ivagljiva committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    3b3f49a View commit details
    Browse the repository at this point in the history

  6. enforce sanity check for directory creation

    @ivagljiva
    ivagljiva committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    6b9e1ff View commit details
    Browse the repository at this point in the history

  7. make class specific to KOfam download mode

    @ivagljiva
    ivagljiva committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    a140ba5 View commit details
    Browse the repository at this point in the history

  8. make db check function more generic

    @ivagljiva
    ivagljiva committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    e48dbd7 View commit details
    Browse the repository at this point in the history

  9. we will move these sanity checks to individual subclasses

    @ivagljiva
    ivagljiva committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    8bce614 View commit details
    Browse the repository at this point in the history

  10. skip_init param that will be accessible in args

    @ivagljiva
    ivagljiva committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    eebf79f View commit details
    Browse the repository at this point in the history

  11. make subclass specific to modules download

    @ivagljiva
    ivagljiva committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    c0404d6 View commit details
    Browse the repository at this point in the history

  12. utilize the new subclasses in setup program

    @ivagljiva
    ivagljiva committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    f52d260 View commit details
    Browse the repository at this point in the history

  13. add missing lambda func and remove stray space

    @ivagljiva
    ivagljiva committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    bb32a87 View commit details
    Browse the repository at this point in the history

  14. we actually need archive path attribute in parent class

    @ivagljiva
    ivagljiva committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    bedcff2 View commit details
    Browse the repository at this point in the history

  15. little init fixies

    @ivagljiva
    ivagljiva committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    144ca5c View commit details
    Browse the repository at this point in the history

Commits on Sep 1, 2023

  1. now this dict stores the modes

    @ivagljiva
    ivagljiva committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    0b38947 View commit details
    Browse the repository at this point in the history

  2. use ozcan's hack for subparser parameters

    @ivagljiva
    ivagljiva committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    3cacc38 View commit details
    Browse the repository at this point in the history

  3. section for mode specific params

    @ivagljiva
    ivagljiva committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    8c5963a View commit details
    Browse the repository at this point in the history

  4. a bit of curation of parameter help

    @ivagljiva
    ivagljiva committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    4e25621 View commit details
    Browse the repository at this point in the history

  5. clarify dir param

    @ivagljiva
    ivagljiva committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    858fe37 View commit details
    Browse the repository at this point in the history

  6. copy pasta error OOPS

    @ivagljiva
    ivagljiva committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    a87a710 View commit details
    Browse the repository at this point in the history

  7. turns out we need this too

    @ivagljiva
    ivagljiva committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    e921d3f View commit details
    Browse the repository at this point in the history

  8. a little fixy for kofam --only-database

    @ivagljiva
    ivagljiva committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    6b7baab View commit details
    Browse the repository at this point in the history

  9. add debug output for db check

    @ivagljiva
    ivagljiva committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    498bf49 View commit details
    Browse the repository at this point in the history

  10. rename only-database arg to only-processing

    @ivagljiva
    ivagljiva committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    4e82c5d View commit details
    Browse the repository at this point in the history

Commits on Sep 5, 2023

  1. switch to sam's newer class for modeling data download

    @ivagljiva
    ivagljiva committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    d5e9e51 View commit details
    Browse the repository at this point in the history

  2. a bit cleaner

    @ivagljiva
    ivagljiva committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    f498190 View commit details
    Browse the repository at this point in the history

Commits on Sep 6, 2023

  1. a better way to list mode descriptions

    @ivagljiva
    ivagljiva committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    954759b View commit details
    Browse the repository at this point in the history

  2. nicer descriptions

    @ivagljiva
    ivagljiva committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    588e7cd View commit details
    Browse the repository at this point in the history

  3. make module download multithreaded

    @ivagljiva
    ivagljiva committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    24d9d18 View commit details
    Browse the repository at this point in the history

  4. verify complete files in a different function

    @ivagljiva
    ivagljiva committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    ea03b03 View commit details
    Browse the repository at this point in the history

  5. KOs -> modules

    @ivagljiva
    ivagljiva committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    71dfbd0 View commit details
    Browse the repository at this point in the history

  6. enable skip_init for subclasses of KeggSetup

    @ivagljiva
    ivagljiva committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    41bbd1b View commit details
    Browse the repository at this point in the history

  7. multithreaded brite download

    @ivagljiva
    ivagljiva committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    cf55991 View commit details
    Browse the repository at this point in the history

  8. confirm json format in brite hierarchies separately from download fun… 

    …ction
    @ivagljiva
    ivagljiva committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    2b36a57 View commit details
    Browse the repository at this point in the history

  9. remove kegg from anvi-setup-protein-ref-db

    @ivagljiva
    ivagljiva committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    c1e38d1 View commit details
    Browse the repository at this point in the history

  10. update provides and requires

    @ivagljiva
    ivagljiva committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    eb84e9c View commit details
    Browse the repository at this point in the history

  11. anvi-setup-kegg- kofams to data name change

    @ivagljiva
    ivagljiva committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    7812718 View commit details
    Browse the repository at this point in the history

  12. reference anvi-setup-kegg-data in reaction-ref-data artifact

    @ivagljiva
    ivagljiva committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    ace50a5 View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2023

  1. some partial changes to the anvi-setup-kegg-data doc

    @ivagljiva
    ivagljiva committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    02b662b View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2023

  1. Revert "can we sneak in metabolism here too?" 

    This reverts commit dfa19aa.

This didn't work.
    @meren
    meren committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    9a0070f View commit details
    Browse the repository at this point in the history

Commits on Sep 21, 2023

  1. merge reaction data into KEGG data dir (now at data/MISC/KEGG/KO_REAC… 

    …TION_NETWORK)
    @ivagljiva
    ivagljiva committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    130c315 View commit details
    Browse the repository at this point in the history

  2. Add meme to environment.yaml

    @FlorianTrigodet
    FlorianTrigodet committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    2457032 View commit details
    Browse the repository at this point in the history

  3. move stuff around so that kegg snapshots are handled by KeggSetup class

    @ivagljiva
    ivagljiva committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    57ec54a View commit details
    Browse the repository at this point in the history

  4. remove KOfam download from modules mode

    @ivagljiva
    ivagljiva committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    be25cca View commit details
    Browse the repository at this point in the history

  5. reorganize params and help output

    @ivagljiva
    ivagljiva committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    14e8d56 View commit details
    Browse the repository at this point in the history

  6. add logic for 'all' mode and make it default

    @ivagljiva
    ivagljiva committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    6870157 View commit details
    Browse the repository at this point in the history

  7. move snapshot arg to the right class

    @ivagljiva
    ivagljiva committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    55c0631 View commit details
    Browse the repository at this point in the history

  8. whoops. init needs args

    @ivagljiva
    ivagljiva committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    f2a3a9a View commit details
    Browse the repository at this point in the history

  9. fix arg parsing error message

    @ivagljiva
    ivagljiva committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    10fa055 View commit details
    Browse the repository at this point in the history

  10. include BRITE in the mode description

    @ivagljiva
    ivagljiva committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    eb43d9b View commit details
    Browse the repository at this point in the history

  11. fix priorities for dir modeling mode param

    @ivagljiva
    ivagljiva committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    76a8810 View commit details
    Browse the repository at this point in the history

  12. allow contigs db only annotated with kofams

    @ivagljiva
    ivagljiva committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    193b866 View commit details
    Browse the repository at this point in the history

  13. just do it flag for anvi-estimate-metabolism

    @ivagljiva
    ivagljiva committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    1252184 View commit details
    Browse the repository at this point in the history

  14. metabolism self-test uses 'all' mode for anvi-setup-kegg-data

    @ivagljiva
    ivagljiva committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    7f88108 View commit details
    Browse the repository at this point in the history

  15. update anvi-run-kegg-kofams doc with conditional modules annotations

    @ivagljiva
    ivagljiva committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    b67f349 View commit details
    Browse the repository at this point in the history

  16. now this extra --dir parameter is properly handled in 'all' mode

    @ivagljiva
    ivagljiva committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    26850e4 View commit details
    Browse the repository at this point in the history

  17. better debug output if archive is not ok

    @ivagljiva
    ivagljiva committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    b34c632 View commit details
    Browse the repository at this point in the history

  18. fixy

    @meren
    meren committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    815d6ca View commit details
    Browse the repository at this point in the history

  19. whoops. these sanity checks should be in KeggSetup class

    @ivagljiva
    ivagljiva committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    3eb8a5e View commit details
    Browse the repository at this point in the history

  20. this num_threads initialization is better for API users

    @ivagljiva
    ivagljiva committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    8cbd3a2 View commit details
    Browse the repository at this point in the history

  21. update doc for anvi-setup-kegg-data with new params and features

    @ivagljiva
    ivagljiva committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    6f2f15b View commit details
    Browse the repository at this point in the history

  22. add indication for no modeling data to KEGG snapshots file

    @ivagljiva
    ivagljiva committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    3d53ebd View commit details
    Browse the repository at this point in the history

  23. smart check for modeling files in kegg archives

    @ivagljiva
    ivagljiva committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    c0722ce View commit details
    Browse the repository at this point in the history

  24. Merge remote-tracking branch 'origin/metabolic-network-storage' into … 

    …kegg_download_consolidation
    @ivagljiva
    ivagljiva committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    51694d1 View commit details
    Browse the repository at this point in the history

  25. rename anvi-setup-protein-reference-database to anvi-setup-modelseed-… 

    …database
    @ivagljiva
    ivagljiva committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    e50000f View commit details
    Browse the repository at this point in the history

  26. Merge branch 'master' into kegg_download_consolidation

    @ivagljiva
    ivagljiva committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    4b490d3 View commit details
    Browse the repository at this point in the history

  27. most recent snapshot has no modeling data associated with it

    @ivagljiva
    ivagljiva committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    5b4dcdc View commit details
    Browse the repository at this point in the history

Commits on Sep 22, 2023

  1. don't reset twice in 'all' mode

    @ivagljiva
    ivagljiva committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    0a36d42 View commit details
    Browse the repository at this point in the history

  2. specify KO database in output

    @ivagljiva
    ivagljiva committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    9bf9bdc View commit details
    Browse the repository at this point in the history

  3. remove references to mode parameter wherever possible in documentatio… 

    …n. we want to encourage people to use the default mode (or read the docs if they really don't want all data so that they can take responsibility for what happens)
    @ivagljiva
    ivagljiva committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    9c80faa View commit details
    Browse the repository at this point in the history

  4. remove references to REACTION and COMPOUND databases in documentation

    @ivagljiva
    ivagljiva committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    7868c64 View commit details
    Browse the repository at this point in the history

  5. whoops, forgot this reference to mode param

    @ivagljiva
    ivagljiva committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    f693593 View commit details
    Browse the repository at this point in the history

  6. update instructions for adding new KEGG snapshot

    @ivagljiva
    ivagljiva committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    ab0b26d View commit details
    Browse the repository at this point in the history

  7. add new default snapshot for anvi'o v8 (now includes modeling data)

    @ivagljiva
    ivagljiva committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    6ab82cc View commit details
    Browse the repository at this point in the history

  8. make --reset work for snapshots and archives

    @ivagljiva
    ivagljiva committed Sep 22, 2023
    Configuration menu
    Copy the full SHA
    423fe1f View commit details
    Browse the repository at this point in the history

Commits on Sep 24, 2023

  1. class variable of expected files

    @semiller10
    semiller10 committed Sep 24, 2023
    Configuration menu
    Copy the full SHA
    d9e1565 View commit details
    Browse the repository at this point in the history

  2. correct docstring

    @semiller10
    semiller10 committed Sep 24, 2023
    Configuration menu
    Copy the full SHA
    a3bf986 View commit details
    Browse the repository at this point in the history

  3. aesthetic

    @semiller10
    semiller10 committed Sep 24, 2023
    Configuration menu
    Copy the full SHA
    50d543a View commit details
    Browse the repository at this point in the history

  4. reference expected files from KODatabase

    @semiller10
    semiller10 committed Sep 24, 2023
    Configuration menu
    Copy the full SHA
    fb10435 View commit details
    Browse the repository at this point in the history