Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release 2.2 #40

Merged
merged 46 commits into from
Jul 26, 2023
Merged

Release 2.2 #40

merged 46 commits into from
Jul 26, 2023

Commits on Dec 19, 2021

  1. Configuration menu
    Copy the full SHA
    406d1a7 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    5e05bf9 View commit details
    Browse the repository at this point in the history

Commits on Dec 20, 2021

  1. Configuration menu
    Copy the full SHA
    49ff515 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    ffcfc48 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    2e3f695 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    8e0b80c View commit details
    Browse the repository at this point in the history

Commits on Apr 5, 2022

  1. Configuration menu
    Copy the full SHA
    933ceeb View commit details
    Browse the repository at this point in the history

Commits on Apr 6, 2022

  1. Configuration menu
    Copy the full SHA
    c096c3e View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    2bc5c29 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    a02c212 View commit details
    Browse the repository at this point in the history

Commits on Apr 12, 2022

  1. πŸŽ¨πŸ“βœ…πŸ’šβ¬†οΈ Add geom_median to env, move favicon assets

    πŸ“πŸŽ¨ Add Bioconda project url to setup.py
    🎨 Add toast when browser storage limits are reached (issue #11)
    :art::memo: Move favicon to automappa/assets
    :arrow_up::art: Add geom-median to environment.yml
    :art: Add get_clusters_geom_medians(...) to celery tasks
    :arrow_up::art: Bump VERSION to 2.2.0
    :see_no_evil: Add data to .gitignore
    evanroyrees committed Apr 12, 2022
    Configuration menu
    Copy the full SHA
    ed8c0d1 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    f79dee0 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    399c2f0 View commit details
    Browse the repository at this point in the history

Commits on Apr 14, 2022

  1. πŸŽ¨πŸ› WIP beginning of using postgres db

    Add TODOs for WIP next steps... To be checked before merge into develop
    evanroyrees committed Apr 14, 2022
    Configuration menu
    Copy the full SHA
    1df246b View commit details
    Browse the repository at this point in the history

Commits on Apr 18, 2022

  1. Refactor callback to viz DataTable and store samples-store State

    πŸŽ¨πŸ› Prevent samples-store data from being overwritten on initial callback
    🎨 samples-store data now persistent with local session
    🎨 Refactor some variables in MAG refinement
    🎨 WIP
    evanroyrees committed Apr 18, 2022
    Configuration menu
    Copy the full SHA
    8a54e12 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    7b5f2d3 View commit details
    Browse the repository at this point in the history

Commits on Apr 20, 2022

  1. Configuration menu
    Copy the full SHA
    991a45f View commit details
    Browse the repository at this point in the history

Commits on Apr 21, 2022

  1. Configuration menu
    Copy the full SHA
    3013b62 View commit details
    Browse the repository at this point in the history

Commits on Apr 22, 2022

  1. 🎨 WIP now fetching all fileuploads and creating files datatable using…

    … MetaData.table.keys()
    
    Should now be able to πŸ”₯ remove all of the data stores to track the fileupload table_id... πŸ”₯
    evanroyrees committed Apr 22, 2022
    Configuration menu
    Copy the full SHA
    ea83e85 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    d0f943f View commit details
    Browse the repository at this point in the history
  3. πŸ”₯🐳🎨🌻🐘 Add docker-services, rm environment/

    🐳🌻🐘🎨 Add docker-services: celery, flower, prometheus, grafana, redis, rabbitmq
    🎨πŸ”₯ Remove environment/*.env files. Consolidate to .env with corresponding prefixes
    🎨 Change pydantic models built on BaseSettings in automappa.settings to use only .env with env_prefix
    🎨🌻 Add working configuration of celery task-queue with redis backend and rabbitMQ broker
    πŸ“ Add resources/references for misc. links used during debugging/troubleshooting
    ⬆️ Add flower. Move celery[redis] to pip install section of environment.yml
    🎨 black formatting to serializers and change server.upload_folder root to server.root_upload_folder
    🎨 blak formatting to home.py
    evanroyrees committed Apr 22, 2022
    Configuration menu
    Copy the full SHA
    cd6d4f8 View commit details
    Browse the repository at this point in the history
  4. Add dbc.Select to home for choosing metagenome annotations (WIP)

    🎨 dbc.Select(...) are now populated with values from db tables
    πŸ›πŸŽ¨ Investigate whether serializer.get_uploaded_files_table() is functional (TODO)
    🐳 Add commented out arg to create_engine(..., pre_pool_ping: Optional[bool]) and db settings
    🐳 Add PRE_POOL_PING to .env
    evanroyrees committed Apr 22, 2022
    Configuration menu
    Copy the full SHA
    6e92c24 View commit details
    Browse the repository at this point in the history
  5. 🎨 WIP refinement page reading user uploaded/selected data

    πŸ”₯ Remove (now deprecated) stores
    🎨 Add serialization of a hash-refinement table when hash-binning file is uploaded (respective to hash-binning)
    🎨 Remove --debug arg from parser. Move statement to .env (now handle read with automappa.settings.server.debug
    πŸ”₯ remove unused imports in index.py
    🎨 Move samples stores from home.py to index.py
    Add dbc.InputGroup(...) with selects for binning, markers and metagenome
    🎨 Add dbc.Button(...) to save table-ids in 'selected-tables-store' to be read from mag_refinement.py
    πŸŽ¨πŸ› Continue refactor of get_scatterplot_2d(...) to use other embedding methods
    🎨 Add get_marker_symbols(...) to tasks.py and automappa.utils.markers (WIP currently being used to *expensively* generate symbols during 2d scatterplot creation
    evanroyrees committed Apr 22, 2022
    Configuration menu
    Copy the full SHA
    c6fd9da View commit details
    Browse the repository at this point in the history
  6. 🎨 Move port and host argparse args to .env

    πŸ”₯ Remove unused (commented out) argparse args
    πŸ”₯🎨🐳 Update automappa-web service command to not use removed argparse args
    evanroyrees committed Apr 22, 2022
    Configuration menu
    Copy the full SHA
    dc5665d View commit details
    Browse the repository at this point in the history
  7. πŸ“ Update TODO list

    evanroyrees committed Apr 22, 2022
    Configuration menu
    Copy the full SHA
    e54108c View commit details
    Browse the repository at this point in the history

Commits on Apr 23, 2022

  1. πŸ³πŸ“Ί provisioned Grafana within docker-compose.

    Will still need to ensure this is working once celery tasks are implemented
    evanroyrees committed Apr 23, 2022
    Configuration menu
    Copy the full SHA
    d35a6cc View commit details
    Browse the repository at this point in the history

Commits on Apr 25, 2022

  1. πŸŽ¨πŸ› Connect mag_summary.layout callbacks to postgres db

    πŸ›πŸ”₯ Fix sankey diagram subsetting bug where selections would not render
    🎨 Change MAG-specific completeness/purity boxplot to barplot
    🎨 Rename save_to_db(...) to file_to_db(...) in serializer
    🎨 Add metric_barplot(...) func to automappa.utils.figures
    evanroyrees committed Apr 25, 2022
    Configuration menu
    Copy the full SHA
    ec9989b View commit details
    Browse the repository at this point in the history
  2. 🎨 Beginning of celery task-queue implementation

    - 🐳🎨 Add mem_limit: 4GB to automappa-web service
    - πŸŽ¨πŸ› Remove ability for user to navigate to other tabs when no data has been uploaded/selected
    - 🎨 Disable 'Refine mags' button if data has not been selected
    - 🎨 Add DataTable(...) to visualize the currently loaded datasets
    - 🎨 Change server settings import so it is clear that the server setting are imported and not the server
    - 🎨πŸ₯• Silence numba and h5py loggers in mag_refinement.py (beginnings of celery task-queue tasks implementation)
     - 🎨 Add logger to automappa.tasks
    - πŸ› Fix logger emitted message during refinement data serialization to postgres db
    evanroyrees committed Apr 25, 2022
    Configuration menu
    Copy the full SHA
    bbc93e9 View commit details
    Browse the repository at this point in the history

Commits on Apr 26, 2022

  1. Configuration menu
    Copy the full SHA
    9917443 View commit details
    Browse the repository at this point in the history

Commits on May 5, 2022

  1. 🎨 WIP Working task-queue with scatterplot-2d views including embeddin…

    …g methods
    
    πŸŽ¨πŸ› Need to replace kmers.embed(...) method in tasks.py to include n_jobs=1 when autometa 2.0.4 release is out
    🎨πŸ₯¦ Add celeryconfig
    🎨πŸ₯¦ Add working kmer freq. analysis parallelize pipeline for use in task-queue
    🎨 Add use of pydantic.BaseModel in models.py of SampleTables, KmerTable, AnnotationTable and misc. others (WIP)
    🎨 Add construction of SeqIO.SeqRecords when retrieving metagenome from table using new func, get_metagenome_seqrecords(...)
    🎨 Refactor get_contig_marker_counts(...)
    🎨 Refactor get_marker_symbols(...) in markers.py
    🎨πŸ”₯ Remove hard-coded values in automappa.utils.figures.get_embedding_traces_df(...)
    🎨 Parse selected_data_tables as SampleTables.parse_raw(selected_data_tables) for shared data
    🎨 Add tasks button to home.py
    🎨 Replace typehints for selected_data_tables to Json[SampleTables]
    evanroyrees committed May 5, 2022
    Configuration menu
    Copy the full SHA
    731ffc1 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    977dbbf View commit details
    Browse the repository at this point in the history

Commits on May 16, 2022

  1. πŸŽ¨πŸ™ˆ WIP for fetching/visualizing data using services

    🎨 Fetch data from postgres service
    🎨πŸ₯¦ Retrieve scatterplot-2d coordinates from celery task-queue results (selected from auto-generated dropdown)
    🎨🌻 Add flower configuration env variable to settings.py
    🎨 generate product of all combinations of kmer pipeline params (kmer_size, norm_method, embed_method) for 2d-coords retrieval
    πŸŽ¨πŸŽπŸ› Retrieve mag summary data from SampleTables properties instead of using get_table(...)
    πŸ”₯πŸ› Remove default value of 'cluster' for mag_summary_cluster_col_dropdown
    🎨πŸ”₯ Remove unnecessary nested lists of Input(...) in mag_summary callbacks
    🎨 Add long_callback_manager to app (WIP: currently not being used)
    🎨 Add handling of kmer param titles and metagenome annotation titles to format_axis_title(...) in figures.py
    🎨🐳🐰 Change rabbitmq image from latest to 3.10.1-management-alpine (w/port) to use amqp management UI
    evanroyrees committed May 16, 2022
    Configuration menu
    Copy the full SHA
    4810ba2 View commit details
    Browse the repository at this point in the history

Commits on May 18, 2022

  1. πŸ’š Add test SampleTables model instance

    πŸ“ Move misc. markdown docs to docs directory
    evanroyrees committed May 18, 2022
    Configuration menu
    Copy the full SHA
    ef3f205 View commit details
    Browse the repository at this point in the history

Commits on May 20, 2022

  1. Configuration menu
    Copy the full SHA
    2a83bfa View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2022

  1. πŸ³πŸ’šπŸ› pin scipy to 1.8

    🐳 Ignore .vscode for docker context
    🐳 Install mamba and prune prior to env update
    evanroyrees committed Aug 2, 2022
    Configuration menu
    Copy the full SHA
    90494cd View commit details
    Browse the repository at this point in the history
  2. 🎨🐳🐎 Add dash-extensions

    πŸ’šπŸŽ Add dash-extensions to enable caching
    πŸ”₯ Reduce kmer tasks to am_clr norm method and kmer_size of 5 (bhsne, densmap and umap embeddings)
    evanroyrees committed Aug 2, 2022
    Configuration menu
    Copy the full SHA
    90b2990 View commit details
    Browse the repository at this point in the history

Commits on Aug 3, 2022

  1. Update .env

    Turn off debug mode by default
    evanroyrees committed Aug 3, 2022
    Configuration menu
    Copy the full SHA
    df22b6e View commit details
    Browse the repository at this point in the history
  2. Update README.md

    πŸ”₯ Change clone of `home-tab` branch to `develop` branch
    evanroyrees committed Aug 3, 2022
    Configuration menu
    Copy the full SHA
    ebaccf4 View commit details
    Browse the repository at this point in the history
  3. Merge pull request #35 from WiscEvan/home-tab

    πŸ³πŸ“ΊπŸŽ¨ Automappa web app with home tab for data upload
    
    - πŸ₯¦ Add task-queue for pre/post-processing tasks with celery
    - πŸŸ₯ Add Redis service for celery backend
    - 🐰 Add RabbitMQ service for celery broker
    - 🐘 Add PostgreSQL for data ingestion
    - 🌷 Add Flower for celery task-queue monitoring
    - 🐍🎨 Add Pydantic for data models
    - 🐳 Add docker-compose file to orchestrate Automappa services
    evanroyrees committed Aug 3, 2022
    Configuration menu
    Copy the full SHA
    c245bc2 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    be091f9 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    31ab494 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    27fea70 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    bc82dd4 View commit details
    Browse the repository at this point in the history

Commits on Jul 21, 2023

  1. Refactor with fixes, features and improved directory structure (#39)

    * 🎨 Apply black formatting
    * πŸŽ¨β¬†οΈπŸ€  refactor to use dash-extensions DashBlueprints
    * Significantly simplified code base with modular components that are each passed their own page's app (DashBlueprint)
    * components and layouts are standardized to use render function that may receive the app with or without arguments
    * πŸ”₯ Remove (now) unnecessary index.py
    * ⬆️ Pin dash-extensions to first major release
    * ⬆️ Pin dash to 2.10
    * ⬆️ Add dash-mantine-components
    * ⬆️ Add dash-iconify
    * πŸ“ Add contributing docs
    * πŸŽ¨πŸ›πŸ“πŸ³ Fix docs, prop id and docker specs
    * 🐳 Add quotes around ports per docker-compose spec
    * πŸ“πŸŽ¨πŸ”₯ add Development resources
    🎨πŸ”₯ Add icons to not found page and remove unnecessary text
    * 🎨 Refactor mag refinement page components
    * 🎨 Refactor marker symbols legend to use icons rather than naming shapes
    * 🎨 Restyle save selections and settings buttons
    * 🎨 Refactor mag-refinement offsettings canvas components
    * 🎨 Add minimum example contig_cytoscape component
    * 🎨 Add specific backend configurations
    * 🎨 Add REDIS_BACKEND details to .env
    * 🎨 Add specific backend params to Serverside(..., backend=...) from automappa.data.db import {redis,file_system}_backend
    * πŸ“πŸ”₯ Refactor dash components to DashProxy components
    * ⬆️ pin dash-mantine-components to 0.12 in env using pip rather than conda
    * 🎨 Add icons in page registry with blueprints
    * 🎨 Add coverage range slider component
    * 🎨 Add Refinement model to track contig cluster refinements
    * 🎨 Implement data source to connect MAG refinements table
    * 🎨 Add methods to get latest user refinements contig headers
    * 🎨 Add interaction b/w coverage range slider and scatterplot2d
    * 🎨 Connect download refinements button to data source
    * 🎨 Hide MAG refinements switch is now disabled when there are no user refinements that have been made
    * 🎨 Refinements table has been styled with new timestamp string and removed 'initial_cluster' column
    * 🎨 Refinements table now only retrieves user refinements (i.e. excludes initial clusters)
    * 🎨 Add refinements clear button
    * Add working data source for completeness/purity boxplot
    * 🎨 Add minimal working AgGrid for MAG Summary
    * 🎨 Replace dbc.Offcanvas with dmc.Drawer
    * 🎨 Replace many of the Dropdowns in Offcanvas with RadioGroup Radio options (less clicks for the user)
    * πŸ₯•πŸ° Add logic for modular task-queue
    * πŸ“ Add documentation for tasks in CONTRIBUTING and reference tasks README.md
    * πŸ₯• Add minimum working example for task submission, progress display for a task status badge
    * πŸ₯• Add task discovery for home page task in celeryconfig.py
    * 🐳🐍 Replace base image of miniforge with mambaforge
    * πŸ₯•πŸŽ¨ Notification system tells user of current state of tasks processing
    * πŸ›πŸŽ¨ Move dcc.Store(...) within respective du.Upload component
    * 🎨 Add tasks store for getting tasks statuses
    * 🐰πŸ₯• Add tasks in home for handling sample data ingestion
    * 🎨 Add mag length sum computations
    * 🎨 Add MIMAG cluster count computations
    * 🎨 Add metadata to sample data card info
    * πŸ› Replace na values from cluster col during contig loader pre-processing
    * 🎨πŸ”₯ Add sample removal feature
    * 🎨 Affix settings and save mag buttons
    evanroyrees committed Jul 21, 2023
    Configuration menu
    Copy the full SHA
    763ec7c View commit details
    Browse the repository at this point in the history

Commits on Jul 26, 2023

  1. Configuration menu
    Copy the full SHA
    1509298 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    eafc892 View commit details
    Browse the repository at this point in the history