Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/find by name #305

Merged
merged 9 commits into from
Dec 14, 2021
Merged

Feature/find by name #305

merged 9 commits into from
Dec 14, 2021

Conversation

genie9
Copy link
Contributor

@genie9 genie9 commented Dec 9, 2021

Description

Adds functionality to filter folders by folder name eg. /folders?name="test string".
The folder collection is updated with the new field text_name which is populated during folder creation from folder name as delimiter separated words e.g 'test_file' becomes 'test file'. This field is indexed so a text search would be possible.
The database is indexed during database container initialization.

Related issues

Feature #280

  • Filter folders by name

Type of change

  • New feature (non-breaking change which adds functionality)

Changes Made

Add indexing on database initialization

  • Add docker-compose configuration for mongo container to run a script on
    newly initialized database.
  • Add js script to build database collections and indexes.

Add new field text_name to folder collection

  • Folder is updated with the new field text_name which is populated
    during folder creation from folder name as delimiter separated words.
  • Schema is updated with a new field.

Add folder querying by name.

Python scripts for database operations

  • Add script mongo_indexes.py for collections and indexes creation to be run if
    the database is destroyed.
  • Update clean_db.py script with new functionality to only delete
    documents from collections. Old functionality ergo dropping database
    can be run with the flag --purge.

Add openapi specs.

Minor changes:

  • Update dev requirements and add the use of env vars in scripts and integration test
    to make it easier to run stuff in the local environment.
  • Update Github workflow file with env vars.
  • Add black formatting argument

Testing

  • Integration Tests

Mentions

@genie9 genie9 force-pushed the feature/find-by-name branch 9 times, most recently from dec2e3c to cd54b09 Compare December 9, 2021 18:49
@blankdots blankdots added this to In progress in SD Submit via automation Dec 9, 2021
Adds docker-compose configuration for mongo container to run script on
newly initialized database.
Adds js script to build database collections and indexes.
Folder is updated with new field text_name which is populated
during folder creation from folder name as delimiter splited words.
This ensures filtering name with string.
Schema is updated with new field.
@genie9 genie9 force-pushed the feature/find-by-name branch 2 times, most recently from cbc448b to 713f3fc Compare December 9, 2021 19:14
@genie9 genie9 requested a review from blankdots December 9, 2021 19:28
Copy link
Contributor

@blankdots blankdots left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

small suggestions of improvement and maybe we want to update the docs with this PR or a follow-up one, namely the https://github.com/CSCfi/metadata-submitter/blob/master/docs/test.rst#integration-testing section

tests/integration/run_tests.py Outdated Show resolved Hide resolved
tests/integration/run_tests.py Outdated Show resolved Hide resolved
tests/integration/clean_db.py Outdated Show resolved Hide resolved
tests/integration/clean_db.py Outdated Show resolved Hide resolved
tests/integration/clean_db.py Outdated Show resolved Hide resolved
metadata_backend/api/operators.py Outdated Show resolved Hide resolved
tests/integration/mongo_indexes.py Outdated Show resolved Hide resolved
tests/integration/mongo_indexes.py Outdated Show resolved Hide resolved
tests/integration/mongo_indexes.py Outdated Show resolved Hide resolved
tests/integration/run_tests.py Show resolved Hide resolved
SD Submit automation moved this from In progress to Reviewer approved Dec 10, 2021
Adds python script for collections and indexes creation to be run if
database is destroyed.
Updates clean_db script with new functionality to only delete
documents from collection. Old functionality ergo dropping database
can be run with flag `--purge`.
Adds dependencies and configs to be able running integration tests on
local environment.
Add env vars to github workflow file as database scripts and
integration tests use environment variables.
Copy link
Contributor

@blankdots blankdots left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks very good.

@genie9 genie9 merged commit 44c4d6d into develop Dec 14, 2021
SD Submit automation moved this from Reviewer approved to Done Dec 14, 2021
@genie9 genie9 deleted the feature/find-by-name branch December 14, 2021 13:01
@blankdots blankdots mentioned this pull request Apr 7, 2022
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
SD Submit
  
Done
Development

Successfully merging this pull request may close these issues.

None yet

2 participants