Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

uses ast.literal_eval() over eval() #61

Merged
merged 2 commits into from
Jul 26, 2021
Merged

uses ast.literal_eval() over eval() #61

merged 2 commits into from
Jul 26, 2021

Conversation

madeinoz67
Copy link
Contributor

static security checking of codebase using Bandit revealed use of unsecure eval() function.

>> Issue: [B307:blacklist] Use of possibly insecure function - consider using safer ast.literal_eval.
   Severity: Medium   Confidence: High
   Location: montydb/types/_nobson.py:163
   More Info: https://bandit.readthedocs.io/en/latest/blacklists/blacklist_calls.html#b307-eval
162                     if not _encoder.key_is_keyword:
163                         key = eval(candidate)
164                         if not isinstance(key, cls._string_types):

Have implemented ast.literal_eval() in its place

@madeinoz67
Copy link
Contributor Author

@davidlatwe can you please check from your side as I'm successful passes tests locally on the specific ones that are failing on the CI so may be something to do with github?


user: seaton on kubu montydb on  fix/unsafe-literal_eval [$] via 🐍 v3.9.6 (.venv) took 5m19s 
🕙[ 17:00:32 ] ❯ pytest --storage memory --use-bson 
===================================================================================== test session starts =====================================================================================
platform darwin -- Python 3.9.6, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
rootdir: /Users/seaton/Documents/src/montydb
plugins: cov-2.12.1
collected 744 items                                                                                                                                                                           

tests/test_client.py ...............                                                                                                                                                    [  2%]
tests/test_collection.py .....................                                                                                                                                          [  4%]
tests/test_cursor.py ........................                                                                                                                                           [  8%]
tests/test_database.py ............                                                                                                                                                     [  9%]
tests/test_invalid_doc.py ..                                                                                                                                                            [  9%]
tests/test_utils.py ....                                                                                                                                                                [ 10%]
tests/test_engine/test_delete.py ..                                                                                                                                                     [ 10%]
tests/test_engine/test_distinct.py ....                                                                                                                                                 [ 11%]
tests/test_engine/test_find.py ...                                                                                                                                                      [ 11%]
tests/test_engine/test_insert.py ...                                                                                                                                                    [ 12%]
tests/test_engine/test_replace.py ....                                                                                                                                                  [ 12%]
tests/test_engine/test_sorting.py .....................                                                                                                                                 [ 15%]
tests/test_engine/test_projection/test_projection_elemMatch.py .......                                                                                                                  [ 16%]
tests/test_engine/test_projection/test_projection_positional.py ........................                                                                                                [ 19%]
tests/test_engine/test_projection/test_projection_regular.py ...                                                                                                                        [ 20%]
tests/test_engine/test_projection/test_projection_slice.py ......                                                                                                                       [ 20%]
tests/test_engine/test_queries/test_querying_none.py ...........................                                                                                                        [ 24%]
tests/test_engine/test_queries/test_queryop_array_all.py ............                                                                                                                   [ 26%]
tests/test_engine/test_queries/test_queryop_array_elemMatch.py ..................                                                                                                       [ 28%]
tests/test_engine/test_queries/test_queryop_array_size.py ......                                                                                                                        [ 29%]
tests/test_engine/test_queries/test_queryop_comparsion_eq.py ............                                                                                                               [ 30%]
tests/test_engine/test_queries/test_queryop_comparsion_gt.py ................................                                                                                           [ 35%]
tests/test_engine/test_queries/test_queryop_comparsion_gte.py ................................                                                                                          [ 39%]
tests/test_engine/test_queries/test_queryop_comparsion_in.py ...............                                                                                                            [ 41%]
tests/test_engine/test_queries/test_queryop_comparsion_lt.py ................................                                                                                           [ 45%]
tests/test_engine/test_queries/test_queryop_comparsion_lte.py ................................                                                                                          [ 50%]
tests/test_engine/test_queries/test_queryop_comparsion_ne.py ..........                                                                                                                 [ 51%]
tests/test_engine/test_queries/test_queryop_comparsion_nin.py ..............                                                                                                            [ 53%]
tests/test_engine/test_queries/test_queryop_element_exists.py .......                                                                                                                   [ 54%]
tests/test_engine/test_queries/test_queryop_element_type.py ................................                                                                                            [ 58%]
tests/test_engine/test_queries/test_queryop_evaluation_mod.py ...............                                                                                                           [ 60%]
tests/test_engine/test_queries/test_queryop_evaluation_regex.py ........                                                                                                                [ 61%]
tests/test_engine/test_queries/test_queryop_logical_and.py ..                                                                                                                           [ 61%]
tests/test_engine/test_queries/test_queryop_logical_nor.py ....                                                                                                                         [ 62%]
tests/test_engine/test_queries/test_queryop_logical_not.py .....                                                                                                                        [ 63%]
tests/test_engine/test_queries/test_queryop_logical_or.py ...                                                                                                                           [ 63%]
tests/test_engine/test_update/test_update.py ........................................                                                                                                   [ 68%]
tests/test_engine/test_update/test_update_addToSet.py .....                                                                                                                             [ 69%]
tests/test_engine/test_update/test_update_currentDate.py .......                                                                                                                        [ 70%]
tests/test_engine/test_update/test_update_inc.py ....................                                                                                                                   [ 73%]
tests/test_engine/test_update/test_update_max.py ........                                                                                                                               [ 74%]
tests/test_engine/test_update/test_update_min.py ........                                                                                                                               [ 75%]
tests/test_engine/test_update/test_update_mod_each.py .......                                                                                                                           [ 76%]
tests/test_engine/test_update/test_update_mod_position.py .....                                                                                                                         [ 77%]
tests/test_engine/test_update/test_update_mod_slice.py ......                                                                                                                           [ 77%]
tests/test_engine/test_update/test_update_mod_sort.py .........                                                                                                                         [ 79%]
tests/test_engine/test_update/test_update_mul.py ....................                                                                                                                   [ 81%]
tests/test_engine/test_update/test_update_pop.py ........                                                                                                                               [ 82%]
tests/test_engine/test_update/test_update_pull.py ............                                                                                                                          [ 84%]
tests/test_engine/test_update/test_update_pullAll.py ......                                                                                                                             [ 85%]
tests/test_engine/test_update/test_update_push.py ...                                                                                                                                   [ 85%]
tests/test_engine/test_update/test_update_rename.py ...........                                                                                                                         [ 87%]
tests/test_engine/test_update/test_update_set.py .....                                                                                                                                  [ 87%]
tests/test_engine/test_update/test_update_setOnInsert.py ...                                                                                                                            [ 88%]
tests/test_engine/test_update/test_update_unset.py ......                                                                                                                               [ 88%]
tests/test_engine/test_update/test_update_upsert.py ...                                                                                                                                 [ 89%]
tests/test_utils.py .                                                                                                                                                                   [ 89%]
tests/test_engine/test_weighted.py ...............                                                                                                                                      [ 91%]
tests/test_client.py .                                                                                                                                                                  [ 91%]
tests/test_configure.py ...                                                                                                                                                             [ 92%]
tests/test_utils.py .....                                                                                                                                                               [ 92%]
tests/test_engine/test_weighted.py ...............                                                                                                                                      [ 94%]
tests/test_engine/test_fieldwalker/test_fieldwalker_get.py ..............................                                                                                               [ 98%]
tests/test_engine/test_fieldwalker/test_fieldwalker_set.py .........                                                                                                                    [100%]

====================================================================================== warnings summary =======================================================================================
montydb/base.py:22
  /Users/seaton/Documents/src/montydb/montydb/base.py:22: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
    from collections import (

montydb/engine/queries.py:4
  /Users/seaton/Documents/src/montydb/montydb/engine/queries.py:4: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
    from collections import Mapping

tests/test_engine/test_queries/test_queryop_array_elemMatch.py::test_qop_elemMatch_18[mongodb-4.2-memory-bson-1]
  /Users/seaton/Documents/src/montydb/.venv/lib/python3.9/site-packages/_pytest/unraisableexception.py:78: PytestUnraisableExceptionWarning: Exception ignored in: <generator object LogicBox._call_elemMatch.<locals>.elem_iter at 0x1049e1350>
  
  Traceback (most recent call last):
    File "/Users/seaton/Documents/src/montydb/montydb/engine/queries.py", line 157, in __call__
      return self._logic[self.theme](fieldwalker)
  KeyError: 'foo'
  
  During handling of the above exception, another exception occurred:
  
  Traceback (most recent call last):
    File "/Users/seaton/Documents/src/montydb/montydb/engine/field_walker.py", line 101, in _iter
      for i, elem in enumerate(value):
  RuntimeError: generator ignored GeneratorExit
  
    warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))

-- Docs: https://docs.pytest.org/en/stable/warnings.html
========================================================================= 744 passed, 3 warnings in 99.18s (0:01:39) ==========================================================================

@davidlatwe
Copy link
Owner

davidlatwe commented Jul 21, 2021 •
edited
Loading

Hey @madeinoz67 , sorry for delay.

I found that these two tests failed when pymongo==3.12.0 and --use-bson is enabled in test:
tests/test_engine/test_insert.py test_insert_invalid_doc_1
tests/test_engine/test_insert.py test_insert_invalid_doc_2

Currently not sure why (not seeing anything related in pymongo changelog yet), but def not because of this PR. Will file an issue and resolve it in another PR.

Cheers 🍻

Update
Ah, pymongo changed the error type for inserting this kind of invalid doc {"$b": 1}, from bson.InvalidDocument to pymongo.errors.BulkWriteError.

@davidlatwe davidlatwe merged commit da04f7f into davidlatwe:master Jul 26, 2021
@davidlatwe davidlatwe mentioned this pull request Jul 26, 2021
Open
@davidlatwe
Copy link
Owner

2.3.12 is out. :)

@madeinoz67 madeinoz67 deleted the fix/unsafe-literal_eval branch August 18, 2021 04:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@madeinoz67 @davidlatwe