Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new lookup plugin: LMDB #24550

Open
wants to merge 1 commit into
base: devel
from

Conversation

@jpmens
Copy link
Contributor

commented May 12, 2017

SUMMARY

This implements a lookup plugin for LMDB databases using a required pip install lmdb. I have called it lmdb_kv because it cannot be named lmdb due to our having to import lmdb and the name goes well (I think) with, say, redis_kv.

This module is able to get specific keys and also enumerate the whole content of a database.

This PR also adds an example to the Ansible lookups Web site documentation.

ISSUE TYPE
  • New Module Pull Request
COMPONENT NAME

lookup/lmdb_kv

ANSIBLE VERSION
ansible 2.3.0.0
  config file = /Users/jpm/.ansible.cfg
  configured module search path = [u'/etc/ansible/library', u'/Users/jpm/Auto/projects/on-github/ansible/pdns_zone/library']
  python version = 2.7.13 (default, Apr 10 2017, 13:51:30) [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)]

ADDITIONAL INFORMATION

create database

#!/usr/bin/env python

import lmdb

map_size = 1024 * 100

env = lmdb.open('jp.mdb', map_size=map_size)

with env.begin(write=True) as txn:
    txn.put('fr', 'France')
    txn.put('nl', 'Netherlands')
    txn.put('es', 'Spain')
    txn.put('lu', 'Luxemburg')

playbook

- hosts: localhost
  connection: local
  gather_facts: False
  vars:
    - benelux: "{{ lookup('lmdb_kv', 'nl', 'be', 'lu', db='jp.mdb') }}"
    - countries: "{{ lookup('lmdb_kv', db='jp.mdb') }}"
  tasks:
  - name: Get specific keys (in terms)
    debug: var=benelux

  - name: Nice country list
    debug: msg="Country code {{ item.0 | upper}} is actually {{ item.1 }}"
    with_items:
      - "{{ countries }}"

  - name: Template Europe
    template: src=europe.in dest=./output.txt

  - debug: msg="Hello from {{ item.0 }} a.k.a. {{ item.1 }}"
    with_lmdb_kv:
      - "n*"

template (europe.in)

{% for cc,country in countries %}
  {{ cc|upper }} is actually {{ country }}
{% endfor %}

playbook run


PLAY [localhost] ***********************************************************************************

TASK [Get specific keys (in terms)] ****************************************************************
ok: [localhost] => {
    "benelux": "Netherlands,Luxemburg", 
    "changed": false
}

TASK [Nice country list] ***************************************************************************
ok: [localhost] => (item=('es', 'Spain')) => {
    "item": [
        "es", 
        "Spain"
    ], 
    "msg": "Country code ES is actually Spain"
}
ok: [localhost] => (item=('fr', 'France')) => {
    "item": [
        "fr", 
        "France"
    ], 
    "msg": "Country code FR is actually France"
}
ok: [localhost] => (item=('lu', 'Luxemburg')) => {
    "item": [
        "lu", 
        "Luxemburg"
    ], 
    "msg": "Country code LU is actually Luxemburg"
}
ok: [localhost] => (item=('nl', 'Netherlands')) => {
    "item": [
        "nl", 
        "Netherlands"
    ], 
    "msg": "Country code NL is actually Netherlands"
}

TASK [Template Europe] *****************************************************************************
ok: [localhost]

TASK [debug] ***************************************************************************************
ok: [localhost] => (item=('nl', 'Netherlands')) => {
    "item": [
        "nl", 
        "Netherlands"
    ], 
    "msg": "Hello from nl a.k.a. Netherlands"
}

PLAY RECAP *****************************************************************************************
localhost                  : ok=4    changed=0    unreachable=0    failed=0   

template output

  ES is actually Spain
  FR is actually France
  LU is actually Luxemburg
  NL is actually Netherlands
@ansibot

This comment has been minimized.

Copy link
Contributor

commented May 12, 2017

@ansibot

This comment has been minimized.

Copy link
Contributor

commented May 12, 2017

The test ansible-test sanity --test pep8 failed with the following errors:

lib/ansible/plugins/lookup/lmdb_kv.py:23:10: E225 missing whitespace around operator
lib/ansible/plugins/lookup/lmdb_kv.py:27:14: E225 missing whitespace around operator
lib/ansible/plugins/lookup/lmdb_kv.py:29:1: E302 expected 2 blank lines, found 1
lib/ansible/plugins/lookup/lmdb_kv.py:65:1: W391 blank line at end of file

click here for bot help

@jpmens jpmens force-pushed the jpmens:lookup_lmdb branch May 12, 2017

@jpmens jpmens force-pushed the jpmens:lookup_lmdb branch 2 times, most recently May 13, 2017

@bcoca bcoca removed the needs_triage label May 15, 2017

@ansibot ansibot added docs and removed docs_pull_request labels Mar 1, 2018

@dharmabumstead

This comment has been minimized.

Copy link
Contributor

commented Apr 5, 2018

Closing this one. Feel free to rebase and reopen.

@abadger

This comment has been minimized.

Copy link
Member

commented Apr 6, 2018

Re-opening as this is a new lookup plugin, no need for closing it.

@abadger abadger reopened this Apr 6, 2018

@ansibot ansibot removed the stale_ci label Apr 6, 2018

@ansibot ansibot added the stale_ci label Apr 16, 2018

@ansibot ansibot removed the stale_ci label May 18, 2018

@ansibot ansibot added the stale_ci label May 26, 2018

@gundalow

This comment has been minimized.

Copy link
Contributor

commented Nov 30, 2018

So we'd prefer not to take in a new lookup plugin without a maintainer or tests, so before this can be merged we are looking for:

  • a maintainer
  • some tests
lib/ansible/plugins/lookup/lmdb_kv.py Outdated Show resolved Hide resolved

@jpmens jpmens force-pushed the jpmens:lookup_lmdb branch Jan 5, 2019

@ansibot ansibot removed the stale_ci label Jan 5, 2019

@ansibot

This comment was marked as resolved.

Copy link
Contributor

commented Jan 5, 2019

The test ansible-test sanity --test pep8 [explain] failed with 1 error:

lib/ansible/plugins/lookup/lmdb_kv.py:96:43: E261 at least two spaces before inline comment

click here for bot help

@ansibot

This comment has been minimized.

Copy link
Contributor

commented Feb 21, 2019

The test ansible-test sanity --test pep8 [explain] failed with 1 error:

lib/ansible/plugins/lookup/lmdb_kv.py:100:33: E117 over-indented

The test ansible-test sanity --test validate-modules [explain] failed with 1 error:

lib/ansible/modules/storage/netapp/na_ontap_vscan_on_demand_task.py:0:0: E325 Argument 'scan_files_with_no_ext' in argument_spec defines type as <class 'bool'> but documentation defines type as 'bool'

click here for bot help

new lookup plugin: LMDB
- allow wildcard searches
- make dbname configurable for with_lmdb_kv:
- added built-in documentation
- incorporate suggested changes by dw
- comment spaces
- outdent for the bot

@jpmens jpmens force-pushed the jpmens:lookup_lmdb branch to ee3190e Feb 21, 2019

@acozine acozine added the easyfix label Mar 6, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.