EBNFSpill

• Create Random Data based on EBNF Syntax description
• Validate EBNF definition (simpleparse's part)
• Validate some data against EBNF (simpleparse's part)

EBNF-Parser: simpleparse (EBNF-Grammars: http://simpleparse.sourceforge.net/simpleparse_grammars.html)

EBNF-Description

Our simple INI-file looks like this:

[MyFirstSection]
myKey	="myValue9_test-test2"
myKey2	="myValue9_test-test2"
myKey3	="myValue9_test-test2"
myKey4	="myValue9_test-test2"

[MySecond-Section]
myKey11	="myValuexx33_test-test2"

EBNF Declaration for a simple INI-File-Syntax:

file              := (section, entry+, '\n\n')+
section           := '[', wordspecial, ']','\n'
entry             := word, '\t=', '"', wordspecial , '"', '\n'
alpha             := [a-zA-Z]
alphanum          := [a-zA-Z0-9]
alphanumspecial   := [a-zA-Z0-9_-]
word              := alpha,alphanum*
wordspecial       := alpha,alphanumspecial*

Description:

'file'           ... our root definition (aka production) which describes the overall file syntax
'section'        ... Ini-file section header
'entry'          ... one key=value entry
'alpha'          ... alpha-chars only
'alphanum'       ... alphanumerical chars
'alphanumspecial'... alphanumerical and _- chars
'word'           ... begins with alpha and continues with none or more alphanums
'wordspecial'    ... begins with alpha and continues with none or more alphanumspecial chars

EBNFSpill Example #1 - Generate Random Data that meets the syntax requirements

EBNFSpill will now take any EBNF defintion that is valid and will try to create random data that matches the described syntax. For the above EBNF declaration this would be random data like this:

Code:

declaration = """
file              := (section, entry+, '\n\n')+

section           := '[', wordspecial, ']','\n'
entry             := word, '\t=', '"', wordspecial , '"', '\n'
alpha             := [a-zA-Z]
alphanum          := [a-zA-Z0-9]
alphanumspecial   := [a-zA-Z0-9_-]
word              := alpha,alphanum*
wordspecial       := alpha,alphanumspecial*
"""
from EBNFSpill import EBNFSpill
s = EBNFSpill()
# process declaration (feeds simpleparse with declaration and defines 'file' as entrypoint)
s.setDeclaration(declaration,production="file")	
# lets generate some random data
print b.generate()

Output:

[JaW-I]
MEk	="RIO_ZC"
KG1HWR0BcYATBF0CpgS5h	="IzOI5ECekj"
c8bDI3ozx3LnwnjpM2lz8	="go8iJsKkJ44JI"
shX3KIJLQkvNEEq1Ja	="E3usLXmAEifbDWOt"


[y7aM2A1QIpcThzPRc0ryFU]
Ng7fMShX4	="Xq"
d43i42sbool3BnI24e	="t1d5GHD"


[EJ4ll]
NuCG	="J0WrZdTnheSQq"
Rm6STRGxllBxlf9fLlEmEtwbgH	="cU82WpNkrHR0CqseBltmGOETdf"
LSadRHDXJ0	="G0Ir0FufaQJdSg9F"
R5T02vYsf	="JaJaPIUPh0zSSYSM4wfA8pjOq"
r6zhJQ7K	="GdYp9hMJjA8"
XMwdm	="wMjUQ0ADRjkT7MPV5zG"


[PXj0N0hrfu9e6dKM9-ujmEHuoU]
d1QFqfbTuqHnVQbK	="wwQ0-sykooiJp201HCwm73fD3"
fQuzvHP2pVwvS8G7UB6s	="PjofofSqLK4Nv25baoAi_RR7D"
jtrOtloyXUHJUKe	="k8P3WxPBRbwPrrh0"
uAuSDmzSVkSHNp0	="MBLM6Lw"
zEAwWEmfNpt	="ZkecJcD"
LcOuVkhLGJkCQYbI	="gnj_n8_FE"
I7TjmZC6	="ci1YzwboZz"


[IMw_fjT1jv]
cIXGM8TQuM29aWnXi9aYK	="UwL_q7RC0JKv7lkFPY"
mmNGyBCO6qTF2yKvelBki77Wz	="b1UZDdc7vT"
TPDcJMTv6mMuOKAoNS1xLf	="TSvbaGyW7K5nV"
aA6qLkQC08	="H3UBl852kRFXlsmoqoX3nO4eGZ"
z7xPM9YLNOro6FKvc1QqkL6zl	="VwiMf6EdCdK6cG7MtCLdof9h"

EBNFSpill Example #2 - show prettyprint of AST-Table, Step-by-Step walk the AST generated by Simpleparse

Code:

declaration = """
file              := (section, entry+, '\n\n')+

section           := '[', wordspecial, ']','\n'
entry             := word, '\t=', '"', wordspecial , '"', '\n'
alpha             := [a-zA-Z]
alphanum          := [a-zA-Z0-9]
alphanumspecial   := [a-zA-Z0-9_-]
word              := alpha,alphanum*
wordspecial       := alpha,alphanumspecial*
"""
from EBNFSpill import EBNFSpill
s = EBNFSpill()
# process declaration (feeds simpleparse with declaration and defines 'file' as entrypoint)
s.setDeclaration(declaration,production="file")	
# lets generate some random data
x= 0        
for i in s.walk():
    x+=1
    # print <number>, <uid>, <human_readable_AST_element> 
    print x,id(i),s.process(i)

Output:

1 36385136 (None, 'MATCH_SUBTABLE', (('section', 203, ((None, 21, '['), ('wordspecial', 203, (('alpha', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'), ('alphanumspecial', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_-', 2, 1), (None, 101, 1, -1, 1))), (None, 21, ']'), (None, 21, '\n'))), ('entry', 203, (('word', 203, (('alpha', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'), ('alphanum', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789', 2, 1), (None, 101, 1, -1, 1))), (None, 21, '\t='), (None, 21, '"'), ('wordspecial', 203, (('alpha', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'), ('alphanumspecial', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_-', 2, 1), (None, 101, 1, -1, 1))), (None, 21, '"'), (None, 21, '\n'))), ('entry', 203, (('word', 203, (('alpha', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'), ('alphanum', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789', 2, 1), (None, 101, 1, -1, 1))), (None, 21, '\t='), (None, 21, '"'), ('wordspecial', 203, (('alpha', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'), ('alphanumspecial', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_-', 2, 1), (None, 101, 1, -1, 1))), (None, 21, '"'), (None, 21, '\n')), 2, 1), (None, 101, 1, -1, 1), (None, 21, '\n\n')))
2 36743928 ('section', 'MATCH_TABLE', ((None, 21, '['), ('wordspecial', 203, (('alpha', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'), ('alphanumspecial', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_-', 2, 1), (None, 101, 1, -1, 1))), (None, 21, ']'), (None, 21, '\n')))
3 36325576 (None, 'MATCH_WORD', '[')
4 36742248 ('wordspecial', 'MATCH_TABLE', (('alpha', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'), ('alphanumspecial', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_-', 2, 1), (None, 101, 1, -1, 1)))
5 36742008 ('alpha', 'MATCH_ISIN', 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ')
6 36746368 ('alphanumspecial', 'MATCH_ISIN', 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_-', 2, 1)
7 36746416 (None, 'MATCH_EOF', 1, -1, 1)
8 36742488 (None, 'MATCH_WORD', ']')
9 36742848 (None, 'MATCH_WORD', '\n')
10 36384896 ('entry', 'MATCH_TABLE', (('word', 203, (('alpha', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'), ('alphanum', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789', 2, 1), (None, 101, 1, -1, 1))), (None, 21, '\t='), (None, 21, '"'), ('wordspecial', 203, (('alpha', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'), ('alphanumspecial', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_-', 2, 1), (None, 101, 1, -1, 1))), (None, 21, '"'), (None, 21, '\n')))
11 36326336 ('word', 'MATCH_TABLE', (('alpha', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'), ('alphanum', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789', 2, 1), (None, 101, 1, -1, 1)))
12 36326216 ('alpha', 'MATCH_ISIN', 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ')
13 36746464 ('alphanum', 'MATCH_ISIN', 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789', 2, 1)
14 36746512 (None, 'MATCH_EOF', 1, -1, 1)
15 36326776 (None, 'MATCH_WORD', '\t=')
16 36326416 (None, 'MATCH_WORD', '"')
17 36326136 ('wordspecial', 'MATCH_TABLE', (('alpha', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'), ('alphanumspecial', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_-', 2, 1), (None, 101, 1, -1, 1)))
18 36292208 ('[RECURSION of Node=36741928]', 'MATCH_RECURSION', 36741928)
19 36385336 (None, 'MATCH_WORD', '"')
20 36385096 (None, 'MATCH_WORD', '\n')
21 36746560 ('entry', 'MATCH_TABLE', (('word', 203, (('alpha', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'), ('alphanum', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789', 2, 1), (None, 101, 1, -1, 1))), (None, 21, '\t='), (None, 21, '"'), ('wordspecial', 203, (('alpha', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'), ('alphanumspecial', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_-', 2, 1), (None, 101, 1, -1, 1))), (None, 21, '"'), (None, 21, '\n')), 2, 1)
22 36292208 ('[RECURSION of Node=36304296]', 'MATCH_RECURSION', 36304296)
23 36746608 (None, 'MATCH_EOF', 1, -1, 1)
24 36384936 (None, 'MATCH_WORD', '\n\n')
25 36746704 (None, 'MATCH_SUBTABLE', (('section', 203, ((None, 21, '['), ('wordspecial', 203, (('alpha', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'), ('alphanumspecial', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_-', 2, 1), (None, 101, 1, -1, 1))), (None, 21, ']'), (None, 21, '\n'))), ('entry', 203, (('word', 203, (('alpha', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'), ('alphanum', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789', 2, 1), (None, 101, 1, -1, 1))), (None, 21, '\t='), (None, 21, '"'), ('wordspecial', 203, (('alpha', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'), ('alphanumspecial', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_-', 2, 1), (None, 101, 1, -1, 1))), (None, 21, '"'), (None, 21, '\n'))), ('entry', 203, (('word', 203, (('alpha', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'), ('alphanum', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789', 2, 1), (None, 101, 1, -1, 1))), (None, 21, '\t='), (None, 21, '"'), ('wordspecial', 203, (('alpha', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'), ('alphanumspecial', 14, 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_-', 2, 1), (None, 101, 1, -1, 1))), (None, 21, '"'), (None, 21, '\n')), 2, 1), (None, 101, 1, -1, 1), (None, 21, '\n\n')), 2, 1)
26 36292208 ('[RECURSION of Node=36746656]', 'MATCH_RECURSION', 36746656)
27 36746752 (None, 'MATCH_EOF', 1, -1, 1)