Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse files

make parser recognize keys with non-ASCII characters

add 1 test
  • Loading branch information...
commit 79b8e13856031f3f95318ae4abea206913aac6c0 1 parent 71285f2
@cruessler cruessler authored
Showing with 21 additions and 4 deletions.
  1. +17 −0 features/issues/non_ascii_keys.feature
  2. +4 −4 lib/bibtex/lexer.rb
View
17 features/issues/non_ascii_keys.feature
@@ -0,0 +1,17 @@
+Feature: Keys containing non-ASCII characters
+ As a hacker who works with bibliographies
+ I want to parse BibTeX entries with keys containing non-ASCII characters
+ Because they can occur in many languages other than English
+
+ Scenario: An entry whose key contains a German umlaut
+ When I parse the following file:
+ """
+ @article{müller.2011,
+ author = {Christian Müller},
+ title = {Important article},
+ journal = {Not so important journal},
+ volume = {5},
+ year = {2011}
+ }
+ """
+ Then my bibliography should contain an article with id "müller.2011"
View
8 lib/bibtex/lexer.rb
@@ -46,8 +46,8 @@ class Lexer
:braces => /\{|\}/o,
:eq => /\s*=\s*/o,
:comma => /\s*,\s*/o,
- :number => /\d+/o,
- :name => /[[:alpha:]\d\/:_!$\?\.%&\*-]+/io,
+ :number => /[[:digit:]]+/o,
+ :name => /[[:alpha:][:digit:]\/:_!$\?\.%&\*-]+/io,
:quote => /\s*"/o,
:unquote => /[\{\}"]/o,
:sharp => /\s*#\s*/o,
@@ -59,8 +59,8 @@ class Lexer
:string => /string/io,
:comment => /comment\b/io,
:preamble => /preamble\b/io,
- :key => /\s*[[:alpha:]\d \/:_!$\?\.%+&\*-]+,/io,
- :optional_key => /\s*[[:alpha:]\d \/:_!$\?\.%+&\*-]*,/io
+ :key => /\s*[[:alpha:][:digit:] \/:_!$\?\.%+&\*-]+,/io,
+ :optional_key => /\s*[[:alpha:][:digit:] \/:_!$\?\.%+&\*-]*,/io
}.freeze
MODE = Hash.new(:meta).merge({
Please sign in to comment.
Something went wrong with that request. Please try again.