Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse files

[api] Added pystring support to Kyuri lexer

  • Loading branch information...
commit 0eb868760b57aac8cf497497220f66618068adda 1 parent 6b97dfd
@indexzero indexzero authored
View
3  examples/simple.feature
@@ -4,6 +4,9 @@ Feature: Addition
I want to be told the sum of two numbers
Scenario: Add two numbers
+ """
+ Hey I'm a pystring, check me out now
+ """
Given I have entered 50 into the calculator
And I have entered 70 into the calculator
When I press add
View
12 lib/kyuri.js
@@ -11,9 +11,9 @@ require.paths.unshift(__dirname);
var kyuri = exports;
kyuri.version = '0.0.1';
-kyuri.compile = require('kyuri/core').compile;
-kyuri.parse = require('kyuri/core').parse;
-kyuri.tokens = require('kyuri/core').tokens;
-kyuri.nodes = require('kyuri/core').nodes;
-kyuri.lexer = require('kyuri/core').lexer;
-kyuri.i18n = require('kyuri/core').i18n;
+kyuri.compile = require('kyuri/core').compile;
+kyuri.parse = require('kyuri/core').parse;
+kyuri.tokens = require('kyuri/core').tokens;
+kyuri.nodes = require('kyuri/core').nodes;
+kyuri.setLanguage = require('kyuri/core').setLanguage;
+kyuri.i18n = require('kyuri/core').i18n;
View
6 lib/kyuri/core.js
@@ -42,6 +42,8 @@ exports.nodes = function (code) {
return parser.parse(lexer.tokenize(code));
};
-exports.i18n = i18n;
+exports.setLanguage = function (language) {
+ lexer = new Lexer(language, i18n);
+};
-exports.lexer = lexer;
+exports.i18n = i18n;
View
28 lib/kyuri/lexer.js
@@ -13,6 +13,9 @@ var helpers = require('./helpers'),
var MULTI_DENT = /^(\t+)(\.)?/,
IS_EXAMPLE_ROW = /^([\|\s+\S+]+\s+\|\s*)$/,
PARSE_EXAMPLE_ROW = /\|\s*(\S+)/gi,
+ PYSTRING = /"""/,
+ TAG = /@(\w+)/i,

That's too restrictive for tags, we allow any character except newline and whitespace. Take a look at the pattern from the ragel grammar:
http://github.com/aslakhellesoy/gherkin/blob/master/ragel/lexer_common.rl.erb#L34

@indexzero Owner

Yeah, you're right. I noticed that myself after I saw a line with two tags on it. I'll update it now and commit

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+ COMMENT = /#([\S+\s+]+)/i,
SENTENCE = /([\S+\s+]+)/i;
var Lexer = function (language, i18n) {
@@ -90,7 +93,7 @@ Lexer.prototype = {
extractNextToken: function () {
var tokens = ['lineToken', 'commentToken', 'featureToken', 'scenarioToken', 'scenarioOutlineToken',
- 'backgroundToken', 'examplesToken', 'examplesRowToken', 'operatorToken', 'sentenceToken'];
+ 'backgroundToken', 'examplesToken', 'examplesRowToken', 'operatorToken', 'pystringToken', 'sentenceToken'];
for (var index in tokens) {
if (this[tokens[index]].apply(this)) {
@@ -163,8 +166,29 @@ Lexer.prototype = {
return true;
},
- commentToken: function() {
+ pystringToken: function () {
+ var match;
+ if (!(match = this.match(PYSTRING))) {
+ return false;
+ }
+ eyes.inspect(match);
+ eyes.inspect(match[0].length);
+
+ this.i += match[0].length;
+ this.token('PYSTRING', 'PYSTRING');
+
+ return true;
+ },
+
+ tagToken: function () {
+ // TODO: Implement comments [http://github.com/nodejitsu/kyuri/issues/5]
+ return false;
+ },
+
+ commentToken: function() {
+ // TODO: Implement comments [http://github.com/nodejitsu/kyuri/issues/6]
+ return false;
},
sentenceToken: function () {
View
6 test/lexer-test.js
@@ -41,15 +41,15 @@ vows.describe('kyuri/lexer').addBatch({
topic: readAllLines(path.join(__dirname, '..', 'examples', 'simple.feature')),
"should lex correctly": function (err, data) {
assert.isNotNull(data.toString());
- eyes.inspect(kyuri.tokens(data.toString()));
+ inspect(kyuri.tokens(data.toString()));
}
- },
+ }/*,
"lexing complex.feature": {
topic: readAllLines(path.join(__dirname, '..', 'examples', 'complex.feature')),
"should lex correctly": function (err, data) {
assert.isNotNull(data.toString());
inspect(kyuri.tokens(data.toString()));
}
- }
+ }*/
}
}).export(module);
@aslakhellesoy

That's too restrictive for tags, we allow any character except newline and whitespace. Take a look at the pattern from the ragel grammar:
http://github.com/aslakhellesoy/gherkin/blob/master/ragel/lexer_common.rl.erb#L34

@indexzero

Yeah, you're right. I noticed that myself after I saw a line with two tags on it. I'll update it now and commit

Please sign in to comment.
Something went wrong with that request. Please try again.