Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Solidity (lexer and examples) #760

Merged
merged 31 commits into from
Apr 5, 2020
Merged
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
afd8be9
spec/lexers/sampes/demos: add Solidity lexer (squashed branch).
veox Aug 19, 2017
672ee0a
lexers/solidity: don't map/merge sets on every invocation - just the …
veox Aug 22, 2017
d5f6378
lexers/solidity: add new keywords mentioned in feedback.
veox Mar 26, 2020
e1b6522
lexers/solidity: add changes up to (mostly) Solidity v0.4.22.
veox Mar 26, 2020
e65c995
lexers/solidity: add new keywords from Solidity v0.6.0.
veox Mar 26, 2020
612c65b
samples/solidity: copy updated sample over from Pygments.
veox Mar 26, 2020
dbec54a
lexers/solidity: mark regexes in rules explicitly.
veox Mar 26, 2020
c219854
spec/solidity: uncomment and fix "test by source".
veox Mar 26, 2020
a8ee009
lexers/solidity: variables can have `$` + `string` is a type.
veox Mar 31, 2020
7692571
Fix alphabetical ordering of types
pyrmont Mar 31, 2020
9ca9556
Replace character ranges with metacharacter
pyrmont Mar 31, 2020
aded449
Fix 'uintb' typo
pyrmont Mar 31, 2020
4ba1cf7
samples/solidity: show `$` is a valid character in the visual sample.
veox Mar 31, 2020
f416146
lexers/solidity: move [u]fixed type to `reserved` + add missing `byte`.
veox Mar 31, 2020
2a14012
lexer/solidity: `receive`, `override`, `virtual` are now actual keywo…
veox Mar 31, 2020
a828d47
lexers/solidity: update `[u]fixed{M}x{N}` types to "new" scheme.
veox Mar 31, 2020
8b09d51
lexers/solidity: add missing `calldata` keyword.
veox Apr 1, 2020
5f81d9b
lexers/solidity: user regex to match [u]fixed{M}x{N} instead.
veox Apr 2, 2020
6f793ba
lexers/solidity: [u]fixed{M}x{N} should have at least one digit for M…
veox Apr 3, 2020
2242a4d
lexers/solidity: use regex for bytes{N} and [u]int{N}, too.
veox Apr 3, 2020
d7ccc70
lexers/solidity: remove duplicate rule for (block|msg|tx).{stuff}.
veox Apr 3, 2020
f19341e
lexers/solidity: catch case of unclosed multi-line comment.
veox Apr 4, 2020
8b42435
spec/solidity: update with less incorrect language syntax.
veox Apr 4, 2020
6d46215
Add nested comments
pyrmont Apr 5, 2020
c8d3f3f
Add state for multiline comments
pyrmont Apr 5, 2020
2e824fa
Simplify demo
pyrmont Apr 5, 2020
b56f33c
Remove nested closing comments
pyrmont Apr 5, 2020
479a6d2
Remove nesting rule from comment state
pyrmont Apr 5, 2020
8b471bb
samples/solidity: make example of comment-to-EOF more wordy.
veox Apr 5, 2020
103718e
lexers/solidity: add missing `abi.decode`.
veox Apr 5, 2020
1555984
samples/solidity: split no-multiline-nesting and multiline-to-eof cases.
veox Apr 5, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions lib/rouge/demos/solidity
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
pragma solidity ~0.4.15;

interface IMirror {
function reflect() external payable returns(bool /* ain't I pretty?.. */);
}

contract Mirror is IMirror {
event logMessage(address indexed sender, uint256 value, uint256 gas, bytes data);

function reflect() external payable returns(bool retval) {
assert(msg.sender != address(this));
require(msg.value != 0);

IMirror rorrim = IMirror(msg.sender);
retval = rorrim.reflect.value(msg.value).gas(msg.gas)();

logMessage(msg.sender, msg.value, msg.gas, msg.data);
return retval;
}

function () { // no funny stuff
revert();
}
}
180 changes: 180 additions & 0 deletions lib/rouge/lexers/solidity.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
# -*- coding: utf-8 -*- #

module Rouge
module Lexers
class Solidity < RegexLexer
title "Solidity"
desc "Solidity, an Ethereum smart contract programming language"
tag 'solidity'
filenames '*.sol', '*.solidity'
mimetypes 'text/x-solidity'

# optional comment or whitespace
ws = %r((?:\s|//.*?\n|/[*].*?[*]/)+)
id = /[a-zA-Z$_][\w$_]*/

def self.detect?(text)
return true if text.start_with? 'pragma solidity'
end

# TODO: seperate by "type"
def self.keywords
@keywords ||= Set.new %w(
abstract anonymous as assembly break catch calldata constant
constructor continue contract do delete else emit enum event
external fallback for function hex if indexed interface
internal import is library mapping memory modifier new
override payable public pure pragma private receive return
returns storage struct throw try type using var view virtual
while
)
end

def self.builtins
return @builtins if @builtins

@builtins = Set.new %w(
now
false true
balance now selector super this
blockhash gasleft
assert require revert
selfdestruct suicide
call callcode delegatecall
send transfer
addmod ecrecover keccak256 mulmod sha256 sha3 ripemd160
)

# TODO: use (currently shadowed by catch-all in :statements)
abi = %w(encode encodePacked encodeWithSelector encodeWithSignature)
veox marked this conversation as resolved.
Show resolved Hide resolved
@builtins.merge( abi.map { |i| "abi.#{i}" } )
block = %w(coinbase difficulty gaslimit hash number timestamp)
@builtins.merge( block.map { |i| "block.#{i}" } )
msg = %w(data gas sender sig value)
@builtins.merge( msg.map { |i| "msg.#{i}" } )
tx = %w(gasprice origin)
@builtins.merge( tx.map { |i| "tx.#{i}" } )
end

def self.constants
@constants ||= Set.new %w(
wei finney szabo ether
seconds minutes hours days weeks years
)
end

def self.keywords_type
@keywords_type ||= Set.new %w(
address bool byte bytes int string uint
)
end

def self.reserved
@reserved ||= Set.new %w(
alias after apply auto case copyof default define final fixed
immutable implements in inline let macro match mutable null of
partial promise reference relocatable sealed sizeof static
supports switch typedef typeof ufixed unchecked
)
end

start { push :bol }

state :expr_bol do
mixin :inline_whitespace

rule(//) { pop! }
end

# :expr_bol is the same as :bol but without labels, since
# labels can only appear at the beginning of a statement.
state :bol do
mixin :expr_bol
end

# TODO: natspec in comments
state :inline_whitespace do
rule %r/[ \t\r]+/, Text
rule %r/\\\n/, Text # line continuation
rule %r(/[*].*?[*]/)m, Comment::Multiline
rule %r(/\*.*(\*/){0})m, Comment::Multiline # open to EOF
pyrmont marked this conversation as resolved.
Show resolved Hide resolved
end

state :whitespace do
rule %r/\n+/m, Text, :bol
rule %r(//(\\.|.)*?\n), Comment::Single, :bol
mixin :inline_whitespace
end

state :expr_whitespace do
rule %r/\n+/m, Text, :expr_bol
mixin :whitespace
end

state :statements do
mixin :whitespace
rule %r/(hex)?\"/, Str, :string_double
rule %r/(hex)?\'/, Str, :string_single
rule %r('(\\.|\\[0-7]{1,3}|\\x[a-f0-9]{1,2}|[^\\'\n])')i, Str::Char
rule %r/\d\d*\.\d+([eE]\d+)?/i, Num::Float
rule %r/0x[0-9a-f]+/i, Num::Hex
rule %r/\d+([eE]\d+)?/i, Num::Integer
rule %r(\*/), Error
rule %r([~!%^&*+=\|?:<>/-]), Operator
rule %r/[()\[\],.]/, Punctuation
rule %r/u?fixed\d+x\d+/, Keyword::Reserved
rule %r/bytes\d+/, Keyword::Type
rule %r/u?int\d+/, Keyword::Type
rule id do |m|
name = m[0]

if self.class.keywords.include? name
token Keyword
elsif self.class.builtins.include? name
token Name::Builtin
elsif self.class.constants.include? name
token Keyword::Constant
elsif self.class.keywords_type.include? name
token Keyword::Type
elsif self.class.reserved.include? name
token Keyword::Reserved
else
token Name
end
end
end

state :root do
mixin :expr_whitespace
rule(//) { push :statement }
# TODO: function declarations
end

state :statement do
rule %r/;/, Punctuation, :pop!
mixin :expr_whitespace
mixin :statements
rule %r/[{}]/, Punctuation
end

state :string_common do
rule %r/\\(u[a-fA-F0-9]{4}|x..|[^x])/, Str::Escape
rule %r/[^\\\"\'\n]+/, Str
rule %r/\\\n/, Str # line continuation
rule %r/\\/, Str # stray backslash
end

state :string_double do
mixin :string_common
rule %r/\"/, Str, :pop!
rule %r/\'/, Str
end

state :string_single do
mixin :string_common
rule %r/\'/, Str, :pop!
rule %r/\"/, Str
end
end
end
end
22 changes: 22 additions & 0 deletions spec/lexers/solidity_spec.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# -*- coding: utf-8 -*- #

describe Rouge::Lexers::Solidity do
let(:subject) { Rouge::Lexers::Solidity.new }

describe 'guessing' do
include Support::Guessing

it 'guesses by filename' do
assert_guess :filename => 'foo.sol'
assert_guess :filename => 'foo.solidity'
end

it 'guesses by mimetype' do
assert_guess :mimetype => 'text/x-solidity'
end

it 'guesses by source' do
assert_guess :source => 'pragma solidity'
end
end
end
Loading