Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Perl 6 Pod lexer #978

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
28 changes: 28 additions & 0 deletions lib/rouge/demos/perl6pod
@@ -0,0 +1,28 @@
#! /usr/bin/env perl6
pyrmont marked this conversation as resolved.
Show resolved Hide resolved

=begin pod

=NAME Test Program for Pod::To::Parser
=AUTHOR Patrick Spek
=VERSION 0.0.1

=begin LICENSE
GNU Affero GPLN<Version 3>
=end LICENSE

=head1 A test program for App::POD::Manual

Now we're reaching the real POD document that I care about. Let's add in some
test cases as well here.

=head2 Text styles

=item1 This text is B<bold>
=item1 This text is I<italic>
=item1 This text is U<underlined>
=item1 This text is C<code>
=item1 This text is L<a link|https://www.tyil.work> to my blog
=item1 This text is normal Z<or is it?>
=item1 I'm running ouf of ideas N<Search the Internet for more!>

=end pod
118 changes: 118 additions & 0 deletions lib/rouge/lexers/perl6pod.rb
@@ -0,0 +1,118 @@
# -*- coding: utf-8 -*- #

module Rouge
module Lexers
class Pod6 < RegexLexer
title "Perl 6 Pod"
desc "Perl 6 Plain' Ol Documentation (perl6.org)"

tag 'perl6pod'
aliases 'pod6'

filenames '*.pod', '*.pod6'

state :root do
rule(/\s*\=begin pod/m, Keyword, :pod6)
rule(/^#.*$/, Comment)
rule(/[^=]*/m, Text)
end

state :pod6 do
rule(/\=end pod/m, Keyword, :pop!)

rule(/\=(?:NAME|AUTHOR|VERSION|TITLE|SUBTITLE)/, Keyword, :semantic)

rule(/\=begin code/, Keyword, :block_code)
rule(/\=begin input/, Keyword, :block_input)
rule(/\=begin output/, Keyword, :block_output)
rule(/\=begin/, Keyword, :block)

rule(/\=head(\d+)\s+/, Keyword, :head)
rule(/\=code/, Keyword, :code)
rule(/\=item(\d+)\s+/, Keyword, :item)
rule(/\=input/, Keyword, :input)
rule(/\=output/, Keyword, :output)
rule(/\=defn/, Keyword)

rule(/^(?:\t|[ ]{4,})/, Other, :code)
rule(/[^=]*/m, Text)
end

state :semantic do
rule(/\n/, Name, :pop!)

rule(/.*/, Name)
end

state :head do
rule(/\n/, Text::Whitespace, :pop!)

rule(/:\w+\</, Name::Attribute, :attribute)
rule(/.*/, Generic::Heading)
end

state :code do
rule(/\n/, Text::Whitespace, :pop!)

rule(/.*/, Generic::Output)
end

state :item do
rule(/\n/, Text::Whitespace, :pop!)

rule(/[^\n]/, Generic)
end

state :input do
rule(/\n/, Text::Whitespace, :pop!)

rule(/.*/, Generic::Inserted)
end

state :output do
rule(/\n/, Text::Whitespace, :pop!)

rule(/.*/, Generic::Output)
end

state :code do
rule(/[^\n]/m, Generic::Output, :pop!)
end

state :attribute do
rule(/\>/, Name::Attribute, :pop!)

rule(/[^>]*/, Name)
end

state :block do
rule(/\=end/, Keyword, :pop!)

# TODO: Check the name of the block, and make sure the =end matches
# the same name

rule(/:\w+\</, Name::Attribute, :attribute)
rule(/.*/, Generic)
end

state :block_code do
rule(/\=end code/, Keyword, :pop!)
rule(/.*/, Generic::Output)
end

state :block_input do
rule(/:\w+\</, Name::Attribute, :attribute)
rule(/\=end input/, Keyword, :pop!)

rule(/.*/, Generic::Inserted)
end

state :block_output do
rule(/:\w+\</, Name::Attribute, :attribute)
rule(/\=end output/, Keyword, :pop!)

rule(/.*/, Generic::Output)
end
end
end
end
123 changes: 123 additions & 0 deletions spec/visual/samples/perl6pod
@@ -0,0 +1,123 @@
#! /usr/bin/env perl6

=begin pod

=NAME Test Program for Pod::To::Parser
=AUTHOR Patrick Spek
=VERSION 0.0.1

=begin LICENSE
GNU Affero GPLN<Version 3>
=end LICENSE

=head1 A test program for App::POD::Manual

Now we're reaching the real POD document that I care about. Let's add in some
test cases as well here.

=head2 Text styles

=item1 This text is B<bold>
=item1 This text is I<italic>
=item1 This text is U<underlined>
=item1 This text is C<code>
=item1 This text is L<a link|https://www.tyil.work> to my blog
=item1 This text is normal Z<or is it?>
=item1 I'm running ouf of ideas N<Search the Internet for more!>

=head2 Lists

=item2 Starting off at level 2
=item5 Progressing to level 5
=item1 Back to level 1
=item2 On to level 2
=item3 On to level 3
=item4 On to level 4

=item8 But what about an item with content that goes well beyond the 80 characters?

=begin item10
Which can easily occur in a block item, for instance. It should wrap around
and bring the next lines on a similar level of indentation.

The question is, does it?
=end item10

=head3 Definition lists

=defn pod
Plain Ol' Documentation

=defn foo
Not the same as bar

=head2 Code blocks

Now on to some code blocks. These are generally used to show code samples.

This is a code block. It is indented by 4 spaces to indicate this.

Code samples are important to easily show a user how to interact with the
program they're using. Nobody wants to read through pages of a manual when they
just want to know how to use it for their particular use case.

=begin code :name<Named code block>
This is a code block by name. It is covered by a begin and end block. It has no limit to line length.
Spaces in the code are preserved .

Similar for newlines, actually.
=end code

=code Code on a single line should also work as expected.

And that concludes the test for code blocks!

=head2 IO blocks

There are blocks to denote program input and output, called IO blocks. They
also have in-line variants:

=item1 This is K<keyboard input>
=item1 This is T<terminal output>

The bigger blocks deserve a test as well, I think:

=head3 Short-hand input and output

=input A simple input line
=output A simple output line

=head3 Big input and output blocks

=begin input
A larger block of input.
=end input

=begin output
A larger block of output.
=end output

=head2 Unicode

And of course, let's throw in some Unicode stuff. I stole these examples from
the documentation site, but I don't think that's a bad source of input for
testing purposes.

Perl 6 makes considerable use of the E<171> and E<187> characters.

Perl 6 makes considerable use of the E<laquo> and E<raquo> characters.N<Not sure why these don't work>

Perl 6 makes considerable use of the E<0b10101011> and E<0b10111011> characters.

Perl 6 makes considerable use of the E<0o253> and E<0o273> characters.

Perl 6 makes considerable use of the E<0d171> and E<0d187> characters.

Perl 6 makes considerable use of the E<0xAB> and E<0xBB> characters.

=head2 Conclusion

And that concludes this test file. If everything looked good in your tests, you
may now be confident about your work.

=end pod