Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proper implementation of Dir.glob #5179

Merged
merged 6 commits into from Jan 11, 2018
Merged

Conversation

straight-shoota
Copy link
Member

@straight-shoota straight-shoota commented Oct 24, 2017

This PR implements Dir.glob with a proper globbing algorithm without employing regular expressions. This results in improved performance and also fixes previously invalid results in some edge cases.

For these simple examples it is faster than the current implementation by an average of 300%:

#{__DIR__}/spec/std/data/dir/*.txt
master  26.94k ( 37.12µs) (± 4.59%)  2.57× slower
pull    69.36k ( 14.42µs) (±21.30%)       fastest

#{__DIR__}/spec/std/data/dir/**/*.txt
master  12.03k ( 83.13µs) (±18.19%)  1.70× slower
pull    20.47k ( 48.85µs) (± 2.24%)       fastest

#{__DIR__}/spec/std/data/{dir,dir/subdir}/*.txt
master    7.4k (135.22µs) (±22.97%)  4.84× slower
pull    35.79k ( 27.94µs) (±16.80%)       fastest

#{__DIR__}/spec/std/data/dir/{**/*.txt,**/*.txx}
master  10.87k ( 92.03µs) (±13.72%)       fastest
pull    10.18k ( 98.21µs) (±11.37%)  1.07× slower

#{__DIR__}/spec/std/data/dir/{?1.*,{f,g}2.txt}
master  27.26k ( 36.69µs) (±20.71%)  1.27× slower
pull    34.72k (  28.8µs) (± 4.89%)       fastest

#{__DIR__}/spec/std/data/dir/*
master  27.64k ( 36.18µs) (± 2.66%)  2.40× slower
pull    66.36k ( 15.07µs) (±15.38%)       fastest

#{__DIR__}/spec/std/data/dir/**
master  12.33k ( 81.13µs) (± 1.46%)  5.13× slower
pull    63.24k ( 15.81µs) (± 3.20%)       fastest

#{__DIR__}/spec/std/data/dir/*/
master  29.73k ( 33.63µs) (± 1.83%)  1.35× slower
pull    40.03k ( 24.98µs) (±16.16%)       fastest

#{__DIR__}/spec/std////data////dir////*.txt
master  12.28k ( 81.46µs) (±11.26%)  5.45× slower
pull    66.89k ( 14.95µs) (± 1.68%)       fastest

The performance improvement is roughly constant for bigger trees.

Fixes #3020

@straight-shoota
Copy link
Member Author

straight-shoota commented Oct 24, 2017

One example is slightly faster in master because this implementation expands curly braces into individual patterns (this was borrowed from Rubinius implementation), so the pattern #{__DIR__}/spec/std/data/dir/{**/*.txt,**/*.txx} is essentially the same as ["#{__DIR__}/spec/std/data/dir/**/*.txt}", "#{__DIR__}/spec/std/data/dir/**/*.txx"], thus traversing through the directories twice.

This could probably be improved by merging identical pattern start sequences, though I'd like to get feedback on the general implementation first.

src/dir/glob.cr Outdated
# * `"c*"` matches all files beginning with `c`.
# * `"*c"` matches all files ending with `c`.
# * `"*c*"` matches all files that have `c` in them (including at the beginning or end).
# * `**` matches firectories recursively or files expansively.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo: firectories

src/dir/glob.cr Outdated
# * `"*c"` matches all files ending with `c`.
# * `"*c*"` matches all files that have `c` in them (including at the beginning or end).
# * `**` matches firectories recursively or files expansively.
# * `?` matches any one charachter.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo: charachter => character

src/dir/glob.cr Outdated
# * `"c*"` matches all files beginning with `c`.
# * `"*c"` matches all files ending with `c`.
# * `"*c*"` matches all files that have `c` in them (including at the beginning or end).
# * `**` matches firectories recursively or files expansively.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

src/dir/glob.cr Outdated
# * `"*c"` matches all files ending with `c`.
# * `"*c*"` matches all files that have `c` in them (including at the beginning or end).
# * `**` matches firectories recursively or files expansively.
# * `?` matches any one charachter.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto

src/dir/glob.cr Outdated
# * `"c*"` matches all files beginning with `c`.
# * `"*c"` matches all files ending with `c`.
# * `"*c*"` matches all files that have `c` in them (including at the beginning or end).
# * `**` matches firectories recursively or files expansively.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto

src/dir/glob.cr Outdated
# * `"*c"` matches all files ending with `c`.
# * `"*c*"` matches all files that have `c` in them (including at the beginning or end).
# * `**` matches firectories recursively or files expansively.
# * `?` matches any one charachter.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto

src/dir/glob.cr Outdated
end
# Yields all files that match against any of *patterns*.
#
# The pattern syntax is similar to shell filename globbing. It may contain the following metacharacters:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pattern syntax is copy/pasted multiple times (with its errors too 😄), maybe it would be best to explain it once at the top, and refer to it when needed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, probably. But I'm not sure how and where.

@asterite
Copy link
Member

Awesome, thanks! I didn't have time to review this, and it will probably take me a while.

I was going to comment that you dropped some features from the original implementation, like character sets ([abc]), negation ([^abc]) and sets ([a-z]) but it seems those were not present in the original implementation.

I'm not sure about the options. In Ruby there's more than just one, they are:

[:FNM_NOESCAPE, :FNM_PATHNAME, :FNM_DOTMATCH, :FNM_CASEFOLD, :FNM_EXTGLOB, :FNM_SYSCASE, :FNM_SHORTNAME]

I don't know what they mean, though, and the documentation is not very good. There's glob in Go too and they don't provide any options, I'm not sure why we need to have these options.

@straight-shoota
Copy link
Member Author

straight-shoota commented Oct 25, 2017

My goal was to provide an improved implementation that matches the features of the current one. I didn't want to spend time on any additions without getting the basics right and approved ;) (Well, the option allow_dots is actually an additional feature, so there is only one)

I guess feature additions should probably better be discussed individually, because they might require some thought about what is needed and what not. Additional features for fnmatch like character sets are certainly benefitial. But I share you'r concerns about adding all these options. Some might be usefull, though. They are btw. explained in File::Constants.

Copy link
Member

@asterite asterite left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't finish reviewing it, just some minor comments

@@ -1,10 +1,5 @@
require "spec"

private def assert_dir_glob(expected_result, *patterns)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What was wrong with this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If find it impractical as it doesn't transmit the location of the failing expectation (all failing specs fail in line 5). This might not be that important because specs should generally match (except when refactoring or improving that code).

It also doesn't do much - essentially it just adds the two sorts - and feels like if anything it is hiding functionality.

src/file.cr Outdated
strlen = path.bytesize

while true
pnext? = preader.current_char != Char::ZERO
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

preader.has_next?

Same in the next line

Also, pnext? as a variable name is an unfortunate accident of the current parser, that shouldn't be legal (at least it's not legal in Ruby). It's quite confusing. So it's better to use another name because that will break in an eventual change/fix.

src/dir/glob.cr Outdated
yield File::SEPARATOR + path
else
yield path
alternatives = [] of String
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe lazily create this array?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I understand where you're going...
Do you mean something like:

alternatives : Array(String)?
# ...
(alternatives ||= [] of String) << pattern.byte_slice(start, reader.pos - start)
# ...
if alternatives

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, like that. I don't know if it affects performance, though.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would rather leave it as is. If it has any impact at all it is totally insignificant. This method is only run once for each glob pattern as long as it doesn't have any curly braces - if it does, this might actually make it worse because of the additional nil checks.

src/dir/glob.cr Outdated
alternatives = [] of String

nest = 0
while char = reader.current_char
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

while reader.has_next?
  reader.next_char
end

or something like that. A crystal string can contain the null character and that doesn't mean it's the end there.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess reader.each is even better.

src/dir/glob.cr Outdated
# characters which must be escaped in a PCRE regex:
escaped = {'$', '(', ')', '+', '.', '[', '^', '|', '/'}
alternatives.each do |alt|
brace_pattern = [front, alt, back].join
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe {front, alt, back}.join

src/dir/glob.cr Outdated
list << DirectoriesOnly.new
else
file = parts.pop
if /^[a-zA-Z0-9._]+$/.match(file)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought regex wasn't used 😄

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not for the globbing itself ^^

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think not using a regex is not that hard in this case. We can use this method, I think:

private def constant_entry?(file)
  file.each_char do |char|
    return false if char == '*' || char == '?'
  end

  true
end

I think we can use that here and a bit below (in the other regex). I'm not sure why the regexes differ. But if you need a difference you can always write two methods with similar but different checks. In my benchmarks this improves things a little. The other bottleneck is File.join, I'll try to optimize that for common cases (here a common case is joining 2 pieces).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that seems to be fine. But we'll need to update the special characters when others get added.

I'd also like this:

SPECIAL = {'?', '*'}
return false if SPECIAL.includes?(c)

But it is a bit slower than the individual comparisons...

src/dir/glob.cr Outdated

last_is_file_separator = char == File::SEPARATOR
while !parts.empty?
dir = parts.pop
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you really need pop here? I think not modifying the array will be faster. If not, you can also do while dir = parts.pop?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think pop is necessary.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In fact, StartRecursiveDirectories was not necessary at all. It was taken from the rubinius implementation but it seems we don't need to distinguish because of a different iteration algorithm.

src/dir/glob.cr Outdated
str << "[^\\" << File::SEPARATOR << "]*"
idx += 2
next
when /^[^\*\?\]]+$/
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if there's a way to avoid regexes at all.

Still, your changes are an improvement over the original code, so it doesn't matter much (it's better if it works well first)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this can be improved. Though I think it is okay to use regexes for these tasks, since compile is only run once per glob run, not for every matching file.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No problem, performance can always be improved later :-)

@straight-shoota
Copy link
Member Author

@asterite Thanks for the (partial) review.
Some comments were marked as outdated by my commit, even if I didn't touch that line - I don't know why:

@Sija
Copy link
Contributor

Sija commented Oct 26, 2017

This 🚢 is ready for ⛵️ !

src/file.cr Outdated
# * `"*c"` matches all files ending with `c`.
# * `"*c*"` matches all files that have `c` in them (including at the beginning or end).
# * `?` matches any one charachter.
def self.fnmatch(pattern, path)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we rename this to just match? fnmatch sounds super cryptic. I know it comes from C, but I don't think keeping C names is good. For example there's no fnmatch in Windows. So a name that's descriptive and applicable independent of the platform is preferred.

Also, please add String type restrictions here.

src/dir/glob.cr Outdated
# * `{a,b}` matches subpattern `a` or `b`.
#
# **options:**
# * *allow_dots*: Match hidden files if `true` (default: `false`).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's keep the options. I was thinking of maybe using an enum but options is nicer here, simpler to specify. I'm not sure about the name, though. In C it's called FNM_PERIOD, in Ruby it's FNM_DOTMATCH but what it means is that it matches hidden files, or files that start with a dot/period. allow_dots suggests either the path or pattern allows dots anywhere. Using "hidden files" isn't totally correct either because in Windows that's a different thing. Maybe just match_leading_dot or similar?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good 👍

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a reminder that allow_dots still needs to be renamed to match_leading_lot.

In fact, since before this PR we didn't have this option, you could actually remove it for now. We can always add it later. I don't know why .foo files should be treated differently.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, sure. I was just waiting if maybe someone would come up with a better name.
What do you think about recurse_hidden_files? I think this would be more descriptive.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current implementation matches hidden files, but the default should be not to. So that's a breaking change. We should have this option right now to allow people to keep the existing behaviour.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not? I just tried this in Go:

package main

import "path/filepath"
import "fmt"

func main() {
	matches, _ := filepath.Glob("./*")
	fmt.Println(matches)
}

Here's part of the output:

[.build .dockerignore .editorconfig .git .gitignore .travis.yml ...

Go doesn't hide dot files by default. Ruby does. That doesn't mean either is correct. But I don't understand why dot files are so important (or not so important) that they should be hidden.

I think we should start with no options, showing all files. Later, in a separate issue/PR, we can discuss options to add to this method.

src/dir/glob.cr Outdated
list << DirectoriesOnly.new
else
file = parts.pop
if /^[a-zA-Z0-9._]+$/.match(file)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think not using a regex is not that hard in this case. We can use this method, I think:

private def constant_entry?(file)
  file.each_char do |char|
    return false if char == '*' || char == '?'
  end

  true
end

I think we can use that here and a bit below (in the other regex). I'm not sure why the regexes differ. But if you need a difference you can always write two methods with similar but different checks. In my benchmarks this improves things a little. The other bottleneck is File.join, I'll try to optimize that for common cases (here a common case is joining 2 pieces).

@@ -1032,4 +1032,26 @@ describe "File" do
end
end
end

describe ".fnmatch" do
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fnmatch -> match?

@straight-shoota
Copy link
Member Author

@asterite Most glob implementations I've come across don't match/recurse hidden files or folders by default, unless explicitly told to, either with a flag or a wildcard prefixed by a dot.

Go and Rust seem to do it the other way around, though Rust has an option not to traverse hidden files/folders.

I'd personally prefer to stick to the behaviour people are used to. When you glob * in a shell, you won't get any hidden files.

@Sija
Copy link
Contributor

Sija commented Oct 31, 2017

I'd keep the option match_leading_dot (default: true).

@RX14
Copy link
Contributor

RX14 commented Oct 31, 2017

I don't really care what the default is but i think it should be match_hidden which ignores things starting with . on unix, and actually uses hidden attribute (probably with the . rule too) on windows.

@straight-shoota
Copy link
Member Author

So, I've added the missing pattern features to File#match?.

The question about what to do with the option is still open, because I'm not sure about how we should proceed. Maybe we can find something about the motivation for Go and Rust to include hidden files by default, despite most others doing it the other way round.

src/file.cr Outdated
inverted = true
preader.next_char
when ']'
raise BadPatternError.new "invalid character set: empty character set"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capitalized exception/error messages, please :)

@Papierkorb
Copy link
Contributor

Bump Besides the hidden-file behaviour, is there something blocking a merge of this?

#
# The pattern syntax is similar to shell filename globbing, see `File.match?` for details.
#
# NOTE: Path separator in patterns needs to be always `/`. The returned file names use system-specific path separators.
Copy link
Contributor

@RX14 RX14 Dec 24, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be fixed to match both seperators (I think).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It matches both, it's just the pattern itself needs to use /. And this is IMHO a good solution because backslasashes are used as escape characters in both Crystal's double-quoted strings and POSIX paths.

src/dir/glob.cr Outdated
count += 1
property? allow_dots

def initialize(@allow_dots = false)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be match_hidden.

src/dir/glob.cr Outdated
# * *allow_dots*: Match hidden files if `true` (default: `false`).
#
# NOTE: Path separator in patterns needs to be always `/`. The returned file names use system-specific path separators.
def self.glob(*patterns, **options, &block : String -> _)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

src/dir/glob.cr Outdated

def self.glob(patterns : Enumerable(String)) : Array(String)
# ditto
def self.glob(patterns : Enumerable(String), **options) : Array(String)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

src/dir/glob.cr Outdated
# * *allow_dots*: Match hidden files if `true` (default: `false`).
#
# NOTE: Path separator in patterns needs to be always `/`. The returned file names use system-specific path separators.
def self.glob(*patterns, **options) : Array(String)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using **options is easier, but please duplicate the definitions so that the documentation looks nice.

* added `**` for matching recursive directories
* `*` and `?` don't match `/`
* added `[abc]`, `[^abc]`, `[a-c]`, `[^a-c]` (including combinations of these) for
  matching a specific set of characters
* added `{sub1,sub2}` for matching subpatterns
* added escaping with `\\`
* added `BadPatternError` exception is raised when a glob pattern is
  invalid
src/dir/glob.cr Outdated
#
# The pattern syntax is similar to shell filename globbing, see `File.match?` for details.
#
# **options:**
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't typically how we document options in crystal.

src/dir/glob.cr Outdated
end
end

private def single_compile(glob)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not name this compile and make it an overload?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not a public API and only called once in the compile method. It could even be directly embedded but I think it's more clear to have a separate method.
But what benefit would it have to name both methods the same? Why add unneccessary work for the type inference? =)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True - it's a style issue. I dislike the compiler performance appeal though, it's not an issue. The benefit to me is it looks nicer.

src/dir/glob.cr Outdated
recursion_depth = ptrn.count(File::SEPARATOR)
# ditto
def self.glob(patterns : Enumerable(String), match_hidden = false, &block : String -> _)
Globber.new(match_hidden: match_hidden).glob(patterns) do |path|
Copy link
Contributor

@RX14 RX14 Dec 24, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the benefit of this struct, it seems to just hold options which could easilly be passed around the methods.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True. I think I was expecting this to be more complicated - or it even might have been so at the beginning, not sure.

@RX14 RX14 added this to the Next milestone Jan 11, 2018
@RX14
Copy link
Contributor

RX14 commented Jan 11, 2018

Does this change the globbing format in any way (i.e. is it a breaking change)?

@RX14 RX14 merged commit bd42727 into crystal-lang:master Jan 11, 2018
@straight-shoota straight-shoota deleted the jm-glob branch January 11, 2018 18:36
@straight-shoota
Copy link
Member Author

straight-shoota commented Jan 11, 2018

It adds features to the pattern language. Glob patterns using any of the added features (like character classes [abc]) will return different results than before.
But apart from that it shouldn't break anything that worked before.

@straight-shoota straight-shoota mentioned this pull request Feb 10, 2018
chris-huxtable pushed a commit to chris-huxtable/crystal that referenced this pull request Apr 6, 2018
chris-huxtable added a commit to chris-huxtable/crystal that referenced this pull request Apr 6, 2018
commit 680d3e0
Author: TSUYUSATO Kitsune <make.just.on@gmail.com>
Date:   Fri Apr 6 08:24:25 2018 +0900

    Format: fix formatting call having trailing comma with block (crystal-lang#5855)

    Fix crystal-lang#5853

commit f22d689
Author: TSUYUSATO Kitsune <make.just.on@gmail.com>
Date:   Fri Apr 6 08:18:23 2018 +0900

    Refactor Colorize#surround (crystal-lang#4196)

    * Refactor Colorize#surround

    This is one of the separations of crystal-lang#3925.

    Remove `surround` and rename `push` to `surround`, now `push` is
    derecated.
    (This reason is dscribed in crystal-lang#3925 (comment))

    * Use #surround instead of #push

    * Apply 'crystal tool format'

    * Remove Colorize#push

commit 12488c2
Author: Benoit de Chezelles <bew@users.noreply.github.com>
Date:   Thu Apr 5 07:49:51 2018 -0700

    Ensure cleanup tempfile after some specs (crystal-lang#5810)

    * Ensure cleanup tempfile after some specs

    * Fix compiler spec

commit ef85244
Author: TSUYUSATO Kitsune <make.just.on@gmail.com>
Date:   Thu Apr 5 20:59:33 2018 +0900

    Format: fix formatter bug on nesting begin/end

commit 8c737a0
Author: TSUYUSATO Kitsune <make.just.on@gmail.com>
Date:   Thu Apr 5 22:54:20 2018 +0900

    Remove duplicated indefinite articles 'a a' in char.cr doc (crystal-lang#5894)

    * Fix duplicated articles 'a a' in char.cr doc

    * Shorten sentence

    crystal-lang#5894 (comment)

commit 73989e8
Author: maiha <maiha@wota.jp>
Date:   Thu Apr 5 22:43:43 2018 +0900

    fix example codes (2018-04) (crystal-lang#5912)

commit b62c4e1
Author: Sijawusz Pur Rahnama <sija@sija.pl>
Date:   Sun Apr 1 16:09:29 2018 +0200

    Refactor out variable name

commit c17ce2d
Author: Sankha Narayan Guria <sankha93@gmail.com>
Date:   Thu Apr 5 01:58:11 2018 -0400

    UUID implements inspect (crystal-lang#5574)

commit 106d44d
Author: TSUYUSATO Kitsune <make.just.on@gmail.com>
Date:   Wed Apr 4 23:01:43 2018 +0900

    MatchData: correct sample code for duplicated named capture

    Ref: crystal-lang#5912 (comment)

commit 011e688
Author: Paul Smith <paulcsmith@users.noreply.github.com>
Date:   Wed Apr 4 13:37:41 2018 -0400

    Small typo fix in bash completion

commit b5a3a65
Author: Johannes Müller <johannes.mueller@smj-fulda.org>
Date:   Wed Apr 4 15:59:33 2018 +0200

    Fix File.join with empty path component (crystal-lang#5915)

commit 9662abe
Author: TSUYUSATO Kitsune <make.just.on@gmail.com>
Date:   Wed Apr 4 16:42:42 2018 +0900

    Fix `String#tr` 1 byte `from` optimization bug

    Ref: crystal-lang#5912 (comment)

    `"aabbcc".tr("a", "xyz")` yields `"xyzxyzbbcc"` currently.
    Of course it is unintentional behavior, in Ruby it yields `"xxbbcc"` and
    on shell `echo aabbcc | tr a xyz` shows `xxbbcc`.

commit ec423eb
Author: TSUYUSATO Kitsune <make.just.on@gmail.com>
Date:   Wed Apr 4 02:38:21 2018 +0900

    Fix crystal-lang#5907 formatter bug (crystal-lang#5909)

    * Fix crystal-lang#5907 formatter bug

    * Apply new formatter

commit 5056859
Author: TSUYUSATO Kitsune <make.just.on@gmail.com>
Date:   Wed Apr 4 02:33:07 2018 +0900

    Pass an unhandled exception to at_exit block as second argument (crystal-lang#5906)

    * Pass an unhandled exception to at_exit block as second argument

    Follow up crystal-lang#1921

    It is better in some ways:

      - it does not need a new exception like `SystemExit`.
      - it does not break compatibility in most cases because block fill up lacking arguments.

    * Add documentation for at_exit block arguments

    * Update `at_exit` block arguments description

    crystal-lang#5906 (comment)
    Thank you @jhass.

commit 82caaf0
Author: TSUYUSATO Kitsune <make.just.on@gmail.com>
Date:   Tue Apr 3 23:18:01 2018 +0900

    Semantic: don't guess ivar type from argument after assigned (crystal-lang#5166)

commit bb5bcd2
Author: TSUYUSATO Kitsune <make.just.on@gmail.com>
Date:   Tue Apr 3 21:00:27 2018 +0900

    Colorize: abstract colors and support 8bit and true color (crystal-lang#5902)

    * Colorize: abstract colors and support 8bit and true color

    Closes crystal-lang#5900

    * Color#fore and #back take io to avoid memory allocation

    * Use character literal instead of 1 length string

commit 7eae5aa
Author: Benoit de Chezelles <bew@users.noreply.github.com>
Date:   Mon Apr 2 16:43:52 2018 -0700

    Use LibCrystalMain.__crystal_main directly (crystal-lang#5899)

commit 60f675c
Author: TSUYUSATO Kitsune <make.just.on@gmail.com>
Date:   Tue Apr 3 08:42:03 2018 +0900

    Format: fix indentation after backslash newline (crystal-lang#5901)

    crystal-lang#5892 (comment)

commit 4d2ad83
Author: TSUYUSATO Kitsune <make.just.on@gmail.com>
Date:   Tue Apr 3 08:21:06 2018 +0900

    Prevent invoking `method_added` macro hook recursively (crystal-lang#5159)

    Fixed crystal-lang#5066

commit e17823f
Author: TSUYUSATO Kitsune <make.just.on@gmail.com>
Date:   Tue Apr 3 07:56:52 2018 +0900

    Format: fix indentation in collection with comment after beginning delimiter (crystal-lang#5893)

commit 49d722c
Author: Chris Hobbs <chris@rx14.co.uk>
Date:   Sat Mar 31 19:30:54 2018 +0100

    Print exception cause when inspecting with backtrace (crystal-lang#5833)

commit 945557b
Author: Sijawusz Pur Rahnama <sija@sija.pl>
Date:   Sat Mar 31 20:30:15 2018 +0200

    Use Char for single char strings (crystal-lang#5882)

commit 4532389
Author: r00ster91 <35064754+r00ster91@users.noreply.github.com>
Date:   Sat Mar 31 15:35:08 2018 +0200

    Fix Random example (crystal-lang#5728)

commit 5cd78fa
Author: Benoit de Chezelles <bew@users.noreply.github.com>
Date:   Fri Mar 30 15:29:30 2018 -0700

    Allow a path to declare a constant (crystal-lang#5883)

    * Allow a path to declare a constant

    * Add spec for type keeping when creating a constant using a Path

commit f33a910
Author: Carl Hörberg <carl.hoerberg@gmail.com>
Date:   Fri Mar 30 20:40:31 2018 +0200

    Enqueue senders in Channel#close (crystal-lang#5880)

    Fixes crystal-lang#5875

commit 0424b22
Author: r00ster91 <35064754+r00ster91@users.noreply.github.com>
Date:   Thu Mar 29 16:17:20 2018 +0200

    Fix HEREDOC error message grammar (crystal-lang#5887)

    * Fix HEREDOC error message grammar

    * Update parser_spec.cr

commit 4927ecc
Author: Benoit de Chezelles <bew@users.noreply.github.com>
Date:   Thu Mar 29 05:06:03 2018 -0700

    Fix typo (crystal-lang#5884)

commit c2efaff
Aut