Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

when using a constant value when case when checking values... 0.0B/op appears. #1

Open
girng opened this issue Sep 28, 2019 · 9 comments

Comments

@girng
Copy link
Owner

girng commented Sep 28, 2019

This seems weird, when constant values are enabled for case when checking (example):

cmd = CMD.from_value 5

instead of

cmd = CMD.from_value CMD_INDEX_VALUES.shuffle[0]

the Benchmark will report:

  case_when_enum 446.48M (  2.24ns) (± 2.90%)  0.0B/op        fastest
case_when_string 440.04M (  2.27ns) (± 4.04%)  0.0B/op   1.01× slower
@girng girng changed the title when using a constant value when case when checking values... 0.0B/op happens. when using a constant value when case when checking values... 0.0B/op appears. Sep 28, 2019
@girng
Copy link
Owner Author

girng commented Sep 28, 2019

How can 0.0B/op be happening here, wtf

@girng girng pinned this issue Sep 28, 2019
@girng
Copy link
Owner Author

girng commented Sep 28, 2019

Actually, I think I understand why.

0.0B/op is short hand for:

Bytes Per Operation, correct? If so, the amount of bytes for a constant conditional.. doesn't take that much. Thus, it probably rounds down to 0.0. Do I got that right?

With that said, it would make sense to use .shuffle. However, I don't know if .shuffle is distorting the actual point of the benchmark (comparing a string vs enum value). Hmmm..

@JacobUb
Copy link

JacobUb commented Sep 29, 2019

Could you post the benchmark code?

@girng
Copy link
Owner Author

girng commented Sep 29, 2019

hello @Exilor

Here is a reduced example:

require "benchmark"

enum CMD
  XFER
end
Benchmark.ips do |x|
  x.report("case_when_enum") { case_when_enum }
  x.report("case_when_string") { case_when_string }
end

def case_when_enum
  cmd = CMD.from_value 0
  if cmd == CMD::XFER
  end
end

def case_when_string
  cmd = "FDGRRGERG1"

  if cmd == "FDGRRGERG1"
  end
end

Output:

  case_when_enum 444.12M (  2.25ns) (± 2.33%)  0.0B/op   1.01× slower
case_when_string 446.90M (  2.24ns) (± 2.95%)  0.0B/op        fastest

@JacobUb
Copy link

JacobUb commented Sep 29, 2019

Check this
The strings being compared are the same object and String#== first checks that, meaning that in this case it won't compare the contents of either string, only their object identity.
I modified the benchmark to account for that:

enum CMD
  XFER
end

TEST_STR = "FDGRRGERG1".reverse.reverse

Benchmark.ips do |x|
  x.report("case_when_enum") { case_when_enum }
  x.report("case_when_string") { case_when_string }
end


def case_when_enum
  cmd = CMD.from_value 0
  cmd == CMD::XFER
end

def case_when_string
  cmd = TEST_STR
  cmd == "FDGRRGERG1"
end

And my results:

case_when_enum 609.35M (  1.64ns) (± 0.79%)  0.0B/op        fastest
case_when_string 309.09M (  3.24ns) (± 6.34%)  0.0B/op   1.97× slower

As to what are B/op... I don't know.

@girng
Copy link
Owner Author

girng commented Sep 29, 2019

@Exilor Maybe.. .shuffle is correct then so it checks against different strings on each iteration? (randomly)? I just didn't want to use shuffle cause maybe it could distort the actual performance of checking an enum value vs a string (cause it has to shuffle the elements, which is a performance factor intrinsically)

edit: One second, thanks for your code, going to try something now.

@girng
Copy link
Owner Author

girng commented Sep 29, 2019

New code:

require "benchmark"

enum CMD
  XFER
  XFER2
  XFER3
end

MAPPED_ENUM_VALUES = [0, 1, 2]
TEST_STRING        = ["XFER", "XFER2", "XFER3"]

Benchmark.ips do |x|
  x.report("case_when_enum") { case_when_enum }
  x.report("case_when_string") { case_when_string }
end

def case_when_enum
  # Ensure some of kind of randomess
  cmd = CMD.from_value MAPPED_ENUM_VALUES.shuffle[0]
  case cmd
  when CMD::XFER
  end
end

def case_when_string
  # Ensure some kind of randomness to bypass return true if same?(other)
  # From: https://github.com/crystal-lang/crystal/blob/master/src/string.cr#L2406

  cmd = TEST_STRING.shuffle[0]
  case cmd
  when "FDGRRGERG1"
  end
end

Output:

  case_when_enum  10.76M ( 92.94ns) (± 4.40%)  48.0B/op   1.14× slower
case_when_string  12.24M ( 81.67ns) (± 5.18%)  64.0B/op        fastest

Now checking Enum values is slower than a string. I don't know how to do this :/

@JacobUb
Copy link

JacobUb commented Sep 29, 2019

Now that benchmark is comparing bytesizes between strings, which is also very cheap like what happened before.
Just use enums for situations that call for enums. They're not slow, certainly not slower than strings.

@girng
Copy link
Owner Author

girng commented Sep 29, 2019

@Exilor I see. Interesting, thanks for the info!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants