fix an edge case in gcd calculation #5444
FIXNUM_P macro means a positive number in range 0...FIXNUM_MAX
any idea for a better name?
Long.MIN_VALUE is -9223372036854775808
mri is unaffected because it doesn't allow to pass negative numbers to i_gcd at all, but their implementation of i_gcd has the same limitation