Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DO NOT MERGE] [stdlib] Random unification #12772

Closed
wants to merge 40 commits into from
Closed
Show file tree
Hide file tree
Changes from 38 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
b30c095
Add shims for stdlib random
Azoy Nov 4, 2017
b1340ea
Initial random api
Azoy Nov 4, 2017
580a670
Use C syscall for I/O
Azoy Nov 4, 2017
0c7ce54
Add shuffle functions
Azoy Nov 4, 2017
7ca2f1c
Add documentation to Random API
Azoy Nov 5, 2017
310e4e1
Fix a few typos within the documentation
Azoy Nov 5, 2017
22ea738
Update API to reflect mailing list discussions
Azoy Nov 15, 2017
40c1433
Update API to match mailing list discussion, add tests
Azoy Nov 23, 2017
a81fe12
Update docs
Azoy Dec 7, 2017
cb6292c
Update from upstream
Azoy Dec 7, 2017
b20b238
Use new Libc naming
Azoy Dec 7, 2017
1b6a3d3
Remove sampling
Azoy Dec 14, 2017
0cee853
Update from upstream
Azoy Dec 14, 2017
7326b68
gybify signed integer creation
Azoy Dec 23, 2017
e4719f8
Make FloatingPoint.random exclusive
Azoy Dec 24, 2017
410e2f7
Update from upstream
Azoy Dec 24, 2017
7a61fef
Refactor {Closed}Range.random
Azoy Dec 24, 2017
3878792
Merge branch 'master' into random-unification
Azoy Jan 8, 2018
63405b4
Fix some doc typos
Azoy Jan 8, 2018
987af08
Make .random a function
Azoy Jan 8, 2018
844c120
Update API to reflect discussion
Azoy Jan 13, 2018
7aabe8c
Merge branch 'master' of https://github.com/apple/swift into random-u…
Azoy Jan 13, 2018
24c29e8
Fix compile errors
Azoy Jan 14, 2018
c31bc1a
Clean up _stdlib_random
Azoy Jan 15, 2018
b0ca3d4
Merge branch 'master' of https://github.com/apple/swift into random-u…
Azoy Jan 26, 2018
573408d
Revert "Merge branch 'master' of https://github.com/apple/swift into …
Azoy Jan 26, 2018
2c5ae76
Fix CollectionAlgorithm conflict
Azoy Jan 26, 2018
e763457
Add shuffle and more tests
Azoy Jan 26, 2018
2e18d36
Provide finishing tests and suggestions
Azoy Feb 2, 2018
4f4f65e
Merge branch 'master' of https://github.com/apple/swift into random-u…
Azoy Feb 3, 2018
a53bec5
Remove refs to Countable ranges
Azoy Feb 3, 2018
8fd12eb
Add `_stdlib_random` for more platforms (#1)
benrimmington Feb 10, 2018
d006c21
Merge branch 'master' of https://github.com/apple/swift into random-u…
Azoy Mar 30, 2018
6f4f1c5
Merge branch 'master' of https://github.com/apple/swift into random-u…
Azoy Mar 30, 2018
b94a5ff
Merge branch 'master' of https://github.com/apple/swift into random-u…
Azoy Mar 31, 2018
47e25db
Fix some implementation details
Azoy Apr 2, 2018
f31c344
Revise documentation, add benchmarks (#3)
natecook1000 Apr 12, 2018
2f4294b
Value type generators
Azoy Apr 20, 2018
6927e9a
Consolidate `_stdlib_random` functions (#2)
benrimmington Apr 22, 2018
7ae4c03
Clean up integerRangeTest
Azoy May 3, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions benchmark/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,8 @@ set(SWIFT_BENCH_MODULES
single-source/Queue
single-source/RC4
single-source/RGBHistogram
single-source/RandomShuffle
single-source/RandomValues
single-source/RangeAssignment
single-source/RangeIteration
single-source/RangeReplaceableCollectionPlusDefault
Expand Down
64 changes: 64 additions & 0 deletions benchmark/single-source/RandomShuffle.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
//===--- RandomShuffle.swift ----------------------------------------------===//
//
// This source file is part of the Swift.org open source project
//
// Copyright (c) 2014 - 2018 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
// See https://swift.org/CONTRIBUTORS.txt for the list of Swift project authors
//
//===----------------------------------------------------------------------===//

import TestsUtils

//
// Benchmark that shuffles arrays of integers. Measures the performance of
// shuffling large arrays.
//

public let RandomShuffle = [
BenchmarkInfo(name: "RandomShuffleDef", runFunction: run_RandomShuffleDef,
tags: [.api], setUpFunction: setup_RandomShuffle),
BenchmarkInfo(name: "RandomShuffleLCG", runFunction: run_RandomShuffleLCG,
tags: [.api], setUpFunction: setup_RandomShuffle),
]

/// A linear congruential PRNG.
struct LCRNG: RandomNumberGenerator {
private var state: UInt64

init(seed: Int) {
state = UInt64(truncatingIfNeeded: seed)
for _ in 0..<10 { _ = next() }
}

mutating func next() -> UInt64 {
state = 2862933555777941757 &* state &+ 3037000493
return state
}
}

var numbers = Array(0...3_000_000)

@inline(never)
func setup_RandomShuffle() {
_ = numbers.count
}

@inline(never)
public func run_RandomShuffleDef(_ N: Int) {
for _ in 0 ..< N {
numbers.shuffle()
blackHole(numbers.first!)
}
}

@inline(never)
public func run_RandomShuffleLCG(_ N: Int) {
var generator = LCRNG(seed: 0)
for _ in 0 ..< N {
numbers.shuffle(using: &generator)
blackHole(numbers.first!)
}
}
87 changes: 87 additions & 0 deletions benchmark/single-source/RandomValues.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
//===--- RandomValues.swift -----------------------------------------------===//
//
// This source file is part of the Swift.org open source project
//
// Copyright (c) 2014 - 2018 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
// See https://swift.org/CONTRIBUTORS.txt for the list of Swift project authors
//
//===----------------------------------------------------------------------===//

import TestsUtils

//
// Benchmark generating lots of random values. Measures the performance of
// the default random generator and the algorithms for generating integers
// and floating-point values.
//

public let RandomValues = [
BenchmarkInfo(name: "RandomIntegersDef", runFunction: run_RandomIntegersDef, tags: [.api]),
BenchmarkInfo(name: "RandomIntegersLCG", runFunction: run_RandomIntegersLCG, tags: [.api]),
BenchmarkInfo(name: "RandomDoubleDef", runFunction: run_RandomDoubleDef, tags: [.api]),
BenchmarkInfo(name: "RandomDoubleLCG", runFunction: run_RandomDoubleLCG, tags: [.api]),
]

/// A linear congruential PRNG.
struct LCRNG: RandomNumberGenerator {
private var state: UInt64

init(seed: Int) {
state = UInt64(truncatingIfNeeded: seed)
for _ in 0..<10 { _ = next() }
}

mutating func next() -> UInt64 {
state = 2862933555777941757 &* state &+ 3037000493
return state
}
}

@inline(never)
public func run_RandomIntegersDef(_ N: Int) {
for _ in 0 ..< N {
var x = 0
for _ in 0 ..< 100_000 {
x &+= Int.random(in: 0...10_000)
}
blackHole(x)
}
}

@inline(never)
public func run_RandomIntegersLCG(_ N: Int) {
for _ in 0 ..< N {
var x = 0
var generator = LCRNG(seed: 0)
for _ in 0 ..< 100_000 {
x &+= Int.random(in: 0...10_000, using: &generator)
}
CheckResults(x == 498214315)
}
}

@inline(never)
public func run_RandomDoubleDef(_ N: Int) {
for _ in 0 ..< N {
var x = 0.0
for _ in 0 ..< 100_000 {
x += Double.random(in: -1000...1000)
}
blackHole(x)
}
}

@inline(never)
public func run_RandomDoubleLCG(_ N: Int) {
for _ in 0 ..< N {
var x = 0.0
var generator = LCRNG(seed: 0)
for _ in 0 ..< 100_000 {
x += Double.random(in: -1000...1000, using: &generator)
}
blackHole(x)
}
}
4 changes: 4 additions & 0 deletions benchmark/utils/main.swift
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,8 @@ import ProtocolDispatch2
import Queue
import RC4
import RGBHistogram
import RandomShuffle
import RandomValues
import RangeAssignment
import RangeIteration
import RangeReplaceableCollectionPlusDefault
Expand Down Expand Up @@ -264,6 +266,8 @@ registerBenchmark(QueueGeneric)
registerBenchmark(QueueConcrete)
registerBenchmark(RC4Test)
registerBenchmark(RGBHistogram)
registerBenchmark(RandomShuffle)
registerBenchmark(RandomValues)
registerBenchmark(RangeAssignment)
registerBenchmark(RangeIteration)
registerBenchmark(RangeReplaceableCollectionPlusDefault)
Expand Down
4 changes: 4 additions & 0 deletions stdlib/public/SwiftShims/LibcShims.h
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,10 @@ __swift_uint32_t _stdlib_cxx11_mt19937(void);
SWIFT_RUNTIME_STDLIB_INTERNAL
__swift_uint32_t _stdlib_cxx11_mt19937_uniform(__swift_uint32_t upper_bound);

// Random number for stdlib
SWIFT_RUNTIME_STDLIB_INTERNAL
void _stdlib_random(void *buf, __swift_size_t nbytes);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this function be SWIFT_RUNTIME_STDLIB_INTERNAL instead?
Should @_inlineable be removed from the Random.next() method?
https://github.com/apple/swift/blob/master/stdlib/public/SwiftShims/Visibility.h

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!


// Math library functions
static inline SWIFT_ALWAYS_INLINE
float _stdlib_remainderf(float _self, float _other) {
Expand Down
24 changes: 24 additions & 0 deletions stdlib/public/core/Bool.swift
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,30 @@ public struct Bool {
public init(_ value: Bool) {
self = value
}

/// Returns a random Boolean
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this be "Boolean value." to match the rest of the file?

///
/// - Parameter generator: The random number generator to use when getting a
/// random Boolean.
/// - Returns: A random Boolean.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto x2: "Boolean value."

@inlinable
public static func random<T: RandomNumberGenerator>(
using generator: inout T
) -> Bool {
return (generator.next() >> 17) & 1 == 0
}

/// Returns a random Boolean
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto x2: "Boolean value."

///
/// - Parameter generator: The random number generator to use when getting a
/// random Boolean.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function doesn't have a generator parameter.

/// - Returns: A random Boolean.
///
/// This uses the standard library's default random number generator.
@inlinable
public static func random() -> Bool {
return Bool.random(using: &Random.default)
}
}

extension Bool : _ExpressibleByBuiltinBooleanLiteral, ExpressibleByBooleanLiteral {
Expand Down
6 changes: 6 additions & 0 deletions stdlib/public/core/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,7 @@ set(SWIFTLIB_ESSENTIAL
Policy.swift
PrefixWhile.swift
Print.swift
Random.swift
RandomAccessCollection.swift
Range.swift
RangeReplaceableCollection.swift
Expand Down Expand Up @@ -192,6 +193,7 @@ if(CMAKE_SYSTEM_NAME STREQUAL "Darwin")
list(APPEND swift_core_link_flags "-all_load")
list(APPEND swift_core_framework_depends Foundation)
list(APPEND swift_core_framework_depends CoreFoundation)
list(APPEND swift_core_framework_depends Security)
list(APPEND swift_core_private_link_libraries icucore)
else()
# With the GNU linker the equivalent of -all_load is to tell the linker
Expand Down Expand Up @@ -226,6 +228,10 @@ if(CMAKE_SYSTEM_NAME STREQUAL "FreeBSD")
${EXECINFO_LIBRARY})
endif()

if(CMAKE_SYSTEM_NAME STREQUAL "Windows")
list(APPEND swift_core_link_flags "$ENV{SystemRoot}/system32/bcrypt.dll")
endif()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@benrimmington Does this look correct?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Azoy, I think the Bcrypt.lib import library is needed.

https://msdn.microsoft.com/en-us/library/windows/desktop/ms681914(v=vs.85).aspx

Are you able to test on Windows? If not, shall we ask @compnerd for help?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this is wrong. You don't link against the dll but the import library. You should be able to just add bcrypt.lib to the link flags.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@compnerd, please see Azoy/swift@b2ef08f in Azoy#2.

It uses #pragma comment(lib, "Bcrypt.lib") to add the import library.

It also uses BCRYPT_SUCCESS instead of NT_SUCCESS, so the extra defines/includes (WIN32_NO_STATUS, <Ntstatus.h>, <Ntdef.h>) aren't needed AFAIK.

option(SWIFT_CHECK_ESSENTIAL_STDLIB
"Check core standard library layering by linking its essential subset"
FALSE)
Expand Down
67 changes: 67 additions & 0 deletions stdlib/public/core/Collection.swift
Original file line number Diff line number Diff line change
Expand Up @@ -790,6 +790,25 @@ public protocol Collection: Sequence where SubSequence: Collection {
/// `endIndex`.
func formIndex(after i: inout Index)

/// Returns a random element of the collection, using the given generator as
/// a source for randomness.
///
/// You use this method to select a random element from a collection when you
/// are using a custom random number generator. For example, call
/// `randomElement(using:)` to select a random element from an array of names.
///
/// let names = ["Zoey", "Chloe", "Amani", "Amaia"]
/// let randomName = names.randomElement(using: &myGenerator)!
/// // randomName == "Amani" (maybe)
///
/// - Parameter generator: The random number generator to use when choosing
/// a random element.
/// - Returns: A random element from the collection. If the collection is
/// empty, the method returns `nil`.
func randomElement<T: RandomNumberGenerator>(
using generator: inout T
) -> Element?

@available(*, deprecated, message: "all index distances are now of type Int")
typealias IndexDistance = Int
}
Expand Down Expand Up @@ -1003,6 +1022,54 @@ extension Collection {
return count
}

/// Returns a random element of the collection, using the given generator as
/// a source for randomness.
///
/// You use this method to select a random element from a collection when you
/// are using a custom random number generator. For example, call
/// `randomElement(using:)` to select a random element from an array of names.
///
/// let names = ["Zoey", "Chloe", "Amani", "Amaia"]
/// let randomName = names.randomElement(using: &myGenerator)!
/// // randomName == "Amani" (maybe)
///
/// - Parameter generator: The random number generator to use when choosing
/// a random element.
/// - Returns: A random element from the collection. If the collection is
/// empty, the method returns `nil`.
@inlinable
public func randomElement<T: RandomNumberGenerator>(
using generator: inout T
) -> Element? {
guard !isEmpty else { return nil }
let random = generator.next(upperBound: UInt(count))
let index = self.index(
startIndex,
offsetBy: numericCast(random)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of using next(upperBound:) and numericCast(_:), would the following work?

let randomDistance = Int.random(in: 0 ..< count, using: &generator)
let randomIndex = index(startIndex, offsetBy: randomDistance)
return self[randomIndex]

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

However, this would be indirectly calling Range.randomElement(using:) in the default implementation of Collection.randomElement(using:), which is a protocol requirement.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@benrimmington The changes you proposed worked, I'm not sure if you want to proceed with it however.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indirectly calling Range.randomElement(using:) inside Collection.randomElement(using:) won't be recursive. But will it be slower than next(upperBound:) and numericCast(_:)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My initial reaction is yes it will be slower, but it should be minuscule however.

)
return self[index]
}

/// Returns a random element of the collection.
///
/// For example, call `randomElement()` to select a random element from an
/// array of names.
///
/// let names = ["Zoey", "Chloe", "Amani", "Amaia"]
/// let randomName = names.randomElement()!
/// // randomName == "Amani" (perhaps)
///
/// This method uses the default random generator, `Random.default`. The call
/// to `names.randomElement()` above is equivalent to calling
/// `names.randomElement(using: &Random.default)`.
///
/// - Returns: A random element from the collection. If the collection is
/// empty, the method returns `nil`.
@inlinable
public func randomElement() -> Element? {
return randomElement(using: &Random.default)
}

/// Do not use this method directly; call advanced(by: n) instead.
@inlinable
@usableFromInline
Expand Down
Loading