Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SR-1124] Drop FileCheck from test/1_stdlib, use StdlibUnittest instead #43737

modocache mannequin opened this issue Apr 1, 2016 · 7 comments

[SR-1124] Drop FileCheck from test/1_stdlib, use StdlibUnittest instead #43737

modocache mannequin opened this issue Apr 1, 2016 · 7 comments
good first issue Good for newcomers improvement standard library Area: Standard library umbrella


Copy link

modocache mannequin commented Apr 1, 2016

Previous ID SR-1124
Radar None
Original Reporter @modocache
Type Improvement
Additional Detail from JIRA
Votes 0
Component/s Standard Library
Labels Improvement, StarterBug, Test
Assignee None
Priority Medium

md5: 5245d992633498d3acacd3655548dd7c

Issue Description:

Swift's test/1_stdlib directory is meant to include tests for the Swift stdlib. Most of these are written using the testing framework designed for this purpose, StdlibUnittest. However, some use an ad-hoc system of printing strings to stdout, then checking the output via FileCheck.

The following tests should be written using StdlibUnittest (and no longer reference FileCheck):

  1. test/1_stdlib/ArrayBridge.swift

  2. test/1_stdlib/BitwiseOperationsType.swift

  3. test/1_stdlib/BridgeNonVerbatim.swift

  4. test/1_stdlib/Bridgeable.swift

  5. test/1_stdlib/CGGeometry.swift

  6. test/1_stdlib/Collection.swift

  7. test/1_stdlib/CollectionOfOne.swift

  8. test/1_stdlib/Float.swift

  9. test/1_stdlib/FloatingPointIR.swift

  10. test/1_stdlib/HeapBuffer.swift

  11. test/1_stdlib/ImplicitlyUnwrappedOptional.swift

  12. test/1_stdlib/KVO.swift

  13. test/1_stdlib/LogicValue.swift

  14. test/1_stdlib/Map.swift

  15. test/1_stdlib/NSObject.swift

  16. test/1_stdlib/NSUndoManager.swift

  17. test/1_stdlib/NewArray.swift.gyb

  18. test/1_stdlib/NewString.swift

  19. test/1_stdlib/NewStringAppending.swift

  20. test/1_stdlib/Nil.swift

  21. test/1_stdlib/Reflection.swift

  22. test/1_stdlib/Reflection_jit.swift

  23. test/1_stdlib/Reflection_objc.swift

  24. test/1_stdlib/SegmentAlignment.c

  25. test/1_stdlib/SpriteKit.swift

  26. test/1_stdlib/StringReallocation.swift

  27. test/1_stdlib/TypeName.swift

  28. test/1_stdlib/UIKit.swift

  29. test/1_stdlib/UIViewControllerAdditions.swift

  30. test/1_stdlib/VarArgs.swift

  31. test/1_stdlib/Zip.swift

  32. test/1_stdlib/alloc_rounding.swift

  33. test/1_stdlib/integer_conversions.swift

  34. test/1_stdlib/sort_integers.swift

  35. test/1_stdlib/subString.swift

Copy link

belkadan commented Apr 1, 2016

I'm a little concerned that we'll lose what little coverage we have for the interpreter; right now there's a (non-buildbot) test mode that turns simple compile-and-run invocations into interpreter invocations. StdlibUnittest currently doesn't support that because it runs all tests out of process by default.

I should probably file a bug for this.

Copy link

belkadan commented Apr 1, 2016

Filed SR-1125 and SR-1126 (neither of which should block this effort).

Copy link

Comment by Kyle Yoon (JIRA)

Is this task mostly changing

// RUN: %target-run %t/ArrayBridge > %t.txt
// RUN: FileCheck %s < %t.txt


// RUN: %target-run-stdlib-swift | FileCheck %s

for the listed files?
Also, does it require you to re-build the toolchain and test every time?

Sorry for all the questions, again!

Copy link
Mannequin Author

modocache mannequin commented Apr 11, 2016

No need to apologize for clarifying the task! On the contrary, thanks for these follow-up questions.

The ideal end state of the listed tests would be the following:

// RUN: %target-run-simple-swift

A good example is test/1_stdlib/NSEnumeratorAPI.swift. Notice that that test does not use FileCheck at all. Instead, it uses the TestSuite construct from StdlibUnittest. This library is capable of making assertions that are not based on textual output, like FileCheck is. We want to avoid text output-based assertions.

Copy link

swift-ci commented Aug 1, 2018

Comment by Sammy Gutierrez (JIRA)

Here is the work I have for this ticket.

I was hoping someone could take a quick look over it before submitting the PR (should I ask for help in the forums instead?). I attempted to remove FileCheck from any files that I could in /test/stdlib but there were a few places where I did not know what to do:

  • Reflection_jit.swift - Could not get Reflection.swift tests to run with JIT (deleted this file for now)

  • SwiftObjectNSObject.swift - Calls SwiftObjectNSObject.m and checks for output.

Copy link

belkadan commented Aug 1, 2018

cc @airspeedswift

Copy link

Resetting assignee for all Starter Bugs not modified since 2018.

I think we lost this one since it wasn't a full PR. In the future, please feel free to make a PR even if it's still WIP.

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
good first issue Good for newcomers improvement standard library Area: Standard library umbrella
None yet

No branches or pull requests

2 participants