Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SR-8678] Possible security fail in implementation of multiline/delimited strings #51192

johnno1962 opened this issue Sep 1, 2018 · 2 comments
bug A deviation from expected or documented behavior Compiler Parser


Copy link

johnno1962 commented Sep 1, 2018

Previous ID SR-8678
Radar None
Original Reporter @johnno1962
Type Bug
Additional Detail from JIRA
Votes 0
Component/s Compiler
Labels Bug, Parser
Assignee None
Priority Medium

md5: 649261c2424597aabda9e63a57de79a9

Issue Description:

As was pointed out in the tail end of the discussion of SE-200 there is a potential security problem with multiple character string delimiters as are used by multiline strings and soon to be implemented “raw/delimited strings” as demonstrated by the following contrived code:

Validating password...
guard user.validatePassword(password) else {
    fatalError("get out!")
Password is valid!

It contains a zero-width unicode character in the first closing delimiter so this code is treated as one string. In Xcode this mitigated by being syntax highlighted correctly but perhaps there is something here that deserves attention. A possible fix for this is included in the last commit on #17668

Copy link
Contributor Author

johnno1962 commented Sep 18, 2018

The code for the implementation of SE-200 was structured so that the function diagnoseZeroWidthMatchAndAdvance() could be updated to resolve the problem - at least for string literal delimiters. A possible solution was removed in the finish which contained the following code:

const static uint32_t ZeroWidthC[] = {
   // "Zero-Width" characters which don't appear to be visible follow.
   0x0001, 0x0002, 0x0003, 0x0004, 0x0005, 0x0006, 0x0007, 0x0008,
   0x000b, 0x000e, 0x000f, 0x0010, 0x0011, 0x0012, 0x0013, 0x0014,
   0x0015, 0x0016, 0x0017, 0x0018, 0x0019, 0x001a, 0x001b, 0x001c,
   0x001d, 0x001e, 0x001f, 0x007f, 0x0080, 0x0081, 0x0082, 0x0083,
   0x0084, 0x0086, 0x0087, 0x0088, 0x0089, 0x008a, 0x008b, 0x008c,
   0x008d, 0x008e, 0x008f, 0x0090, 0x0091, 0x0092, 0x0093, 0x0094,
   0x0095, 0x0096, 0x0097, 0x0098, 0x0099, 0x009a, 0x009b, 0x009c,
   0x009d, 0x009e, 0x009f, 0x00ad, 0x0300, 0x0301, 0x0302, 0x0303,
   0x0304, 0x0306, 0x0307, 0x0308, 0x0309, 0x030a, 0x030b, 0x030c,
   0x030f, 0x0311, 0x031b, 0x0323, 0x0324, 0x0325, 0x0326, 0x0327,
   0x0328, 0x032d, 0x032e, 0x0330, 0x0331, 0x0332, 0x034f, 0x055f,
   0x0610, 0x0611, 0x0612, 0x0613, 0x0614, 0x0615, 0x0616, 0x0617,
   0x0618, 0x0619, 0x061a, 0x061c, 0x064b, 0x064c, 0x064d, 0x064e,
   0x064f, 0x0650, 0x0651, 0x0652, 0x0653, 0x0654, 0x0655, 0x0656,
   0x0657, 0x0658, 0x0659, 0x065a, 0x065b, 0x065c, 0x065d, 0x065e,
   0x065f, 0x0670, 0x06d6, 0x06d7, 0x06d8, 0x06d9, 0x06da, 0x06db,
   0x06dc, 0x06df, 0x06e0, 0x06e1, 0x06e2, 0x06e3, 0x06e4, 0x06e6,
   0x06e7, 0x06e8, 0x06ea, 0x06eb, 0x06ec, 0x06ed, 0x070f, 0x0711,
   0x0730, 0x0731, 0x0732, 0x0733, 0x0734, 0x0735, 0x0736, 0x0737,
   0x0738, 0x0739, 0x073a, 0x073b, 0x073c, 0x073d, 0x073e, 0x073f,
   0x0740, 0x0741, 0x0742, 0x0743, 0x0744, 0x0745, 0x0746, 0x0747,
   0x0748, 0x0749, 0x074a, 0x07a6, 0x07a7, 0x07a8, 0x07a9, 0x07aa,
   0x07ab, 0x07ac, 0x07ad, 0x07ae, 0x07af, 0x07b0, 0x07eb, 0x07ec,
   0x07ed, 0x07ee, 0x07ef, 0x07f0, 0x07f1, 0x07f2, 0x07f3, 0x0816,
   0x0817, 0x0818, 0x0819, 0x081b, 0x081c, 0x081d, 0x081e, 0x081f,
   0x0820, 0x0821, 0x0822, 0x0823, 0x0825, 0x0826, 0x0827, 0x0829,
   0x082a, 0x082b, 0x082c, 0x082d, 0x0858, 0x0859, 0x085a, 0x085b,
   0x08d5, 0x08d6, 0x08d7, 0x08d8, 0x08d9, 0x08e0, 0x08e1, 0x08e2,
   0x08e3, 0x08e4, 0x08e5, 0x08e6, 0x08e7, 0x08e8, 0x08e9, 0x08ea,
   0x08eb, 0x08ec, 0x08ed, 0x08ee, 0x08ef, 0x08f0, 0x08f1, 0x08f2,
   0x08f3, 0x08f4, 0x08f5, 0x08f6, 0x08f7, 0x08f8, 0x08f9, 0x08fb,
   0x08fc, 0x08fd, 0x08fe, 0x08ff, 0x0f18, 0x0f19, 0x0f35, 0x0f37,
   0x0f39, 0x0f72, 0x0f7a, 0x0f7b, 0x0f7c, 0x0f7d, 0x0f7e, 0x0f80,
   0x0f82, 0x0f83, 0x0f84, 0x0f86, 0x0f87, 0x0fc6, 0x115f, 0x1160,
   0x1712, 0x1713, 0x1714, 0x1732, 0x1733, 0x1752, 0x1753, 0x1772,
   0x1773, 0x17b4, 0x17b5, 0x180b, 0x180c, 0x180d, 0x180e, 0x1920,
   0x1921, 0x1922, 0x1927, 0x1928, 0x192a, 0x1932, 0x193a, 0x193b,
   0x1a17, 0x1a18, 0x1a1b, 0x1a55, 0x1a56, 0x1a59, 0x1a5a, 0x1a5b,
   0x1a5c, 0x1a5d, 0x1a5e, 0x1a60, 0x1a62, 0x1a65, 0x1a66, 0x1a67,
   0x1a68, 0x1a69, 0x1a6a, 0x1a6c, 0x1a73, 0x1a74, 0x1a75, 0x1a76,
   0x1a77, 0x1a78, 0x1a79, 0x1a7a, 0x1a7b, 0x1a7c, 0x1a7f, 0x1abe,
   0x1b80, 0x1b81, 0x1ba1, 0x1ba2, 0x1ba3, 0x1ba4, 0x1ba5, 0x1ba8,
   0x1ba9, 0x1bac, 0x1bad, 0x1be6, 0x1be8, 0x1be9, 0x1bed, 0x1bee,
   0x1bef, 0x1bf0, 0x1bf1, 0x1c2c, 0x1c2d, 0x1c2e, 0x1c2f, 0x1c30,
   0x1c31, 0x1c32, 0x1c33, 0x1c36, 0x1c37, 0x1ce1, 0x1cf2, 0x1cf3,
   0x1cf7, 0x200b, 0x200c, 0x200d, 0x200e, 0x200f, 0x202a, 0x202b,
   0x202c, 0x202d, 0x202e, 0x2060, 0x2061, 0x2062, 0x2063, 0x2064,
   0x2065, 0x2066, 0x2067, 0x2068, 0x2069, 0x206a, 0x206b, 0x206c,
   0x206d, 0x206e, 0x206f, 0x2cef, 0x2cf0, 0x2cf1, 0x2d7f, 0x3164,
   0xa6f0, 0xa6f1, 0xa802, 0xa806, 0xa80b, 0xa825, 0xa826, 0xa8b6,
   0xa8c4, 0xa948, 0xa949, 0xa94a, 0xa94b, 0xa94c, 0xa94d, 0xa94e,
   0xa94f, 0xa950, 0xa951, 0xa953, 0xa961, 0xa962, 0xa963, 0xa964,
   0xa965, 0xa966, 0xa967, 0xa968, 0xa969, 0xa96a, 0xa96b, 0xa96c,
   0xa96d, 0xa96e, 0xa96f, 0xa970, 0xa971, 0xa972, 0xa973, 0xa974,
   0xa975, 0xa976, 0xa977, 0xa978, 0xa979, 0xa97a, 0xa97b, 0xa97c,
   0xa9e5, 0xaa7b, 0xaa7c, 0xaa7d, 0xaab0, 0xaab2, 0xaab3, 0xaab4,
   0xaab7, 0xaab8, 0xaabe, 0xaabf, 0xaac1, 0xaaec, 0xaaed, 0xaaf6,
   0xabe5, 0xabe8, 0xabe9, 0xabea, 0xabed, 0xd7b1, 0xd7b2, 0xd7b3,
   0xd7b4, 0xd7b5, 0xd7b6, 0xd7b7, 0xd7b8, 0xd7b9, 0xd7ba, 0xd7bb,
   0xd7bc, 0xd7bd, 0xd7be, 0xd7bf, 0xd7c0, 0xd7c1, 0xd7c2, 0xd7c3,
   0xd7c4, 0xd7c5, 0xd7c6, 0xd7cc, 0xd7cd, 0xd7ce, 0xd7cf, 0xd7d0,
   0xd7d1, 0xd7d2, 0xd7d3, 0xd7d4, 0xd7d5, 0xd7d6, 0xd7d7, 0xd7d8,
   0xd7d9, 0xd7da, 0xd7db, 0xd7dc, 0xd7dd, 0xd7de, 0xd7df, 0xd7e0,
   0xd7e1, 0xd7e2, 0xd7e3, 0xd7e4, 0xd7e5, 0xd7e6, 0xd7e7, 0xd7e8,
   0xd7e9, 0xd7ea, 0xd7eb, 0xd7ec, 0xd7ed, 0xd7ee, 0xd7ef, 0xd7f0,
   0xd7f1, 0xd7f2, 0xd7f3, 0xd7f4, 0xd7f5, 0xd7f6, 0xd7f7, 0xd7f8,
   0xd7f9, 0xd7fa, 0xd7fb, 0xf850, 0xf85f, 0xf860, 0xf861, 0xf862,
   0xf863, 0xf864, 0xf865, 0xf866, 0xf867, 0xf868, 0xf869, 0xf86a,
   0xf86b, 0xf86c, 0xf86d, 0xf86e, 0xf86f, 0xf884, 0xf885, 0xf886,
   0xf887, 0xf888, 0xf889, 0xf88a, 0xf88b, 0xf88c, 0xf88d, 0xf88e,
   0xf88f, 0xf890, 0xf891, 0xf892, 0xf893, 0xf894, 0xf895, 0xf896,
   0xf897, 0xf898, 0xf899, 0xf89f, 0xfbb2, 0xfbb3, 0xfbb4, 0xfbb5,
   0xfbb6, 0xfbb7, 0xfbb8, 0xfbb9, 0xfbba, 0xfbbb, 0xfbbd, 0xfbbe,
   0xfbbf, 0xfbc1, 0xfc5e, 0xfc5f, 0xfc60, 0xfc61, 0xfc62, 0xfc63,
   0xfe0f, 0xfe20, 0xfe21, 0xfe22, 0xfe23, 0xfeff, 0xffa0, 0xfff0,
   0xfff1, 0xfff2, 0xfff3, 0xfff4, 0xfff5, 0xfff6, 0xfff7, 0xfff8,
   0xfff9, 0xfffa, 0xfffb, 0xfffc, 0x10a01, 0x10a02, 0x10a03, 0x10a05,
   0x10a06, 0x10a0c, 0x10a0d, 0x10a0e, 0x10a0f, 0x10a38, 0x10a39, 0x10a3a,
   0x11038, 0x11039, 0x1103a, 0x1103b, 0x1103c, 0x1103d, 0x1103e, 0x1103f,
   0x11040, 0x11041, 0x11042, 0x11043, 0x11044, 0x11045, 0x11046, 0x11080,
   0x11081, 0x110b1, 0x110b3, 0x110b4, 0x110b5, 0x110b6, 0x110b9, 0x110ba,
   0x11a01, 0x11a02, 0x11a03, 0x11a04, 0x11a05, 0x11a06, 0x11a07, 0x11a08,
   0x11a09, 0x11a0a, 0x11a33, 0x11a35, 0x11a36, 0x11a37, 0x11a38, 0x11a39,
   0x11a3b, 0x11a3c, 0x11a3d, 0x11a3e, 0x11a51, 0x11a52, 0x11a53, 0x11a54,
   0x11a55, 0x11a56, 0x11a57, 0x11a58, 0x11a59, 0x11a5a, 0x11a5b, 0x11a8a,
   0x11a8b, 0x11a8c, 0x11a8d, 0x11a8e, 0x11a8f, 0x11a90, 0x11a91, 0x11a92,
   0x11a93, 0x11a94, 0x11a95, 0x11a96, 0x11a97, 0x11a98, 0x11d31, 0x11d32,
   0x11d33, 0x11d34, 0x11d35, 0x11d36, 0x11d3a, 0x11d3c, 0x11d3d, 0x11d3f,
   0x11d40, 0x11d41, 0x11d43, 0x11d47, 0x1bc9d, 0x1bca0, 0x1bca1, 0x1bca2,
   0x1bca3, 0x1d173, 0x1d174, 0x1d175, 0x1d176, 0x1d177, 0x1d178, 0x1d179,
   0x1d17a, 0x1da00, 0x1da01, 0x1da02, 0x1da03, 0x1da04, 0x1da05, 0x1da06,
   0x1da07, 0x1da08, 0x1da09, 0x1da0a, 0x1da0b, 0x1da0c, 0x1da0d, 0x1da0e,
   0x1da0f, 0x1da10, 0x1da11, 0x1da12, 0x1da13, 0x1da14, 0x1da15, 0x1da16,
   0x1da17, 0x1da18, 0x1da19, 0x1da1a, 0x1da1b, 0x1da1c, 0x1da1d, 0x1da1e,
   0x1da1f, 0x1da20, 0x1da21, 0x1da22, 0x1da23, 0x1da24, 0x1da25, 0x1da26,
   0x1da27, 0x1da28, 0x1da29, 0x1da2a, 0x1da2b, 0x1da2c, 0x1da2d, 0x1da2e,
   0x1da2f, 0x1da30, 0x1da31, 0x1da32, 0x1da33, 0x1da34, 0x1da35, 0x1da36,
   0x1da3b, 0x1da3c, 0x1da3d, 0x1da3e, 0x1da3f, 0x1da40, 0x1da41, 0x1da42,
   0x1da43, 0x1da44, 0x1da45, 0x1da46, 0x1da47, 0x1da48, 0x1da49, 0x1da4a,
   0x1da4b, 0x1da4c, 0x1da4d, 0x1da4e, 0x1da4f, 0x1da50, 0x1da51, 0x1da52,
   0x1da53, 0x1da54, 0x1da55, 0x1da56, 0x1da57, 0x1da58, 0x1da59, 0x1da5a,
   0x1da5b, 0x1da5c, 0x1da5d, 0x1da5e, 0x1da5f, 0x1da60, 0x1da61, 0x1da62,
   0x1da63, 0x1da64, 0x1da65, 0x1da66, 0x1da67, 0x1da68, 0x1da69, 0x1da6a,
   0x1da6b, 0x1da6c, 0x1da75, 0x1da84, 0x1da9b, 0x1da9c, 0x1da9d, 0x1da9e,
   0x1da9f, 0x1daa1, 0x1daa2, 0x1daa3, 0x1daa4, 0x1daa5, 0x1daa6, 0x1daa7,
   0x1daa8, 0x1daa9, 0x1daaa, 0x1daab, 0x1daac, 0x1daad, 0x1daae, 0x1daaf,

 /// diagnoseZeroWidthMatchAndAdvance - Error zerowidth characters in delimiters.
 /// A non visible character in the middle of a delimter can be used to extend
 /// the literal beyond what it would appear creating potential security bugs.
 static bool diagnoseZeroWidthMatchAndAdvance(char Target, const char *&CurPtr,
                                              DiagnosticEngine *Diags) {
   //  A way needs to be found to find the complete set of zero width chars or
   //  this security mitigation will be in vain. Current list is generated using
   //  the display width of attributed strings checking when it does not change.
   //  The font used was SF Mono, 11pt, the default font of the Xcode editor.
   static std::vector<uint32_t> ZeroWidthV;
   if (!ZeroWidthV.size())
                       ZeroWidthC + sizeof ZeroWidthC/sizeof ZeroWidthC[0]);

   const char *TmpPtr = CurPtr;
   while (true) {
     uint32_t NextChar = validateUTF8CharacterAndAdvance(TmpPtr, TmpPtr + 6);
     if (NextChar != (uint32_t)Target &&
         (NextChar == ~0U || (NextChar >= 0xe0000 && NextChar <= 0xe0fff) ||
          std::binary_search(ZeroWidthV.begin(), ZeroWidthV.end(), NextChar))) {
       if (Diags && *TmpPtr == Target)
       CurPtr = TmpPtr;

     return *CurPtr == Target && CurPtr++;

Copy link

swift-ci commented Oct 24, 2020

Comment by Michael Verges (JIRA)

Note for future implementation:

The pull request for SR-10011 adds a test case containing zero-width characters within a character sequence that resembles the multi-line delimiter. The implementation of diagnoseZeroWidthMatchAndAdvance may (or may not?) break this test case. If you run into that problem, revisit advanceIfMultilineDelimiter.

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
bug A deviation from expected or documented behavior Compiler Parser
None yet

No branches or pull requests

2 participants