S2N prevents building on Linux

[ugocottin]

Describe the bug

Building on Linux distro throw an error caused by aws-common-runtime/s2n submodule.
This error does not occur on macOS.

Expected Behavior

Building should be successful on Linux distro

Current Behavior

error: 'src': target 'S2N' has invalid header layout: umbrella header found at '/src/aws-common-runtime/s2n/api/S2N.h', but directories exist next to it: /src/aws-common-runtime/s2n/api/unstable; consider removing them

Reproduction Steps

On Linux distro, clone project with submodule:

git clone --recurse-submodules https://github.com/awslabs/aws-crt-swift.git

Build project:

swift build

Possible Solution

No response

Additional Information/Context

S2N submodule version: v1.3.35

AWS CRT SWIFT version used

0.6.1

Compiler and Version used

Swift 5.7

Operating System and version

Amazon Linux 2 (4.14), Ubuntu 22.04

[waahm7]

Hi,

Thank you for creating the issue. I have tried it on both Amazon Linux 2 and Ubuntu 22.04 but could not reproduce the issue. It works fine for both. Could you please try cloning it again and building it? Also, you will need to install OpenSSL, please do the following.

For Ubuntu:

apt-get install openssl
apt-get install libssl-dev

For Amazon Linux 2:

yum install openssl
yum install openssl-devel

[ugocottin]

Hi,
Thank you for your answer.
I tried with the swift:5.7-amazonlinux2 container image, and I succeed to reproduce the bug, but only if the repository was cloned on the host and mounted within a volume inside the container.

git clone --recurse https://github.com/awslabs/aws-crt-swift.git
cd aws-crt-swift
podman run -it --rm --volume "$(pwd)/:/src" --workdir "/src/" swift:5.7-amazonlinux2 swift build

I cannot reproduce if I clone the repository in the container.
I had the same ref for aws-crt-swift (0.6.1), and s2n submodule (v1.3.35)
Any ideas ?

[waahm7]

Hi,

Thank you for your answer. I have tried using both podman and docker to mount the source code, and I am able to do so without encountering any build issues. Could you please try cloning the repository again on the host to see if that resolves the problem? Thank you.

[ugocottin]

Hi,

I cloned again the repository (now on commit ccec2f7).

I've tried again on my machine (macOS 13.2.1 on Apple Silicon M1 Pro), and on another Mac (macOS 13.0.1, Intel), both with podman version 4.4.1 with vm Fedora CoreOS 37.X, unsuccessfully.

I've also tried with various versions of the container image swift:5.X-amazonlinux2, unsuccessfully.

[waahm7]

Thank you for the response. Can you please explain a bit more about the steps that you are taking to encounter this issue, or what am I missing below?

1. Create an AL2 or Ubuntu EC2 instance.
2. SSH into the instance and clone the aws-crt-swift repository recursively using git clone --recurse https://github.com/awslabs/aws-crt-swift.git.
3. Install Podman/Docker.
4. Run a swift:5.X-amazonlinux2 container using Podman/Docker and mount the code directory.
5. SSH into the container.
6. do yum install openssl-devel.
7. do Swift build, and it works.

[ugocottin]

The main and only notable difference between your steps and mine is that you are using AL2 or Ubuntu as host, and I'm using macOS.

But I finally found out the cause of my issue: my file system is case-insensitive  😠.
On macOS, default APFS volume is case-insensitive. When I did my steps with the mounted folder located in a case-sensitive APFS volume, I succeed in compiling aws-crt-swift.

This is due to this code in SPM:

        let umbrellaHeader = publicHeadersDir.appending(component: moduleName + ".h")
        if fileSystem.isFile(umbrellaHeader) {
            // In this case, 'PublicHeadersDir' is expected to contain no subdirectories.
            if directories.count != 0 {
                diagnosticsEmitter.emit(.umbrellaHeaderHasSiblingDirectories(targetName: targetName, umbrellaHeader: umbrellaHeader, siblingDirs: directories))
                return .none
            }
            return .umbrellaHeader(umbrellaHeader)
        }

https://github.com/apple/swift-package-manager/blob/bfcff5f3fd65dacad08db9c7b065b78d8dc72ee5/Sources/PackageLoading/ModuleMapGenerator.swift#L130-L137

SPM will look for the S2N.h file, which is the same as s2n.h on a case-insensitive file system, but not on a case-sensitive file system like on AL2 or Ubuntu.

[waahm7]

Thank you, that's great debugging! I was able to successfully reproduce the issue, and I'm currently looking into possible solutions.

[waahm7]

Hi, we have fixed this issue in 0.12.0 release. Please feel free to reopen the issue if it still exists.

[jbelkins]

FYI I was able to verify that this issue is fixed when I use 0.12.0 in AWS SDK for Swift