build.sh error: Unable to parse all content from SOE Zip exit status 1

[alanzablocki]

Describe the bug

Trying to perform a docker build in a VM running Ubuntu 18.04.2 as guest OS , on Windows 10 host. Upon running the build script I get an SOE Error:

#To Reproduce
Run the build script with SOE zip

``
work@work:~/sas-container-recipes-19m04$ ./build.sh --type single --zip /home/work/SAS_Viya_deployment_data.zip --addons "auth-demo"

Building Docker Build Container

Sending build context to Docker daemon 2.136MB
Step 1/19 : FROM golang:1.12.0
---> cee68f119e19
Step 2/19 : ARG USER_UID=1000
---> Using cache
---> 1076921135f2
Step 3/19 : ARG DOCKER_GID=997
---> Using cache
---> 372931cdf39d
Step 4/19 : RUN apt-get update && apt-get install -y openjdk-8-jdk-headless ansible && rm -rf /var/lib/apt/lists/*
---> Using cache
---> 007dcfe51d92
Step 5/19 : RUN go get -x -u gopkg.in/yaml.v2 github.com/docker/docker/api/types github.com/docker/docker/client
---> Using cache
---> c50ca82c0101
Step 6/19 : RUN groupadd --gid ${DOCKER_GID} docker
---> Using cache
---> eb966abc16fb
Step 7/19 : RUN useradd --uid ${USER_UID} --gid ${DOCKER_GID} --create-home --shell /bin/bash sas
---> Using cache
---> 936eb8d0efcf
Step 8/19 : WORKDIR /sas-container-recipes
---> Using cache
---> f4d4c4558464
Step 9/19 : RUN chmod 777 /sas-container-recipes
---> Using cache
---> e721803871f8
Step 10/19 : COPY --chown=sas:docker addons ./addons
---> Using cache
---> 205663975502
Step 11/19 : COPY --chown=sas:docker samples ./samples
---> Using cache
---> ed27f1bb7f52
Step 12/19 : COPY --chown=sas:docker tests ./tests
---> Using cache
---> 197aa6afc60a
Step 13/19 : COPY --chown=sas:docker util ./util
---> Using cache
---> aa47b06ac46d
Step 14/19 : COPY --chown=sas:docker *.yml *.go ./
---> Using cache
---> 9bcfbc1d49a2
Step 15/19 : RUN chown -R ${USER_GID}:${DOCKER_GID} /sas-container-recipes
---> Using cache
---> 2934bab35e02
Step 16/19 : USER sas
---> Using cache
---> 7adc395018fd
Step 17/19 : ENTRYPOINT ["/usr/local/go/bin/go", "run", "main.go", "container.go", "order.go"]
---> Using cache
---> 008f4214070c
Step 18/19 : LABEL sas.recipe=true
---> Using cache
---> 21ba965ccbe8
Step 19/19 : LABEL sas.recipe.builder.version=19.04.0-20190510135948-no-git-sha
---> Running in de3b467b8811
Removing intermediate container de3b467b8811
---> e25b3f7bbbf9
Successfully built e25b3f7bbbf9
Successfully tagged sas-container-recipes-builder:19.04.0-20190510135948-no-git-sha

==============================
Running Docker Build Container

2e37b777abe3eeeb8f8cfb3d349d47e28c865e73c15482f019c122c0cc65dc70
2019/05/10 18:59:58

	Build Variables

==================================================
'Base Image ': centos:7
'Mirror URL ': https://ses.sas.download/ses/
'Virtual Host ': myvirtualhost.mycompany.com
'Docker Namespace ': ""
'Docker Registry ': ""
'Deployment Type ': single
'Platform ': redhat
'Project Name ': sas-viya
'Tag Override ': 19.04.0-20190510135948-no-git-sha
'AddOns ':

• addons/auth-demo/
  'Debug Containers ': []
  'Worker Count ': 1
  'Verbose ': false
  'Skip Mirror Validation ': true
  'Skip Docker Validation ': false
  'Generate Manifests Only ': false
  'Build Only ': []
  'Timestamp Tag ': 2019-05-10-18-59-58
  ==================================================

2019/05/10 18:59:58 Reading sitedefault.yml ...
2019/05/10 18:59:58 Skipping loading sitedefault.yml
2019/05/10 18:59:58 Fetching orchestration tool ...
2019/05/10 18:59:58 Reading Software Order Email Zip ...
2019/05/10 18:59:58 Connecting to the Docker daemon ...
2019/05/10 18:59:58 Finished connecting to Docker daemon
2019/05/10 18:59:58 Pulling base container image 'centos:7' ...

2019/05/10 18:59:58 Unable to parse all content from SOE zip
exit status 1
sas-container-recipes-builder-19.04.0-20190510135948-no-git-sha
``

Expected Behaviour

get the server address and be able to deploy

#Additional Context

We are therefore hitting this part of the build:

// Make sure all required files were loaded into the order if len(order.License) == 0 || len(order.MeteredLicense) == 0 || len(order.CA) == 0 || len(order.Entitlement) == 0 { fail <- "Unable to parse all content from SOE zip" return }

Any idea what the issue is with SOE? Could it be because we have a partner license?

Since the build complained about the SOE, I looked into the SOE structure as shown in
https://github.com/sassoftware/sas-container-recipes/blob/master/order.go

The only odd thing I saw, is that it seems like it is expecting a folder called license, but in our .zip, the folder is called licenses. Changing folder name did not fix the issue.

Also I noticed that while a windows Viya SOE had a jwt file, our Linux Viya does not. Could that be the issue?

#Environment

Our docker version and VM os details:
work@work:$ uname -a
Linux work 4.18.0-15-generic #1618.04.1-Ubuntu SMP Thu Feb 7 14:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
work@work:~$ docker version
Client:
Version: 18.09.6
API version: 1.39
Go version: go1.10.8
Git commit: 481bc77
Built: Sat May 4 02:35:57 2019
OS/Arch: linux/amd64
Experimental: false

Server: Docker Engine - Community
Engine:
Version: 18.09.6
API version: 1.39 (minimum version 1.12)
Go version: go1.10.8
Git commit: 481bc77
Built: Sat May 4 01:59:36 2019
OS/Arch: linux/amd64
Experimental: false

[g8sman]

The missing JWT file could be the issue. Is your order Viya 3.3 based? You could adjust the line you quoted and remove the check for order.MeteredLicense and see if that gets you further.

[alanzablocki]

Thank you. We got a new order zip (that now has the jwt file) for SAS Viya 3.4 on Linux and were able to get past the SOE Error, so a fresh zip with a jwt was the solution.

I had found that a different approach using the mirror manager listed in the SAS-Viya-3-4-for-Containers Deployment Guide (see Pre-installation tasks -> Download the image Step 3) returned no output and it could not find the latest docker tag. This SAS Communities post
confirms that the issue was with an old order zip.

Yesterday, I was able to finally build the docker image (build took 70 minutes) and deploy it locally, but I had to do the build in a Centos VM not Ubuntu VM. The build on Ubuntu goes quite far along before it fails when the script starts using yum commands, instead of apt-get.

Also I found that the root (/) directory had to be huge ( more than 50GB ) to deal with the intermediate build steps. My initial VM had plenty of space in /home, but not enough in root.

[g8sman]

Good stuff. Glad that you were able to get an updated order and had success in building the images. What we have seen in the past is that the space specifically for /var/lib/docker needs to have enough to cover the products that have been ordered. This is covered in the System Requirements. As for base image support, we only support RedHat based images. In the How to build section we hint that this is the case but do not explicitly state it. The reason gets back to the Supported platforms for SAS Viya 3.4.