Could not interpret 'experimental_external_s3' as global setting: No internal setting named 'experimental_external_s3' exists

[spark131008]

Hello, I'm trying to create .hyper file from S3 natively by following an example here.

and I'm getting following error:

Caused by:
The Hyper server process exited during startup with exit code: 1
	Command-line: "/opt/python/tableauhyperapi/bin/hyper/hyperd run --date-style=MDY --date-style-lenient=false --experimental-external-s3=true --init-user=tableau_internal_user --language=en_US --log-config= --log-dir=/tmp/ --no-password=true --skip-license=true --telemetry-opt-in= --listen-connection tab.domain:///tmp/domain/auto --callback-connection tab.domain:///tmp/domain/3b5f57d9d4bf42b38beca85975adcd08"
	Child process' stderr:
	Error(s) while applying config file or command line parameters.
	Error. Could not interpret 'experimental_external_s3' as global setting: No internal setting named 'experimental_external_s3' exists.
	Critical setting errors encountered, shutting down.

I'm running this in AWS Lambda, Runtime 3.9, x86_64.
I have built a custom layer with this command:
mkdir -pv /build/layer/ && pip install -t /build/layer/python tableauhyperapi>=0.0.14946 tableauserverclient>=0.19.0

Below is the code I'm running:

from tableauhyperapi import HyperProcess, Connection, Telemetry, CreateMode, SqlType, TableDefinition, TableName, Nullability, Inserter, escape_string_literal
import os

database_path = '/tmp/ls1_kpi_15min.hyper'
CURRENT_DATASET = escape_string_literal("s3://mybucket/ymd=20230201/581b0d96eb7a46508ca22dca0702sdf9530.snappy.parquet")

def lambda_handler(event, context):
    list_files = os.listdir('/opt/python/tableauhyperapi/bin/hyper')
    print(list_files)
    
    # Start up a local Hyper process. 
    with HyperProcess(telemetry=Telemetry.SEND_USAGE_DATA_TO_TABLEAU, parameters={'log_config': '', 'log_dir':'/tmp/', "experimental_external_s3": "true"}) as hyper:

        with Connection(endpoint=hyper.endpoint, database=database_path, create_mode=CreateMode.CREATE_AND_REPLACE) as connection:
            table_name=TableName("ls1", "kpi_15min")
            
            cmd = f"CREATE TABLE {table_name}" \
                  f" AS ( SELECT * FROM EXTERNAL(S3_LOCATION({CURRENT_DATASET} FORMAT => 'parquet')))"
            connection.execute_command(cmd)
            
            row_count = connection.execute_scalar_query(f"SELECT COUNT(*) FROM {table_name}")
            print (f"Loaded {row_count} rows")

Any idea why this job is failing to interpret 'experimental_external_s3' as global setting?

[jkammerer]

Hello,

From the release notes (v0.0.15735):

Support for reading external files from S3 is now enabled by default. (The experimental_external_s3 setting has been removed. Specifying it now causes an unknown setting error.)

Simply remove the setting from your parameter list and it should work.