Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for ALB #124

Merged
merged 2 commits into from
Jun 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ In order to deploy both the stacks the user needs to provide a set of required a
| certificateArn | Optional | string | Add ACM certificate to the any listener (OpenSearch or OpenSearch-Dashboards) whose port is mapped to 443. e.g., `--context certificateArn=arn:1234`|
| mapOpensearchPortTo | Optional | integer | Load balancer port number to map to OpenSearch. e.g., `--context mapOpensearchPortTo=8440` Defaults to 80 when security is disabled and 443 when security is enabled |
| mapOpensearchDashboardsPortTo | Optional | integer | Load balancer port number to map to OpenSearch-Dashboards. e.g., `--context mapOpensearchDashboardsPortTo=443` Always defaults to 8443 |
| loadBalancerType | Optional | string | The type of load balancer to deploy. Valid values are nlb for Network Load Balancer or alb for Application Load Balancer. Defaults to nlb. e.g., `--context loadBalancerType=alb` |

* Before starting this step, ensure that your AWS CLI is correctly configured with access credentials.
* Also ensure that you're running these commands in the current directory
Expand Down Expand Up @@ -169,7 +170,7 @@ All the ec2 instances are hosted in private subnet and can only be accessed usin

## Port Mapping

The ports to access the cluster are dependent on the `security` parameter value
The ports to access the cluster are dependent on the `security` parameter value and are identical whether using an Application Load Balancer (ALB) or a Network Load Balancer (NLB):
* If `security` is `disable` (HTTP),
* OpenSearch 9200 is mapped to port 80 on the LB
* If `security` is `enable` (HTTPS),
Expand Down
214 changes: 158 additions & 56 deletions lib/infra/infra-stack.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,13 @@
SubnetType,
} from 'aws-cdk-lib/aws-ec2';
import {
ApplicationListener,
ApplicationLoadBalancer,
ApplicationProtocol,
BaseApplicationListenerProps,
BaseListener,
BaseLoadBalancer,
BaseNetworkListenerProps,
ListenerCertificate,
NetworkListener, NetworkLoadBalancer, Protocol,
} from 'aws-cdk-lib/aws-elasticloadbalancingv2';
Expand All @@ -55,6 +62,11 @@
ARM64='arm64'
}

export enum LoadBalancerType {
NLB = 'nlb',
ALB = 'alb'
}

const getInstanceType = (instanceType: string, arch: string) => {
if (arch === 'x64') {
if (instanceType !== 'undefined') {
Expand Down Expand Up @@ -133,10 +145,13 @@
readonly mapOpensearchPortTo ?: number
/** Map opensearch-dashboards port on load balancer to */
readonly mapOpensearchDashboardsPortTo ?: number
/** Type of load balancer to use (e.g., 'nlb' or 'alb') */
readonly loadBalancerType?: LoadBalancerType
}

export class InfraStack extends Stack {
public readonly nlb: NetworkLoadBalancer;
public readonly elb: NetworkLoadBalancer | ApplicationLoadBalancer;
public readonly elbType: LoadBalancerType;

private instanceRole: Role;

Expand Down Expand Up @@ -200,8 +215,6 @@

constructor(scope: Stack, id: string, props: InfraProps) {
super(scope, id, props);
let opensearchListener: NetworkListener;
let dashboardsListener: NetworkListener;
let managerAsgCapacity: number;
let dataAsgCapacity: number;
let clientNodeAsg: AutoScalingGroup;
Expand Down Expand Up @@ -398,11 +411,28 @@

const certificateArn = `${props?.certificateArn ?? scope.node.tryGetContext('certificateArn')}`;

this.nlb = new NetworkLoadBalancer(this, 'clusterNlb', {
vpc: props.vpc,
internetFacing: (!this.isInternal),
crossZoneEnabled: true,
});
// Set the load balancer type, defaulting to NLB if not specified
const loadBalancerTypeStr = scope.node.tryGetContext('loadBalancerType') ?? 'nlb'
this.elbType = props?.loadBalancerType ?? LoadBalancerType[(loadBalancerTypeStr).toUpperCase() as keyof typeof LoadBalancerType];
switch (this.elbType) {
case LoadBalancerType.NLB:
this.elb = new NetworkLoadBalancer(this, 'clusterNlb', {
vpc: props.vpc,
internetFacing: (!this.isInternal),
crossZoneEnabled: true,
});
break;
case LoadBalancerType.ALB:
this.elb = new ApplicationLoadBalancer(this, 'clusterAlb', {
vpc: props.vpc,
internetFacing: (!this.isInternal),
crossZoneEnabled: true,
securityGroup: props.securityGroup,
});
break;
default:
throw new Error('Invalid load balancer type provided. Valid values are ' + Object.values(LoadBalancerType).join(', '));
}

const opensearchPortMap = `${props?.mapOpensearchPortTo ?? scope.node.tryGetContext('mapOpensearchPortTo')}`;
const opensearchDashboardsPortMap = `${props?.mapOpensearchDashboardsPortTo ?? scope.node.tryGetContext('mapOpensearchDashboardsPortTo')}`;
Expand All @@ -428,37 +458,30 @@
+ ` Current mapping is OpenSearch:${this.opensearchPortMapping} OpenSearch-Dashboards:${this.opensearchDashboardsPortMapping}`);
}

if (!this.securityDisabled && !this.minDistribution && this.opensearchPortMapping === 443 && certificateArn !== 'undefined') {
opensearchListener = this.nlb.addListener('opensearch', {
port: this.opensearchPortMapping,
protocol: Protocol.TLS,
certificates: [ListenerCertificate.fromArn(certificateArn)],
});
} else {
opensearchListener = this.nlb.addListener('opensearch', {
port: this.opensearchPortMapping,
protocol: Protocol.TCP,
});
}
const useSSLOpensearchListener = !this.securityDisabled && !this.minDistribution && this.opensearchPortMapping === 443 && certificateArn !== 'undefined';
const opensearchListener = InfraStack.createListener(
this.elb,
this.elbType,
'opensearch',
this.opensearchPortMapping,
(useSSLOpensearchListener) ? certificateArn : undefined
);

let dashboardsListener: NetworkListener | ApplicationListener;
if (this.dashboardsUrl !== 'undefined') {
if (!this.securityDisabled && !this.minDistribution && this.opensearchDashboardsPortMapping === 443 && certificateArn !== 'undefined') {
dashboardsListener = this.nlb.addListener('dashboards', {
port: this.opensearchDashboardsPortMapping,
protocol: Protocol.TLS,
certificates: [ListenerCertificate.fromArn(certificateArn)],
});
} else {
dashboardsListener = this.nlb.addListener('dashboards', {
port: this.opensearchDashboardsPortMapping,
protocol: Protocol.TCP,
});
}
const useSSLDashboardsListener = !this.securityDisabled && !this.minDistribution
&& this.opensearchDashboardsPortMapping === 443 && certificateArn !== 'undefined';
dashboardsListener = InfraStack.createListener(
this.elb,
this.elbType,
'dashboards',
this.opensearchDashboardsPortMapping,
(useSSLDashboardsListener) ? certificateArn : undefined
);
}

if (this.singleNodeCluster) {
console.log('Single node value is true, creating single node configurations');
singleNodeInstance = new Instance(this, 'single-node-instance', {

Check warning on line 484 in lib/infra/infra-stack.ts

View workflow job for this annotation

GitHub Actions / build

Unexpected console statement
vpc: props.vpc,
instanceType: singleNodeInstanceType,
machineImage: MachineImage.latestAmazonLinux2023(
Expand All @@ -483,19 +506,23 @@
});
Tags.of(singleNodeInstance).add('role', 'client');

opensearchListener.addTargets('single-node-target', {
port: 9200,
protocol: Protocol.TCP,
targets: [new InstanceTarget(singleNodeInstance)],
});
// Disable target security for now, can be provided as an option in the future
InfraStack.addTargetsToListener(
opensearchListener,
this.elbType,
'single-node-target',
9200,
new InstanceTarget(singleNodeInstance),
false);

if (this.dashboardsUrl !== 'undefined') {
// @ts-ignore
dashboardsListener.addTargets('single-node-osd-target', {
port: 5601,
protocol: Protocol.TCP,
targets: [new InstanceTarget(singleNodeInstance)],
});
InfraStack.addTargetsToListener(
dashboardsListener!,
this.elbType,
'single-node-osd-target',
5601,
new InstanceTarget(singleNodeInstance),
false);
}
new CfnOutput(this, 'private-ip', {
value: singleNodeInstance.instancePrivateIp,
Expand Down Expand Up @@ -660,23 +687,27 @@
Tags.of(mlNodeAsg).add('role', 'ml-node');
}

opensearchListener.addTargets('opensearchTarget', {
port: 9200,
protocol: Protocol.TCP,
targets: [clientNodeAsg],
});
// Disable target security for now, can be provided as an option in the future
InfraStack.addTargetsToListener(
opensearchListener,
this.elbType,
'opensearchTarget',
9200,
clientNodeAsg,
false);

if (this.dashboardsUrl !== 'undefined') {
// @ts-ignore
dashboardsListener.addTargets('dashboardsTarget', {
port: 5601,
protocol: Protocol.TCP,
targets: [clientNodeAsg],
});
InfraStack.addTargetsToListener(
dashboardsListener!,
this.elbType,
'dashboardsTarget',
5601,
clientNodeAsg,
false);
}
}
new CfnOutput(this, 'loadbalancer-url', {
value: this.nlb.loadBalancerDnsName,
value: this.elb.loadBalancerDnsName,
});

if (this.enableMonitoring) {
Expand Down Expand Up @@ -1013,4 +1044,75 @@

return cfnInitConfig;
}

/**
* Creates a listener for the given load balancer.
* If a certificate is provided, the protocol will be set to TLS/HTTPS.
* Otherwise, the protocol will be set to TCP/HTTP.
*/
private static createListener(elb: BaseLoadBalancer, elbType: LoadBalancerType, id: string, port: number,
certificateArn?: string): ApplicationListener | NetworkListener {
const useSSL = !!certificateArn;

let protocol: ApplicationProtocol | Protocol;
switch(elbType) {
case LoadBalancerType.ALB:
protocol = useSSL ? ApplicationProtocol.HTTPS : ApplicationProtocol.HTTP;
break;
case LoadBalancerType.NLB:
protocol = useSSL ? Protocol.TLS : Protocol.TCP;
break;
default:
throw new Error('Unsupported load balancer type.');
}

const listenerProps: BaseApplicationListenerProps | BaseNetworkListenerProps = {
port: port,
protocol: protocol,
certificates: useSSL ? [ListenerCertificate.fromArn(certificateArn)] : undefined,
};

switch(elbType) {
case LoadBalancerType.ALB: {
const alb = elb as ApplicationLoadBalancer;
return alb.addListener(id, listenerProps as BaseApplicationListenerProps);
}
case LoadBalancerType.NLB: {
const nlb = elb as NetworkLoadBalancer;
return nlb.addListener(id, listenerProps as BaseNetworkListenerProps);
}
default:
throw new Error('Unsupported load balancer type.');
}
}

/**
* Adds targets to the given listener.
* Works for both Application Load Balancers and Network Load Balancers.
*/
private static addTargetsToListener(listener: BaseListener, elbType: LoadBalancerType, id: string, port: number, target: AutoScalingGroup | InstanceTarget,
securityEnabled: boolean) {
switch(elbType) {
case LoadBalancerType.ALB: {
const albListener = listener as ApplicationListener;
albListener.addTargets(id, {
port: port,
protocol: securityEnabled ? ApplicationProtocol.HTTPS : ApplicationProtocol.HTTP,
targets: [target],
});
break;
}
case LoadBalancerType.NLB: {
const nlbListener = listener as NetworkListener;
nlbListener.addTargets(id, {
port: port,
protocol: securityEnabled ? Protocol.TLS : Protocol.TCP,
targets: [target],
});
break;
}
default:
throw new Error('Unsupported load balancer type.');
}
}
}
40 changes: 40 additions & 0 deletions test/infra-stack-props.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -317,3 +317,43 @@ test('Throw error on invalid CPU Arch', () => {
expect(error.message).toEqual('distributionUrl parameter is required. Please provide the OpenSearch distribution artifact url to download');
}
});

test('Throw error on invalid load balancer type', () => {
const app = new App({
context: {
distVersion: '1.0.0',
securityDisabled: false,
minDistribution: false,
cpuArch: 'x64',
singleNodeCluster: false,
dashboardsUrl: 'www.example.com',
distributionUrl: 'www.example.com',
serverAccessType: 'ipv4',
restrictServerAccessTo: 'all',
additionalConfig: '{ "name": "John Doe", "age": 30, "email": "johndoe@example.com" }',
additionalOsdConfig: '{ "something.enabled": "true", "something_else.enabled": "false" }',
loadBalancerType: 'invalid-type',
},
});

try {
// WHEN
const networkStack = new NetworkStack(app, 'opensearch-network-stack', {
env: { account: 'test-account', region: 'us-east-1' },
});

// @ts-ignore
const infraStack = new InfraStack(app, 'opensearch-infra-stack', {
vpc: networkStack.vpc,
securityGroup: networkStack.osSecurityGroup,
env: { account: 'test-account', region: 'us-east-1' },
});

// eslint-disable-next-line no-undef
fail('Expected an error to be thrown');
} catch (error) {
expect(error).toBeInstanceOf(Error);
// @ts-ignore
expect(error.message).toEqual('Invalid load balancer type provided. Valid values are nlb, alb');
}
});
Loading
Loading