Introduction
In modern cloud environments, efficient management and analysis of logs is crucial. However, balancing security and accessibility can be challenging, especially when working with sensitive data. This article will guide you through setting up OpenSearch in a private subnet to securely transfer logs while enabling public access to the OpenSearch dashboard.
OpenSearch Configuration
Login to AWS Console:
Navigate to the AWS Management Console.
Access OpenSearch Service and click Domains.
Select Create Domain.
2. Enter Domain Name:
Provide a unique domain name.
Choose Standard Create.
3. Select Domain Type:
Choose the Domain with Standby option for fault tolerance and scalability.
4. Configure Instances:
Select the appropriate Instance Type based on your workload.
Specify the Number of Nodes (recommended: 3).
5. EBS Storage Settings:
Choose the EBS Volume Type according to your performance needs.
Set EBS Storage Size per Node for scalable data storage.
6. Private OpenSearch Configuration:
Enable VPC Access.
Select IP Address Type, VPC ID, Subnet IDs, and Security Group for enhanced security.
7. Create Master User Credentials:
Set up master user credentials for secure access control.
8. Launch the Domain:
Click Create to initialize the OpenSearch domain.
9. Obtain OpenSearch Endpoint:
After successful domain creation, retrieve the OpenSearch Endpoint for use in subsequent configurations.
10. Configure ECS Task Definition:
Pass the retrieved OpenSearch details into the ECS task definition for integration.
11. Verify Connectivity:
Connect to an EC2 instance within the same VPC.
Ping the OpenSearch endpoint three times to obtain the associated IP addresses.
Network Load balancer Configuration:
1. Create a Load Balancer:
Go to Load Balancers in the AWS Management Console.
Select Create Load Balancer.
2. Choose Load Balancer Type:
Select Network Load Balancer and click Create.
3. Basic Configuration:
Enter a descriptive Load Balancer Name.
Choose Internet-facing for public access.
4. VPC and Subnet Selection:
Select the VPC and Public Subnets for internet-facing access.
5. Port and Protocol Settings:
Choose TCP as the protocol and set the Port to 443 (HTTPS).
6. Create Target Group:
After creating the load balancer, click Create Target Group.
Select IP Addresses under Basic Configuration.
7. Target Group Details:
Provide a Target Group Name.
Specify the Port Number and VPC.
Register Target IPs obtained from the OpenSearch endpoint.
Click Include as Pending Below, then Create Target Group.
8. Associate Target Group:
Select the created target group and finalize the load balancer creation.
Access OpenSearch Dashboard:
Once the configuration is complete, access the OpenSearch login page using:https://<load-balancer-dns>/_dashboards
Conclusion:
This setup offers a secure and scalable logging solution, combining private communication for log transfers with public access to the OpenSearch dashboard. By using a Network Load Balancer, carefully configuring subnets, and defining security groups, you ensure that your logging infrastructure remains secure, highly available, and easily accessible. This strategy strengthens your cloud environment while enhancing operational visibility and performance.
