Cloud Project: Azure and AWS connection through VPC

Project Title: Multi-Cloud Architecture for Fintech Firm

10/19/2024 - 11/3/2024

Objective: This capstone project was completed independently as part of my cloud engineering bootcamp. The objective was to design and implement a multi-cloud architecture using Azure and AWS to allow secure communication between resources without exposing data to the public internet. This project aimed to demonstrate my proficiency in cloud infrastructure, network security, and multi-cloud solutions.

Scenario: A fintech firm has a global clientele. To ensure the availability of its services, the chief solutions architect has suggested keeping the services running on a multi cloud environment. They want to deploy app resources in Azure and AWS and allow them to communicate with each other without exposing them to public access as the data on these resources will be confidential and should not be compromised. Create an architecture and the step by step guide to provide a solution for this scenario.

Technology Stack:

Azure VPN Gateway, AWS VPN Gateway, EC2, Virtual Machines, Azure Local Network Gateway, AWS Customer Gateway, Site-toSite VPN

-----------------------------------------------------------------------------------------------------------------------------------------------------

As the sole contributor to this project, I was responsible for every aspect of the design, implementation, and deployment of the multi-cloud architecture. My responsibilities included:

- Requirement Analysis: Identified the key requirements for secure, cross-cloud communication for a fintech application.

- Network Design: Developed the architecture for secure communication between Azure and AWS using site-to-site VPN.

- Configuration and Implementation: Set up Azure Virtual Machines (VMs) and AWS EC2 instances, and configured Azure VPN Gateway, AWS VPN Gateway, Customer Gateway, and Local Network Gateway for encrypted traffic.

- Network Security: Applied security rules, managed routing tables, and ensured compliance with industry standards for data protection.

- Testing and Validation: Independently tested the setup to ensure that communication between the two environments was secure and encrypted, with no exposure to the public internet.

Architecture Diagram https://drive.google.com/file/d/1a412QmA8BEcwshZJUcdXTqj_JEgPc1Wu/view?usp=drive_link

Step 1: MMulti-Cloud Environment Setup

• The project involves setting up a multi-cloud environment for a fintech firm that serves global clients. The services need to run on both Azure and AWS cloud platforms, and the two environments should be able to communicate securely without exposing any resources to the public internet due to the confidential nature of the data.

Step 2: Azure Network Setup

• Azure Virtual Machine (VM): Deploy a VM within an Azure virtual network (VNet)

• Azure VPN Gateway: Set up a VPN Gateway to handle the secure communication between the Azure VNet and the external AWS network. This gateway will handle the encrypted traffic across the site-to-site VPN.

• Create a Azure Gateway subnet

• Azure Local Network Gateway: Configure the Local Network Gateway, which holds the information about the external network (in this case, AWS). This defines the AWS VPC's public IP address and IP range, enabling Azure to understand where it is connecting.

Step 3: AWS Network Setup

• AWS EC2 Instance: Deploy an EC2 instance within a Virtual Private Cloud (VPC) in AWS.

• AWS Customer Gateway: Set up a Customer Gateway in AWS to hold the information about the external network (Azure). This is similar to the Azure Local Network Gateway and enables AWS to recognize the Azure VPN.

• AWS VPN Gateway: Configure a VPN Gateway to establish and manage the secure VPN connection between the AWS VPC and the Azure network.

Step 4: Establish Site-to-Site VPN Connection

• Establish a site-to-site VPN between the Azure VPN Gateway and the AWS VPN Gateway. This will create an encrypted connection between the two cloud environments (Azure and AWS), allowing the two networks to securely communicate.
• The site-to-site VPN connection ensures that sensitive data does not traverse the public internet and is fully encrypted while traveling between the two clouds.

Step 5: Configure Routing and Security Rules

• On both Azure and AWS, update the routing tables and network security groups (NSGs) to allow traffic over the VPN. Ensure the correct IP ranges are used for both sides to route traffic between the cloud platforms.
• On Azure: Add routes to allow traffic destined for AWS over the VPN Gateway.
• On AWS: Add routes to allow traffic to Azure using the VPN Gateway.

Step 6: Deploy Application Resources

• Deploy the necessary application resources in both Azure and AWS. The resources can communicate securely through the VPN connection, allowing the multi-cloud architecture to function as a unified system without exposing sensitive data to the internet.

Step 7: Test and Validate the Connection

• Test the setup by initiating traffic between the Azure and AWS environments. For example, ping the EC2 instance from the Azure VM to verify that the VPN connection is functioning correctly.
• Ensure that all data transfer is secure and that neither side is exposed to public internet traffic.

Step 8: Monitor and Manage the Environment

• Set up monitoring and logging in both Azure and AWS to track the performance and health of the VPN connection.
• Regularly check the VPN Gateway logs for any errors or unusual activity and monitor the performance of the deployed applications in both cloud environments.

Cloud Project: Deployment of Multicloud App

 Project name: Deployment of Multicloud App

Objectives: To deploy a multicloud app using two load balancer setups on AWS and Azure platforms

Skills: AWS, Azure, High Availability, Cloud Architect

Scenario

A logistics company wants to ensure high availability for its end users using their web application which can be accessed from anywhere in the world. Since the web application gets huge traffic everyday, the company cannot afford to have any downtime. So they have decided to deploy the web application on two cloud platforms to maintain resiliency.

Create an architecture and the step-by-step guide to provide a solution for this scenario.

Object:

Deploy the web application on both AWS and Azure so that if one platform experiences downtime, the application remains available on the other

Diagram

Step

1. Set up Azure VMs (VM1, VM2)

2. Set up AWS EC2 instances (two)

3. Set up Window web servers (4)

4. Azure Load Balancer setup

5. Azure Traffic manager setup

 

6. Health Check and Test

Cloud Project: Resilient Mult-Timer Web App with AWS SQS

Project name: Deploy a multi-timer web app using SQS

Skills: AWS, Azure

Scenario: A B2B web application deployed in Azure platform has been observing decreased number of leads/enquires being generated on their website. Upon investigating the logs generated in Azure Monitor service, they found that the app servers (backend) are going down for few minutes on a daily basis. Since they are on a tight budget, they are looking for a solution which can help them in retaining the forms filled via the front-end so that even if the backend servers are down the forms data submitted by the end users are not lost. They want to apply this solution part on AWS to retain the high availability of their app.  

Create an architecture and the step-by-step guide to provide a solution for the above problem statement.

Objective: To set up decoupling of a multi-timer web application deployed on Azure platform using the SQS service offered by AWS

The scenario involves deploying a B2B web application in Azure, where the backend servers frequently go down, leading to a potential loss of form data submitted by end users. The goal is to create an architecture that ensures form data is retained, even when the backend is temporarily down, by using AWS services.

Diagram

Step

Step 1:  Setup frontend VM (+Vnet)

Step 2: Setup backend VM (+Vnet)

Step 3: Create AWS SQS

Step4: Create RDS (mySQL)

- Add MySQL to inbound rules

Step 5: Configure connection by CLI 

-log in to frontend and backend VM

 -ssh azureadmin@public ip address

1. To install MySQL CLI use this command in cloudshell:

- sudo apt-get update && sudo apt-get install mysql-client

2. Use this below code to connect between mysql server:

- mysql -u admin -p -h amanpdb.c297f9nhmqb5.ap-south-1.rds.amazonaws.com --port=3306

3. Run the following query to create DB:

- CREATE DATABASE customerdb;

USE customerdb;

CREATE TABLE customers (

name VARCHAR(30) NOT NULL,

address VARCHAR(30) NOT NULL

);

****************************************************************************************************************************

Python Scripts to set up VMs as backend and frontend:

For frontend & backend:-

1. sudo apt-get update

2. sudo apt-get install awscli

3. sudo su

3. aws configure

***************

For frontend:-

1. sudo su

2. apt-get update

3. apt-get install -y python3

4. apt install python3-pip

5. pip install boto3

6. sudo nano send_message.py

7. Paste the code and change the queue URL of your VMs:

/////

import sys

import boto3

sqs = boto3.client('sqs')

queue_url = 'https://sqs.ap-south-1.amazonaws.com/001232840143/simplequeue'

response = sqs.send_message(

QueueUrl=queue_url,

MessageBody=(sys.argv[1])

)

print(response['MessageId'])

/////

8. python3 send_message.py Peter,USA    send 3 more messages

**************

For backend:-

1. sudo su

2. apt-get update

3. apt-get install -y python3

4. apt install python3-pip

5. pip install boto3

1. pip install mysql-connector-python

2. sudo nano get_message.py

4. Paste the below code:

/////

import time

import boto3

import mysql.connector

queue_url = 'https://sqs.ap-south-1.amazonaws.com/001232840143/simplequeue'

#Specify the database details

host = 'amanpdb.c297f9nhmqb5.ap-south-1.rds.amazonaws.com'

user = 'admin'

password = 'simplilearn'

database = 'customerdb'

#Create a SQS Client

sqs = boto3.client('sqs')

#Connect to the RDS MySQL Instance

mydb = mysql.connector.connect(host=host, user=user, password=password, database=database)

mycursor = mydb.cursor()

# Receive message from SQS queue

response = sqs.receive_message(QueueUrl=queue_url)

message = response['Messages'][0]

# Delete received message from queue

receipt_handle = message['ReceiptHandle']

sqs.delete_message(

QueueUrl=queue_url,

ReceiptHandle=receipt_handle

)

print('Received and deleted message: %s' % message["Body"])

#Get the customer name and address from the message

customerDetails = message["Body"]

customerDetailsList = customerDetails.split(',')

name = customerDetailsList[0]

address = customerDetailsList[1]

#Write the record to the database

val = (name, address)

sql = "INSERT INTO customers (name, address) VALUES (%s, %s)"

mycursor.execute(sql, val)

mydb.commit()

print("Record inserted in the DB")

////

5. Enter the default region as ap-south-1

6. In the backend use the command as sudo nano get_message.py

7. Type in python3 get_message.py to run the script, Send three more messages