Alexander Abakumov
15e744da49
- Added comprehensive course structure with 11 modules - Included theoretical content, practical examples, and tasks - Multi-language code examples (Java, Rust, Node.js, Python) - Detailed analysis of architectural approaches and trade-offs - Complete caching strategies documentation - Architecture examples from real-world systems (Twitter, Netflix, Uber) - Structured navigation and learning path - Ready for educational use
562 lines
16 KiB
Markdown
562 lines
16 KiB
Markdown
# AWS (Amazon Web Services)
|
|
|
|
## Навигация
|
|
- [🏠 Главная страница курса](../../../README.md)
|
|
- [📚 Модуль 8](../README.md)
|
|
|
|
---
|
|
|
|
## Основные сервисы AWS
|
|
|
|
### 1. Compute Services
|
|
|
|
#### EC2 (Elastic Compute Cloud)
|
|
- Виртуальные серверы в облаке
|
|
- Различные типы инстансов (t3, m5, c5, r5, etc.)
|
|
- Автомасштабирование с Auto Scaling Groups
|
|
- Spot Instances для экономии
|
|
|
|
```yaml
|
|
# EC2 Instance Configuration
|
|
Resources:
|
|
WebServerInstance:
|
|
Type: AWS::EC2::Instance
|
|
Properties:
|
|
ImageId: ami-0c02fb55956c7d316 # Amazon Linux 2
|
|
InstanceType: t3.micro
|
|
KeyName: my-key-pair
|
|
SecurityGroupIds:
|
|
- !Ref WebServerSecurityGroup
|
|
UserData:
|
|
Fn::Base64: !Sub |
|
|
#!/bin/bash
|
|
yum update -y
|
|
yum install -y docker
|
|
systemctl start docker
|
|
systemctl enable docker
|
|
usermod -a -G docker ec2-user
|
|
docker run -d -p 80:80 nginx
|
|
Tags:
|
|
- Key: Name
|
|
Value: WebServer
|
|
- Key: Environment
|
|
Value: Production
|
|
```
|
|
|
|
#### Lambda (Serverless Computing)
|
|
- Выполнение кода без управления серверами
|
|
- Оплата только за время выполнения
|
|
- Автоматическое масштабирование
|
|
- Интеграция с другими AWS сервисами
|
|
|
|
```python
|
|
# Lambda Function Example
|
|
import json
|
|
import boto3
|
|
|
|
def lambda_handler(event, context):
|
|
"""
|
|
AWS Lambda function для обработки заказов
|
|
"""
|
|
# Получение данных из события
|
|
order_data = event.get('order', {})
|
|
|
|
# Валидация данных
|
|
if not validate_order(order_data):
|
|
return {
|
|
'statusCode': 400,
|
|
'body': json.dumps({'error': 'Invalid order data'})
|
|
}
|
|
|
|
# Обработка заказа
|
|
order_id = process_order(order_data)
|
|
|
|
# Отправка уведомления через SNS
|
|
sns = boto3.client('sns')
|
|
sns.publish(
|
|
TopicArn='arn:aws:sns:us-east-1:123456789012:order-notifications',
|
|
Message=f'Order {order_id} has been processed',
|
|
Subject='Order Processed'
|
|
)
|
|
|
|
return {
|
|
'statusCode': 200,
|
|
'body': json.dumps({
|
|
'orderId': order_id,
|
|
'status': 'processed'
|
|
})
|
|
}
|
|
|
|
def validate_order(order_data):
|
|
"""Валидация данных заказа"""
|
|
required_fields = ['customerId', 'items', 'total']
|
|
return all(field in order_data for field in required_fields)
|
|
|
|
def process_order(order_data):
|
|
"""Обработка заказа"""
|
|
# Логика обработки заказа
|
|
return f"ORD-{order_data.get('customerId')}-{int(time.time())}"
|
|
```
|
|
|
|
#### Elastic Beanstalk
|
|
- Платформа для развертывания приложений
|
|
- Автоматическое управление инфраструктурой
|
|
- Поддержка различных языков и фреймворков
|
|
|
|
```yaml
|
|
# Elastic Beanstalk Configuration
|
|
Resources:
|
|
WebAppEnvironment:
|
|
Type: AWS::ElasticBeanstalk::Environment
|
|
Properties:
|
|
ApplicationName: !Ref WebApplication
|
|
EnvironmentName: !Sub "${AWS::StackName}-env"
|
|
SolutionStackName: "64bit Amazon Linux 2 v3.4.0 running Docker"
|
|
VersionLabel: !Ref ApplicationVersion
|
|
OptionSettings:
|
|
- Namespace: aws:autoscaling:launchconfiguration
|
|
OptionName: InstanceType
|
|
Value: t3.micro
|
|
- Namespace: aws:autoscaling:asg
|
|
OptionName: MinSize
|
|
Value: "1"
|
|
- Namespace: aws:autoscaling:asg
|
|
OptionName: MaxSize
|
|
Value: "3"
|
|
```
|
|
|
|
### 2. Storage Services
|
|
|
|
#### S3 (Simple Storage Service)
|
|
- Объектное хранилище для файлов
|
|
- Высокая доступность и долговечность
|
|
- Различные классы хранения
|
|
- Интеграция с другими AWS сервисами
|
|
|
|
```yaml
|
|
# S3 Bucket Configuration
|
|
Resources:
|
|
MyBucket:
|
|
Type: AWS::S3::Bucket
|
|
Properties:
|
|
BucketName: my-app-bucket
|
|
VersioningConfiguration:
|
|
Status: Enabled
|
|
PublicAccessBlockConfiguration:
|
|
BlockPublicAcls: true
|
|
BlockPublicPolicy: true
|
|
IgnorePublicAcls: true
|
|
RestrictPublicBuckets: true
|
|
LifecycleConfiguration:
|
|
Rules:
|
|
- Id: TransitionToIA
|
|
Status: Enabled
|
|
Transitions:
|
|
- StorageClass: STANDARD_IA
|
|
TransitionInDays: 30
|
|
- Id: DeleteOldVersions
|
|
Status: Enabled
|
|
NoncurrentVersionTransitions:
|
|
- StorageClass: GLACIER
|
|
TransitionInDays: 30
|
|
NoncurrentVersionExpirationInDays: 90
|
|
```
|
|
|
|
#### EBS (Elastic Block Store)
|
|
- Блочное хранилище для EC2 инстансов
|
|
- Высокая производительность
|
|
- Различные типы томов (gp3, io2, st1, sc1)
|
|
|
|
```yaml
|
|
# EBS Volume Configuration
|
|
Resources:
|
|
MyVolume:
|
|
Type: AWS::EC2::Volume
|
|
Properties:
|
|
Size: 100
|
|
VolumeType: gp3
|
|
Iops: 3000
|
|
Throughput: 125
|
|
Encrypted: true
|
|
AvailabilityZone: !GetAZs[0]
|
|
Tags:
|
|
- Key: Name
|
|
Value: MyDataVolume
|
|
```
|
|
|
|
#### Glacier
|
|
- Архивное хранилище для долгосрочного хранения
|
|
- Низкая стоимость
|
|
- Различные классы доступа
|
|
|
|
### 3. Database Services
|
|
|
|
#### RDS (Relational Database Service)
|
|
- Управляемые реляционные базы данных
|
|
- Поддержка MySQL, PostgreSQL, Oracle, SQL Server
|
|
- Автоматические бэкапы и патчи
|
|
- Multi-AZ для высокой доступности
|
|
|
|
```yaml
|
|
# RDS Configuration
|
|
Resources:
|
|
MyDatabase:
|
|
Type: AWS::RDS::DBInstance
|
|
Properties:
|
|
DBInstanceIdentifier: my-database
|
|
DBInstanceClass: db.t3.micro
|
|
Engine: postgres
|
|
EngineVersion: "13.7"
|
|
MasterUsername: admin
|
|
MasterUserPassword: !Ref DatabasePassword
|
|
AllocatedStorage: 20
|
|
StorageType: gp2
|
|
VPCSecurityGroups:
|
|
- !Ref DatabaseSecurityGroup
|
|
DBSubnetGroupName: !Ref DBSubnetGroup
|
|
BackupRetentionPeriod: 7
|
|
MultiAZ: true
|
|
StorageEncrypted: true
|
|
DeletionProtection: true
|
|
```
|
|
|
|
#### DynamoDB
|
|
- NoSQL база данных
|
|
- Управляемый сервис
|
|
- Автоматическое масштабирование
|
|
- Глобальные таблицы
|
|
|
|
```yaml
|
|
# DynamoDB Table Configuration
|
|
Resources:
|
|
MyTable:
|
|
Type: AWS::DynamoDB::Table
|
|
Properties:
|
|
TableName: my-table
|
|
AttributeDefinitions:
|
|
- AttributeName: id
|
|
AttributeType: S
|
|
- AttributeName: timestamp
|
|
AttributeType: N
|
|
KeySchema:
|
|
- AttributeName: id
|
|
KeyType: HASH
|
|
- AttributeName: timestamp
|
|
KeyType: RANGE
|
|
BillingMode: PAY_PER_REQUEST
|
|
StreamSpecification:
|
|
StreamViewType: NEW_AND_OLD_IMAGES
|
|
GlobalSecondaryIndexes:
|
|
- IndexName: GSI1
|
|
KeySchema:
|
|
- AttributeName: status
|
|
KeyType: HASH
|
|
Projection:
|
|
ProjectionType: ALL
|
|
```
|
|
|
|
#### Redshift
|
|
- Data warehouse для аналитики
|
|
- Columnar storage
|
|
- Масштабирование кластера
|
|
|
|
### 4. Networking Services
|
|
|
|
#### VPC (Virtual Private Cloud)
|
|
- Изолированная виртуальная сеть
|
|
- Полный контроль над сетевой конфигурацией
|
|
- Интеграция с on-premises инфраструктурой
|
|
|
|
```yaml
|
|
# VPC Configuration
|
|
Resources:
|
|
MyVPC:
|
|
Type: AWS::EC2::VPC
|
|
Properties:
|
|
CidrBlock: 10.0.0.0/16
|
|
EnableDnsHostnames: true
|
|
EnableDnsSupport: true
|
|
Tags:
|
|
- Key: Name
|
|
Value: MyVPC
|
|
|
|
PublicSubnet:
|
|
Type: AWS::EC2::Subnet
|
|
Properties:
|
|
VpcId: !Ref MyVPC
|
|
CidrBlock: 10.0.1.0/24
|
|
AvailabilityZone: !Select [0, !GetAZs '']
|
|
MapPublicIpOnLaunch: true
|
|
Tags:
|
|
- Key: Name
|
|
Value: PublicSubnet
|
|
|
|
PrivateSubnet:
|
|
Type: AWS::EC2::Subnet
|
|
Properties:
|
|
VpcId: !Ref MyVPC
|
|
CidrBlock: 10.0.2.0/24
|
|
AvailabilityZone: !Select [1, !GetAZs '']
|
|
Tags:
|
|
- Key: Name
|
|
Value: PrivateSubnet
|
|
|
|
InternetGateway:
|
|
Type: AWS::EC2::InternetGateway
|
|
Properties:
|
|
Tags:
|
|
- Key: Name
|
|
Value: MyIGW
|
|
|
|
InternetGatewayAttachment:
|
|
Type: AWS::EC2::VPCGatewayAttachment
|
|
Properties:
|
|
InternetGatewayId: !Ref InternetGateway
|
|
VpcId: !Ref MyVPC
|
|
```
|
|
|
|
#### CloudFront
|
|
- CDN для глобального распространения контента
|
|
- Интеграция с S3 и другими источниками
|
|
- SSL/TLS сертификаты
|
|
|
|
```yaml
|
|
# CloudFront Distribution
|
|
Resources:
|
|
MyDistribution:
|
|
Type: AWS::CloudFront::Distribution
|
|
Properties:
|
|
DistributionConfig:
|
|
Origins:
|
|
- Id: S3Origin
|
|
DomainName: !GetAtt MyBucket.RegionalDomainName
|
|
S3OriginConfig:
|
|
OriginAccessIdentity: !Sub 'origin-access-identity/cloudfront/${MyOriginAccessIdentity}'
|
|
DefaultCacheBehavior:
|
|
TargetOriginId: S3Origin
|
|
ViewerProtocolPolicy: redirect-to-https
|
|
AllowedMethods: [GET, HEAD]
|
|
CachedMethods: [GET, HEAD]
|
|
ForwardedValues:
|
|
QueryString: false
|
|
Cookies:
|
|
Forward: none
|
|
Enabled: true
|
|
DefaultRootObject: index.html
|
|
PriceClass: PriceClass_100
|
|
```
|
|
|
|
### 5. Security Services
|
|
|
|
#### IAM (Identity and Access Management)
|
|
- Управление пользователями и доступом
|
|
- Роли и политики
|
|
- MFA и временные credentials
|
|
|
|
```yaml
|
|
# IAM Role Configuration
|
|
Resources:
|
|
EC2Role:
|
|
Type: AWS::IAM::Role
|
|
Properties:
|
|
AssumeRolePolicyDocument:
|
|
Version: '2012-10-17'
|
|
Statement:
|
|
- Effect: Allow
|
|
Principal:
|
|
Service: ec2.amazonaws.com
|
|
Action: sts:AssumeRole
|
|
Policies:
|
|
- PolicyName: S3Access
|
|
PolicyDocument:
|
|
Version: '2012-10-17'
|
|
Statement:
|
|
- Effect: Allow
|
|
Action:
|
|
- s3:GetObject
|
|
- s3:PutObject
|
|
Resource: !Sub '${MyBucket}/*'
|
|
ManagedPolicyArns:
|
|
- arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy
|
|
|
|
EC2InstanceProfile:
|
|
Type: AWS::IAM::InstanceProfile
|
|
Properties:
|
|
Roles:
|
|
- !Ref EC2Role
|
|
```
|
|
|
|
#### KMS (Key Management Service)
|
|
- Управление ключами шифрования
|
|
- Шифрование данных
|
|
- Интеграция с другими сервисами
|
|
|
|
### 6. Monitoring and Logging
|
|
|
|
#### CloudWatch
|
|
- Мониторинг ресурсов и приложений
|
|
- Логи и метрики
|
|
- Алерты и автоматические действия
|
|
|
|
```yaml
|
|
# CloudWatch Alarms
|
|
Resources:
|
|
HighCPUAlarm:
|
|
Type: AWS::CloudWatch::Alarm
|
|
Properties:
|
|
AlarmName: HighCPUUtilization
|
|
AlarmDescription: Alarm when CPU exceeds 80%
|
|
MetricName: CPUUtilization
|
|
Namespace: AWS/EC2
|
|
Statistic: Average
|
|
Period: 300
|
|
EvaluationPeriods: 2
|
|
Threshold: 80
|
|
ComparisonOperator: GreaterThanThreshold
|
|
Dimensions:
|
|
- Name: InstanceId
|
|
Value: !Ref MyInstance
|
|
AlarmActions:
|
|
- !Ref ScaleUpPolicy
|
|
|
|
LowCPUAlarm:
|
|
Type: AWS::CloudWatch::Alarm
|
|
Properties:
|
|
AlarmName: LowCPUUtilization
|
|
AlarmDescription: Alarm when CPU is below 20%
|
|
MetricName: CPUUtilization
|
|
Namespace: AWS/EC2
|
|
Statistic: Average
|
|
Period: 300
|
|
EvaluationPeriods: 2
|
|
Threshold: 20
|
|
ComparisonOperator: LessThanThreshold
|
|
Dimensions:
|
|
- Name: InstanceId
|
|
Value: !Ref MyInstance
|
|
AlarmActions:
|
|
- !Ref ScaleDownPolicy
|
|
```
|
|
|
|
#### CloudTrail
|
|
- Аудит API вызовов
|
|
- Логирование действий пользователей
|
|
- Соответствие требованиям
|
|
|
|
### 7. Application Services
|
|
|
|
#### API Gateway
|
|
- Управление API
|
|
- Аутентификация и авторизация
|
|
- Rate limiting и throttling
|
|
|
|
```yaml
|
|
# API Gateway Configuration
|
|
Resources:
|
|
MyAPI:
|
|
Type: AWS::ApiGateway::RestApi
|
|
Properties:
|
|
Name: MyAPI
|
|
Description: My API Gateway
|
|
EndpointConfiguration:
|
|
Types:
|
|
- REGIONAL
|
|
|
|
MyResource:
|
|
Type: AWS::ApiGateway::Resource
|
|
Properties:
|
|
RestApiId: !Ref MyAPI
|
|
ParentId: !GetAtt MyAPI.RootResourceId
|
|
PathPart: myresource
|
|
|
|
MyMethod:
|
|
Type: AWS::ApiGateway::Method
|
|
Properties:
|
|
RestApiId: !Ref MyAPI
|
|
ResourceId: !Ref MyResource
|
|
HttpMethod: GET
|
|
AuthorizationType: NONE
|
|
Integration:
|
|
Type: AWS_PROXY
|
|
IntegrationHttpMethod: POST
|
|
Uri: !Sub 'arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${MyLambdaFunction.Arn}/invocations'
|
|
```
|
|
|
|
#### SNS (Simple Notification Service)
|
|
- Уведомления и сообщения
|
|
- Push, SMS, Email
|
|
- Интеграция с приложениями
|
|
|
|
```yaml
|
|
# SNS Topic Configuration
|
|
Resources:
|
|
MyTopic:
|
|
Type: AWS::SNS::Topic
|
|
Properties:
|
|
TopicName: my-topic
|
|
DisplayName: My Topic
|
|
Subscription:
|
|
- Protocol: email
|
|
Endpoint: admin@example.com
|
|
- Protocol: sqs
|
|
Endpoint: !GetAtt MyQueue.Arn
|
|
```
|
|
|
|
#### SQS (Simple Queue Service)
|
|
- Очереди сообщений
|
|
- Асинхронная обработка
|
|
- Dead letter queues
|
|
|
|
```yaml
|
|
# SQS Queue Configuration
|
|
Resources:
|
|
MyQueue:
|
|
Type: AWS::SQS::Queue
|
|
Properties:
|
|
QueueName: my-queue
|
|
VisibilityTimeoutSeconds: 30
|
|
MessageRetentionPeriod: 1209600
|
|
ReceiveMessageWaitTimeSeconds: 20
|
|
RedrivePolicy:
|
|
deadLetterTargetArn: !GetAtt MyDeadLetterQueue.Arn
|
|
maxReceiveCount: 3
|
|
|
|
MyDeadLetterQueue:
|
|
Type: AWS::SQS::Queue
|
|
Properties:
|
|
QueueName: my-dlq
|
|
MessageRetentionPeriod: 1209600
|
|
```
|
|
|
|
## Лучшие практики AWS
|
|
|
|
### 1. Security
|
|
- Используйте IAM роли вместо ключей доступа
|
|
- Включайте шифрование для всех данных
|
|
- Регулярно ротируйте ключи
|
|
- Используйте VPC для изоляции ресурсов
|
|
|
|
### 2. Cost Optimization
|
|
- Используйте Spot Instances для не критичных workload
|
|
- Настройте Lifecycle policies для S3
|
|
- Мониторьте использование ресурсов
|
|
- Используйте Reserved Instances для стабильных workload
|
|
|
|
### 3. Monitoring
|
|
- Настройте CloudWatch алерты
|
|
- Используйте CloudTrail для аудита
|
|
- Мониторьте затраты с помощью Cost Explorer
|
|
- Настройте автоматическое масштабирование
|
|
|
|
### 4. Backup and Recovery
|
|
- Включите автоматические бэкапы для RDS
|
|
- Используйте Cross-Region replication для критичных данных
|
|
- Тестируйте процедуры восстановления
|
|
- Документируйте disaster recovery планы
|
|
|
|
### 5. Networking
|
|
- Используйте приватные подсети для баз данных
|
|
- Настройте NAT Gateway для исходящего трафика
|
|
- Используйте VPC Endpoints для AWS сервисов
|
|
- Настройте Security Groups с минимальными правами
|