EC2 Instance Storage

 

I. What's  an EBS Volume?

  • An EBS (Elastic Block Store) Volume is a network drive you can attached to your instances while they run
  • It's allows your instances to persitt data, even after they termination
  • They can only be mounted to one instance at a time (at CCP level)
  • they are bound to a specific availability zone
  • Analogy: think of them as a "network usb stick"
  • free tier: 30GB of free EBS storage of type General Purpose(SSD) or Magnetic per month
  • It's a netwok drive (not a physical drive)
    • It uses the network to communicate the instance, which mean there might be a bit of latency
    • It's can be detached from an EC2 instance and attached to another one quickly
  • It's locked to an AZ
    • An EBS volume in us-east-1a can not be attached to us-east-1b
    • to move a volume across, you fist need to snapshot it
  • Have a provisioned capacity (size in GBs and IOPS)
    • you get billed for all the provisioned capacity
    • you can increase the capacity of the dirve over time 

1. EBS Volume - Example


2. EBS delete on Termination attribute

  • Controls the EBS behaviour when EC2 instances in termitates
    • by default, the root EB2 volume is deleted (attribute enabled) 
    • by default, any other attached EBS volume is not deleted (attribute disabled)
  • this can be controlled by the AWS console/ AWS CLI
II. EBS Hands on
Select an EC2 instance to which you want to attach more Volumes

copy EC2 instance's AZ to create new Volume


attach volume


To actually use this new block device: AWS Documentation

III. EBS Snapshots

  • make a backup (snapshot) for your EBS volume at a point in time
  • Not necessary to detach volume to do snapshot, but recommended
  • Can copy snapshot across AZ or region

1. EBS Snapshots Features

  • EBS sapshots archive
    • Move a snapshot to an "archive tier" that is 75% cheaper
    • Takes within 24 to 72 hours for restoring the archive
  • Recycle Bin for EBS Snapshots
    • Setup rules to retain deleted snapshots so you can recover them after an accidental deletion
    • specify retention (from 1 day to one year)
  • Fast snapshot restore (FSR)
    • force full initialization of snapshot to have no latency on the first use ($$$)

    IV. EBS Snapshots - Hands on

    1.create snapshot



    2. Copy snapshot Into any destination region that you want

    3. Create volume from snapshot

    4. Recycle Bin
    AWS Documentation
    • Create Retention Rule

    • Delete snapshot
    • Recovery snapshot

    V. AMI Overview

    • AMI = Amazon Machine Image
    • AMI are a customization of an EC2 Instance
    • you add your own software, configuration, operating system, monitoring, ...
    • Faster boot/ configuration time because all your software is pre-packaged
    • AMIs are built for a specific region (and can be copied across regions)
    • You can lunch EC2 instances from:
      • A public AMI: AWS provided
      • Your own AMI: you make and maintain them yourseft
      • An AWS Marketplace AMI: an AMI someone else made (and potentially sells)
    AMI Process ( from EC2 instance)
    • start an EC2 instance and customize it
    • stop the instance (for data integrity)
    • Build an AMI - this will also create EBS snapshots
    • Launch instances from other AMIs

    VI. AMI - Hands on

    1. Create Image from EC2 instance


    2. lunch Instance from AMIs

      VII. EC2 instance Store

      • EBS volumes are network drives with good but "limited" performance 
      • if you a need a high performance hardware disk => use EC2 instance store
      • Better I/O performance
      • EC2 instance store lose their storage if they're stopped
      • Good for buffer, cache, scratch data, temporary content
      • Risk of data loss if hardware fails
      • Backups and Replication are your responsibility
      Local EC2 instance store

        VIII. EBS Volume Types
        1. Amazon EBS volume types
        • EBS Volumes come in 6 types
          • gp2/gp3 (SSD): General purpose SSD volume that balances price and performance for a wide variety of workloads
          • io1/ io2 (SSD): Highest-performace SSD volume for mission-critical low-latency or hight-throughput workloads
          • st1 (HDD): low cost HDD volume designed for frequently accessed, throuhput-intensive workloads
          • sc1 (HDD): lowest cost HDD volume designed for less frequently accessed workloads
        • EBS Volumes are characterized in Size | Throughput | IOPS (I/O Ops per sec)
        • when in doubt always consult the AWS documentation - it's good!
        • Only gp2/gp3 and io1/io2 can be used as boot volumes
        2. EBS Volume use cases:
        General purpose SSD
        • Cost effective storage, low-latency
        • System boot volumes, Vitural Desktops, Development and test environments,
        • 1Gib - 16 TiB
        • gp3:
          • Baseline of 3,000 IOPS and throughput of 125 MiB/s
          • Can increase IOPS upto 16,000 and throughput up to 1000 MiB/s independently
        • gp2:
          • Small gp2 volumes can burst IOPS to 3,000
          • Size of the volume and IOPS are linked, max IOPS is 16,000
          • 3 IOPS per GB, means at 5334GB we are at the max IOPS
        Provisioned IOPS (PIOPS) SSD
        • critical business applications with sustained IOPS performance
        • Or applications that need more than 16,000 IOPS
        • Great for databases workloads (sensitive to storage perf and consistency)
        • io1 /io2 (4Gib - 16 Tib)
          • Max PIOPS: 64,000 for Nitro EC2 instance & 32,000 for other
          • Can increase PIOPS independently from storage size
          • io2 have more durability and more IOPS per GiB (at the same price as io1)
        • io2 Block Express (4GiB -64TiB):
          • Sub-milisecond latency
          • Max PIOPS: 256,000 with an IOPS: GiB ratio of 1,000:1
        • Supports multi-attach
        Hard Disk Drives (HDD)
        • Cannot be a boot volume
        • 125GiB to 16TiB
        • Throughput Optimized HDD (st1)
          • Big Data, Data Warehouses, Log Processing
          • Max throughtput 500 Mib/s - max IOPS 500
        • Cold HDD (sc1):
          • for data that is infrequently accessed
          • Scenarios where lowest cost is important
        3. EBS - Volume Types Summary


        IX. EBS Multi Attach
        • EBS Multi Attach - io1/io2 family
        • Attach the same EBS volume to multipe EC2 Instances in the same AZ
        • Each instance has full read & write permissions to the hight-performace volume
        • use-case:
          • Achieve higher application availability in clustered Linux applications (ex: teradata)
          • Applications must manage concurrent write operations
        • Up to 16 EC2 instances at the time
        • Must use a file system that's cluster-aware (not XFS, EXT4, etc...)
        X. Amazon EFS
        1. Amazon EFS - Elastic File System
        • Usecases: content management, web serving, data sharing, wordpress
        • Uses NFSv4.1 protocol
        • Uses security group to control access to EFS
        • compatible with Linux based AMI (not windows)
        • Encyption at rest using KMS
        • POSIX file system (~Linux) that has a standard file API
        2. EFS - Performance & Storage Classes
        • EFS scale 
          • 1000s of concurrent NFS clients, 10GB+ /s throughtput
          • Grow to Petabyte-scale network file system, automatically
        • Performace mode (set at EFS creation time)
          • General purpose (default) - latency-sensitive use cases (web server, CMS, etc, ...)
          • Max I/O - higher latency, throughput, hight parallel (big data, media processing)
        • Throughtput Mode
          • Bursting - 1TB = 50MiB/s + burst of up to 1000MiB/s
          • Provisioned - set your throughtput regardless of storage size, ex: 1GiB/s for 1 TB storage
          • Elastic - automatically scales throughput up or down based on your workloads 
            • up to 3GiB/s for reads and 1GiB/s for writes
            • used for unpredictable workloads
        3. EFS - Storage Classes
        • Storage Tiers (lifecycle management feature - move file after N days)
          • Standard: for frequently accessed files
          • Infrequent access (EFS-IA): cost to retrieve files, lower price to store. Enable EFS-IA with a Lifecycle Policy
        • Availability and durability
          • standard: Multi-AZ, great for prod
          • one zone: One AZ, great for dev, backup enable by default, compatible with IA (EFS One Zone-IA)
        • Over 90% in cost savings
        XI. Amazon EFS - Hands on

        XII. EBS vs EFS
        1. EBS vs EFS - Elastic Block Storage
        • EBS volumes
          • one instance (except multi-attach io1/io2)
          • are locked at the Availability Zone (AZ) level
          • gp2: IO increases if the disk size increases
          • gp3 & io1: can increase IO independently
        • To migrate an EBS volume across AZ
          • take a snapshot
          • Restore the snapshot to another AZ
          • EBS Backups use IO and you shouldn't run them while your application is handling a lot of traffic
        • Root EBS volumes of instances get terminated by default if the EC2 instance get terminated
        2. EBS vs EFS - Elastic File System
          • Mounting 100s of instances across AZ
          • EFS share website files (WordPress)
          • Only for Linux instances (POSIX)
          • EFS has a higher price point than EBS
          • Can leverage EFS-IA for cost savings
          • Remember: EFS vs EBS vs Instance Store


          Reference: Content based on "Ultimate AWS Certified Developer Associate 2023 NEW DVA-C02."

          Comments

          Post a Comment

          Popular posts from this blog

          IAM & AWS CLI

          Image
           I. IAM introduction: Users. Groups, Policies 1. IAM: Users & Groups IAM = indentity and Access Management, Global service. Root account create by default, shouldn't be used or shared (not best practice to use the root account) Users are people within organization, and can be grouped. Groups only contain users, not other groups. Users don't have to belong to a group, and user can belong to mutiple groups 2. IAM: Permissions Users or Groups can be assigned JSON ducuments called policies. These policies define the permissions of the users In AWS you apply the least privilege principle: don't give more permissions than a user needs. II. IAM Users & Groups Hands On create User  create or select one Group existed III. IAM Policies A policy is an object in AWS that, when associated with an identity or resource, defines their permissions. Permissions in the policies determine whether the request is allowed or denied. AWS Document :  Policies and permissions in IAM 1...
          Read more

          EC2 Fundamentals

          Image
            I. AWS Budget Setup Bills Free tier To get alerts about your upcomming costs, the best thing to do is to go into creating an AWS Budget. II. EC2 basics. 1. Amazon EC2. EC2 is one of the most popular of AWS's offering. EC2 = Elastic Compute Cloud = infrastructure as a service it mainly consists in the capbility of: Renting Virtual machines (EC2) Storing data on virtual drives (EBS) Distributing load across machines (ELB) Scaling the services using an auto-scaling group (ASG) Knowing EC2 is fundamental to understand how to Cloud works 2. EC2 rizing & and configuration options. Operation System (OS): Linux, MacOS, Window. How much compute power & cores (CPU)? How much ramdom-access memory(RAM) How much storage space: Network-attached (EBS &EFS) hardware  (EC2 instance storage) Network card: speed of the card, Public IP address. Firewall rules: security group Bootstrap script ( configure at the first launch): UC2 User Data 3. EC2 User Data it's possible to bootstrap ...
          Read more

          AWS Fundamentals: ELB + ASG

          Image
          I. Elastic Load Balacing overview 1. What is load balacing? Load Balances are servers that forward traffic to multiple servers(e.g., EC2 instances) downstream 2. Why use a load balacer? Spread load across multiple downstream instances Expose of single point of access (DNS) to your application Seamlessly hanle failures of downstream instances Do regular health checks to your instances Provide SSL termination (HTTPS) for your websites Enforce stickiness with cookies High availability across zones Separate public traffic from private traffic 3. Why use an Elastic load balancer? An Elastic load balancer is a managed load balancer AWS guarantees that it will be working AWS takes care of upgrades, maintenance, hight availability AWS provides only a few configuration knobs It costs less to setup your own load balancer but it will be a lot more effort on your end It's intergrated with many AWS offerings/ services EC2, EC2 Auto Scaling Groups, Amazon ECS AWS Certificate Manager (ACM), Cloud...
          Read more