Storage Gateway basics
AWS Storage Gateway is a hybrid storage service: an appliance on premises stores data in AWS.
How it works
- Deploy the gateway appliance on premises.
- Applications use it as local storage.
- The gateway stores the data in AWS.
Exam cues
- If the question says hybrid storage connecting on-premises apps to AWS storage, think Storage Gateway.
Remember
- On-premises appliance, cloud storage.
Trap: Storage Gateway keeps apps on premises; DataSync moves data in bulk.
Da documentação da AWS (3)
- AWS Storage Gateway connects an on-premises software appliance with AWS cloud storage so on-premises applications can use it.
- S3 File Gateway exposes S3 objects as files through Network File System (NFS) or Server Message Block (SMB) shares.
- Volume Gateway exposes block volumes through iSCSI; Tape Gateway exposes a virtual tape library through iSCSI.
File gateways
S3 File Gateway lets existing applications use shared files while storing their data as S3 objects. Network File SystemNFS is a network protocol that lets clients access shared files. (NFS) and Server Message BlockSMB is a network protocol that lets clients access shared files. (SMB) are protocols for accessing files over a network. Think of the gateway as a file interface backed by an S3 bucket.
How it works
- Deploy and activate the gateway as a virtual appliance on premises or an EC2 image. Associate a file share with an S3 bucket.
- Clients access the NFS or SMB share. Each file becomes an S3 object, and its path becomes the object keyAn object key identifies an S3 object and corresponds to its file path through S3 File Gateway., the object’s name within the bucket.
- The gateway uploads changes asynchronously, after the local file operation. A local cacheA cache keeps recently accessed data locally to reduce retrieval latency. keeps recently accessed data nearby.
- For an SMB share, join the gateway to the domain to authenticate users with Microsoft Active Directory, or use guest access.
Exam cues
- If the question says retain NFS or SMB access while storing accessible S3 objects, think S3 File Gateway, because it maps files to objects.
- If the question says Windows users need corporate logins for an SMB share, think Active Directory authentication, because the gateway can join the domain.
- If the question says block volumes or virtual tape libraryA virtual tape library presents software tape drives and a media changer to backup applications., think Volume Gateway or Tape Gateway, because both expose iSCSIiSCSI carries storage commands over an IP network so a server can access gateway devices. interfaces.
Remember
-
Private VPC endpoints allow gateway connections and data transfers to AWS without public internet access.
-
S3 data can use versioning, replication and lifecycle management.
-
FSx File Gateway closed to new customers in October 2024. Existing customers can continue using it.
Trap: FSx File Gateway is closed to new customers, although existing customers can continue using it.
Write files
Upload asynchronously
Da documentação da AWS (8)
- S3 File Gateway exposes S3 objects as files through Network File System (NFS) or Server Message Block (SMB) shares.
- A file maps to an S3 object and its file path maps to the object key; gateway changes are uploaded asynchronously.
- S3 File Gateway keeps recently accessed data in a local cache for low-latency access.
- S3 File Gateway can run as an on-premises virtual appliance or as an EC2 image.
- S3 File Gateway data is directly accessible through S3 and can use S3 versioning, replication and lifecycle management.
- FSx File Gateway closed to new customers in October 2024; existing customers can continue using it.
- An S3 File Gateway SMB file share authenticates users with Microsoft Active Directory (on-premises or AWS Managed Microsoft AD) after the gateway joins the domain, or with guest access.
- S3 File Gateway can use private VPC endpoints for its AWS connections and data transfers without requiring public internet access.
Volume and Tape Gateways
Volume Gateway gives existing servers block storage, which they access as disks. Tape Gateway gives backup software a virtual tape libraryA virtual tape library presents software tape drives and a media changer to backup applications., a software replacement for tape drives and a tape changer. Both use iSCSIiSCSI carries storage commands over an IP network so a server can access gateway devices., a protocol that sends storage commands over an IP network.
How it works
- For block storage, deploy Volume Gateway and present iSCSI volumes to application servers. Choose where the primary dataset must live.
- Cached volumes keep primary data in S3 and recently accessed data locally. An upload bufferAn upload buffer is local staging storage for data waiting to be transferred to AWS. stages writes before transfer to AWS.
- Stored volumes keep the entire primary dataset locally and asynchronously create cloud backups as EBS snapshotsA snapshot is a point-in-time backup that can be used to recover a volume.. A snapshot is a point-in-time backup; incremental snapshots retain changed data.
- For tape backups, connect existing backup software to Tape Gateway’s virtual drives and media changer. The gateway stages writes locally and uploads tape data asynchronously to S3.
- Eject a virtual tape through the backup software to archive it in S3 Glacier Flexible Retrieval or Deep Archive. Retrieve an archived tape to the gateway before reading it.
| Requirement | Choice | Primary access |
|---|---|---|
| Reduce local block storage | Cached Volume Gateway | iSCSI, hot data cached locally |
| Keep all block data local with cloud backups | Stored Volume Gateway | iSCSI, full local dataset |
| Replace physical tape infrastructure | Tape Gateway | iSCSI virtual tape library |
Exam cues
- If the question says all block data must remain local for low latency, think stored volumes, because only backups move to AWS asynchronously.
- If the question says reduce local block storage while retaining fast hot-data reads, think cached volumes, because S3 holds the primary dataset.
- If the question says preserve tape backup software while removing physical tapes, think Tape Gateway, because it presents a compatible virtual tape library.
Remember
- EBS snapshots support gateway restoration or EC2 recovery through an EBS volume.
- Typical archived tape retrieval: 3 to 5 hours from Flexible Retrieval, 12 hours from Deep Archive.
Trap: Volume Gateway stores data in S3 but does not expose it as ordinary S3 objects for direct S3 API access. That file-to-object access model belongs to S3 File Gateway.
Write virtual tapes
Eject to archive
Da documentação da AWS (12)
- Volume Gateway exposes cloud-backed block storage to application servers as iSCSI devices, which carry storage commands over IP networks.
- Cached volumes store primary data in S3 and keep frequently accessed data locally.
- Stored volumes keep the complete primary dataset on premises and asynchronously back it up to S3 as EBS snapshots.
- An upload buffer stages volume writes before the gateway uploads them to AWS.
- Volume Gateway snapshots are incremental point-in-time EBS backups that can restore gateway volumes or create EBS volumes for EC2 recovery.
- Cached Volume Gateway data and snapshots cannot be read with the S3 API or S3 console even though they are stored in S3.
- Tape Gateway presents a virtual tape library (VTL) with tape drives and a media changer as iSCSI devices to existing backup software.
- Tape Gateway stores writes locally before asynchronously uploading virtual tape data to S3.
- Ejecting a tape through backup software moves it to an archive backed by S3 Glacier Flexible Retrieval or S3 Glacier Deep Archive.
- Archived tapes cannot be read directly; retrieve them to Tape Gateway before reading.
- Archived tape retrieval typically takes 3 to 5 hours from S3 Glacier Flexible Retrieval or 12 hours from S3 Glacier Deep Archive.
- A virtual tape is 100 GiB to 15 TiB; a Tape Gateway holds up to 1,500 tapes and 1 PiB of tapes in total.
Uploads, cache and volume limits
S3 File Gateway upload notifications and cacheA cache keeps recently accessed data locally to reduce retrieval latency. refreshes help coordinate file workflows with S3. Volume Gateway limits determine the size and number of volumes available in each mode.
How it works
- After writing a working file set to a share, call
NotifyWhenUploaded. The response returns a notification ID without waiting for the uploads to finish. - Once all files written to the share up to the request time have reached S3, EventBridge sends a completion notification. It can route the event to SNS or Lambda.
- For changes made directly in S3, use
RefreshCacheto update the gateway’s cached object inventory. This does not load file contents into the local cache. - Alternatively, configure a time to liveFor cache refresh, TTL is the time since a directory refresh after which access triggers another refresh. (TTL) for cache refresh. After it expires, access to a directory triggers a refresh of that directory and waits until the refresh finishes. The refresh does not include subdirectories.
Exam cues
- If the question says a workflow must wait for a working file set to reach S3, think
NotifyWhenUploadedand the EventBridge completion notification, because the API response does not wait for the uploads. - If the question says each file upload needs its own notification, think
NotificationPolicy, because it reports individual uploads rather than a working file set. - If the question says objects changed outside the gateway are missing from a share, think
RefreshCacheor TTL-based refresh, because they update the cached inventory.
Remember
- Cached volumes: 1 GiB to 32 TiB per volume, up to 32 volumes per gateway.
- Stored volumes: 1 GiB to 16 TiB per volume, up to 32 volumes per gateway.
Trap: A
NotifyWhenUploadedresponse is not upload completion. Wait for the EventBridge notification. TTL refresh also does not scan every directory: it refreshes only an accessed directory after its TTL expires, and access waits for the refresh.
Da documentação da AWS (8)
- NotifyWhenUploaded requests an EventBridge notification after all files written to a file share up to the request time have uploaded to S3.
- NotifyWhenUploaded returns a NotificationId and does not wait synchronously for file upload completion.
- EventBridge can route the upload completion notification to targets such as SNS or Lambda.
- RefreshCache updates the cached inventory of S3 objects after changes outside the gateway; it does not import file contents into local cache storage.
- TTL-based cache refresh runs when a directory is accessed after its time to live has expired; it is non-recursive and blocks that directory access while refreshing.
- NotificationPolicy requests individual file upload notifications, whereas NotifyWhenUploaded tracks a working file set.
- Cached volumes support 1 GiB to 32 TiB per volume and up to 32 volumes per gateway.
- Stored volumes support 1 GiB to 16 TiB per volume and up to 32 volumes per gateway.
Flashcards
Nada para revisar aqui agora. Escolha outros tópicos ou outro filtro.
·
Todos os 17 flashcards em lista
On-premises applications must keep using local file or block storage while the data is stored in AWS. Which service fits?
AWS Storage Gateway.
Which AWS Storage Gateway interface fits file shares, block volumes and tape backup software?
S3 File Gateway for NFS or SMB files stored as S3 objects; Volume Gateway for iSCSI block volumes; Tape Gateway for an iSCSI virtual tape library.
An on-premises application must keep using SMB while AWS applications read its files as S3 objects. Which Storage Gateway type fits?
S3 File Gateway. It maps files to S3 objects and retains recently accessed data locally. Uploads happen asynchronously.
An on-premises application needs an SMB file share backed by Amazon S3 with low-latency access to recent data. Which AWS Storage Gateway option fits?
S3 File Gateway. It exposes S3 objects through SMB shares and caches recently accessed data locally.
Windows users must reach an S3 File Gateway share with their corporate logins. What is required?
An SMB file share with Active Directory authentication, with the gateway joined to the domain.
An on-premises S3 File Gateway must connect to AWS and transfer data without public internet access. Which service connectivity option fits?
Private VPC endpoints configured for the gateway's AWS connections and data transfers.
How do AWS Volume Gateway cached and stored volumes differ?
Cached volumes keep primary data in S3 and frequently accessed data locally. Stored volumes keep the entire primary dataset locally and asynchronously back it up as EBS snapshots.
A datacenter must replace physical tape hardware while retaining supported tape backup software. Which AWS Storage Gateway type fits?
Tape Gateway. Its iSCSI virtual tape library accepts backup writes and archives ejected tapes to S3 Glacier Flexible Retrieval or Deep Archive.
An on-premises block application needs low-latency access to its entire dataset and offsite backups. Which AWS Volume Gateway mode fits?
Stored volumes. The complete primary dataset remains local and EBS snapshots back it up asynchronously to AWS.
An application using AWS Volume Gateway needs less local primary storage and mostly reads a hot subset of data. Which volume mode fits?
Cached volumes. Primary data lives in S3 while frequently accessed data stays in the local cache.
Can an application use the S3 API to read data from an AWS Volume Gateway cached volume?
No. Use the iSCSI volume interface. The S3 backing data is not directly accessible through the S3 API or console.
An on-premises application needs iSCSI block storage but has limited local capacity. Which AWS Storage Gateway volume type fits?
Cached volumes. Volume Gateway stores primary data in S3 and keeps frequently accessed data locally.
A workflow must run only after an AWS S3 File Gateway working file set reaches S3. What should it use?
Call NotifyWhenUploaded after writing the file set and handle the later EventBridge completion notification. The API response itself is not upload completion.
An application writes objects directly into the S3 bucket backing an S3 File Gateway share. How should the gateway discover those changes?
Refresh the cached inventory with RefreshCache, or configure TTL-based refresh for directory access. Refreshing inventory does not preload file contents.
Does AWS S3 File Gateway TTL-based cache refresh scan every directory periodically?
No. It refreshes only the directory accessed after its TTL expires, without recursively refreshing subdirectories. Access waits for the refresh.
How do NotificationPolicy and NotifyWhenUploaded differ in AWS S3 File Gateway?
NotificationPolicy requests a notification for each completed file upload. NotifyWhenUploaded requests completion for all files written to the share up to the request time.
What are the per-volume size ranges for AWS Volume Gateway cached and stored volumes?
Cached: 1 GiB to 32 TiB. Stored: 1 GiB to 16 TiB. Each mode supports up to 32 volumes per gateway.