Easy ReviewDOP-C02
SDLC Automation11

FSx for Lustre

Use high-performance file storage for compute workloads and linked S3 datasets.

Este conteúdo de estudo ainda não foi traduzido, então aparece em inglês. Ajude a traduzir

Parallel storage and S3 integration

Amazon FSx for Lustre provides a managed high-performance file system for workloads such as machine learning and high-performance computing. Linux applications can use its POSIXA family of standards for operating system interfaces used by applications. file interface, a standard way to access files.

How it works

  1. Choose scratch for temporary processing or persistent for longer-lived workloads. Scratch does not replicate data; persistent replicates data and replaces failed servers.
  2. Select a storage class. SSD provides low latency for the full dataset. Intelligent-Tiering moves data between storage tiers and can use an SSD read cache. HDD provides single-digit millisecond latency.
  3. Link an S3 data repository. Its objects appear as files, and a data repository associationA link between a file system directory and an S3 bucket or prefix. can import their metadataInformation about a file, such as its ownership, permissions and timestamps., such as file names and attributes.
  4. Mount the file system from EC2 using the Lustre client, or use the FSx for Lustre CSI driver with EKS. Applications process the files.
  5. To return changes to S3, use automatic export or an on-demand export task. The console enables automatic export of new, changed and deleted files by default when adding a repository. Automatic export sends changes asynchronously after the application finishes modifying a file.

Exam cues

  • If the question says a temporary compute job can recreate its input, think scratch, because its data is not replicated.
  • If the question says long-running processing must survive a file server failure, think persistent, because FSx replaces failed servers and replicates data.
  • If the question says process S3 objects through a Linux file interface, think FSx for Lustre with an S3 association, because objects are presented as files.
  • If the question says results must reach S3, think automatic export or an export task, because linking alone does not ensure export.
  • If the question says an S3-linked file system must read a large dataset, think lazy loading, because file content loads from S3 on first access.
  • If the question says an FSx for Lustre file system needs backups, think persistent without an S3 link, because scratch and S3-linked file systems do not support backups.

Remember

  • Persistent SSD and HDD replicate within one Availability Zone; Intelligent-Tiering replicates across multiple Availability Zones.
  • Lustre 2.10 and Scratch 1 do not support data repository associations or automatic export.
  • New console-created repository associations enable automatic export by default; check the settings when using another creation method.
  • File content loads from S3 on first access; later reads come from the file system.
  • Backups are supported only on persistent file systems that are not linked to S3.

Trap: A linked S3 bucket is not an immediate, conflict-free mirror. Configure export and coordinate concurrent writes to the same file in S3 and the file system.

Transfer between S3 and Lustre
12S3 repositoryFSx for LustreSame repository
  1. Import metadata for linked S3 objects into the file system.

  2. Export file changes back to the linked repository using configured automatic export or a task.

Da documentação da AWS (17)
  • FSx for Lustre provides a managed high-performance Lustre file system for workloads such as machine learning, high-performance computing and video processing.
  • FSx for Lustre is POSIX-compliant and provides a native file system interface for Linux applications.
  • EC2 instances access FSx for Lustre using the Lustre client; EKS can use the FSx for Lustre CSI driver.
  • Scratch file systems suit temporary processing; data is not replicated and does not persist if a file server fails.
  • Persistent file systems are designed for longer-term workloads, replicate data and replace failed file servers.
  • Persistent SSD and HDD data is replicated within the file system's Availability Zone; Intelligent-Tiering data is replicated across multiple Availability Zones.
  • SSD provides low-latency access to the full dataset, Intelligent-Tiering automatically tiers data and offers an optional SSD read cache, and HDD provides single-digit millisecond latency.
  • A linked S3 data repository presents S3 objects as files on FSx for Lustre.
  • Data repository associations and automatic export are unavailable on Lustre 2.10 or Scratch 1 file systems.
  • A data repository association can import metadata at creation, through an import task, or automatically as S3 objects change.
  • Changes can be exported to a linked S3 repository by automatic export or an on-demand export task.
  • FSx for Lustre automatic export sends changes asynchronously after a file is modified; adding a data repository through the console enables automatic export of new, changed and deleted files by default.
  • FSx for Lustre does not prevent conflicting writes to the same file in both the file system and S3.
  • With a linked S3 repository, FSx for Lustre loads a file content from S3 the first time an application accesses it; later reads come from the file system.
  • FSx for Lustre Intelligent-Tiering file systems cannot link to an S3 data repository.
  • FSx for Lustre backups are supported only on persistent file systems not linked to S3; scratch and S3-linked file systems have no backups.
  • On-premises clients can mount FSx for Lustre over Direct Connect or Site-to-Site VPN.

S3 association settings and exports

An FSx for Lustre data repository associationA link between a file system directory and an S3 bucket or prefix. links a file system directory to an S3 bucket or prefix. Its automatic import and export defaults depend on how the repository is added.

How it works

  1. Link a file system directory to an S3 bucket or prefix with a data repository association. A file system supports up to eight associations.
  2. When adding a repository, check the automatic import and export settings. The console enables new, changed and deleted events by default; the CLI and API disable automatic import and export by default.
  3. Account for the export method: automatic export and export data repository tasks cannot run at the same time.

Exam cues

  • If the question says an S3 association was created through the API, think disabled automatic import and export by default, because the API and console have different defaults.
  • If the question says an export task must run while automatic export is running, think they cannot run at the same time, because FSx for Lustre does not allow both to run concurrently.
  • If the question says link a file system directory to an S3 prefix, think data repository association, because it connects the directory to the bucket or prefix.

Remember

  • Up to eight data repository associations per file system.
  • The console enables automatic import and export for new, changed and deleted events by default; the CLI and API disable them by default.

Trap: An export task cannot run at the same time as automatic export.

Da documentação da AWS (3)
  • Automatic import and export of new, changed and deleted files are enabled by default when adding a repository in the console but disabled by default through the CLI or API.
  • Automatic export and export data repository tasks cannot run at the same time.
  • A data repository association links a file system directory to an S3 bucket or prefix, with up to eight associations per file system.

Flashcards

toque para virar
Resposta

·

Todos os 11 flashcards em lista

Which file system fits HPC or machine learning workloads that process S3 data as files?

FSx for Lustre. Link an S3 repository and choose scratch for temporary work or persistent storage for longer-lived workloads.

A short-lived FSx for Lustre processing job can recreate its data after a server failure. Which deployment fits?

Scratch. It is designed for temporary processing and does not replicate data.

A long-running workload needs an FSx for Lustre file system that replaces failed servers. Which deployment fits?

Persistent. Its data is replicated and failed file servers are replaced.

Does every persistent FSx for Lustre storage class replicate data only within one Availability Zone?

No. SSD and HDD replicate within the same Availability Zone, while Intelligent-Tiering replicates across multiple Availability Zones.

Does linking an S3 bucket to FSx for Lustre guarantee that file changes immediately appear in S3?

No. Automatic export is asynchronous. The console enables it by default for new repository associations; check its configuration or run an export task.

How should applications handle concurrent writes to the same data in S3 and an FSx for Lustre file system?

Coordinate writes at the application level. FSx for Lustre does not prevent conflicting writes.

An HPC job starts on FSx for Lustre linked to a 200 TB S3 bucket. Does the file system copy all object data before the job can read files?

No. Metadata is imported, and file content loads from S3 on first access. Later reads come from the file system.

A team needs FSx backups for a scratch FSx for Lustre file system used by a nightly job. Is that supported?

No. Backups work only on persistent file systems not linked to S3. Keep durable data in S3 or use a persistent deployment.

What are the default automatic import and export settings for an FSx for Lustre S3 association created through the console versus the API?

The console enables new, changed and deleted events by default; the API disables automatic import and export by default.

Can an FSx for Lustre export task run while automatic export is running?

No. Automatic export and export tasks cannot run at the same time.

What links an FSx for Lustre directory to an S3 bucket or prefix, and how many links are supported?

A data repository association. A file system can have up to eight.

Você concluiu todos os tópicos da DOP-C02!

Seu badge ficou dourado. Próximo passo: agendar a prova na AWS. Depois de passar, adicione-se ao Quem passou.

Compartilhar

Conte que você concluiu esta certificação

Concluí todos os tópicos da AWS Certified DevOps Engineer - Professional (DOP-C02) no Easy Review. Venha estudar também!