AWS Glue is a serverless data integration service for discovering, preparing, moving and integrating data from many sources.
Glue DataBrew
AWS Glue DataBrew is point-and-click data preparation: profile a dataset, spot quality problems and apply ready-made transformations, with no code and no clusters.
How it works
- Connect a dataset, for example in Amazon S3.
- Profile it and review suggested data quality fixes.
- Build a recipe from ready-made transformations and run it as a job.
Exam cues
- If the question says visual, no-code data cleaning for analytics or ML, think AWS Glue DataBrew.
Remember
- Serverless, over 250 transformations.
Trap: DataBrew prepares data; AWS Glue ETL jobs are the code-based option for large pipelines.
Data integration and catalog
AWS Glue prepares data for analytics. It discovers data in many sources, keeps a central catalog of it, and runs ETL jobs that clean and move data into a data lake, without servers to manage.
How it works
- Connect Glue to data sources; it records their structure in the Data Catalog.
- Build ETL pipelines, visually or in code, to transform data and load it into a data lake.
- Query the cataloged data with Amazon Athena, Amazon EMR or Amazon Redshift Spectrum.
| Analytics need | Service |
|---|---|
| Prepare and catalog data (ETL) | AWS Glue |
| Query data in S3 with SQL | Amazon Athena |
| Stream data in real time | Amazon Kinesis |
| Dashboards and BI | Amazon Quick (QuickSight) |
| Data warehouse | Amazon Redshift |
Exam cues
- If the question says ETL, data catalog or data preparation, think AWS Glue.
Remember
- Serverless; 70+ data sources; central Data Catalog.
Trap: Glue prepares and catalogs data; querying it with SQL is Amazon Athena.
Da documentação da AWS (5)
- AWS Glue is a serverless data integration service for discovering, preparing, moving and integrating data from multiple sources for analytics, machine learning and application development.
- With AWS Glue you connect to more than 70 data sources and manage data in a centralized data catalog that Amazon Athena, Amazon EMR and Amazon Redshift Spectrum can query.
- AWS Glue lets you visually create, run and monitor extract, transform and load (ETL) pipelines that load data into data lakes.
- Glue job bookmarks persist what a previous ETL job run already processed, so scheduled reruns process only new data instead of reprocessing old data.
- AWS Glue streaming ETL jobs run continuously, read from Kinesis Data Streams, Apache Kafka or Amazon MSK, and load cleaned data into S3 or JDBC stores.
Flashcards
Nada para revisar aqui agora. Escolha outros tópicos ou outro filtro.
·
Todos os 7 flashcards em lista
A business analyst must clean and normalize a messy dataset for ML training without writing code. Which AWS service fits?
AWS Glue DataBrew, a visual, serverless data preparation tool.
Does AWS Glue DataBrew need clusters to be provisioned?
No. DataBrew is serverless.
A company needs serverless ETL jobs to clean data from several sources and load it into an S3 data lake. Which AWS service fits?
AWS Glue, a serverless data integration and ETL service.
What does the AWS Glue Data Catalog help an analytics team do?
It lets the team manage data in a centralized catalog that analytics services can query.
Does AWS Glue require provisioning servers for ETL jobs?
No. AWS Glue is serverless.
What can AWS Glue do to prepare data for a data lake?
AWS Glue can create and run extract, transform and load pipelines that load data into data lakes.
Which AWS service helps discover and prepare data from multiple sources for machine learning?
AWS Glue is a serverless data integration service for discovering and preparing data for machine learning and analytics.