The correct answer is Create an AWS Glue extract, transform, and load (ETL) job to convert the .csv files to Parquet format and place the output files into an S3 bucket. Create an AWS Lambda function for each S3 PUT event to invoke the ETL job.

This solution uses AWS Glue, a fully managed ETL service designed for large-scale data processing and format conversion. Glue can easily handle 1 GB file sizes and supports CSV-to-Parquet conversion natively. Triggering the ETL job with a Lambda function based on S3 PUT events ensures automation without requiring constant polling or infrastructure management. This setup minimizes operational overhead and takes advantage of AWSs serverless architecture.

Create an AWS Lambda function to download the .csv files, convert the files to Parquet format, and place the output files in an S3 bucket. Invoke the Lambda function for each S3 PUT event is incorrect because Lambda has limitations on memory, storage, and execution time. Processing 1 GB files inside a Lambda function is not practical and could result in timeouts or throttling.

Create an Apache Spark job to read the .csv files, convert the files to Parquet format, and place the output files in an S3 bucket. Create an AWS Lambda function for each S3 PUT event to invoke the Spark job could work but requires managing Spark infrastructure (e.g., Amazon EMR or AWS Glue Studio with Spark), which increases operational complexity. Glue ETL already uses Spark under the hood with far less overhead to manage.

Create an AWS Glue table and an AWS Glue crawler for the S3 bucket where the application places the .csv files. Schedule an AWS Lambda function to periodically use Amazon Athena to query the AWS Glue table, convert the query results into Parquet format, and place the output files into an S3 bucket is an unnecessarily complex and inefficient workaround. Athena is meant for ad hoc querying, not high-throughput file format transformation.

Why this question matters: It tests your ability to select appropriate serverless data transformation services based on data volume, automation, and operational efficiency. Knowing the limits of Lambda and the strengths of Glue for data conversion is essential in modern AWS data pipelines.

Tip: When dealing with large file transformations in S3, consider AWS Glue ETL as the first optionespecially when the files are structured like CSV and need conversion to columnar formats like Parquet.

Full AWS Practitioner Certification Question