The correct answer is: Set up an AWS Glue crawler to catalog the data and configure an AWS Glue ETL job to convert the CSV files to Parquet, writing the output to the target S3 bucket.

AWS Glue is a serverless data integration service that requires minimal development effort. You can use a Glue crawler to automatically detect the schema of incoming CSV files, and an ETL job can be visually designed or written with minimal code to convert the files into Parquet format. Glue handles job orchestration, parallel processing, and integration with S3, making it ideal for this use case.

Why the other options are incorrect:

Creating an Amazon EMR cluster and writing a Spark application provides fine-grained control and performance, but it involves substantial setup and maintenance overhead. It is not the most development-efficient option for this relatively straightforward transformation task.

Using AWS Batch with a Bash script to transform the data requires writing and maintaining custom scripts, managing job definitions, and orchestrating the process manually. This adds complexity and operational burden, which contradicts the requirement for minimal development effort.

Developing an AWS Lambda function to process and convert the files is feasible for small datasets, but Lambda has runtime limitations and memory constraints that can be problematic with large or frequent file transformations. It also requires more custom development compared to Glues native ETL capabilities.

Why this question is important: It evaluates your understanding of AWSs managed data processing services and your ability to choose solutions based on development effort and operational simplicityimportant considerations in real-world, high-volume data pipelines.

Tip: When the goal is to transform structured data at scale with minimal coding or server management, AWS Glue is often the best-fit solution, especially when converting between formats like CSV and Parquet.

Full AWS Practitioner Certification Question