In this post we will see how to create a data pipeline in AWS which picks data from S3 csv file and inserts records in RDS MySQL table. I am using below csv file which contains a list of passengers. CSV Data stored in the file Passenger.csv Upload Passenger.csv file to S3 bucket using AWS ClI In below screenshot I am connecting the RDS MySQL instance I have created in AWS and the definition of the table that I have created in the database testdb. Once we have uploaded the csv file we will create the data pipeline. There are 2 ways to create the pipeline. Using "Import Definition" option under AWS console. We can use import definition option while creating the new pipeline. This would need a json file which contains the definition of the pipeline in the json format. You can use my Github link below to download the JSON definition: JSON Definition to create the Data Pipeline Using "Edit Architect" ...
AWS, Database, ETL, Integration, Data Reporting & Automation Expert
Really well done, lots of good tips. Visit this link for more: Digital Marketing Company in Medavakkam
ReplyDelete