Reading from Amazon S3 using NiFi

Reading from Amazon S3 using NiFi

1) First create a bucket on Amazon S3 and create public and private keys from IAM in AWS

2) Proper permission should be provided so that users with the public and private keys can access the bucket

3) Use some S3 client tool to test that the files are accessible.

4) Create the dataflow on Nifi using ListS3 , FetchS3Object and PutS3 object as shown in the diagrams below.

5) Setting of ListS3 is listed below. S3-kannadiga is bucket name in US-East region. The access key and secret key is entered in this processor. 

6) Setting of Fetch S3 is given below. $(s3.bucket) is the setting to read from List S3 processor.

 7) The setting of PutFile is given below.