Today, I will discuss about Athena APIs which can be used in automation using shell scripting to fetch the Table data to perform validations like count, date, sum (aggregate functions), duplicate .I will explain the APIs using count validation in Athena. Get the Total records Details of the Athena tables using Athena APIs.STEP 1 : Use the below Command to get the query execution id for count Query of particular Table.API Used : start-query-executionaws athena start-query-execution –query-string “SELECT count(*) FROM $S3-Schema.$S3-Table” | grep [a-z] |awk -F “:” ‘{print $2}’|sed -e…
Tag: Automation in AWS
Automate the existence of S3 files through shell scripting
Today, I will discuss about “How to automate the existence of files in S3 bucket through shell script”. Here , i will share pseudo code for the same in the form of steps.* create one config file which we will have below details with pipe delimiter.S3 bucket Name|project_folder_name|Relative path of sub folderTechie-1|ABC|prj1/UC1/ #### Here techie-1 is the bucket Name, the immediate folder to this bucket is ABC and project Use-case folder is prj1/UC1/ ####* This[config] file will have list of Use-case folder names for which you need to test the…