Today, I will discuss about ” How to automate the process where in you can check entire row duplicate record in hive”. As I have mentioned in all Automation blogs, I will share the pseudo code.STEP1: In hive , use “desc table_name” , this command will give you column names along with datatype and data length. Store the output of this command in a file , say HIVE_TABLE_DDL.txt STEP2 : Read the file HIVE_TABLE_DDL.txt using “cat” command. cat HIVE_TABLE_DDL.txt | awk ‘{print $1}’ ORS=’,’ | sed ‘s/,$//’ * awk'{print $1}’…
Tag: Accelerators
shell script to check whether logs have errors or not
Today, I will discuss about “how to check logs files at regular interval and identify if any errors are there or not and if yes, then send email to the concerned team members. As mentioned in my previous blogs , I will be sharing pseudo code only. STEP1 : Create a parameterized file which has all the details of log file names. For example, consider filename is (logs.config) and below is the content of the fileUSECASE_NAME|SERVER_LOGNAME_PATTERNRECHARGE_USSD|rechargeUssdMode STEP2 : Create the shell script in such a way that when you trigger…
check whether process is running or not
Today, I will discuss about existence of a process in the linux server through shell script. I will share the pseudo code for the same in the form of steps.* Create a shell script which runs in infinite loop using while 1==1 * Inside this while loop, check the existence of a process on every 15th minute which can be achieved by dividing the minutes part of current timestamp by 15 using MOD function. If it returns 0 , go to next statements else come out of If statement. *…
Automate the existence of S3 files through shell scripting
Today, I will discuss about “How to automate the existence of files in S3 bucket through shell script”. Here , i will share pseudo code for the same in the form of steps.* create one config file which we will have below details with pipe delimiter.S3 bucket Name|project_folder_name|Relative path of sub folderTechie-1|ABC|prj1/UC1/ #### Here techie-1 is the bucket Name, the immediate folder to this bucket is ABC and project Use-case folder is prj1/UC1/ ####* This[config] file will have list of Use-case folder names for which you need to test the…