BEELINE COMMAND LINE IN HIVE

Today, I will discuss about the beeline command line which we use to call the SQL Query through Linux. But what if same thing needs to be called through Shell Scripting. First of all, we need to call Sql Query through Beeline command line inside shell Scripting using below command. beeline -u “jdbc:hive2://localhost:10000/default;principal=hive/localhost“ -n “username” -p “password” –hivevar var1=$col1_hive –hivevar var2=$schema_name –hivevar var3=$table_name –hivevar var4=$col1_value -f sql_script.sql > text.log Here $col1_hive is the column name of a table. $table_name is the table name. $schema_name is the Schema Name where that…

Remove the extra delimiters in the file using shell script

Today, i will discuss about the “How to remove extra delimiters in a file” using shell script. Below is the content of the file. a|b||c d||e|||f Post execution of shell script, the output will be a|b|c d|e|f Below is the script which is created to remove extra delimiter in a file. #file is assigned to variable INPUT . Here we used two files text1.txt and text2.txt to manipulate the source file data. INPUT=”Path of the file”/file.txt #Truncating text1.txt. Make sure this file exists. >text1.txt #Reading character by character using while…

Shell script to run for x hours using while Loop

Today i will discuss about the shell script code snippet which will run for x hours. Below is the code snippet which will help you to run code for x hours. ## Here I am using 10 seconds , in order to run for 1 hours, mention the same in seconds which is 3600. $vartime value will be set to 3600 initially. vartime=$((SECONDS+10)) echo $vartime ## $SECONDS tells you the time period for which script is running.Initially it will be 0 seconds when script starts. So, 0 will be compared…

Shell script to check files changed in last 24 hours

Today , i will discuss about the how to get all files which are changed in last  24 hours in shell script. Below  is the code for the same. ## STEP-1 first of all get the yesterday date using below command. DATE=`date -d “yesterday” ‘+%Y%m%d%H%M’` echo $DATE ## STEP-2 Below command will create file date.txt as per yesterday timings. touch -t $DATE /opt/A1/date.txt ## STEP-3 Below are the paths where we need to check the changed files.Here I have considered four directories. FILES_PATH_A=”/opt/A” FILES_PATH_B=”/opt/B” FILES_PATH_C=”/opt/C” FILES_PATH_D=”/opt/D” ## STEP-4 Using find…

Shell Script to Merge multiple files into one file and Archive the source files.

Today i will discuss about the shell script which will merge the multiple files into one file  and post merging , archive the source files. Below is the code for the same. Here, i am passing four arguments  namely source file location,Archive file location,merged file name pattern,source file name pattern. Using date_1 variable which converts  the date into format YYYYMMDD_HHMMSS. Using “cd” command to go source file location. Count number of files in that location. If count is equal to 1 , that means there is no need to merge.…