Cloudera CDH/CDP 및 Hadoop EcoSystem, Semantic IoT등의 개발/운영 기술을 정리합니다. gooper@gooper.com로 문의 주세요.
#!/bin/bash
kinit -k -t /var/lib/keytab/hadoop.keytab hadoop@GOOPER.COM
usage="Usage: del_diff.sh [days] [delete target path]"
if [[ $# != 2 ]]
then
echo $usage
exit 1
fi
path=${2}
echo "delete target path : " $path
now=$(date +%s)
echo "old file date point : " $now
hdfs dfs -ls $path | while read f; do
dir_date=`echo $f | awk '{print $6}'`
#180일
difference=$(( ( $now - $(date -d "$dir_date" +%s) ) / (24 * 60 * 60) ))
if [ $difference -gt $1 ]; then
if [[ `echo $f | awk '[ print $8 }'` == *$path* ]]; then
hdfs dfs -rm `echo $f| awk '{ print $8 }'`;
else
echo 'else - target path is [' `echo $f| awk '{ print $8 }'` '] not contain this path [' $path ']'
fi
fi
done