Skip to content

Commit

Permalink
added compute for sitelinks and upload
Browse files Browse the repository at this point in the history
  • Loading branch information
athalhammer committed Feb 13, 2024
1 parent f9d6c84 commit db086ef
Showing 1 changed file with 7 additions and 0 deletions.
7 changes: 7 additions & 0 deletions rpi/raspberry-danker.sh
Original file line number Diff line number Diff line change
Expand Up @@ -36,3 +36,10 @@ aws s3 cp "$INDEX_FILE" s3://"$S3_BUCKET"/ --grants read=uri=http://acs.amazonaw
aws s3 cp "$filename.rank.bz2" s3://"$S3_BUCKET"/ --grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers
aws s3 cp "$filename.stats.txt" s3://"$S3_BUCKET"/ --grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers
rm "$filename"

# Prepare sitelinks
filename="${filename%.*}".sitelinks.count
sort -k1,1 ./*.site.links | cut -f 1 | uniq -c | awk '{print $2 "\t" $1}' > "$filename"
bzip2 "$filename"
aws s3 cp "$filename".bz2 s3://"$S3_BUCKET"/ --grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers
rm ./*.site.links

0 comments on commit db086ef

Please sign in to comment.