Takuhai downloads files from remote server by ssh/scp command.
After download, you can add any logic for each line.
※Takuhai means deliver.
- You want to download files and any edit or check in regularly. It's boiler template work!
- Public key authentication settings must be done.
Add this line to your application's Gemfile:
gem 'takuhai'
And then execute:
$ bundle
Or install it yourself as:
$ gem install takuhai
Fist, generate Takuhaifile template.
$ takuhai init
Second, edit your Takuhaifile file.
Third, execute. You get converted file from remote.
$ takuhai
or
$ takuhai collect
You can confirm help
$ takuhai help
$ takuhai h
You can confirm version
$ takuhai version
$ takuhai v
This file defines download ,convert and output spec.
Template have following contents.
You can start to edit from this template.
But if you want to write DSL-text from new file, you can.
You can confirm dsl syntax from this template file's comments.
# encoding: utf-8
# write your ssh login user name. this attribute is required.
# ex:user "app"
user "TODO: user"
# write your ssh login server name or server ip. this attribute is required.
# ex1:server "some_server"
# ex2:user "192.168.10.11"
server "TODO: server"
# write your targets. this attribute is required.
# ex1:targets "app.log-2011-10-*"
# ex2:targets "app.log-2011-10-01 app.log-2011-10-02 app.log-2011-10-05"
targets "TODO: targets"
# write your log directory. this attribute is required.
# ex:target_dir "/home/app/some_system/log/production"
target_dir "TODO: target_dir"
# write your output_base_name. this value is using in tarball file name and output file name. this attribute is required.
# ex:output_base_name "server_log"
output_base_name "output"
# write output file's extension. this attribute is required.
# ex:output_extension "tsv"
output_extension "txt"
# if you want to execute some logic for line data, edit following code.
# ex: if line have "ERROR", append !!ERROR!! to line String.
# block do |file, line|
# line = "!!ERROR!!" + line if line.include?("ERROR")
# line
# end
block do |file, line|
line
end
- You get files[file1.txt, file2.txt, file3.txt] from remote server.
- file1.txt's contents is following.
[user = test1]
[operation = search]
[password = test1pass]
- file2.txt's contents is following.
[user = test2]
[operation = search]
[password = test2pass]
- file3.txt's contents is following.
[user = test3]
[operation = search]
[password = test3pass]
- If file contain [password = some_password], you mask password.
- Before execute takuahi, public key authentication settings must be done.
- Remote user is app
- Remote server is 192.168.11.11
- Target directory is /home/app/some_product/log
- Output directory is ./masked_files
- Output base name is masked_files
- Output extension is txt
Create Template
$ takuhai init
Edit Takuhaifile
# encoding: utf-8
user "app"
server "192.168.11.11"
targets "file*txt"
# targets "file1.txt file2.txt file3.txt"
target_dir "/home/app/some_product/log"
output_base_name "masked_files"
output_extension "txt"
block do |file, line|
if line.match /\[password = (.*)\]/
line.gsub!($1, '*'*$1.size)
end
line
end
output file tree is ...
│ Takuhaifile
│ masked_files20131129222450.txt
│
└─masked_files20131129222450
file1.txt
file2.txt
file3.txt
masked_files.20131129222450.tar.gz
masked_files20131129222450.txt contents is ...
[user = test1]
[operation = search]
[password = *********]
[user = test2]
[operation = search]
[password = *********]
[user = test3]
[operation = search]
[password = *********]
- This tool have not good performance. So if target file size is huge, you must not use this gem.
-
If you want to use Takuhaifile-DSL snippet, you can use snippet/takuhai/*.sublime-snippet.
-
These files is generated by sublime_sunippetter gem.
- Fork it
- Create your feature branch (
git checkout -b my-new-feature
) - Commit your changes (
git commit -am 'Add some feature'
) - Push to the branch (
git push origin my-new-feature
) - Create new Pull Request