Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

This branch is even with id774:master

README

automaticruby

Name
 Automatic Ruby - Ruby framework for the general-purpose automatic processing

Syntax
$ automatic

Specify the start of any recipe
$ automatic -c <recipe>

To view the version number
$ automatic -v

Description
 This is a general-purpose automatic processing
ruby framework which can extend the functionality
by plug-ins.


============
Installation
============

[Stable]
$ gem install automatic
$ automatic scaffold
(Make ~/.automatic on your home directory.)
$ automatic -c ~/.automatic/config/example/feed2console.yml
(This process will be output my blog feed to your terminal.)


[Development]
$ git clone git://github.com/automaticruby/automaticruby.git
$ cd automaticruby
$ bundle install --path vendor/gems
$ bin/automatic scaffold
$ bin/automatic -c ~/.automatic/config/example/feed2console.yml
(The same as above.)


===========
Get Started
===========

 Create of any recipe (see below).
$ automatic -c your_recipe.yml

 -c option to run with the manual. If there is no problem
to automate, register it to cron.

 If you specify the-c argument of a simple file name, load
the file located in ~ /.automatic/config. If not exist,
load file as specified in the path.

 If ~/.automatic/db exists, plug-ins save the file to this
directory.

 If ~/.automatic/plugins exists, the plug-ins be loaded
 as well.


 This is the easy recipe to simply output blog's feed to
only console. It can be used to check the operation of
this software.
$ automatic -c ~/.automatic/config/example/feed2console.yml


=======
Recipes
=======

 Automatic Ruby parses configuration file that was written
in the form of YAML which including variety of information
of associated plug-ins.

 This YAML file is called "Recipe".

 You can use -c option for specify a file name.

[Example]
$ automatic -c your_recipe.yml


=====================
How to write a Recipe
=====================

 The Recipe has an implicit naming convention.

[Syntax]

plugins:
  - module: MODULE_NAME
    config:
      VARIABLES


 The following is a example.

plugins:
  - module: SubscriptionFeed
    config:
      feeds:
        - http://reretlet.tumblr.com/rss

  - module: StorePermalink
    config:
      db: tumblr.db

  - module: FilterImage

  - module: FilterTumblrResize

  - module: StoreFile
    config:
      path: /Users/yourname/Desktop/
      interval: 1

In the case of sample this recipe.
1. Subscribe to RSS feeds of Tumblr by SubScriptionFeed.
2. Save permalink to database by StorePermalink.
3. Specify the URL of the image by FilterImage and FilterTumblrResize.
4. Downloading image file of Tumblr by StoreFile.


Showing another example as follows.

plugins:
  - module: SubscriptionFeed
    config:
      feeds:
        - http://example.com/rss2
        - http://hogefuga.com/feed

  - module: FilterIgnore
    config:
      link:
        - hoge
        - fuga

  - module: StorePermalink
    config:
      db: permalink.db

  - module: PublishHatenaBookmark
    config:
      username: your_hatena_id
      password: your_password
      interval: 5

In the case of sample this recipe.
1. Subscribe to 2 RSS feeds by SubScriptionFeed.
2. Feed ignore the keyword by FilterIgnore.
3. Save permalink to database by StorePermalink.
4. Social Bookmarking by PublishHatenaBookmark.

(Hatena Bookmark is a most popular social bookmark service
in Japan.)


These have been realized by a combination of plug-ins
a series of processes.

In this way, by simply writing to the recipe and
configuration information for plug-ins, free processing can
be achieved by a combination of multiple plug-ins.

The possibility of infinite depending on the combination
of plug-ins can be realized.

All depending on whether you make a plug-in looks like!


=======
Plug-in
=======

Plug-ins from the plugins directory (see below) will be loaded.

If ~ /.automatic/plugins exists, is loaded into them as well.

Put your own plug-ins in your home directory.
(How to make the plug-in will be described later.)

You can automatically generate a sub-directory for your own plug-ins
with the following command.

$ automatic scaffold
(For a description of automatic subcommands will be described later.)


============================
Directory and file structure
============================
.
|
+-+ bin
| |
| +-- automatic
|   The main file for run.
|
+-- config
|
|   Describe the recipe information in the form of yaml.
|   They called in the argument -c option of Automatic Ruby.
|   If you write a new recipe, locate it in the config directory.
|
|
+-+ plugins
| |
| +-- subscription
| | Plug-ins to subscribe to feeds.
| |
| +-- customfeed
| | Plug-ins to generate a custom feed.
| |
| +-- filter
| | Plug-ins to filter the information.
| |
| +-- store
| | Plug-ins to store the information to internally.
| |
| +-- notify
| | Plug-ins to notify the information.
| |
| +-- publish
|   Plug-ins to send information to external.
|
| If you write a new plug-in, locate it in the plugins directory.
|
|
+-+ lib
| |
| +-- automatic.rb
| | To the definition of automatic module.
| |
| +-+ automatic
|   |
|   +-- environment.rb
|   | Read the environmental information.
|   |
|   +-- pipeline.rb
|   | To use an object called a pipeline
|   | sequentially processing plug-ins in recipes.
|   |
|   +-- feed_parser.rb
|   | To parse the feed.
|   |
|   +-- log.rb
|   | To output logs.
|   |
|   +-- recipe.rb
|     Read a recipes.
|
+-- db
|
| If ~/.automatic/db not exist, save db file here.
|
+-- assets
|
| There are some files that needed in a plug-ins.
| If ~/.automatic/assets exist, takes precedence it.
|
+-+ script
| |
| +-- build
|   Run Integration Test that carried out on CI.
|
+-- spec
|   Directory for unit tests (Use RSpec).
|
+-+ test
| |
| +-- fixtures
| | Fixtures for test.
| |
| +-- integration
|   Directory for the Integration Test.
|
+-+ vendor
| |
| +-- gems
|   Bundle installed gem packages.
|
+---+ doc
    |
    +-- COPYING
    | The license for this software.
    |
    +-- ChangeLog
    | Update history.
    |
    +-- PLUGINS
    | Documentation for plug-ins.
    |
    +-- README
      This document.



===========
Development
===========

Repository
https://github.com/automaticruby/automaticruby

Issues
https://github.com/automaticruby/automaticruby/issues

RubyForge
http://rubyforge.org/projects/automatic/

RubyGems.org
https://rubygems.org/gems/automatic

CI
http://jenkins.id774.net/jenkins/


===========
How to Join
===========

1. Fork the repository.
2. Write new plugin or existing code improvement.
3. Send pull request!

After that, we will give you the right to commit for a quick fix finely.


===========
Coding Rule
===========

Coding standards should be completed in 1 file.
Remove trailing spaces.
{} is recommended than do end (To avoid end end end,,).
RDoc Header is written by creator of the file.
Write tests with RSPec.
Aim at 100% coverage.


===================
Automatic::Pipeline
===================

 This framework sequentially analyze a plug-ins on yaml recipe.
At this time, pipeline passed instantance as an argument. And then,
pipeline backs again as the return value.

 Code is as follows.

pipeline = []
recipe.each_plugin {|plugin|
  mod = plugin.module
  load_plugin(mod)
  klass = Automatic::Plugin.const_get(mod)
  pipeline = klass.new(plugin.config, pipeline).run
}

 In this iteration, sequential processing can be realized.

 This mechanism called "Automatic::Pipeline".

 The contents of the pipeline is the array of the feeds.


=============================
Implementation of the plug-in
=============================

 Plug-in is loaded by the constructor of YAML.load
Class is recieved the array of hashes as the instance
variable of pipelines object.

Example implementation of the constructor that is recommended
is as follows.

def initialize(config, pipeline=[])
  @config = config
  @pipeline = pipeline
end

Run an instance method is called automatically. The return value
is on the instance variable @pipeline. And then, be handed over
to the next plug-in in the recipe.

@pipeline's data format should be combined. If the preceding
plug-in will return an RSS feed, you need to write handle the
RSS feed. If the one will return as HTML, you need to write
code to handle HTML.


=========
Unit Test
=========

Unit testing is performed in RSpec.

Test of the plug-in, pass the pipeline which will be the argument,
to check the return value.


============
Binding Test
============

To combine more than one recipe under the test/integration.
Put the recipe for the binding test.

Before you check-in to the repository, run following
command with ruby 1.9.
$ script/build

By this, all RSpec tests and binding tests will be conducted.
You can check-in to the repository after all test successed.
Otherwise, CI will fail.


======================
Continuous Integration
======================

CI is performed in Jenkins.
http://jenkins.id774.net/jenkins/


===========================
Description of the plug-ins
===========================

See doc/PLUGINS.



===========
SubCommands
===========

automatic has the auxiliary tool in subcommands.

[Syntax]
$ automatic
 Show help and list of sub-command.

$ automatic scaffold
 Make ~/.automatic directory in your home directory.
 This operation makes 'plugins', 'db', 'config', 'assets' directories.
 These takes precedence the origin directory.

$ automatic unscaffold
 Delete ~/.automatic directory.

$ automatic autodiscovery <url>
 Return the URL of the feed of target detected by auto discovery.
(ex.)
$ automatic autodiscovery http://blog.id774.net/post
["http://blog.id774.net/post/feed/", "http://blog.id774.net/post/comments/feed/"]

$ automatic opmlparser <opml path>
 Output the URLs to parse the OPML file.
(ex.)
$ automatic opmlparser opml.xml > feeds.txt

$ automatic feedparser <url>
 Return the contents to parse the feed.

$ automatic inspect <url>
 Return the URL of the feed of target detected by auto discovery.
 Return the contents to parse the first feed further.
 This inspects verify the target subscriptionable.

$ automatic log <level> <message>
 Output log messages in the form of Automatic Ruby.



==============
Update History
==============

 See doc/ChangeLog.


====
TODO
====

 Refer to the issues of github.
 https://github.com/automaticruby/automaticruby/issues


===========
Environment
===========

 After Ruby 1.9.

 Depend on the packages that described in Gemfile.


=====
Notes
=====

To avoid excessive scraping, use in the bounds of common sense.
This software licensed under the GNU GENERAL PUBLIC LICENSE, Version 3.0.


Something went wrong with that request. Please try again.