smoke is a Ruby based DSL that allows you to take data from YQL, RSS / Atom (and more, if you think of a useful source). This "data" can then be re-represented, sorted and filtered. You can collect data from a multiude of sources, sort them on a common property and return a plain old ruby object or json (You could add in something to output XML too)
- Presentation from Melbourne #roro
- Early screencast to get developer / peer feedback
The concept comes from using Yahoo Pipes to make little mash ups, get a list of tv shows for my torrent client, compile a recipe book or make tools to give me a list of albums that artists in my music library are about to be release.
- Test everything you do
- Add a way to output (XML, anyone?)
- Examples of queries you'd like to be able to do (email / github message them to me)
# This will use yahoo search to get an array of search results about Ruby
Smoke.yql(:ruby) do
select :all
from "search.web"
where :query, "ruby"
discard :title, /tuesday/i
end
Smoke.yql(:python) do
select :all
from "search.web"
where :query, "python"
end
Smoke.join(:ruby, :python)
or even
Smoke.join(:python, :ruby) do
emit do
sort :title
rename :shit_name => :title
end
end
Integrity is running for smoke
- YQL Definitions
- Consider variable injection (eg, allow to query sending a username within some aspect of the request)
- Implement a disaptch / request method that will actually thread property (event-machine)
- Checkout experimental fakeweb version to stub out the yql specs
- Allow for sources to explicitly set the content type being returned for those stupid content providers
- YQL w/oAuth
- YQL Subqueries?
Copyright (c) 2009 Ben Schwarz. See LICENSE for details.