Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add vidscraper-cmd. #7

Closed
wants to merge 2 commits into from
Closed

Add vidscraper-cmd. #7

wants to merge 2 commits into from

Conversation

willkg
Copy link
Contributor

@willkg willkg commented Apr 3, 2012

This adds the beginnings of a vidscraper-cmd. It supports auto-scraping
videos, but nothing else, yet. However, the code is written such that
adding support for other actions is easy to do.

The video subcommand supports specifying fields to be returned and also
api keys (though the latter is untested).

It returns data in JSON format--which required adding items() and
to_json() methods to the Video class.

Also adds a new topic in the documentation.

Closes #6.

@paulswartz
Copy link
Contributor

  • can't be automatically merged.
  • tests? At the very least, the new video methods should be tested, but it shouldn't be hard to test the CommandHandler either.
  • I don't see anything new in the documentation.

# fetchvideo -> auto_scrape(url, fields, api_keys)


class CommandHandler(object):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For clarity, perhaps VidscraperCommandHandler?

@paulswartz
Copy link
Contributor

Comments aside, I'm excited that other people find Vidscraper useful; thanks for the request!

@willkg
Copy link
Contributor Author

willkg commented Apr 3, 2012

Cool. Sounds like the general structure is good. I'll fix up the issues you mentioned and clean some other things up (and include the topic file I wrote but forgot to add to the commit).

@paulswartz
Copy link
Contributor

Yes, the structure looks good. I like that you've made it easy to extend with new commands in the future.

@melinath
Copy link
Contributor

melinath commented Apr 3, 2012

Looks good. Only thing that's bothering me atm is the use of mem as a filler variable where a more verbose name would be more readable - for example, for mem in self.fields could be rewritten as for field in self.fields.

@willkg
Copy link
Contributor Author

willkg commented Apr 5, 2012

Rebased from master so it should apply cleanly now.

Addressed other issues from your comments.

@melinath
Copy link
Contributor

melinath commented Apr 5, 2012

The branch needs to be rebased onto develop, not master, unfortunately. :-)

@willkg
Copy link
Contributor Author

willkg commented Apr 5, 2012

Whoops. This project is different than all the other ones I work on and I forgot.

@willkg
Copy link
Contributor Author

willkg commented Apr 5, 2012

Rebased against develop and re-pushed.

@paulswartz
Copy link
Contributor

Are you sure it's rebased against our develop? Github says it still can't be merged, and the list of commits (https://github.com/pculture/vidscraper/pull/7/commits) includes a bunch that have already happened.


class VideoTestCase(unittest.TestCase):
def test_items(self):
video = auto_scrape("http://www.youtube.com/watch?v=J_DV9b0x7v4")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yay tests! Doing it this way, though, requires that Vidscraper actually go to YouTube and grab some data, and you aren't actually looking at the data. Can you test this by just making a plain Video instance?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Show me an example of how to do that with your system and I'll change the code.

I'm really running out of time I can spend on the nit-picky stuff.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

vidscraper.videos.Video(url) won't actually load the URL.

This adds the beginnings of a vidscraper-cmd. It supports auto-scraping
videos, but nothing else, yet. However, the code is written such that
adding support for other actions is easy to do.

The video subcommand supports specifying fields to be returned and also
api keys (though the latter is untested).

It returns data in JSON format--which required adding ``items()`` and
``to_json()`` methods to the Video class.

Also adds a new topic in the documentation.

Closes #6.
* adds missing files
* improves documentation
* adds tests for new Video methods
* handles datetime.datetime serialization issues with json.dumps.
@willkg
Copy link
Contributor Author

willkg commented Apr 5, 2012

That should fix the rebasing issue.

I'm kind of out of time I can spend on upstreaming this. So if it's not good as is, I'll just maintain my fork with this and future things I want to do. Then you can cherry-pick whatever is interesting to you.

@paulswartz
Copy link
Contributor

Still not mergeable, but I've rebased it and filed #8 with the changes.

@paulswartz paulswartz closed this Apr 5, 2012
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants