New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parse Score Breakdown from Twitter #722

Closed
wolfgang42 opened this Issue Aug 23, 2013 · 12 comments

Comments

Projects
None yet
5 participants
@wolfgang42
Copy link

wolfgang42 commented Aug 23, 2013

The @frcfms twitter feed provides a breakdown of the scores for each match (foul points, auto score, teleop disk points, etc). Is it possible to add this information to the API? I don't have a twitter account, so I can no longer access this information via the Twitter API.

@gregmarra

This comment has been minimized.

Copy link
Contributor

gregmarra commented Aug 24, 2013

We'd need to add Twitter Datafeeds, which we don't currently have, then probably extend the Match model to store a JSON blob of the breakdown, since it changes year to year. Once we do that, we could expose it via the API.

What do people use the score breakdowns for?

@wolfgang42 , is it easier for you to get a Twitter account than wait for us to maybe eventually build this? :)

@brandondean

This comment has been minimized.

Copy link
Member

brandondean commented Aug 24, 2013

If I remember correctly one of the issues with using Twitter as another source of match information is that they don't tweet every single match. Although, I suppose it's better to have a few matches with this information rather than none at all I don't think it should be high priority because it won't benefit every match.

I can't seem to find the email thread where this got discussed a while back. Maybe I'm just imagining things.

@wolfgang42

This comment has been minimized.

Copy link
Author

wolfgang42 commented Aug 24, 2013

@gregmarra: I've sent an email to FIRST asking them to provide an alternative way of accessing the data. We'll see what happens.
I was planning to use the score breakdowns to provide a scouting tool: by looking at the average number of disk points for all the matches each team was in, for instance, you can get an approximate rank for how good each team is at scoring. Obviously this isn't perfect, but it does seem to work quite well. Unfortunately, twitter now requires authentication to use their API, which means that I can't access the feed anymore. I'm reluctant to get an account, though I could probably get my team to set one up if I really needed to.

@brandondean: Matches can get tweeted twice (when a rematch occurs), which does complicate things, but I've never encountered anything where the match isn't tweeted at all.

@fangeugene

This comment has been minimized.

Copy link
Member

fangeugene commented Aug 25, 2013

You can only get the last 3500 tweets from twitter. Also, many events don't Tweet properly. Adding score breakdown has too many edge cases IMO.

@fangeugene

This comment has been minimized.

Copy link
Member

fangeugene commented Aug 25, 2013

On another note, we do have a Twitter datafeed that we can manually run to get results for offseason events. We only look at the final scores though.

@wolfgang42

This comment has been minimized.

Copy link
Author

wolfgang42 commented Aug 25, 2013

@fangeugene What do you mean by "many events don't tweet properly"? I've never encountered a misformatted tweet, if that's what you mean.

I'm not necessarily requesting that the old matches be updated with the data--though it would be convenient. Rather, I'm suggesting that future matches have this information. Old matches would be a bonus. (I've also asked FIRST if they can make the old data available outside of the Twitter API.)

@synth3tk

This comment has been minimized.

Copy link
Member

synth3tk commented Aug 25, 2013

Some events have spotty internet, or the FMS system just doesn't work. So
sometimes we may not get any tweets at all.

On Sun, Aug 25, 2013 at 12:01 AM, Wolfgang notifications@github.com wrote:

@fangeugene https://github.com/fangeugene What do you mean by "many
events don't tweet properly"? I've never encountered a misformatted tweet,
if that's what you mean.

I'm not necessarily requesting that the old matches be updated with the
data--though it would be convenient. Rather, I'm suggesting that future
matches have this information. Old matches would be a bonus. (I've also
asked FIRST if they can make the old data available outside of the Twitter
API.)


Reply to this email directly or view it on GitHubhttps://github.com/gregmarra/the-blue-alliance/issues/722#issuecomment-23221514
.

@brandondean brandondean removed the API label Feb 24, 2014

@wolfgang42

This comment has been minimized.

Copy link
Author

wolfgang42 commented Dec 3, 2014

I contacted FIRST, and they said: "Unfortunately, we don’t yet have the resources available to make these changes at this time. However, we will definitely consider your input and suggestions for the @frcfms twitter feed once it becomes active again." So, that's out the window for now.

I got a Twitter account and set up a program on my server to pull the data. The code is at https://github.com/wolfgang42/frcfms-python, and you can download a CSV file from http://www.linestarve.com/frc/fms/.

I could have sworn that I also screen-scraped Twitter at some point and got all of the historical tweets, but I can't for the life of me find them anymore. I'll have to look in my backups.

At some point in the future, I'll see if I can set up some nice parsing and correlate tweets with events. In the meantime, I'm putting the links above in for anyone who may find this issue in a web search.

(This response has been delayed rather badly; I forgot about this issue.)

@fangeugene

This comment has been minimized.

Copy link
Member

fangeugene commented Dec 3, 2014

With FIRST's new upcoming API, this may no longer be relevant for future events.
It may still be useful to add in match breakdown for past events though.

@wolfgang42

This comment has been minimized.

Copy link
Author

wolfgang42 commented Dec 4, 2014

@fangeugene: New API? Are you referring to this? It looks like it doesn't provide as detailed of a breakdown (for instance, it doesn't seem to split teleop from autonomous). In addition, it requires a private API key, though I have no idea why.

@gregmarra

This comment has been minimized.

Copy link
Contributor

gregmarra commented Apr 26, 2015

The FRC API will be the new way we get data, FRCFMS may not exist in the future.

@gregmarra gregmarra closed this Apr 26, 2015

@fangeugene

This comment has been minimized.

Copy link
Member

fangeugene commented Apr 26, 2015

Does not exist anymore*

On Sun, Apr 26, 2015 at 7:53 PM, Greg Marra notifications@github.com
wrote:

Closed #722
#722.


Reply to this email directly or view it on GitHub
#722 (comment)
.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment