Permalink
Browse files

Classifier works with ~65% accuracy in 3-fold validation

  • Loading branch information...
1 parent ad860ce commit 6124344e628dd6b7fb986771f27b449cc692575f @pranjalv123 pranjalv123 committed Oct 31, 2011
Showing 2,005 changed files with 64,837 additions and 27 deletions.
View
@@ -0,0 +1 @@
+*.pyc
View
@@ -1,4 +1,6 @@
import random
+import data
+from numpy import *
"""
A classifier is initialized with a training set of feature vectors
@@ -22,14 +24,50 @@ def classify(self, point):
class BayesClassifier:
def __init__(self, trainingset) :
- self.cl = {}
- for row in trainingset.T:
- if row[-1] in self.cl:
- v, c = cl[row[-1]]
- self.cl[row[-1]] = (v + row[:-1], c + sum(row[:-1]))
+ self.classes = {}
+ self.lengths = {}
+ for row in trainingset.asMatrix().T:
+ cls = row[-1]
+ pt = row[:-1]
+ if cls in self.classes:
+ self.classes[cls] += pt - ones(len(pt))
+
else:
- self.cl[row[-1]] = (row[:-1], sum(row[:-1]))
+ self.classes[cls] = pt
+
+ for cls in self.classes:
+ self.lengths[cls] = sqrt(float(dot(self.classes[cls], self.classes[cls])))
+ self.classes[cls] = log(self.classes[cls])
def classify(self, point):
- for c in self.cl:
-
-
+ mx = 0
+ mx_cls = 0
+ for cls in self.classes:
+ dotprod = dot(self.classes[cls], log(point)) - log(self.lengths[cls])
+ if dotprod > mx:
+ mx = dotprod
+ mx_cls = cls
+ return mx_cls
+
+class BayesPresenceClassifier(BayesClassifier):
+ def classify(self, point):
+ mx = 0
+ mx_cls = 0
+ for cls in self.classes:
+ dotprod = dot(self.classes[cls], log(point.clip(max=2))) - log(self.lengths[cls])
+ if dotprod > mx:
+ mx = dotprod
+ mx_cls = cls
+ return mx_cls
+
+
+def test_bayes():
+ trainingset = data.Data(array([[2, 2, 2, 1],
+ [1, 1, 2, 0],
+ [1, 1, 2, 0],
+ [2, 1, 1, 0]]).T)
+ bc = BayesClassifier(trainingset)
+ print bc.classify(array([2, 2, 2]))
+ print bc.classify(array([3, 1, 1]))
+
+if __name__ == "__main__":
+ test_bayes()
View
@@ -0,0 +1,22 @@
+import data
+import ngrams
+import validate
+import classifier
+import os
+from numpy import *
+classif = classifier.BayesPresenceClassifier
+
+def read_reviews():
+ print "Reading and parsing files..."
+ pos_files = [ngrams.ngrams(1, open("pos/"+i).read()) for i in os.listdir("pos")]
+ neg_files = [ngrams.ngrams(1, open("neg/"+i).read()) for i in os.listdir("neg")]
+ classes = [1] * len(pos_files) + [0] * len(neg_files)
+ print "Creating matrix..."
+ mat = ngrams.ngrams_to_matrix(pos_files + neg_files, classes)
+ print "Running classifier..."
+ print validate.kfold(3, classif, mat)
+ print validate.kfold(5, classif, mat)
+ print validate.kfold(10, classif, mat)
+
+if __name__ == "__main__":
+ read_reviews()
View
@@ -0,0 +1,35 @@
+plot : two teen couples go to a church party , drink and then drive .
+they get into an accident .
+one of the guys dies , but his girlfriend continues to see him in her life , and has nightmares .
+what's the deal ?
+watch the movie and " sorta " find out . . .
+critique : a mind-fuck movie for the teen generation that touches on a very cool idea , but presents it in a very bad package .
+which is what makes this review an even harder one to write , since i generally applaud films which attempt to break the mold , mess with your head and such ( lost highway & memento ) , but there are good and bad ways of making all types of films , and these folks just didn't snag this one correctly .
+they seem to have taken this pretty neat concept , but executed it terribly .
+so what are the problems with the movie ?
+well , its main problem is that it's simply too jumbled .
+it starts off " normal " but then downshifts into this " fantasy " world in which you , as an audience member , have no idea what's going on .
+there are dreams , there are characters coming back from the dead , there are others who look like the dead , there are strange apparitions , there are disappearances , there are a looooot of chase scenes , there are tons of weird things that happen , and most of it is simply not explained .
+now i personally don't mind trying to unravel a film every now and then , but when all it does is give me the same clue over and over again , i get kind of fed up after a while , which is this film's biggest problem .
+it's obviously got this big secret to hide , but it seems to want to hide it completely until its final five minutes .
+and do they make things entertaining , thrilling or even engaging , in the meantime ?
+not really .
+the sad part is that the arrow and i both dig on flicks like this , so we actually figured most of it out by the half-way point , so all of the strangeness after that did start to make a little bit of sense , but it still didn't the make the film all that more entertaining .
+i guess the bottom line with movies like this is that you should always make sure that the audience is " into it " even before they are given the secret password to enter your world of understanding .
+i mean , showing melissa sagemiller running away from visions for about 20 minutes throughout the movie is just plain lazy ! !
+okay , we get it . . . there
+are people chasing her and we don't know who they are .
+do we really need to see it over and over again ?
+how about giving us different scenes offering further insight into all of the strangeness going down in the movie ?
+apparently , the studio took this film away from its director and chopped it up themselves , and it shows .
+there might've been a pretty decent teen mind-fuck movie in here somewhere , but i guess " the suits " decided that turning it into a music video with little edge , would make more sense .
+the actors are pretty good for the most part , although wes bentley just seemed to be playing the exact same character that he did in american beauty , only in a new neighborhood .
+but my biggest kudos go out to sagemiller , who holds her own throughout the entire film , and actually has you feeling her character's unraveling .
+overall , the film doesn't stick because it doesn't entertain , it's confusing , it rarely excites and it feels pretty redundant for most of its runtime , despite a pretty cool ending and explanation to all of the craziness that came before it .
+oh , and by the way , this is not a horror or teen slasher flick . . . it's
+just packaged to look that way because someone is apparently assuming that the genre is still hot with the kids .
+it also wrapped production two years ago and has been sitting on the shelves ever since .
+whatever . . . skip
+it !
+where's joblo coming from ?
+a nightmare of elm street 3 ( 7/10 ) - blair witch 2 ( 7/10 ) - the crow ( 9/10 ) - the crow : salvation ( 4/10 ) - lost highway ( 10/10 ) - memento ( 10/10 ) - the others ( 9/10 ) - stir of echoes ( 8/10 )
View
@@ -0,0 +1,13 @@
+the happy bastard's quick movie review
+damn that y2k bug .
+it's got a head start in this movie starring jamie lee curtis and another baldwin brother ( william this time ) in a story regarding a crew of a tugboat that comes across a deserted russian tech ship that has a strangeness to it when they kick the power back on .
+little do they know the power within . . .
+going for the gore and bringing on a few action sequences here and there , virus still feels very empty , like a movie going for all flash and no substance .
+we don't know why the crew was really out in the middle of nowhere , we don't know the origin of what took over the ship ( just that a big pink flashy thing hit the mir ) , and , of course , we don't know why donald sutherland is stumbling around drunkenly throughout .
+here , it's just " hey , let's chase these people around with some robots " .
+the acting is below average , even from the likes of curtis .
+you're more likely to get a kick out of her work in halloween h20 .
+sutherland is wasted and baldwin , well , he's acting like a baldwin , of course .
+the real star here are stan winston's robot design , some schnazzy cgi , and the occasional good gore shot , like picking into someone's brain .
+so , if robots and body parts really turn you on , here's your movie .
+otherwise , it's pretty much a sunken ship of a movie .
View
@@ -0,0 +1,23 @@
+it is movies like these that make a jaded movie viewer thankful for the invention of the timex indiglo watch .
+based on the late 1960's television show by the same name , the mod squad tells the tale of three reformed criminals under the employ of the police to go undercover .
+however , things go wrong as evidence gets stolen and they are immediately under suspicion .
+of course , the ads make it seem like so much more .
+quick cuts , cool music , claire dane's nice hair and cute outfits , car chases , stuff blowing up , and the like .
+sounds like a cool movie , does it not ?
+after the first fifteen minutes , it quickly becomes apparent that it is not .
+the mod squad is certainly a slick looking production , complete with nice hair and costumes , but that simply isn't enough .
+the film is best described as a cross between an hour-long cop show and a music video , both stretched out into the span of an hour and a half .
+and with it comes every single clich ? .
+it doesn't really matter that the film is based on a television show , as most of the plot elements have been recycled from everything we've already seen .
+the characters and acting is nothing spectacular , sometimes even bordering on wooden .
+claire danes and omar epps deliver their lines as if they are bored , which really transfers onto the audience .
+the only one to escape relatively unscathed is giovanni ribisi , who plays the resident crazy man , ultimately being the only thing worth watching .
+unfortunately , even he's not enough to save this convoluted mess , as all the characters don't do much apart from occupying screen time .
+with the young cast , cool clothes , nice hair , and hip soundtrack , it appears that the film is geared towards the teenage mindset .
+despite an american 'r' rating ( which the content does not justify ) , the film is way too juvenile for the older mindset .
+information on the characters is literally spoon-fed to the audience ( would it be that hard to show us instead of telling us ? ) , dialogue is poorly written , and the plot is extremely predictable .
+the way the film progresses , you likely won't even care if the heroes are in any jeopardy , because you'll know they aren't .
+basing the show on a 1960's television show that nobody remembers is of questionable wisdom , especially when one considers the target audience and the fact that the number of memorable films based on television shows can be counted on one hand ( even one that's missing a finger or two ) .
+the number of times that i checked my watch ( six ) is a clear indication that this film is not one of them .
+it is clear that the film is nothing more than an attempt to cash in on the teenage spending dollar , judging from the rash of really awful teen-flicks that we've been seeing as of late .
+avoid this film at all costs .
View
@@ -0,0 +1,19 @@
+ " quest for camelot " is warner bros . ' first feature-length , fully-animated attempt to steal clout from disney's cartoon empire , but the mouse has no reason to be worried .
+the only other recent challenger to their throne was last fall's promising , if flawed , 20th century fox production " anastasia , " but disney's " hercules , " with its lively cast and colorful palate , had her beat hands-down when it came time to crown 1997's best piece of animation .
+this year , it's no contest , as " quest for camelot " is pretty much dead on arrival .
+even the magic kingdom at its most mediocre -- that'd be " pocahontas " for those of you keeping score -- isn't nearly as dull as this .
+the story revolves around the adventures of free-spirited kayley ( voiced by jessalyn gilsig ) , the early-teen daughter of a belated knight from king arthur's round table .
+kayley's only dream is to follow in her father's footsteps , and she gets her chance when evil warlord ruber ( gary oldman ) , an ex-round table member-gone-bad , steals arthur's magical sword excalibur and accidentally loses it in a dangerous , booby-trapped forest .
+with the help of hunky , blind timberland-dweller garrett ( carey elwes ) and a two-headed dragon ( eric idle and don rickles ) that's always arguing with itself , kayley just might be able to break the medieval sexist mold and prove her worth as a fighter on arthur's side .
+ " quest for camelot " is missing pure showmanship , an essential element if it's ever expected to climb to the high ranks of disney .
+there's nothing here that differentiates " quest " from something you'd see on any given saturday morning cartoon -- subpar animation , instantly forgettable songs , poorly-integrated computerized footage .
+ ( compare kayley and garrett's run-in with the angry ogre to herc's battle with the hydra .
+i rest my case . )
+even the characters stink -- none of them are remotely interesting , so much that the film becomes a race to see which one can out-bland the others .
+in the end , it's a tie -- they all win .
+that dragon's comedy shtick is awfully cloying , but at least it shows signs of a pulse .
+at least fans of the early-'90s tgif television line-up will be thrilled to find jaleel " urkel " white and bronson " balki " pinchot sharing the same footage .
+a few scenes are nicely realized ( though i'm at a loss to recall enough to be specific ) , and the actors providing the voice talent are enthusiastic ( though most are paired up with singers who don't sound a thing like them for their big musical moments -- jane seymour and celine dion ? ? ? ) .
+but one must strain through too much of this mess to find the good .
+aside from the fact that children will probably be as bored watching this as adults , " quest for camelot " 's most grievous error is its complete lack of personality .
+and personality , we learn from this mess , goes a very long way .
View
@@ -0,0 +1,37 @@
+synopsis : a mentally unstable man undergoing psychotherapy saves a boy from a potentially fatal accident and then falls in love with the boy's mother , a fledgling restauranteur .
+unsuccessfully attempting to gain the woman's favor , he takes pictures of her and kills a number of people in his way .
+comments : stalked is yet another in a seemingly endless string of spurned-psychos-getting-their-revenge type movies which are a stable category in the 1990s film industry , both theatrical and direct-to-video .
+their proliferation may be due in part to the fact that they're typically inexpensive to produce ( no special effects , no big name stars ) and serve as vehicles to flash nudity ( allowing them to frequent late-night cable television ) .
+stalked wavers slightly from the norm in one respect : the psycho never actually has an affair ; on the contrary , he's rejected rather quickly ( the psycho typically is an ex-lover , ex-wife , or ex-husband ) .
+other than that , stalked is just another redundant entry doomed to collect dust on video shelves and viewed after midnight on cable .
+stalked does not provide much suspense , though that is what it sets out to do .
+interspersed throughout the opening credits , for instance , a serious-sounding narrator spouts statistics about stalkers and ponders what may cause a man to stalk ( it's implicitly implied that all stalkers are men ) while pictures of a boy are shown on the screen .
+after these credits , a snapshot of actor jay underwood appears .
+the narrator states that " this is the story of daryl gleason " and tells the audience that he is the stalker .
+of course , really , this is the story of restauranteur brooke daniels .
+if the movie was meant to be about daryl , then it should have been called stalker not stalked .
+okay . so we know who the stalker is even before the movie starts ; no guesswork required .
+stalked proceeds , then , as it begins : obvious , obvious , obvious .
+the opening sequence , contrived quite a bit , brings daryl and brooke ( the victim ) together .
+daryl obsesses over brooke , follows her around , and tries to woo her .
+ultimately rejected by her , his plans become more and more desperate and elaborate .
+these plans include the all-time , psycho-in-love , cliche : the murdered pet .
+for some reason , this genre's films require a dead pet to be found by the victim stalked .
+stalked is no exception ( it's a cat this time -- found in the shower ) .
+events like these lead to the inevitable showdown between stalker and stalked , where only one survives ( guess who it invariably always is and you'll guess the conclusion to this turkey ) .
+stalked's cast is uniformly adequate : not anything to write home about but also not all that bad either .
+jay underwood , as the stalker , turns toward melodrama a bit too much .
+he overdoes it , in other words , but he still manages to be creepy enough to pass as the type of stalker the story demands .
+maryam d'abo , about the only actor close to being a star here ( she played the bond chick in the living daylights ) , is equally adequate as the " stalked " of the title , even though she seems too ditzy at times to be a strong , independent business-owner .
+brooke ( d'abo ) needs to be ditzy , however , for the plot to proceed .
+toward the end , for example , brooke has her suspicions about daryl .
+to ensure he won't use it as another excuse to see her , brooke decides to return a toolbox he had left at her place to his house .
+does she just leave the toolbox at the door when no one answers ?
+of course not .
+she tries the door , opens it , and wanders around the house .
+when daryl returns , he enters the house , of course , so our heroine is in danger .
+somehow , even though her car is parked at the front of the house , right by the front door , daryl is oblivious to her presence inside .
+the whole episode places an incredible strain on the audience's suspension of disbelief and questions the validity of either character's intelligence .
+stalked receives two stars because , even though it is highly derivative and somewhat boring , it is not so bad that it cannot be watched .
+rated r mostly for several murder scenes and brief nudity in a strip bar , it is not as offensive as many other thrillers in this genre are .
+if you're in the mood for a good suspense film , though , stake out something else .
Oops, something went wrong.

0 comments on commit 6124344

Please sign in to comment.