Skip to content
Permalink
Branch: master
Find file Copy path
Find file Copy path
1 contributor

Users who have contributed to this file

5522 lines (3118 sloc) 441 KB

1. https://www.rogerebert.com/reviews/the-heir-apparent-largo-winch-2011

Root claim: In the review, the reviewer describes the movie as an action thriller packed with cliches.

Generator credence: 98% -- While it’s not explicitly stated that the film is an action thriller (apart from the sidebar), it’s clear from the description of the movie’s plot and tropes.

Tree judgement: 82%

Ensembled step judgement: 90% (10 reviews)

Subclaim 1: In the review, the reviewer explains that the film’s plot involves fights in exotic locations as the hero attempts to gain control of the corporation he inherited

Subclaim 2: In the review, the reviewer claims the film is so cliched it’s like they’ve seen it before.

Subclaim 3: In the review, the reviewer lists the action thriller cliches that occur in the film

Rebuttal: a second heir apparent, who I dare not describe, because that would spoil a very tiny secret. Much interest is added by the beautiful Lea (Melanie Thierry, reminding me of Maud Adams)

Judge reviews:

  1. 55% -- The supporting claims provide substantiation for the main claim. The rebuttal provides evidence of one possible counter: the "very tiny secret", which either indicates that there is at least one thing that's not a cliche, but doesn't address whether the movie might still be full of cliches.

  2. 90% -- The claim is well supported and not rebutted.

  3. 90% -- The supporting quote is quite strong, the rebuttal does not offer contradicting information.

  4. 90% -- There are two parts to the claim: the reviewer describes the movie as... A) an action thriller B) packed with cliches Supporting claim #1 provides very strong evidence for part A -- a movie with fights in exotic locations and a hero trying to gain control of his inheritance is highly likely to be an action thriller Supporting claim #2 provides direct proof for B Supporting claim #3 provides further proof for both A and B The rebuttal seems to be irrelevant. The only way I can see for the supporting claims to be true but the root claim to be false is if the movie actually isn't an action thriller (despite being packed with action thriller cliches?), but the rebuttal provides no evidence of this.

  5. 92% -- the three supporting claims verify all statements of the claim.

  6. 95% -- 'fights in exotic locations', 'so cliched' and 'lists the action thriller cliches' are strong evidence for the claim. The rebuttal is basically irrelevant.

  7. 98% -- thans to supporting quote 2 and 3 I have very strong credence it is true, "a very tiny secret" doesnt contradict that it is mostly clichéd

  8. 100% -- solid evidence

  9. 100% -- The rebuttal doesn't address the claim nearly as well as the supporting claims do

  10. 100% -- In addition to the supporting claims covering all the relevant sub-claims, there is an included #3 that points to examples provided by the reviewer. The Rebuttal appears to only talk about the beginning of the plot instead of more global comments on the film.

Subclaim 1: In the review, the reviewer explains that the film’s plot involves fights in exotic locations as the hero attempts to gain control of the corporation he inherited

Generator credence: 96%

Ensembled step judgement: 82% (7 reviews)

Quotes: The hero…is the secretly adopted son of a billionaire Did I care if Largo Winch won his struggle for control of Winch International? fight to the death high atop, yes, the Hong Kong skyscraper.

Rebuttal: Their contest involves complex financial instruments, and indeed I began to fear the whole movie would play out right there in the boardroom

Judge reviews:

  1. 60% -- Support is a bit weak, but not rebutted.
  2. 80% -- Good rebuttal, but the part "indeed I began to fear" does not mean that the movie ended up playing out in the boardroom.
  3. 84% -- The supporting quotes appear in support of the claim as there is references to a corporation and a Hong Kong skyscraper would be an exotic location. I am given great pause with 'The hero ...is the..' as the location of an ellipses just after a noun like that means virtually any action in the text can be now connected to it. The rebuttal appears to offer tangential support for there being a corporation or fights for control of one but lacks any support for exotic locations. At 197 characters the ellipses might have been necessary in the supporting quotes; still I have to give it much less credence than I otherwise would.
  4. 87% -- i guess a ' Hong Kong skyscraper' could be called 'exotic'. Fight for the corporation is verified
  5. 90% -- The supporting quote is very strong, the rebuttal offers contradicting information but I can imagine it as followed by part where the author contradicts it.
  6. 90% -- There are 2 parts to the claim: the reviewer explains that... A) The film's plot involves fights in exotic locations B) the hero attempts to gain control of a corporation he inherited The supporting quote establishes that there is at least one fight in an exotic location (atop a Hong Kong skyscraper). This technically doesn't prove that there are fights in exotic locations (plural), but I doubt the difference is significant for the tree overall, or from the perspective of someone deciding whether to see the movie. Overall, part A is strongly supported. The supporting quotes also establish that the hero is the son of a billionaire, and that someone called Largo Winch is struggling for control of an eponymous corporation; taken together, this is very strong evidence for part B. The rebuttal quote "I began to fear the movie would play out right there in the boardroom" appears to be an argument against "fights in exotic locations". However, it's very weak evidence: "I began to fear that the movie woudl take place in the boardroom" doesn't mean that it actually did, and the supporting quote proves otherwise.
  7. 98% -- solid evidence

Subclaim 2: In the review, the reviewer claims the film is so cliched it’s like they’ve seen it before.

Generator credence: 97%

Ensembled step judgement: 86.5% (6 reviews)

Quotes: Do you sometimes get the feeling that you haven't seen a film before, but you might as well have? all of his action and dialogue were shunted into narrow corridors of movie formulas

Rebuttal: a second heir apparent, who I dare not describe, because that would spoil a very tiny secret. Much interest is added by the beautiful Lea (Melanie Thierry, reminding me of Maud Adams)

Judge reviews:

  1. 70% -- The claim is more or less supported, and not rebutted.
  2. 85% -- The first supporting qutoe is devoid of context, so it's fairly weak evidence for the claim. The reviewer is more likely to make a statement like the 1st supporting quote if they're reviewing a cliched movie, but it's not direct proof. The second supporting quote is similar; overall the supporting quotes are slightly weaker than I would expect to see if the claim were true. The rebuttal seems to be irrelevant, substantially raising my credence in the claim. If the claim were false, the first supporting quote would have to be about something other than the movie being reviewed, which should be easy to prove in the rebuttal. A similar argument applies to the 2nd supporting quote, so my overall credence in the claim is quite high.
  3. 91% -- yes, good evidence for the claim
  4. 95% -- The supporting quote is quite strong, the rebuttal does not offer contradicting information.
  5. 100% -- direct evidence
  6. 100% -- The supporting quotes contain a continuous sentence that on its own would substantiate the claim followed by another continuous quote that serves as a example of cliches. The rebuttal lacks comments on the film's originality and instead references the beginning of the plot and early introduced characters.

Subclaim 3: In the review, the reviewer lists the action thriller cliches that occur in the film

Generator credence: 96%

Ensembled step judgement: 83.5% (6 reviews)

Quotes: You need a male lead with a permanent short stubble. A sensual woman… Flashbacks to inexplicable locations…giant sinister corporation…the rooftop struggle…movie cliche he called "The Climbing Killer

Rebuttal: a second heir apparent, who I dare not describe, because that would spoil a very tiny secret. Much interest is added by the beautiful Lea (Melanie Thierry, reminding me of Maud Adams)

Judge reviews:

  1. 65% -- The supporting quotes show that there are mentions of action thriller cliches in the film, but not that the reviewer "lists the cliches". This distinction is somewhat important because "lists the cliches" implies that the reviewer is singling them out for criticism, rather than neutrally describing a plot which happens to contain things that readers would recognize as cliches. "Cliche" is only used once in the supporting quotes: "movie cliche he called "The Climbing Killer". The use of "he" in the quote suggests that the reviewer theirself doesn't describe the movie as cliched, in which case the claim would be misleading. The rebuttal is irrelevant (in fact, it's the same rebuttal as all the other workspaces I've seen so far in this tree....). This slightly raises my credence in the claim because it means that the ellipses in the supporting quotes probably don't leave out important context that would falsify the claim. Overall, the claim seems more likely to be true than not, but the ambiguity in the supporting quotes somewhat reduces my confidence. My specific credence of 65% relies on vague priors about how likely the reviewer is to describe a movie as cliched given that it has the various features listed in the supporting quotes.
  2. 80% -- The claim is supported and not rebutted.
  3. 94% -- supporting quote verifies the claim.
  4. 95% -- The supporting quote is quite strong, the rebuttal does not offer contradicting information.
  5. 98% -- The supporting quotes contains several examples of what might fall into a movie cliche category and one verbatim referenced cliche. While there are many ellipses used, with a total input of 198 used I would imagine the ellipses as needed for brevity over misdirection. The rebuttal speaks to the plot but lacks specific reference to 'action thriller cliches.'
  6. 100% -- direct evidence

2. https://www.rogerebert.com/reviews/tears-of-the-sun-2003

Root claim: In the review, the reviewer argues that while the screenplay is not great and the ending is uncompelling, Bruce Willis’s performance and the mood and texture of the movie mostly overcome those problems

Generator credence: 95% -- It’s a pretty complicated claim (many subparts), but also has very strong support

Tree judgement: 70%

Ensembled step judgement: 82.25% (8 reviews)

Subclaim 1: In the review, the reviewer argues that Willis acts well in the movie

Subclaim 2: In the review, the reviewer implies the screenplay is not great

Subclaim 3: In the review, the reviewer argues that the movie’s ending is uncompelling

Subclaim 4: In the review, the reviewer argues that the mood and texture of the movie are effective

Subclaim 5: In the review, the reviewer argues that the cinematography and Willis mostly outweigh the poor screenplay

Rebuttal: `Willis, for example, has a scene in the movie where, as a woman approaches a river, he emerges suddenly from beneath the water to grab her, silence her... This scene is laughable``

Judge reviews:

  1. 49% -- 2 problems: bad screenplay and uncompelling ending. Bruce Willis overcomes the poor screenplay according to the supporting quotes. But does anything overcome the "uncompelling ending"? or is the "uncompelling ending" part of the screenplay (which is better thanks to bruce willis). I think that the main claim treats it as two separate problems, therefore it is not fully supported

  2. 50% -- I'm not sure

  3. 93% -- The subclaims support the claim well, except for possibly outwighing the screenplay but not the poor ending. The rebuttal may indicate that the screenplay is bad.

  4. 95% -- I'm not sure what the rebuttal is getting at. Why is the scene laughable?

  5. 95% -- The supporting claims nicely support each part of the main claim. Rebuttal only supports supporting claim 2.

  6. 95% -- Assuming all the supporting claims are true, then the rebuttal would have to contain something that completely unwinds one supporting claim or add a degree of doubt to several. Here the rebuttal only talks about action in one of the scenes for Bruce Willis' character instead of providing something that can be attributed to the film globally.

  7. 95% -- The supporting claims are quite compeling, the rebuttal has missing text in a critical place...

  8. 98% -- if the reviewers were well-convinced of all 5 claims, that's a pretty overwhelming chunk of evidence

Subclaim 1: In the review, the reviewer argues that Willis acts well in the movie

Generator credence: 95%

Ensembled step judgement: 70% (5 reviews)

Quotes: some actors who couldn't say that dialogue without risking laughter from the audience. Willis is not one of them...If we could fully understand how he does what he does, we would know a great deal

Rebuttal: `Willis, for example, has a scene in the movie where, as a woman approaches a river, he emerges suddenly from beneath the water to grab her, silence her... This scene is laughable``

Judge reviews:

  1. 60% -- 'laughable' is slightly ambiguous, especially behind ellipses. I lean towards believing for this reason
  2. 70% -- The quote seems like it appreciates the gravitas Willis brings to the role, but it's not clear what's going on.
  3. 95% -- The supporting quotes appear to speak well of Willis' acting ability. Yeah, there is an ellipse in the supporting quote but at 196 characters it was likely necessary. The rebuttal only talks about a scene that Willis is in and perhaps the impact of the scene but lacks strong reference to Willis' acting ability.
  4. 95% -- from what I know about willis and what is said about him in the quotes I am pretty sure the main claim is true. The rebuttal seems manipulated(taken out of the context)
  5. 100% -- Just because he has one laughable scene, doesn't mean he is acting poorly. The supporting quotation is very strong

Subclaim 2: In the review, the reviewer implies the screenplay is not great

Generator credence: 99%

Ensembled step judgement: 90% (6 reviews)

Quotes: These materials are sufficient to build a film almost as good as if there had been a better screenplay. story consultants and the like, who found it their duty to dumb it down

Rebuttal: close combat done well in a film that is really about it This film, in this way, from beginning to end... really amounted to something This scene is laughable, but effective but it is wisely left as a subtext

Judge reviews:

  1. 58% -- The rebuttal contradicts the claim somewhat, but not completely.
  2. 90% -- if there had been a better screenplay is direct support of the claim. Rebuttal shows praise for the film, not screenplay
  3. 90% -- "story consultants... dumbed it down" and "as good as if there ad been a better screenplay" - good evidence
  4. 95% -- The supporting quotes are compeling, the rebuttal is omitting important parts of the text...
  5. 96% -- if there had been a better screenplay is pretty damning
  6. 100% -- The rebuttal is difficult to follow and not nearly as direct as the supporting quote

Subclaim 3: In the review, the reviewer argues that the movie’s ending is uncompelling

Generator credence: 99%

Ensembled step judgement: 80% (7 reviews)

Quotes: story consultants and the like, who found it their duty to dumb it down by cobbling together a conventional action climax. The last half hour of "Tears of the Sun," with its routine gun battles

Rebuttal: close combat done well in a film that is really about it This film, in this way, from beginning to end... really amounted to something `This scene is laughable, but effective``

Judge reviews:

  1. 70% -- The supporting quote wins dues to the last sentence.
  2. 80% -- 'Conventional action climax' in film about 'close combat' seems really uncompelling. Not more confident, because the continuation of supporting quote might give it a different meaning. Rebuttal is not directly contradictory.
  3. 80% -- The quotes make it clear that the ending is conventional, while the ellipsis in the rebuttal means that I can't really trust the one quote that addresses the ending.
  4. 90% -- The supporting quotes are very good, the rebuttal is omitting important parts of the text...
  5. 95% -- I give the supporting quotes more weight as they are continuous and maintain the same context across sentences. The rebuttal has separate phrases disconnected from subjects and loose associations to a verbs action.
  6. 95% -- The rebuttal quotes are tepid. Sounds like the climax is too.
  7. 96% -- In the rebuttal the only quote that's about the ending is the second one, and its convenient placement of the ellipsis makes it suspect. Whereas the supporting quote is solid. I'm leaving the leeway because I imagine the ending doesn't have to be uncompelling to everyone, although it would be to me if the support is context-accurate.

Subclaim 4: In the review, the reviewer argues that the mood and texture of the movie are effective

Generator credence: 95%

Ensembled step judgement: 87.5% (7 reviews)

Quotes: There is a way in which movies like "Tears of the Sun" can be enjoyed for their very texture., the editor, Conrad Buff, creates a minimalist mood in setup scenes of terse understatement

Rebuttal: `Willis, for example, has a scene in the movie where, as a woman approaches a river, he emerges suddenly from beneath the water to grab her, silence her... This scene is laughable``

Judge reviews:

  1. 65% -- Quote shows reviewer mentions mood and texture, but not necessarily as effective. I give more than 50 because the rebuttal does not attack this point (and I assume it would if it wasn't true)
  2. 85% -- Assuming the movie is "Tears of the Sun".
  3. 90% -- Rebuttal is challenging the effective mood claim, but the ellipsis is a touch bit too convenient. Still, there is apparently a scene that's "laughable". Is that good or bad? IDK, but it's not too strong a piece of evidence.
  4. 93% -- "can be enjoyed for their texture" -means the texture is effective, "creates minimalist mood..." means the mood is effective
  5. 95% -- I don't think the scene described in Rebuttal disproves its effective mood/texture - one would need to see it in context. And the supporting quotes directly call out texture and mood in positive comments.
  6. 100% -- supporting quote is very clearly directly related to the claim, the rebuttal is irrelevant because one scene is not generalizable
  7. 100% -- The supporting quotes speak well to the mood and texture of the movie and why they are effective. The rebuttal instead of commenting on mood and texture look at the acting and action of Willis' character.

Subclaim 5: In the review, the reviewer argues that the cinematography and Willis mostly outweigh the poor screenplay

Generator credence: 95%

Ensembled step judgement: 90% (5 reviews)

Quotes: of the Sun" is a film constructed out of rain, cinematography and the face of Bruce Willis. These materials are sufficient to build a film almost as good as if there had been a better screenplay.

Rebuttal: I intuit "input" from producers, studio executives, story consultants and the like, who found it their duty to dumb it down Willis... has a scene in the movie... This scene is laughable

Judge reviews:

  1. 90% -- 'cinematography and Willis' part are well supported. 'almost as good as ... a better screenplay' does seem equivalent to 'mostly outweigh the poor screenplay'. Rebuttal only seems to support 'poor screenplay'.
  2. 90% -- The supporting quote is quite compeling, the rebuttal does not seem to be much relevant...
  3. 95% -- The supporting quotes contain all the sentiment needed for the root claim. It shows what the film excelled in and notes how that compensated for the poor screen play. The rebuttal shows a good amount of opinion from the reviewer but the concluding quote to back up the primary/first/ core quote's context is spaced out with context destroying ellipses.
  4. 95% -- the positive and negative comments match
  5. 100% -- I agree with teh supporting quotes because they mentioned the "face" of Bruce Willis, not his acting skills, which kind of detracts from the Rebuttal mentioning Willis' one poorly-acted scene

3. https://www.rogerebert.com/reviews/dead-snow-2-red-vs-dead-2014

Root claim: In the review, the reviewer argues that the movie’s zombie fights work better than the human protagonists.

Generator credence: 95% -- mentioned in a couple of places

Tree judgement: 70%

Ensembled step judgement: 90% (9 reviews)

Subclaim 1: In the review, the reviewer describes the zombie fight in the climax as joyful and funny.

Subclaim 2: In the review, the reviewer argues that the human protagonists are unlikable and don’t work well.

Subclaim 3: In the review, the reviewer asserts that the section of the film focused on the human protagonists is boring compared to the zombie fights.

Rebuttal: Turn back now if you're not already chuckling. The makers of "Dead Snow 2" rely heavily on the inherent absurdity of their film's premise, and expect you to do the rest. Several comedic scenes that

Judge reviews:

  1. 75% -- The supporting claims substantiate the main claim and the rebuttal appears to render some support rather than to contradict the claim.

  2. 90% -- "the section of the film focused on the human protagonists is boring compared to the zombie fights" almost equals to "zombie fights work better than the human protagonists." in the main claim, the rebuttal is not convincing

  3. 90% -- Supporting quote is sufficient.

  4. 91% -- supporting quote establish that the zombie fights work well and the characters don't. This is good evidence for the claim.

  5. 92% -- Claim well supported by single subclaim, rebuttal not contradicting

  6. 95% -- The supporting claims are very strong. The rebuttal not so.

  7. 95% -- Rebuttal isn't addresing the claim.

  8. 99% -- 1 and 2 are by itself sufficient. combined the chance is very high

  9. 100% -- All three supporting claims together provide complete evidence for the claim while 1. and 2. alone are weaker evidence than 3.

Subclaim 1: In the review, the reviewer describes the zombie fight in the climax as joyful and funny.

Generator credence: 80%

Ensembled step judgement: 85% (6 reviews)

Quotes: most novel twist: what if … zombie Nazis fight zombie Communists? That joyfully moronic question is answered in a comically drawn-out battle royale All creative roads lead to … film's big climax

Rebuttal: Turn back now if you're not already chuckling. The makers of "Dead Snow 2" rely heavily on the inherent absurdity of their film's premise, and expect you to do the rest. Several comedic scenes that

Judge reviews:

  1. 70% -- But is that epic (and very important!) fight the climax? I hope it is, but these quotes don't show that.
  2. 85% -- There is an ironic undertone to the claim, but otherwise is seems to support the claim
  3. 85% -- The suppporting quotes do make the claim very likely but don't provide complete evidence for the claim that the zombie fight takes place in the climax of the film.
  4. 88% -- The battle royale does not need to be in the climax, but the prior is it usually is. Rebuttal does not contradict.
  5. 90% -- Supporting quote is sufficient.
  6. 95% -- The supporting quotes are strong, the rebuttal not so.

Subclaim 2: In the review, the reviewer argues that the human protagonists are unlikable and don’t work well.

Generator credence: 95%

Ensembled step judgement: 70% (7 reviews)

Quotes: Zombie Squad's members … they're so gawky and earnest that you keep … hoping that one of them gets dispatched by a brain-nibbling … Daniel … a greasy, trenchcoat-clad geek, is the most obnoxious

Rebuttal: Wirkola and co. do right by their preposterous stars by making them pose like punch-drunk pro-wrestlers on their way to a steel cage match.

Judge reviews:

  1. 65% -- I'm confused about the use of word 'dispatched' which may not mean killed in the context. The second part hints characters are obnoxious. Rebuttal suggests the roles are good, not necessarily likable. The obnoxious part dominates.
  2. 70% -- That Daniel is awful apparently. But does that apply to all the protagonists?
  3. 70% -- Supporting quote is sufficient.
  4. 75% -- Even though the supporting quote is convincing, the discontinuities make me unsure.
  5. 75% -- The supporting quote is highly cut by omissions but clear in essence.
  6. 84% -- they are described as unlikable but maybe that is intentionally and they do work well.
  7. 90% -- The supporting quotes are very strong, even though a lot of text is missing. The rebuttal does offer some contradicting information but it is not enough.

Subclaim 3: In the review, the reviewer asserts that the section of the film focused on the human protagonists is boring compared to the zombie fights.

Generator credence: 90%

Ensembled step judgement: 92% (7 reviews)

Quotes: made zombie Nazis fight zombie Communists? That joyfully moronic question is answered in a comically drawn-out battle royale. But, until then, viewers are stuck watching unmoving human protagonists

Rebuttal: Turn back now if you're not already chuckling. The makers of "Dead Snow 2" rely heavily on the inherent absurdity of their film's premise, and expect you to do the rest. Several comedic scenes that

Judge reviews:

  1. 90% -- Claim somewhat open to interpretation, but well supported by single quote
  2. 90% -- Supporting quote is sufficient.
  3. 94% -- 'stuck watching unmoving human protagonists' is good evidence for the claim
  4. 95% -- The supporting quotes are very strong, the rebuttal does not offer much.
  5. 97% -- Pretty good evidence
  6. 100% -- The supporting quote provides full evidence for the claim with the second sentence.
  7. 100% -- Awesome supporting quote meets ... a rebuttal that's rebutting something else I guess.

4. https://www.rogerebert.com/reviews/chennai-express-2013

Root claim: In the review, the reviewer claims the movie takes a progressive stance on gender relations.

Generator credence: 98% -- Clear from quotes.

Tree judgement: 64%

Ensembled step judgement: 64% (9 reviews)

Subclaim 1: In the review, the reviewer claims the movie cuts against the typical conventions because it has an important female character (Meena) play the heroic role of repeatedly saving an important male character (Rahul).

Subclaim 2: In the review, the reviewer suggests that an expert viewer will recognize that the heroine has a more compelling and powerful role.

Rebuttal: The only time it ever really goes off the rails… is with some of the broader ethnic stereotyping of South Indians. the very model of old fashioned cinematic and social value

Judge reviews:

  1. 30% -- The rebuttal is quite compelling but I am not sure of the context of it to give it high credence. On the other hand, supporting claims seems to be strong, but I am not sure if the fact that the woman saves male is enough for the movie to be considered a progressive. Also, I am not sure what the second supporting claim exactly means - it is not exactly clear with who is the heroine compared.

  2. 60% -- Rebuttal seems weak

  3. 64% -- is a female lead who rescues a male character enough to establish a progressive stance on gender relations. Not quite, i think.

  4. 65% -- The supporting claims substantiate the main claim; the rebuttal appears to speak of ethnic stereotyping but it is unclear whether this extends to gender stereotyping as well; that seems plausible.

  5. 74% -- I interpret the rebuttal as rather supporting the claim (only time...off the rails...streotyping seems more compatible with overall not stereotyping). The latter part of rebuttal is out of context and doesn't affect my credence. important female...saving...male can plausibly be interpreted as progressive...gender relations, but the claim could also be misleadingly strong.

  6. 80% -- Strong supporting claims, the rebuttal quotes could be easily taken out of context

  7. 85% -- The supporting claim #1 clearly establishes that the reviewer says it goes against typical conventions on gender roles. It's highly likely but not totally certain that this means that the reviewer claims that the movie takes a progressive role. I don't understand why supporting claim #2 is relevant for the truth of the root claim given supporting claim #1. The rebuttal seems cherrypicked and irrelevant: ethnic stereotyping of South Asians is irrelevant to gender relations and it's unclear what "...old fashioned cinematic..." refers to.

  8. 90% -- rebuttal could have been taken out of context (probably was, which increases my credence), but the supporting quotes are not too convincing.

  9. 92% -- oh wait the Claim is that reviewer SAYS it's progressive. Yeah I guess they do. [really, it takes an 'expert viewer' to tell she is powerful? And she saves an dude (likely the protagonist) repeatedly? I don't see any challenge to gender roles here]

Subclaim 1: In the review, the reviewer claims the movie cuts against the typical conventions because it has an important female character (Meena) play the heroic role of repeatedly saving an important male character (Rahul).

Generator credence: 98%

Ensembled step judgement: 71.25% (6 reviews)

Quotes: The twist here is that rather than Rahul discovering an inner heroism and saving the imperiled damsel from the fates that beset her, Meena spends the rest of the film bailing him out of mortal danger

Rebuttal: Much of the overt action belongs to Shahrukh Khan Khan can be relied on for any one thing, it's to deliver a Movie Star Moment the very model of old fashioned cinematic and social value

Judge reviews:

  1. 40% -- The supporting quote shows that a female character repeatedly saves a male character, but doesn't show that either character is important. More importantly, it doesn't show that the reviewer claims that this cuts against typical conventions! If the claim is true, it might be hard to find concise quotes showing that Rahul and Meena are important characters, but it should be possible to find a quote about the movie cutting against typical conventions. However, "The twist here..." does imply it to some extent, and makes it more likely that the claim is true in the context of the entire review. The rebuttal is weak evidence against the claim. If "much of the overt action belongs to Shahrukh Khan", it makes it slightly less likely that Rahul is an important character, and Khan reliably delivering "Movie Star Moment[s]" is weak evidence against the film subverting traditional gender roles. Overall, the direct evidence is somewhat weaker than I would expect if the claim is true, but not damningly so, and the rebuttal is fairly weak.
  2. 70% -- Supporting quote seems compelling
  3. 75% -- this claim is supported well by the quote. Alas, the rebuttal casts doubt on me, but the quotes could mean the opposite in different contexts.
  4. 87% -- based on context I think Shahrukh Khan is the male actor
  5. 95% -- solid evidence
  6. 95% -- The supporting claim is very strong, the rebuttal does not seem to be relevant in this context.

Subclaim 2: In the review, the reviewer suggests that an expert viewer will recognize that the heroine has a more compelling and powerful role.

Generator credence: 90%

Ensembled step judgement: 75% (7 reviews)

Quotes: "Chennai Express" very quietly becomes her movie, before we even realize it's happening the most compelling element would be the subtlety of the heroine's string-pulling

Rebuttal: Much of the overt action belongs to Shahrukh Khan Khan can be relied on for any one thing, it's to deliver a Movie Star Moment the very model of old fashioned cinematic and social value

Judge reviews:

  1. 60% -- The 2nd supporting quote is fairly good evidence for the heroine having a more compelling role, but there is no clear evidence that she has a "more powerful" role. "very quietly becomes her movie" is more likely to refer to the strength of the actor's performance than the power of the character within the film. The rebuttal doesn't seem very strong. In particular, the inclusion of "overt" in the first quote suggests that the reviewer may be setting up a contrast with a different character who is important in other ways. The direct quotes don't say anything about an "expert viewer" in particular, but that part is reasonably likely to be true if the rest of the claim is well-supported.
  2. 70% -- top claim makes a lot of claims that the supporting quote does support, but not fully
  3. 80% -- Rebutting quotes don't really speak against the claim
  4. 80% -- yes the reviewer does suggest that
  5. 80% -- The quotes seem to directly support the claim ('expert viewer' part may be a bit imprecise, but I don't find it very relevant). In the light of supporting quotes, rebuttal does not really contradict.
  6. 82% -- 'subtlety of the heroine's string-pulling' is good evidence for the claim.
  7. 90% -- The supporting quotes are very compelling, the rebuttal is not relevant enough to change my mind, I can imagine both of the quotes be true in the same context.

5. https://www.rogerebert.com/reviews/my-life-without-me-2003

Root claim: In the review, the reviewer finds the actions of the main character, a terminally ill woman, to be despicable.

Generator credence: 96% -- The claim is well supported, except for possible ambiguity over the harshness of the word ‘despicable’.

Tree judgement: 63.75%

Ensembled step judgement: 67.5% (10 reviews)

Subclaim 1: In the review, the reviewer states that the main character is a terminally ill woman.

Subclaim 2: In the review, the reviewer condemns the main character’s actions.

Subclaim 3: In the review, the reviewer argues that a female character’s actions hurt those around her

Rebuttal: on good days works through a checklist of "10 Things to Do Before I Die." several scenes of Ann making tape recordings intended to be played by her daughters on their birthdays until they're 18.

Judge reviews:

  1. 60% -- The supporting claims evidence the main claim. The rebuttal seems to imply that a terminally ill woman is unlikely to be doing despicable things that hurt others if she is making recordings for her daughters; however, there is a possibility that these recordings are hurtful (the quote may be taken out of context), in which case it would support the claim of the actions being despicable.

  2. 65% -- The claim is not /absolutely established, and the supporting claims could be consistent with the claim being true /or false.

  3. 65% -- "Condemns" and "hurt those around her" are a little less strong than "despicable", but still suggestive of it. The rebuttal doesn't offer much to counter this. Would have higher confidence, except I seem to be nearly always putting true, so I'm probably overestimating somewhere.

  4. 75% -- If the jump 'condems' -> 'despicable' is plausible, the claim is well supported by supporting claims 1-2, 3 is vague so mostly irrelevant. Rebuttal doesn't seem very relevant either. The result is from my priors for supporting claims somewhat discounting for the ->'despicable' jump

  5. 80% -- The supporting quotes are quite strong. Also, I can imagine the rebuttal being true at the same time as the claim. Especially because of the first part "on good days ..."

  6. 83% -- Supporting quote offer medium evidence for the claim. The reviewer condemns her actions, but does he find them despicable? No strong support for that.

  7. 85% -- The supporting claims #1 and #2 I think would be adequate to support the claim as condemning actions follows in line with finding those actions despicable. The #3 supporting claim can be in reference to the terminally ill woman that is the lead or to another different female character, so I question its inclusion here. The rebuttal shows some of the actions of Ann presumably the terminally ill lead that on the surface appear neutral as opposed to malicious. The evidence of the fictitious expert reviewers points toward full endorsement but there is a loose end here that does not fit.

  8. 85% -- 1 & 2 are strong enough. 3 is sketchy - 'a female character'? Who? Rebuttal shows good acts, but despicable people do normal stuff too. Evil isn't always on.

  9. 87% -- "condemns the main character’s actions"- equals to "finds them despicable."

  10. 95% -- To condemn is not exactly the same as being despicable, but otherwise the claims are a coherent decomposition of the claim

Subclaim 1: In the review, the reviewer states that the main character is a terminally ill woman.

Generator credence: 99%

Ensembled step judgement: 70% (7 reviews)

Quotes: The heroine of "My Life Without Me," on the other hand, engineers her death as a soap opera that would be mushy… but is shameless because it is by her. Told she has inoperable ovarian cancer

Rebuttal: but I can't deny that death has been on my mind these days. Having surgery for cancer concentrates the mind wonderfully. I have some notions of what I would do if I had little time to live,

Judge reviews:

  1. 40% -- Quite unsure in this one. I'm unsure what is engineered in "engineers her death as a soap opera" the death itself or the soap opera. And then there is the conflict of "inoperable" and "surgery". A lot of misleading information overall.
  2. 60% -- The rebuttal and the supporting quote read like they've been flipped. The rebuttal implies the main character has cancer - whether it will kill them or not, isn't clear.
  3. 80% -- The supporting quote would on the surface appear to validate the claim as there is mention of death and inoperable ovarian cancer. However, it states 'the heroine...on the other hand' leading me to believe there was a prior comparison made. I then wonder if the remaining parts are related to the same heroine or if they are connected to something else. The rebuttal offers contrasting evidence or perhaps evidence of a comparison in the way of 'having surgery for cancer'. I can say that yes there is cancer, death, and women but some unreliability as to how those elements relate and diffuse notions of how terminal plays in.
  4. 80% -- The quote and rebuttal are contradictory - quote says 'inoperable...cancer', rebuttal shows that at least at some point there was a chance for surgery. Overall even if there is a chance of surgery, I still find the interpretation of cancer as 'terminally ill' more plausible.
  5. 90% -- The supporting quote provides substantiation for the claim, and provides a possible explanation for the rebuttal in that it might be a part of the 'soap opera' orchestrated; the supporting quote is incomplete and might be misleading.
  6. 90% -- all these quotes are clearly from the story of a person soon to die
  7. 98% -- only the title "my life without me" might be enough as evidence :) . "inoperable ovarian cancer`" makes it crystal clear

Subclaim 2: In the review, the reviewer condemns the main character’s actions.

Generator credence: 98%

Ensembled step judgement: 69.25% (6 reviews)

Quotes: I would not, for example, do any of the things that are done by Ann, the heroine of "My Life Without Me," who would be a cruel egocentric if she weren't so obviously just a fictional pawn.

Rebuttal: there is truth here, too, and a convincing portrait of working-class lives. These people don't stagger under some kind of grim proletarian burden, but are smart and resourceful

Judge reviews:

  1. 50% -- That's not quite a condemnation in the Supporting quotes.
  2. 69% -- Rebuttal is not very helpful. As for support, I'm not sure what to make of the 'just a fictional pawn' part. Without it, the claim seems indirectly supported.
  3. 70% -- "Condemn" seems a bit harsh given the text, but the idea is there.
  4. 75% -- "who would be a cruel egocentric " seems like a condemnation
  5. 79% -- "The heroine" is probably the main character, and calling a character "a cruel egocentric" is condemnation.
  6. 100% -- The supporting quote is a continuous sentence from the reviewer stating the opinions in the claim about the main character. The rebuttal is speaking towards an understanding that while the film is fictional that it does speak to some real life truths. The rebuttal lacks any effective challenge to the claim.

Subclaim 3: In the review, the reviewer argues that a female character’s actions hurt those around her

Generator credence: 98%

Ensembled step judgement: 63.75% (6 reviews)

Quotes: Her egocentric decision puts enormous pressure on the kind doctor... And on her kids… And on her husband, her mother, her best friend at work, and even the woman (also named Ann)...

Rebuttal: several scenes of Ann making tape recordings intended to be played by her daughters on their birthdays until they're 18.

Judge reviews:

  1. 30% -- To put pressure on is not the same as to hurt
  2. 60% -- Unsure. "Hurt" seems too strong a word to depict the supporting quote.
  3. 75% -- "egocentric decision puts enormous pressure" is fairly strong evidence for "hurt". The fact that the claim is generic, "a female character", means I don't have to worry that it's misleadingly referring to a different character. The rebuttal suggests only that at least one of her actions is presumably reasonable.
  4. 87% -- 'puts enormous pressure on' can be interpreted as hurting others.
  5. 93% -- The quote quite directly supports the claim, rebuttal does not contradict
  6. 94% -- The supporting quotes provides several examples of hurtful actions of a female character by identifying the people that are hurt. At 181 characters or so some use of ellipses is expected. The rebuttal recounts some actions in the plot taken by Ann that may or may not be interpreted as hurtful and may or may not be in reference to the same Ann mentioned in the supporting quotes.

6. https://www.rogerebert.com/reviews/nothing-but-the-truth-2009

Root claim: In the review, the reviewer finds that the screenwriter compellingly presented a court case.

Generator credence: 97% -- The review is very positive about the movie and screenwriter, in particular. For example, review details how the judge’s speech is so well written, it moved the reviewer to patriotism.

Tree judgement: 60%

Ensembled step judgement: 69% (9 reviews)

Subclaim 1: In the review, the reviewer praises the movie’s screenwriter

Subclaim 2: In the review, the reviewer writes positively of the movie overall

Subclaim 3: In the review, the reviewer praises a courtroom performance

Rebuttal: The movie is above all a drama about the people involved, and his actors are effective playing personalities, not symbols. Kate Beckinsale is Rachel Armstrong, the reporter for the "Capitol Sun

Judge reviews:

  1. 60% -- that is strongly implied. but although the courtroom performance and the writing in general were praised, the writing of the court scene in particular was not.

  2. 60% -- it seems the movie is about a court. the movie is good. the screenwriter is good. therefore his writting about the court case is good too?

  3. 69% -- only 3. presents an argument for the truth of the claim, but it is weak. It only indicates that there is one scene in court. not a compelling court case.

  4. 70% -- The claim that a courtroom performance and the movie are good doesn't quite support the idea that the courtroom part was well presented. But the rebuttal doesn't mention it either. Is this because the reviewer just doesn't talk about how the court case was presented?

  5. 70% -- Using the guidelines in the instructions regarding the definition of "true", I would only count the claim as true if the court case were central to the plot of the movie. If the reviewer had said e.g. "the movie was awful but this one scene about a court case was compelling", I would consider the claim to be quite misleading from the perspective of someone deciding whether to watch the movie. In reality, the supporting claims suggest that the positive tone of the root claim is warranted, but do not show that the compelling "courtroom performance" was central to the plot. Overall, the supporting claims are more likely to be true if the court case was central to the plot, but not overwhelmingly so (so they're not knockdown bayesian evidence). I don't understand why the rebuttal is relevant.

  6. 80% -- Claim 3 is the lychpin of the overall claim. If true the overall claim is true. If faked the overall statement is false. The others don't matter.

  7. 80% -- Both the supporting claims and the rebuttal indicate general positivity towards the work. The screenwriter is praised, as well as the courtroom performace. The rebuttal indicates it may be a bit more about the people than the court case, but if there were bigger criticism, I'd expect to see them.

  8. 89% -- Caim well supported by supporting claims, even though not literally. Rebuttal does not contradict

  9. 99% -- I can't see a situation in which supporting claims are true and claim isn't

Subclaim 1: In the review, the reviewer praises the movie’s screenwriter

Generator credence: 98%

Ensembled step judgement: 92.25% (8 reviews)

Quotes: `speech was written by Rod Lurie, the writer and director of the film” “Lurie, who is a powerful screenwriter, is freed by fiction to do two very interesting things”

Rebuttal: Alan Alda has a scene in "Nothing But the Truth" where he reads a dissenting Supreme Court opinion defending the right of journalists to protect confidential sources.

Judge reviews:

  1. 85% -- Lurie, who is a powerful screenwriter,. The rebuttal just explains more about the speech.
  2. 90% -- The supporting quote directly praises the screenwriter, and the rebuttal seems irrelevant. The most likely way for the claim to be false would be if omitted quotes contradict the supported quotes, but it would be strange if those weren't included in teh rebuttal. I'm not 100% sure if "writer and director of the film" implies "screenwriter of the film", but it seems likely (and again, the rebuttal should've been stronger if the screenwriter were someone other than Lurie)
  3. 93% -- 'powerful screenwriter' seems like a praise.
  4. 95% -- 'powerful screenwriter' is quite clearly a 'praise'. Rebuttal does not contradict
  5. 95% -- the claim is fully supported by the quotes
  6. 98% -- solid evidence
  7. 99% -- 'powerful screenwriter' is praise. the reviewer wrote it.
  8. 100% -- Explicitly states the premise in the quotes.

Subclaim 2: In the review, the reviewer writes positively of the movie overall

Generator credence: 98%

Ensembled step judgement: 80% (7 reviews)

Quotes: `Nothing But the Truth" is a finely-crafted film of people and ideas” “It is far above the "straight-to-DVD" category”

Rebuttal: trapped the film in a quicksand of complications., positioned as the villain, but objectively he is only doing his job, and Dillon says he played the role as if he were the film's good guy.

Judge reviews:

  1. 64% -- 'finely-crafted' and 'above "straight-to-dvd"' are only weak indications for a general positive description of the movie.
  2. 75% -- It's hard to evaluate the overall tone of the review without seeing every claim made, but the supporting (positive) quotes seem more applicable to the overall film, whereas the rebuttal quotes seem more likely to be only about specific parts.
  3. 85% -- positioned as the villain, but objectively he is only doing his job, and Dillon says he played the role as if he were the film's good guy. doesn't seem all that negative about the film.
  4. 86% -- supporting quotes are quite positive. Rebuttal isn't even reviewer writing negatively.
  5. 88% -- Quote supports the claim. Most likely interpretation of rebuttal refers to film's production rather than overall rating, the second part does not contradict
  6. 90% -- Explicitly give examples of the reviewer praising the movie. The 10 is due to doubts raised in the rebuttal.
  7. 100% -- solid evidence

Subclaim 3: In the review, the reviewer praises a courtroom performance

Generator credence: 70%

Ensembled step judgement: 60% (5 reviews)

Quotes: `a scene in "Nothing But the Truth" where he reads a dissenting Supreme Court opinion… It was so soundly grounded in American idealism that I felt a patriotic stirring”

Rebuttal: opinion defending the right of journalists to protect confidential sources. I assumed the speech was genuine, and was surprised to learn that the case inspiring the film was not heard by the Supreme

Judge reviews:

  1. 30% -- I dont know if an opinion defending the right of journaalist to protect confidential sources could cause "patriotic stirring"
  2. 60% -- From the quote it is not at all clear whether the court opinion was actually read at a courtroom or somewhere else. I expect this may be hard to rebut. The rebuttal is not conclusive either way. What dominates is my 'inner simulator' in which I feel the claim's interpretation as slightly more likely.
  3. 70% -- The supporting quote doesn't actually say that the reviewer praises the performance (he could feel that the speech was grounded in american idealism but it was uncompelling overall for other reasons). However, if the claim were false I would expect a somewhat stronger rebuttal; "assuming the speech was genuine" sounds like the reviewer is impressed by the quality of the writing. However, it's still entirely possibel that the reviewer doesn't directly evaluate the courtroom performance as positive or negative (just patriotic), in which case I wouldn't describe the claim as true.
  4. 93% -- even the rebuttal supports the claim.
  5. 97% -- saying a scene elicited a patriotic stirring in this reviewer's heart is high praise indeed. Rebuttal doesn't contradict Claim.

7. https://www.rogerebert.com/reviews/beaches-1989

Root claim: In the review, the reviewer suggests the movie is a conventional drama (in terms of genre).

Generator credence: 99% -- It is classified as a drama in the metadata and has various standard features.

Tree judgement: 60%

Ensembled step judgement: 60% (9 reviews)

Subclaim 1: In the review, the reviewer describes a predictable plot about the life trajectories of two female friends.

Subclaim 2: In the review, the reviewer argues that the plot is cliched and one of the main characters is conventional.

Subclaim 3: In the review, the reviewer states that one of the main emotional tones of the movie is bittersweet poignancy

Rebuttal: C C is a trouper even at the age of 12, and we follow her through a show-biz audition in which her mother (Lainie Kazan) cheerleads from the front of the stalls. She doesn't get the job

Judge reviews:

  1. 50% -- Supporting claim #1 isn't strong evidence for the root claim because it doesn't explicitly establish that the reviewer suggests that the movie is conventional. This could be an oversight, but given the adversarial nature of factored evaluation, I think it's plausibly intentional. Supporting claim #2 is moderately strong that the movie is conventional. After looking up the definition of a "drama movie" per instructions, I think that claim #3 is evidence that the movie is a drama (but I'm very uncertain because I don't know anything about movies!). I don't understand how the rebuttal is relevant to either part of the claim ("conventional" or "drama"). It could be intended as evidence that C C is not conventional, but that doesn't say much about whether the movie is conventional overall. Overall, I'm highly unsure if the claim is true or not given the supporting claims. My main hesitation is that none of the supporting claims explicitly support either of the two parts of the root claim. I would expect malicious experts to try a strategy like "prove that the movie has conventional and drama-like features, then claim that the reviewer suggests that the movie is a conventional drama", making it more likely that this is a false claim.

  2. 55% -- The supporting claims seem to describe a conventional drama, but I feel highly uncertain about what that term means. Ordinarily, I would look up "conventional drama" online, but I also feel unsure of what others' level of knowledge on the term. It's not like I don't have a running definition in my head; I certainly do. But are the supporting quotes sufficient to describe it?

  3. 60% -- The supporting quotes do not contain anything precise about that genre is drama and they say that one of the main characters is conventional, which does not says much about the genre. But the rebuttal also does not offer strong contradiction information.

  4. 65% -- The supporting claims do no proof that the movie is a drama.

  5. 75% -- The supporting claims substantiate the main claim, the rebuttal has a quote that does not directly contradict the claim and could be interpreted to support it.

  6. 80% -- Supporting claims and rebuttal points to the claim being likely aka the movie being a conventional drama about the life of two women.

  7. 80% -- "predictable plot" even "clichéd plot" suggest it is conventional. the main emotional tone is bittersweet poignancy - it sounds like a drama.

  8. 95% -- I could easily see how the movie could still be unconventional while mantaining these what is said in the supporting quotes. However, the fact that the rebuttal actually shows something that is also pretty cliche, it makes it me very confident that the claim is correct.

  9. 100% -- The supporting claims are in line with the expectations of a 'conventional drama', especially with the noting of cliche plot and a conventional main character. The rebuttal talks about elements of the plot but fails to comment on the genre of the film. The rebuttal contains some dramatic content and partially supports the main claim.

Subclaim 1: In the review, the reviewer describes a predictable plot about the life trajectories of two female friends.

Generator credence: 99%

Ensembled step judgement: 62.5% (7 reviews)

Quotes: We're way ahead of the characters on the screen. We know the two women will meet again as young adults ... will fall in love with the same man ... some big fights but ... friendship will endure.

Rebuttal: C C is a trouper even at the age of 12, and we follow her through a show-biz audition in which her mother (Lainie Kazan) cheerleads from the front of the stalls. She doesn't get the job

Judge reviews:

  1. 50% -- "We're way ahead of the characters on the screen" is it because the plot is so predictable? or is it because some relevant information was revealed to us in advance? the plot doesnt sound so predictable to me.
  2. 55% -- I think the supporting quotes strongly imply the root claim. But us knowing what will happen does not mean the plot is necessarily predictable. It may be that the film itself showed a surprising future-set scene early in the film. But the rebutter doesn't say this. So I'm preferencing that the claim is true.
  3. 70% -- Supporting quotes validate the claim. Even though there are gaps in the quotes, they seem to omit portion inorder to summarise the story rather than to misinterpret information.
  4. 85% -- If the ellipses in the quotes aren't leaving out important context, the supporting quotes seem like very strong evidence for the claim. The main uncertainty is whether the plot is "predictable"; "we're way ahead of the characters..." is strong evidence for predictability but not ironclad proof. The rebuttal seems irrelevant. In particular, the rebuttal doesn't seem to provide context for the supporting quotes that was omitted by the ellipses, which suggests that there weren't in fact important omissions.
  5. 95% -- solid evidence
  6. 98% -- The supporting quotes appear to spell out what is said in the claim. I was given some pause by the use of ellipses. However at 198 characters they were likely necessary and appear to be functioning primarily as abbreviation. The rebuttal offers some insight on one of the characters but lacks any commentary on the whole plot.
  7. 100% -- The supporting quote is direct evidence for the claim.

Subclaim 2: In the review, the reviewer argues that the plot is cliched and one of the main characters is conventional.

Generator credence: 99%

Ensembled step judgement: 70% (7 reviews)

Quotes: story is set up so completely in terms of ancient movie cliches, we don't expect her to be portraying a character completely dictated by convention

Rebuttal: Maybe the problem is with the flashbacks. Maybe if the whole story had simply been told from beginning to end, it would have felt less like one of those 1950s tearjerkers with the rain blowing in

Judge reviews:

  1. 40% -- "the plot is cliched" is well supported but "one of the main characters is conventional" is not . I am trying to figure out how could this sentence continue "we don't expect her to be portraying a character completely dictated by convention' . If it was in the past tense - "we didnt expect her to be portraying a character..." - that would mean that she really made it conventional. but it present...I dont know
  2. 60% -- The first supporting quote seems like strong evidence that the story is cliched (though there's a possibility that it's taken out of context). The second supporting quote is fairly ambiguous; it could be saying something like "we wouldn't expect such a good actor to be stuck playing such a conventional character" , or something like "we don't expect her to be portraying a character completely dictated by convention, but who would have thought she'd be this wild". Including such an ambiguous quote would normally be a warning sign, but for something like "one of the main characters is conventional" I can see the quote in question being the strongest evidence, even if the claim is true. I don't understand what's going on with the rebuttal, as it seems to provide additional support for the claim that the movie is cliched. Overall, the supporting quotes for "one of the main characters is conventional" are slightly weaker than I'd expect if the claim were true, but the lack of an effective rebuttal makes me think that the claim is somewhat more likely to be true than not.
  3. 80% -- The supporting quotes and rebuttal validate the claim
  4. 85% -- The rebuttal seems to be in favor of the claim. But I don't find strong evidence that the main character is conventional, but there is incidental evidence to it.
  5. 90% -- Supporting quote is sufficient.
  6. 100% -- The supporting quotes while separate in two parts, still is very adequate in substantiating cliches and conventional characters. The rebuttal while loosely referring to plot speaks more to the timeline.
  7. 100% -- The claim is provable by the supporting quotes.

Subclaim 3: In the review, the reviewer states that one of the main emotional tones of the movie is bittersweet poignancy

Generator credence: 95%

Ensembled step judgement: 96% (6 reviews)

Quotes: "Beaches" begins on a note of impending doom, and that colors everything else with an undertone of bittersweet poignancy and, believe me, there is only so much bittersweet poignancy I can take

Rebuttal: C C is a trouper even at the age of 12, and we follow her through a show-biz audition in which her mother (Lainie Kazan) cheerleads from the front of the stalls. She doesn't get the job

Judge reviews:

  1. 85% -- The supporting quotes seem like strong evidence for the claim: "bittersweet poignancy" is a verbatim quote from the text, and "more bittersweet poignancy than I can take" strongly suggests that it's a main emotional theme in the film. The rebuttal seems irrelevant or week evidence for the claim (the quote could be interpreted as an example of bittersweet poignancy).
  2. 95% -- Supporting quote is sufficient.
  3. 99% -- direct evidence
  4. 100% -- The supporting quote is continuous and rather definitive with regards to the presence of bittersweet poignancy. The rebuttal offers only a description of one plot point over a commentary on the film's overall tone.
  5. 100% -- Both supporting quotes and rebuttal validates the claim.
  6. 100% -- The supporting quotes offer full evidence for the claim.

8. https://www.rogerebert.com/reviews/love-letters-1984

Root claim: In the review, the suggests describes the movie as an intellectually sophisticated drama.

Generator credence: 98% -- It’s clearly a drama and it’s described as being intellectually stimulating, having good ideas

Tree judgement: 60%

Ensembled step judgement: 70% (9 reviews)

Subclaim 1: In the review, the reviewer says one of the actors has a dramatic role, suggesting the movie is a drama.

Subclaim 2: In the review, the reviewer asserts the movie involves a woman whose mother dies and who falls in love with a married man

Subclaim 3: In the review, the reviewer says the movie explores intellectual ideas rather than conveying a simple message.

Subclaim 4: In the review, the reviewer suggests the movie is subtle

Rebuttal: Love Letters was written and directed by Amy Jones, whose previous credit was "The Slumber Party Massacre". … exploitation movies… all her horror pictures

Judge reviews:

  1. 60% -- "the slumber party massacre" :-D lol. The supporting claims are very good(I would even say that they fully support the claim) but so is the rebbutal. Maybe the director just wanted to try something different?

  2. 62% -- "one of the actors has a dramatic role, suggesting the movie is a drama": this is weak. According to the rebuttal, Amy Jones made some number of exploitation or horror movies. What about this one though? I'm bumping this positively because of "intellectually sophisticated".

  3. 70% -- claim 1 supports it is 'drama' but not directly. Rebuttal suggests it could e a horror movie. I still give more than 50 because if it were horror movie, I would expect a more direct mention in rebuttal

  4. 75% -- The Rebuttal states that the director directed horror films in the past, but that does not disprove any of the claims which strongly support the main claim.

  5. 85% -- The supporting claims indicate high credence on "drama" and "the movie explores intellectual ideas", which is close to synonymous with "intellectually sophisticated drama". The rebuttal tries to suggest that it may be a horror and/or lowbrow movie, but does appear probable that in context that these quotes will be brought up primarily for contrast reasons.

  6. 85% -- I'm specifically assuming that both parts of #1 are known to be true (the actor has a dramatic role AND this suggests the movie is a drama). Normally, I wouldn't expect "actor has a dramatic role" to suggest a movie is a drama. However, if true in entirety, #1 establishes that the movie is a drama, and #3 and #4 strongly suggest that the reviewer describes the movie as intellectually sophisticated. I don't really understand what role claim #2 plays. The rebuttal suggests taht the director of the movie in question is known for exploitation movies, but doesn't really show that this movie is an exploitation movie.

  7. 89% -- The subclaims support the main claim well, and the rebuttal seems fishy.

  8. 91% -- good evidence ( especially 3.)

  9. 97% -- The supporting claims fall well in line with verifying drama and enough of the connotations paired with 'intellectually sophisticated.' The rebuttal while pointing towards the director's prior horror films fails to provide evidence to the contrary of the film being a drama. I believe if it was a horror film named 'Love Letters' minimally there would be some commentary on that sort of title for a horror film.

Subclaim 1: In the review, the reviewer says one of the actors has a dramatic role, suggesting the movie is a drama.

Generator credence: 98%

Ensembled step judgement: 60% (5 reviews)

Quotes: And she gives Jamie Lee Curtis the best dramatic role of her career; this role… shows her with a range we couldn't have guessed from all her horror pictures.

Rebuttal: She meets a photographer (James Keach), who is a sensitive, intelligent, married man, and feels powerfully drawn to him. soap opera romance

Judge reviews:

  1. 20% -- I don't think that an actor having a dramatic role "suggests that the movie is a drama" (though I know very little about movie genres). The rebuttal suggests that th emovie is a "soap opera romance" rather than a drama; the fact that "soap opera romance" is provided with no contextualizing quotes is suspicious, but the remainder of the rebuttal sounds more like a romance than a drama. Also, "a range we couldn't have guessed from horror pictures" sounds more compatible with a romance than a drama (aren't horror pictures dramatic? I could be wrong on this). Also, if the movie were a drama, the expert should have been able to find a much stronger supporting quotes.
  2. 60% -- Dramas can have photographers. But how does that suggest the movie is a drama? (Also, all daytime soaps are about drama AFAIK - I assume that's a contractual requirement between creators and the universe)
  3. 85% -- The rebuttal strengthens the supporting quote.
  4. 91% -- on actor has a dramatic role. and it does suggest that it is a drama.
  5. 98% -- The supporting quote validates the claim with '...best dramatic role of her career...'. The rebuttal mostly addresses the action of the plot and provides some tangential support for the film being a drama with 'soap opera romance.'

Subclaim 2: In the review, the reviewer asserts the movie involves a woman whose mother dies and who falls in love with a married man

Generator credence: 95%

Ensembled step judgement: 87.25% (6 reviews)

Quotes: her mother dies, and she gets a job offer ... She doesn't take the job ... because something else happens. She meets a photographer ... who is a ... married man, and feels powerfully drawn to him.

Rebuttal: Love Letters was written and directed by Amy Jones, whose previous credit was "The Slumber Party Massacre". … exploitation movies… all her horror pictures

Judge reviews:

  1. 85% -- According to the rebuttal exploitation / horror movies can't have female characters whose mother dies and they fall for married men. Other genres get to do that I guess? Support has some leeway in "feels powerfully drawn": this doesn't have to be love.
  2. 87% -- The supporting quote straightforwardly says "her mother dies" and she meets a "married man" who she is "powerfully drawn to". The claim could be false if she is powerfully drawn to the man but doesn't love him, or if "she" refers to different people in the two different parts of the quote. However, neither of those seem very likely on priors.
  3. 88% -- Not more because there is some context missing. Not less because Rebuttal doesn't attempt to add that context.
  4. 90% -- The supporting quotes, if not taken maliciously out of context, strongly supports the claim. If taken out of context, I would expect the rebuttal to be able to include this, but iinstead it seems fairly irrelevant.
  5. 90% -- good evidence, but a lot of ellipses in the supporting quote
  6. 98% -- The rebuttal seems easily dismissed as it speaks to Amy Jones' work within horror films. The Supporting quotes do speak to the aspects of the claim but do so with a hefty serving of ellipses which usually gives me some pause. I think the necessity of the ellipses is driven here by the 200 character limit.

Subclaim 3: In the review, the reviewer says the movie explores intellectual ideas rather than conveying a simple message.

Generator credence: 95%

Ensembled step judgement: 72.75% (6 reviews)

Quotes: What she accomplishes here is wonderful. She creates a story of passion that is as absorbing as a thriller. She makes a movie of ideas that never, ever, seems to be just a message picture.

Rebuttal: Love Letters was written and directed by Amy Jones, whose previous credit was "The Slumber Party Massacre". … exploitation movies… all her horror pictures

Judge reviews:

  1. 70% -- Supporting quotes indicate "movie of ideas" and not "just a message" picture, but "intellectual ideas" isn't said explicitly (they might be "emotional" ideas, although "ideas" usually implies some element of intellectualism). The rebuttal implies it could be lowbrow horror, but I think this could be to show contrast.
  2. 72% -- 'a movie of ideas' =/ 'explores intellectual ideas'
  3. 75% -- The quote seems to support 'ideas ... simple message', not necessarily 'intellectual'. The rebuttal doesn't contradict, though, and still seems desperate to omit context.
  4. 85% -- "movie of ideas" is very well supported (but there's some chance that they aren't "intellectual" ideas?), and "never, ever, seems to be just a message picture" suggests that the movie isn't conveying a simple message. My credence would be higher, but "story of passion" sounds weakly incompatible with "intellectual ideas".
  5. 100% -- The last sentence of the supporting quote is adequate to substantiate the root claim as it contains some similar language and is referential to the whole film like the claim is. The rebuttal has a lack of global critiques and instead offers commentary on the directors previous works in horror.
  6. 100% -- This movie is Love Letters, not The Slumber Party Massacre and apparently doesn't belong to the set "exploitation / horror movies directed by Amy Jones".

Subclaim 4: In the review, the reviewer suggests the movie is subtle

Generator credence: 99%

Ensembled step judgement: 79% (7 reviews)

Quotes: All of this is handled with as much subtlety as Ingmar Bergman brought to similar situations in Scenes from a Marriage. This isn't a soap opera romance

Rebuttal: Love Letters was written and directed by Amy Jones, whose previous credit was "The Slumber Party Massacre". … exploitation movies… all her horror pictures

Judge reviews:

  1. 40% -- The supporting quote is quite unclear without knowing whether Ingmar Bergman's scenes in "Scenes from a Marriage" were subtle or not. The phrasing of the sentence weakly suggests they were not subtle, and reading the first page of google results about "Scenes from a Marriage" also weakly suggests it wasn't subtle. However, it's unclear to me why this would have been picked as a supporting quote if Ingmar Bergman were known as an unsubtle director. Since I know much less about movies or directors compared to the intended audience of the review (or the Ought judging pool, probably), this makes me wonder if I misinterpreted the evidence (and Scenes from a Marriage is indeed known as a subtle film). The rebuttal doesn't say anything about whether themovie is subtle, but it does provide weak evidence that it's not (by suggesting the director specializes in exploitation movies, which are not generally subtle AFAIK).
  2. 78% -- Quote supports the claim unless it's meant ironically. Rebuttal doesn't show it is and altogether seems a bit desperate to omit context
  3. 80% -- "As much subtlety as" could imply subtlety, or an ironic lack of subtlety if the comparison is unsubtle. I know Ingmar Bergman is a famous film director, so it's probably an earnest comparison. "This isn't a soap opera romance" implies subtlety, as soap operas are usually considered overdramatic and hence unsubtle. The rebuttal leads weak evidence against it, but they could be referencing her past works as a contrast..
  4. 83% -- 'handled with as much subtlety as ...' one has to know the comparison, but i suggest that the movie is subtle (as the claim claims)
  5. 85% -- The rebuttal is off topic. The supporting claim is strong.
  6. 85% -- This movie is different than previous ones, induction disproven? Supporting quote implies subtle, doesn't demonstrate or state.
  7. 100% -- The supporting quotes verify the claim verbatim with 'as much subtlety...brought to similar situations...'. The rebuttal is commentary on the directors previous work in horror films.

9. https://www.rogerebert.com/reviews/da-sweet-blood-of-jesus-2015

Root claim: In the review, the reviewer finds the movie’s messages varied and incoherent, but argues that this is okay because its genre, horror, is more about evoking a mood than presenting a clear argument.

Generator credence: 97% -- The claim is is explicitly stated over several paragraphs.

Tree judgement: 60%

Ensembled step judgement: 89.75% (8 reviews)

Subclaim 1: In the review, the reviewer states that the movie’s messages are varied and incoherent

Subclaim 2: In the review, the reviewer describes the movie as horror

Subclaim 3: In the review, the reviewer argues it’s OK for a horror film to be incoherent, as horror is about sustaining a feeling

Subclaim 4: In the review, the reviewer states that lack of coherence doesn’t really hurt the film because the film is more about evoking a type of mood

Rebuttal: not once does Lee trouble himself with questions of likability, much less the possibility that he's following his own muse to the point where he's losing people. This will prove either maddening

Judge reviews:

  1. 85% -- Rebuttal could have a point, but it undermines itself with the last sentence "this will prove either maddening". It's likely that that sentence is followed by "or won't bother at all - horror is about feels yadda la da vidda amigo"

  2. 89% -- rebuttal seems to cut off right before a positive claim. Supporting claims are valid as heck and present a quite coherent argument... unlike the film

  3. 90% -- Supporting claims are sufficient.

  4. 92% -- the supporting claims establish the claim.

  5. 93% -- The supporting claims, of which there are four, cover nicely the majority of the root claim. The rebuttal speaks of the presumed director Lee's choices around character development. The rebuttal appears to offer tangential evidence to #1 with "where he is loosing people."

  6. 95% -- the claim is fully supported and the rebuttal doenst say anything relevant - the fact that some people might not like his movie doesnt contradict the main claim

  7. 95% -- The supporting claims fully justify the primary claim. The rebuttal isn't doesn't provide real counter evidence and seems like it's taken out of context anyway.

  8. 100% -- coherent decomposition of claim

Subclaim 1: In the review, the reviewer states that the movie’s messages are varied and incoherent

Generator credence: 95%

Ensembled step judgement: 60% (7 reviews)

Quotes: Lee's most persistent problem, an inability to unify his messages and make them cohere Ideas bleed into other ideas and morph into third, tangentially related ideas

Rebuttal: The most impressive thing about "Jesus" is its unfailing control of tone. This is manifested in Lee's immaculate and uncharacteristically clean, even sterile, widescreen compositions

Judge reviews:

  1. 40% -- I'm confused. What does "widescreen compositions" refer to? Support and rebuttal together give me the impression that Lee usually does lots of free association in his films but not in this film? If widescreen compositions were instead "scenes", I'd be more confident.
  2. 60% -- incoherent = doesn't cohere. but control of tone implies Claim is false...
  3. 60% -- The rebuttal is strong, but it is not clear that it is talking specifically about the "messages", so I trust the supporting quotes more.
  4. 90% -- The claim is pretty well supported by the quote. The rebuttal just shows that the reviewer thinks the tone is coherent, not the messages of the movie
  5. 94% -- 'inability to unify his messages and make them cohere' establishes the claim.
  6. 99% -- The supporting quote contains some verbatim support for the root claim. The rebuttal talks about unity and control of tone but lacks referencing the movie's message.
  7. 100% -- solid evidence

Subclaim 2: In the review, the reviewer describes the movie as horror

Generator credence: 90%

Ensembled step judgement: 86% (7 reviews)

Quotes: the film is a hypnotically nightmarish mood piece almost unbearably savage violence Hess transformed into a nightcrawling loner with a thirst for human blood not, strictly speaking, a vampire movie

Rebuttal: dancer Charles "Li'L Buck" Riley gliding, twisting, spiraling and twirling across an array of Red Hook, Brooklyn locations—and keeps it going through a gospel performance late in the movie

Judge reviews:

  1. 55% -- My prior for horror musicals with gospel songs is pretty low, but it could certainly exist, and this appears to be one from the quotes.
  2. 85% -- Movie seems horror-adjacent at least, maybe more along the lines of grotesque violence. first supporting quote is carrying the claim. I'm not sure what the rebuttal is saying about the claim.
  3. 87% -- nightmarish mood piece - horrifying. Dancing isn't NOT horror (see Thriller)
  4. 90% -- The quotes pretty much support the claim, to a degree it's reasonable to say it's highly likely to be true, in my opinion. The rebuttal just brings up some potentially comedic and/or musical aspects of the movie, which doesn't disqualify the claim. If there was no better evidence, the claim is probably true.
  5. 92% -- solid evidence, could still not be a horror though
  6. 92% -- nightmarish, savage violence, nightcrawling loner, hirst for human blood: sounds like a horror movie.
  7. 94% -- The supporting quotes provides 4 examples of how the movie might fall into a horror movie reference. I find it mildly strange that if the reviewer described the film as horror then as its a popular genre why wouldn't that word horror be used and included in these quotes. The rebuttal talks about filming locations.

Subclaim 3: In the review, the reviewer argues it’s OK for a horror film to be incoherent, as horror is about sustaining a feeling

Generator credence: 98%

Ensembled step judgement: 91.25% (6 reviews)

Quotes: the horror genre is one of the only genres where it's OK, or at least acceptable, to be imprecise, because it's more often about sustaining a feeling or a mood than making a case.

Rebuttal: not once does Lee trouble himself with questions of likability, much less the possibility that he's following his own muse to the point where he's losing people. This will prove either maddening

Judge reviews:

  1. 75% -- Imprecise != incoherent, but if that's the best rebuttal I'm more likely to believe the claim
  2. 90% -- Supporting quote is sufficient.
  3. 95% -- All points in the claim are supported by the quote, and the rebuttal doesn't provide counter evidence
  4. 96% -- clear evidence in the quote!
  5. 100% -- The supporting quote is a continuous sentence in support of the root claim. The rebuttal appears to offer evidence of the imprecise nature of the film.
  6. 100% -- solid evidence...weird that this quote would make me much more confident in the previous workspace.

Subclaim 4: In the review, the reviewer states that lack of coherence doesn’t really hurt the film because the film is more about evoking a type of mood

Generator credence: 98%

Ensembled step judgement: 89.25% (6 reviews)

Quotes: an inability to unify his messages and make them cohere, doesn't really hurt him in "Da Sweet Blood of Jesus" because the film is a hypnotically nightmarish mood piece more than anything else

Rebuttal: not once does Lee trouble himself with questions of likability, much less the possibility that he's following his own muse to the point where he's losing people. This will prove either maddening

Judge reviews:

  1. 80% -- Supporting quote is sufficient.
  2. 89% -- good evidence
  3. 90% -- 100 credence for "reviewer states this", downgraded because as a potential viewer that might be an issue.
  4. 95% -- The quote supports the claim on all points, and the rebuttal brings up the possibility that other things hurt the film. That isn't part of the claim though, so it doesn't matter
  5. 100% -- The supporting quote almost mirrors the root claim.
  6. 100% -- solid evidence

10. https://www.rogerebert.com/reviews/crave-2013

Root claim: In the review, the reviewer explains that the movie is about a mentally unstable loner who snaps and commits murder.

Generator credence: 95% -- The reviewer doesn’t actually state what happens in the climax, but it’s heavily implied that Aiden kills at least one person.

Tree judgement: 55%

Ensembled step judgement: 80% (9 reviews)

Subclaim 1: In the review, the reviewer calls the main character a mentally unstable loner.

Subclaim 2: In the review, the reviewer explains that the main character fantasizes about killing people who offend him.

Subclaim 3: In the review, the reviewer explains the main character snaps after getting dumped

Subclaim 4: In the review, the reviewer implies the movie’s climax involves the main character killing people

Rebuttal: Aiden as he successfully keeps his daydreams from bleeding into his real life `Aiden (...) suggests a shy and somewhat awkward thirty-something guy who hasn't fully committed to adulthood``

Judge reviews:

  1. 40% -- the supporting claims say that he kills some people not that he murders them (deliberatly). The supporting claim says he fantasizes about killing people - but the rebuttal says that he keeps his daydreams from bleeding into his real life. the fact that he "snaps" after getting dumped is not convincing either

  2. 70% -- The supporting claims validate the claim. The rebuttal could be taken from the context setting portion of the review, also validating the mentally unstable part.

  3. 80% -- The supporting quotes are quite strong and the rebuttal also not strongly contradict the claim but it seems to imply it instead.

  4. 80% -- The rebuttal doesn't fully contradict the claim, and to an extent supports it.

  5. 85% -- Claim is a conjunction of subclaims, unless they refer to multiple main characters. Rebuttal does not contradict

  6. 94% -- The supporting claims collectively support the main claim nearly verbatim. The rebuttal is consistent with the main claim being true. I suppose there's some ambiguity over whether the climax involves murder, but it seems likely given this context.

  7. 95% -- Rebuttal's information is not mutually exclusive with the supporting claims. supporting claims strongly support the claim.

  8. 97% -- the truth of the supporting claims implicate the truth of the claim.

  9. 99% -- With all the supporting claims in mind the claim is well supported.

Subclaim 1: In the review, the reviewer calls the main character a mentally unstable loner.

Generator credence: 98%

Ensembled step judgement: 85.5% (6 reviews)

Quotes: He is a loner Aiden hears voices in his head that urge him to commit gruesome acts of violence Sensing his grip on reality is slipping, Aiden tells…

Rebuttal: Aiden as he successfully keeps his daydreams from bleeding into his real life Aiden (...) suggests a shy and somewhat awkward thirty-something guy who hasn't fully committed to adulthood

Judge reviews:

  1. 52% -- he doesnt "call" him "mentally unstable" - he only suggests something with a very similar meaning, I dont know if I should consider this is just as a detail
  2. 85% -- Supporting quotes are much more strong than the rebuttal
  3. 87% -- The 'loner' part and that Aiden is main character are not well supported because quote is out of context. The rebuttal supports it, though.
  4. 90% -- There is very good evidence for the claim in the supporting quotes.
  5. 96% -- quotes provide direct evidence for the claim.
  6. 100% -- Strong supporting claims validate the claim

Subclaim 2: In the review, the reviewer explains that the main character fantasizes about killing people who offend him.

Generator credence: 98%

Ensembled step judgement: 61.25% (6 reviews)

Quotes: Aiden hears voices in his head…urge him to commit gruesome acts of violence against those who offend his standards He fantasizes about a kind of street justice that involves outlandishly gory payback

Rebuttal: Aiden as he successfully keeps his daydreams from bleeding into his real life `Aiden (...) suggests a shy and somewhat awkward thirty-something guy who hasn't fully committed to adulthood``

Judge reviews:

  1. 20% -- "offend him" doesnt equal "offend his standards"
  2. 55% -- The claim is mostly true, but the quotes don't support the claim that he fantasizes 'killing' specifically
  3. 80% -- Quotes support violence, not necessarily 'killing'. The rebuttal does not contradict
  4. 93% -- supporting quotes offer very good support.
  5. 95% -- Rebuttal appears to support the supporting quote? Daydreams ≈ fantasies
  6. 100% -- The claim is true based on the information in the supporting quotes.

Subclaim 3: In the review, the reviewer explains the main character snaps after getting dumped

Generator credence: 95%

Ensembled step judgement: 83.75% (6 reviews)

Quotes: the relationship unfortunately emboldens Aiden…to act upon his destructive and ultimately deadly impulses with less and less remorse. He reaches the breaking point after Virginia dumps him

Rebuttal: `The climax, which takes its sweet time in arriving and features Edward Furlong as Virginia's abusive ex-boyfriend, is more Grand Guignol comedy of errors than expected.``

Judge reviews:

  1. 78% -- 'reaching his breaking point' could be interpreted as 'snaps'.
  2. 80% -- The supporting quotes provide strong evidence for the claim 'breaking point'
  3. 95% -- Rebuttal is not mutually exclusive with the supporting quote. e.g. could be a fatal comedy of errors
  4. 95% -- Well supported by single quote, not contradicted by rebuttal
  5. 95% -- "He reaches the breaking point after Virginia dumps him" is strong evidence for the claim. The rebuttal doesn't do much to mitigate this
  6. 100% -- The supporting quote provide complete evidence for the claim

Subclaim 4: In the review, the reviewer implies the movie’s climax involves the main character killing people

Generator credence: 96%

Ensembled step judgement: 55% (4 reviews)

Quotes: Virginia dumps him emboldens Aiden…to act upon his…ultimately deadly impulses The climax…features…Virginia's abusive ex-boyfriend, is more…comedy of errors than expected. But it still isn't pretty.

Rebuttal: Aiden as he successfully keeps his daydreams from bleeding into his real life `Aiden (...) suggests a shy and somewhat awkward thirty-something guy who hasn't fully committed to adulthood``

Judge reviews:

  1. 40% -- The supporting quotes indicates that main character (Aiden) becomes abusive, but does not support that claim that he kills people
  2. 60% -- Unclear if the acting on the deadly impulses happens in Aiden's day dreams or not.
  3. 70% -- The rebuttal makes more sense now. There is not so much evidence in this supporting claim that Aiden actually committed the crime.
  4. 72% -- the supporting quote is hard to judge, due to many ellipses, and thus offers only medium support for the claim.

11. https://www.rogerebert.com/reviews/filth-and-wisdom-2008

Root claim: In the review, the reviewer digresses to explain his view that strippers are only erotic to somewhat depraved men.

Generator credence: 90% -- he says he’s grateful to avoid strip clubs and that they are erotic for men who enjoy seeing women humilitate themselves.

Tree judgement: 53.25%

Ensembled step judgement: 53.25% (10 reviews)

Subclaim 1: In the review, the reviewer digresses from discussing the movie to give his personal view on strippers.

Subclaim 2: In the review, the reviewer asserts strippers are only erotic to men who enjoy seeing women humiliated and demeaned.

Rebuttal: Aren't we all way beyond being shocked by sexual fetishes I mention this because Madonna still gets intrigued, I guess, simply by regarding a stripper sliding down a pole.

Judge reviews:

  1. 35% -- The supporting claims do fall in line with the content of the claim. However, "aren't we all way beyond being shocked by sexual fetishes' would lead me to believe the reviewer feels that most anyone, including depraved men, would have some level of familiarity with sexual content in films. So the rebuttal appears to refute strippers as especially erotic and questions the specific audience of 'depraved men.' I think this is likely to be false or misleading.

  2. 45% -- The supporting claims indicate the viewer does "digress", and also asserts "strippers are only erotic to men who enjoy seeing women humiliated and demeaned." The key point is whether "enjoy[s] seeing women humiliated and demeaned" equates to "somewhat depraved". I think this is a common view, although "Aren't we all way beyond being shocked by sexual fetishes" does suggest the reviewer may not think so (though this could for something else). Although, there's a distinction between finding a fetish "shocking" and thinking it "somewhat depraved".

  3. 51% -- The rebuttal is pretty strong here. This is because the key link between the supporting claims and the root claim is whether the reviewer thinks that men who enjoy seeing women demeaned are somewhat depraved. But the rebuttal has the reviewer saying that we are beyond being shocked by sexual fetishes. So would the reviewer think that men enjoy seeing women demeaned are somewhat depraved? I keep going back and forth on this. I've found myself putting in 85% confidence in one direction, and then, not two minutes later, changing my mind to 50%. So do I put in an average of my credences? Or put what I think now, after having typed this out? I think that, given how I keep changing my mind here, I'm going to go with 50%. Although that's my minimum credence, it seems to be what my credence over time is moving toward as I think this through more and more deeply. ... Well. Maybe 51% to indicate that I do think the root claim is true. Just not by much.

  4. 60% -- The claim can be broken into 2 parts: A) The reviewer digresses to impart his view on strippers B) The reviewer's view is that strippers are only erotic to somewhat depraved men Part A is strongly supported by supporting claim #1 (which asserts it nearly verbatim). Part B is potentially supported by supporting claim #2 if "men who enjoy seeing women humiliated and demeaned" are "somewhat depraved" in the opinion of the reviewer On priors, it's likely that the reviewer would see someone who enjoys "seeing women humiliated and demeaned" as "somewhat depraved". However, the quote "Aren't we all way beyond being shocked by sexual fetishes" seems to suggest otherwise (seeing women humiliated and demeaned can be a sexual fetish, so in context the quote makes it less likely that the reviewer sees people with that fetish as depraved). I don't see why the 2nd part of the rebuttal is relevant. Overall I'm quite unsure, but I think the claim is slightly more likely than not given the supporting claims.

  5. 80% -- hinges on claim that depraved = enjoy seeing women humiliated. Sure. Rebuttal could almost be a supporting claim

  6. 85% -- Claim seems well supported. Not sure what rebuttal points at, maybe the 'digresses' part, but I find that rather weak

  7. 90% -- "enjoy seeing women humiliated and demeaned" supports "somewhat depraved".

  8. 90% -- The point about Madonna doesn't contradict anything about the supporting claims.

  9. 95% -- the supporting claims are very compeling, the rebuttal does not seem relevant.

  10. 100% -- direct evidence. 2 would be enough for me to have >95% confidence, 1 just explains that he digresses. But I don't even think it was really needed.

Subclaim 1: In the review, the reviewer digresses from discussing the movie to give his personal view on strippers.

Generator credence: 98%

Ensembled step judgement: 85% (7 reviews)

Quotes: A.K. is the landlord … steers Holly into stripping… I saw my first strip show at the old Follies Burlesque on South State... … They're erotic only to men who enjoy… But I wander

Rebuttal: `

Judge reviews:

  1. 20% -- The supporting quotes supports a personal view on the male viewers, not the strippers.
  2. 80% -- The ellipses are very suspicious, but the first person "I saw my first strip show..." seems like fairly strong evidence. Assuming the blank rebuttal is evidence that there is no contradictory quote, it seems like the ellipses do not obscure important context. If this is true, then the claim is highly likely to be true.
  3. 90% -- Claim seems well supported by quote, unless the missing context is important. The rebuttal does not provide any, though.
  4. 90% -- It seems the supporting quotes are really talking about his personal views but it is hard to tell. the rebuttal is missing so I give it higher credence.
  5. 90% -- Without a rebuttal, the supporting quotes are sufficient.
  6. 99% -- No rebuttal
  7. 100% -- 'I saw my first strip show ...' is adequate to substantiate the reviewer sharing his personal views on strippers. Here the rebuttal field shows up blank.

Subclaim 2: In the review, the reviewer asserts strippers are only erotic to men who enjoy seeing women humiliated and demeaned.

Generator credence: 90%

Ensembled step judgement: 87.5% (8 reviews)

Quotes: Today's strippers leap onstage already almost naked and perform contortions that gotta hurt … Ugh. They're erotic only to men who enjoy seeing women humiliate themselves

Rebuttal: Aren't we all way beyond being shocked by sexual fetishes I mention this because Madonna still gets intrigued, I guess, simply by regarding a stripper sliding down a pole.

Judge reviews:

  1. 55% -- Uncertain here. Feels like there could have been some misdirection in the quote discontinuity. The rebuttal goes against the claim as well.
  2. 80% -- The claim is supported but not rebutted.
  3. 90% -- Supporting quote is sufficient.
  4. 90% -- My prior on the claim is very low (why is the reviewer unprofessionally giving his opinion of strippers when he's supposed to be reviewing movies??) but the supporting quotes are very strong evidence. The most likely way that the claim could be false, given the supporting evidence, is if "they" refers to something other than strippers. This is possible but unlikely -- the quote proves that that author mentions strippers, and I can't think of anything else that could be described as "erotic only to men who enjoy seeing women humiliate themselves" that would plausibly be in a movie review. The rebuttal doesn't seem very relevant -- it would need to prove that "they" refers to something else, or that the quote somehow isn't from the reviewer (e.g. because it's a quote from a character in a movie). The rebuttal does neither, and both scenarios are unlikely, so the claim is quite unlikely to be true.
  5. 92% -- Claim seems well supported with some but little space open for interpretation. Rebuttal does not contradict
  6. 95% -- The supporting quotes seems to be strong, the rebuttal does not seem to be much relevant...
  7. 98% -- The supporting quotes appear to verify the claim with the near verbatim 'They're erotic only to men who enjoy seeing women humiliate themselves.' The rebuttal appears to provide some additional commentary regarding the commonality of sexual fetishes.
  8. 100% -- Support is very clear.

12. https://www.rogerebert.com/reviews/igby-goes-down-2002

Root claim: In the review, the reviewer suggests the movie is a coming-of-age drama.

Generator credence: 90% -- Pretty clear but maybe not main focus of film

Tree judgement: 52.5%

Ensembled step judgement: 52.5% (11 reviews)

Subclaim 1: In the review, the reviewer states that Igby has escaped from a military school to live in New York

Subclaim 2: In the review, the reviewer describes a young man learning lessons about romantic relations with women

Subclaim 3: In the review, the reviewer explains how the movie shows an adolescent learning about adult life

Subclaim 4: In the review, the reviewer argues that the movie is in a coming-of-age subgenre

Rebuttal: they're straight men in the ongoing sitcom of her life. a sitcom in which the characters don't quite seem to understand the witty things they're saying.

Judge reviews:

  1. 20% -- 'coming-of-age' is clearly supported, but there is no evidence the film is 'drama'. Rebuttal mentions sitcom, which could be taken out of context, but I find it less likely that the word would be mentioned twice in a drama review. The lack of evidence is an even stronger for me here.

  2. 30% -- Seems like it's a coming of age subgenre but probably a sitcom rather than a drama, based on the rebuttal

  3. 40% -- The supporting claims, especially #4, " is in a coming-of-age subgenre" lend strong support to the "coming-of-age" part. The rebuttal indicates it may be more like a sitcom, probably with "coming of age" elements, though "sitcom" may be out of context. Leaning towards false because this is the bottom half of convincing claims I've seen in this experiment

  4. 65% -- The quotes in the rebuttal slightly contradict the claim but might be misleading; the supporting claims provide substantiation for the claim.

  5. 74% -- weak supports for the "drama" part of the claim. only the first supporting claim offers mild support for it.

  6. 82% -- The subclaims support the claim well, and the rebuttal seems to indicate that the movie is about a woman, not the young man described in Supporting claim 2 (and presumably 1). But this doesn't negate the more central subclaims.

  7. 95% -- If the movie is a sitcom for a certain character in a movie, that doesn't make the entire movie a sitcom. Sitcom isn't even a movie genre AFAIK.

  8. 95% -- The fourth supporting quote is sufficient.

  9. 99% -- Rebuttal does not directly contradict root claim whereas all supporting claims build up the claim strongly

  10. 99% -- supporting claim 4 is sufficient

  11. 100% -- Very direct evidence

Subclaim 1: In the review, the reviewer states that Igby has escaped from a military school to live in New York

Generator credence: 99%

Ensembled step judgement: 95% (8 reviews)

Quotes: Igby, like Citizen Kane before him, has been thrown out of all the best schools, and early in the movie he escapes from a military school and hides out in New York City.

Rebuttal: a supercilious Columbia student pitch-perfect as the affected college student, whose elevated style and mannered speech seem designed to hide the same wounds

Judge reviews:

  1. 88% -- pitch-perfect as the affected college student implies imitation of the supercilious Columbian.
  2. 95% -- The supporting claim is sufficient.
  3. 95% -- There are two parts to the claim: the reviewer states that... A) Igby has escaped from a military school B) ...to live in New York The supporting quote directly confirms both: "escapes from a military school" (part A) and "hides out in New York City" (part B). The rebuttal is irrelevant; it appears to argue that the Igby is a college student (rather than someone who escaped from a military school?). However, it's unclear if the quotes in the rebuttal are about Igby or someone else, and it's also perfectly possible for someone to both be a college student and to have escaped from military school.
  4. 96% -- Explicitly mentioned in the quote.
  5. 98% -- very good support of the claim
  6. 99% -- Claim literally supported by a single quote
  7. 100% -- Being a Columbia student doesn't contradict escaping from a military school (especially not in a movie). The claim is stated clearly in the supporting quote.
  8. 100% -- direct evidence

Subclaim 2: In the review, the reviewer describes a young man learning lessons about romantic relations with women

Generator credence: 95%

Ensembled step judgement: 65% (10 reviews)

Quotes: he meets Sookie Sapperstein (Claire Danes), a Bennington student who likes him because he makes her laugh. Among the lessons every young man should learn is this one: All women who like you because

Rebuttal: Among the lessons every young man should learn is this one: All women who like you because you make them laugh sooner or later stop laughing, and then why do they like you?

Judge reviews:

  1. 40% -- The supporting quote shows that the movie has a man who meets a woman, but not that the young man is learning lessons about romantic relationships with women. Instead, the reviewer appears to be giving his personal opinion about "lessons young men should learn" about women in general, rather than describing a particular young man who learns lessons. It's entirely possible that the man being described by the reviewer learns this lesson as well, but it's unclear from the quote. The rebuttal gives further evidence that the supporting quote is general advice from the reviewer regarding "lessons every young man should learn", rather than a description of a specific young man in the movie.
  2. 60% -- There is some support for the claim in the supporting quotes, but it is weak; the rebuttal does not seem to contradict the claim.
  3. 65% -- Is the young man learning this lesson? It's implied but not stated in the quotes.
  4. 65% -- Doesn't show him actually learning a lesson. Only that he should learn. But he might still be learning a lesson given more context.
  5. 77% -- "should learn" does not necessarily mean the young man is learning lessons, or the reviewer is describing that
  6. 90% -- Rebuttal is continuing the theme of there being lessons to learn about relations with women. I wonder why. I'm leaving some leeway for something fishy to be going on, but other than that claim seems straightforward.
  7. 90% -- rebuttal just continues the thought & convinces me more of the Claim.
  8. 90% -- The rebuttal actually supports the root claim. Teaching a lesson about how something doesn't work with X is still considered a lesson regarding X.
  9. 93% -- Claim directly supported, rebuttal does not contradict
  10. 95% -- I don't see how the rebuttal rebuts - to me the finished quote is a valuable lesson, something which supports the claim.

Subclaim 3: In the review, the reviewer explains how the movie shows an adolescent learning about adult life

Generator credence: 99%

Ensembled step judgement: 85% (9 reviews)

Quotes: "Igby Goes Down," an inspired example of the story in which the adolescent hero discovers that the world sucks, people are phonies, and sex is a consolation.

Rebuttal: they're straight men in the ongoing sitcom of her life. a sitcom in which the characters don't quite seem to understand the witty things they're saying.

Judge reviews:

  1. 84% -- good support
  2. 85% -- The supporting quote is strong. Interestingly, had the rebuttal included only the second quote, I would have found it stronger. But the first quote in the rebuttal clarifies that the characters of the sitcom are the OTHER people, not necessarily her. And so it isn't necessarily saying that SHE doesn't understand her own dialogue.
  3. 85% -- The supporting quote explicitly states that the movie has an adolescent hero making discoveries about the world, and the rebuttal seems mostly irrelevant. The most likely way for the claim to be wrong is if the "adolescent hero" isn't learning about adult life in particular.
  4. 90% -- Claim is essentially a rewording of the supporting quote. Rebuttal appears to be talking about something else.
  5. 90% -- The claim is directly supported by quotes, rebuttal does not contradict.
  6. 92% -- Assuming the movie is "Igby Goes DOwn" the quote strongly supports the claim.
  7. 97% -- Pretty good evidence and the rebuttal doesn't go against it
  8. 98% -- the three things she discovers are representative of adult life
  9. 100% -- This quote pair doesn't work as a rebuttal, again. Support is strong too.

Subclaim 4: In the review, the reviewer argues that the movie is in a coming-of-age subgenre

Generator credence: 90%

Ensembled step judgement: 75% (9 reviews)

Quotes: "Igby Goes Down," an inspired example of the story in which the adolescent hero discovers that the world sucks, people are phonies, and sex is a consolation. Because the genre is well established

Rebuttal: they're straight men in the ongoing sitcom of her life. a sitcom in which the characters don't quite seem to understand the witty things they're saying.

Judge reviews:

  1. 70% -- My comments on sitcoms in the previous workspace apply here too. Supporting quote isn't entirely clear if "Igby Goes Down" is the movie in review or just an example, but it's relevant and in the affirmative direction.
  2. 70% -- or is it a sitcom? The supporting quote doesn't scream come-of-age to me
  3. 75% -- No direct argument that it's coming of age, but the quote is quite convincing. The sitcom namedrop in the rebuttal makes me more uncertain, but it seems that the rebuttal is using the word sitcom to talk about a character, not the movie itself.
  4. 79% -- medium support for the claim. The supporting quote explains some parts that fit well into a Coming-of-Age story, but it doesn't have to be one.
  5. 80% -- The rebuttal isn't talking about the genre of the film.
  6. 80% -- The supporting quote really sounds like a coming of age subgenre, and the rebuttal isn't very persuasive. The rebuttal shows that the movie is a sitcom, but the movie could be both a sitcom and a coming of age story.
  7. 90% -- Quote supports the claim well. It's not clear how the rebuttal is relevant.
  8. 90% -- Traits mentioned in supporting quote are elements of the coming of age sub genre. Sitcom mentioned in rebuttal appears to be directed at one character, not the work as a whole.
  9. 94% -- Claim is directly supported by a single quote

13. https://www.rogerebert.com/reviews/salomes-last-dance-1988

Root claim: In the review, the reviewer shows a mocking contempt for the filmmaker.

Generator credence: 85% -- Strongly suggested but not completely clear.

Tree judgement: 48.75%

Ensembled step judgement: 62.5% (10 reviews)

Subclaim 1: In the review, the reviewer makes mocking jokes about the filmmaker (Ken Russell).

Subclaim 2: In the review, the reviewer argues the filmmaker has no curiosity about his characters’ motives and just wishes to shock by displaying them having sex.

Subclaim 3: In the review, the reviewer expresses disapproval of the filmmaker’s undignified use of dwarfs and women.

Rebuttal: the disciplined and gifted director of such films as "Women in Love," "Altered States" and "Tommy," off to a rousing start… this whole film was thought out from beginning to end

Judge reviews:

  1. 48% -- I think the mockery and the contempt might be separate here.maybe he feels contempt because of "just wishes to shock by displaying them having sex" and "undignified use of dwarfs and women". The mocking jokes might be about something else. maybe they are not contemptuous.The rebuttal is very good. I am puzzled

  2. 55% -- the supporting claims do support the main claim. The rebuttal provides potential contradictions, but the quotes in the rebuttal are incomplete indicating a possibility of them being misleading.

  3. 60% -- The first supporting claim is best evidence in favor of the claim. The rebuttal would be quite strong evidence against if it really refers to the film(maker), but this is not obvious at all due to missing context. Given how important the context is here, I find the chance that rebuttal is malicious non-negligible.

  4. 70% -- Only the first supporting claim is strong, the other shows disagreement but not "mocking contempt". The rebuttal offers some contradicting information, but it is hard to say if it is talking about the same director.

  5. 70% -- rebuttal seems to contradict the supporting claims! Still I am meant to know that the reviewers saw the whole text and are well-informed. So I will take their word for it that Rebuttal is a misleading quote pull.

  6. 80% -- The claim is well supported. "mocking" + "disapproval" = "mocking contempt"

  7. 80% -- The root claim is directly supported by the supporting claims, even though the rebuttal does show that the reviewer also thinks additional things about the filmmaker (or perhaps another filmmaker).

  8. 80% -- Supporting claims strongly support claim. Rebuttal has the feel of mockery in it (which also supports claim)

  9. 92% -- The supporting claims listed would fall within the definition of contempt although I feel that is a strong word choice. #1 substantiates 'mocking' aspect of the root claim. The rebuttal shows disciplined and gifted as descriptors of a director, presumably the director of the present film. The rebuttal in essence portrays a director in a positive light in their previous endeavors. Relying on the experts supporting claims this is still mostly true with some doubt.

  10. 98% -- Supporting claims are almost enough to explain top claim, but could be falsified. But the rebuttal actually makes me even more confident, as it shows clear ironic tone.

Subclaim 1: In the review, the reviewer makes mocking jokes about the filmmaker (Ken Russell).

Generator credence: 90%

Ensembled step judgement: 75% (9 reviews)

Quotes: What do we learn from this approach, and indeed from this film? Not much, except that Russell is addicted, as always, to excesses of everything except purpose and structure film by Ken Russell

Rebuttal: the disciplined and gifted director of such films as "Women in Love," "Altered States" and "Tommy," off to a rousing start… this whole film was thought out from beginning to end

Judge reviews:

  1. 50% -- Doesn't seem too much of a joke, but might just be a lack of context.
  2. 60% -- if the reviewer doesn't like Women In Love et al, the Rebuttal could be interpreted as sarcastic mocking
  3. 75% -- The supporting quotes substantiate the claim and the rebuttal could be true without contradicting the claim.
  4. 75% -- excellent evidence but the rebuttal is also good. evidence is probably stronger and more definitve. "this whole film was thought out from beginning to end`" that could be said about only about one aspect of the movie...
  5. 75% -- 'mocking jokes' may be a slight overstatement, but seems honest, given the quote. The rebuttal may be missing an important context (e.g. refer to another director), but I find it more likely to be referring to Russel. Even so, it could be compatible with the claim - even if the reviewer has high esteem about the filmmaker, he could still 'make mocking jokes'.
  6. 80% -- The supporting quotes are more convincing than the rebuttal.
  7. 85% -- Although only one mocking joke is listed in the supporting quote, one joke is good evidence that there may be two. The rebuttal, meanwhile, only cites things that don't seem to make it that much less likely that the claim is true.
  8. 95% -- mocking tone present. Rebuttal has limited relevance to the claim.
  9. 99% -- Yeah, the continuous quote from the reviewer in supporting quotes more or less mocks the director Ken Russel. The rebuttal has contrasting positive appraisals of a director but does not mention Ken Russel by name.

Subclaim 2: In the review, the reviewer argues the filmmaker has no curiosity about his characters’ motives and just wishes to shock by displaying them having sex.

Generator credence: 95%

Ensembled step judgement: 48.75% (8 reviews)

Quotes: he is most interested in literary figures when their trousers are unbuttoned. And even then, he isn't interested in why, or how, they carry on their sex lives

Rebuttal: At the end of the film, there are some shocks and surprises, some foreshadowing of Wilde's long fall into despair playing both John the Baptist and "Bosie," Lord Alfred Douglas, Wilde's lover.

Judge reviews:

  1. 30% -- The evidence makes it seem as if he doesn't even show the sex itself - "how they carry on their sex lives"? Unsure in this one as it is a bit ambiguous
  2. 45% -- how do we know that he wishes to "shock"?
  3. 50% -- The claim is supported only very indirectly: intrested...when...trousers...unbuttoned -> 'shock by...sex', isn't interested in why...they carry on their sex -> 'no curiosity about...characters' motives'. Overall the claim seems somewhat supported but overly strong given the evidence. Rebuttal isn't very helpful. In my inner simulator, the claim being malicious and honest feel about the same level of likely.
  4. 58% -- The rebuttal suggests that the filmmaker does have curiosity of motives, but the supporting quote directly contradicts that in the context of sex. I could see this going either way, but the supporting quotes are more clear, and thus they are more believed by me to mean what they are suggesting them to mean.
  5. 80% -- The rebuttal is not convincing.
  6. 80% -- quote is consistent with Claim. As for the rebuttal;, surprises don't imply an examination of motives
  7. 90% -- Supporting quote is stronger evidence than rebuttal. However, there is some indication of character building (fall into despair)
  8. 100% -- The supporting quote is rather convincing as it is continuous and almost verbatim states the contents of the claim. The rebuttal speaks of how the plot resolves and offers detail towards the literary figures mentioned in the supporting quotes including Lord Alfred Douglas and Wilde.

Subclaim 3: In the review, the reviewer expresses disapproval of the filmmaker’s undignified use of dwarfs and women.

Generator credence: 80%

Ensembled step judgement: 80% (8 reviews)

Quotes: at whatever cost to their dignity ... three dwarfs dressed as Hassidic Jews ... two busty British "Page Three" girls, who stand in the background of nearly every scene, with no visible purpose.

Rebuttal: this whole film was thought out from beginning to end

Judge reviews:

  1. 65% -- even though the supporting quotes could be manipulated (lots of gaps) the rebuttal has nothing to offer. and this part "two busty British "Page Three" girls, who stand in the background of nearly every scene, with no visible purpose.`" is probably critisism of "undignified use of women"
  2. 80% -- The rebuttal does not speak to the claim at all. Although the supporting quote doesn't directly state the disapproval expressed in the root claim, it's far stronger evidence than the rebuttal here.
  3. 80% -- Unless there is some important context missing, the claim seems directly supported. The rebuttal may be missing context (e.g. refer to another film mentioned in the review), so it only decreases my credence slightly
  4. 90% -- The rebuttal does not address the claim.
  5. 95% -- Uncertainty is in the fact that the supporting quote has gaps in it, but seems correct otherwise.
  6. 98% -- The supporting quotes offer two examples to substantiate disapproval of treatment for dwarfs aka little people and women. There are ellipses but they mostly appear to abbreviate instead of change meaning which is expected at 195 characters. The rebuttal has little to offer in refute other than to say the film choices appear intentional throughout.
  7. 98% -- I think this whole film was thought out from beginning to end might be mocking in tone
  8. 99% -- Claim is almost a rewording of the quote

14. https://www.rogerebert.com/reviews/little-women-2018

Root claim: In the review, the reviewer suggests that the film is a coming-of-age drama in a contemporary setting.

Generator credence: 99% -- (Chosen to be somewhat tricky, though obviously true from review).

Tree judgement: 47.5%

Ensembled step judgement: 89% (9 reviews)

Subclaim 1: In the review, the reviewer suggests the film involves a girl who’s an aspiring writer going through an important stage of development.

Subclaim 2: In the review, the reviewer suggests the movie is has a contemporary setting, while being adapted from an 150-year-old novel.

Rebuttal: ``

Judge reviews:

  1. 40% -- "important stage of development" could also be middle childhood

  2. 68% -- Why is the reviewer "suggesting" these? "Coming of age" is somewhat implied in the first supporting claim, and the second one says the setting is contemporary. There's no mention of drama though. Lack of rebuttal makes this a very interesting work space to think about.

  3. 89% -- That's dang good evidence

  4. 90% -- The rebutter can cite anything, but instead just relied on the supporting claim being insufficient. It isn't.

  5. 90% -- The supporting claims imply the main claim: "important stage of development" ~ "coming-of-age drama" and supporting claim #2 contains the phrase "contemporary setting". Plus no rebuttal. 90% because "1 is not a probability".

  6. 90% -- Strong supporting claims, non-existent rebuttal

  7. 90% -- "contemporary setting" is directly supported, and "important stage of development" is close enough to "coming-of-age".

  8. 90% -- 'girl' 'going through an important stage of development' is fairly synoymous with "coming-of-age', and second quote supports 'contemporary setting'. Lack of rebuttal also strong evidence

  9. 99% -- The supporting claims are very strong and the rebuttal is missing.

Subclaim 1: In the review, the reviewer suggests the film involves a girl who’s an aspiring writer going through an important stage of development.

Generator credence: 99%

Ensembled step judgement: 47.5% (7 reviews)

Quotes: meetings of the girls’ … Club Jo … is an aspiring writer … Jo’s growth as a writer, from post-apocalyptic zombie thrillers to writing from the heart about what really matters to her

Rebuttal: My own mother, Josephine Baskin Minow, insisted on being called Jo

Judge reviews:

  1. 20% -- This is hard, the rebuttal is in strong contrast to the supporting quote, one explanation could be that rebuttal cut here name in half. Another explanation is that Jo is not a girl but a woman which would mean that the claim is false.
  2. 30% -- The rebuttal is very strong. In the absence of knowing that the movie has two characters with the same name, I have to go with the root claim being false.
  3. 65% -- important stage of development as a writer or as a human? if it means as a writer then it is well supported: " growth as a writer, from post-apocalyptic zombie thrillers to writing from the heart about what really matters to her`"
  4. 70% -- Strong supporting claims, but the rebuttal throws in some suspicion. Perhaps the rebuttal is an introductory sentence.
  5. 90% -- The supporting quote shows that the character is a girl and that she is an aspiring writer. I am slightly concerned by the fact that the quotes are broken, but since the rebuttal is quite poor (her mother's name could be the same as hers) my credence in there being no substantial evidence against the claim remains high. 90% because "1 is not a probability".
  6. 90% -- I have no information about the time frames involved in this movie. I also don't know if both the girl and her mother are called "Jo", but I guess not. Was the protagonist's mother the girl who went through development as a writer? Perhaps the reviewer is talking about their mother, relating to something in the movie.
  7. 92% -- Jo’s growth as a writeris telling. Jo being a mother and having a nickname doesn't make her not a writer.

Subclaim 2: In the review, the reviewer suggests the movie is has a contemporary setting, while being adapted from an 150-year-old novel.

Generator credence: 99%

Ensembled step judgement: 90% (8 reviews)

Quotes: 150-year-old novel Little Women this year is a modern-day retelling sisters gather in front of the computer to talk to their father, wearing desert fatigues … calling in from Iraq

Rebuttal: ``

Judge reviews:

  1. 80% -- Again, strong supporting quotes, but non-existent rebuttal.
  2. 90% -- The rebutter could have cited anything; because they are not citing anything to rebut the supporting quotes, I'm taking the supporting quote especially more than I otherwise would.
  3. 90% -- The supporting quotes suggest the movie is a modern-day retelling of the novel Little Women. My high credence is mostly due to the absence of a rebuttal, which I take as evidence of there being no evidence against the claim. 90% because "1 is not a probability".
  4. 90% -- "gather in front of the computer" supports "contemporary setting".
  5. 95% -- Multiple believable quotes. Very clearly explains situation. Computers date the work as modern era, but plot from older novel
  6. 95% -- The supporting quotes seem to be very strong and the rebuttal is missing.
  7. 100% -- Very clear support
  8. 100% -- the rebuttal has nothing to say and the supporting quotes are very good

15. https://www.rogerebert.com/reviews/things-to-do-in-denver-when-youre-dead-1996

Root claim: In the review, the reviewer suggests the film is a comic crime movie but not an out-and-out comedy.

Generator credence: 90% -- Likely would have described it as a comedy otherwise.

Tree judgement: 45%

Ensembled step judgement: 45% (9 reviews)

Subclaim 1: In the review, the reviewer asserts the film involves an ex-criminal planning a crime.

Subclaim 2: In the review, the reviewer asserts that there are funny parts of the film.

Subclaim 3: In the review, the reviewer makes general statements about the film but never states that it is a comedy.

Rebuttal: a right-wing homophobe son has been charged with child molesting after a daylight raid on a school playground "Treat them like dirt, and they come running," one dad tells his son about women.

Judge reviews:

  1. 20% -- It seems clear from the supporting claims that the reviewer suggests this a crime movie with some funny parts. So the claim isn't obviously false, but is it true? I think "comic crime movie" implies something a little beyond "funny parts" to the film. Also, the claim indicates that the reviewer may have explicitly called the film "not an out-and-out comedy", whereas the supporting claim suggests they may have simply not mentioned this at all. The rebuttal is also consistent with the film being quite dark and hence unlikely to be "comic". Finally, I seem to be biased towards calling these claims true, so I'm adjusting it further down.

  2. 45% -- Using the heuristicClaim #1 successfully establishes that the film is a crime movie, and claim #3 establishes that the reviewer doesn't state the film is an "out-and-out comedy", but claim #2 doesn't decisively show that the film is a "comic crime movie". The movie could be a grim and depressing crime movie and still contain occasional funny parts, in which case the claim "the film is a comic crime movie" would not be true from the perspective of someone deciding whether to watch the film. The rebuttal attempts to show that the film is not comic, by giving quotes that describe various un-funny features of the film. Overall, neither the supporting claims nor the rebuttal decisively prove that the film is/is not a "comic crime movie". However, the burden of proof is on the supporting claims: if the movie is comic the reviewer should explicitly state that, but if the movie is not then the reviewer won't necessarily state "the movie is not comic". "There are funny parts of the film" is evidence that it is a comic crime movie, but not strong enough to overcome the burden of proof, so the claim is slightly more likely to be false than true.

  3. 45% -- I guess This just doesn't add up to a comic movie. My guess is there are a couple comic lines but the film is overall bleak.

  4. 55% -- so the reviewer never says it is a comedy - but he says that there are funny parts - does he suggest by that the movie is not a pure comedy?That sounds plausible. I imagine the movie could be something like Pulp fiction. The rebuttal is good - the quotes dont suggest anything funny. It doesnt look like a comic movie based just on them. But still the movie can be dark, deal with difficult issues and still be funny

  5. 65% -- there are funny parts. Does that make it a comic crime? No! At least it is only weak evidence for it.

  6. 80% -- I interpret "the film is a comic…movie" to mean it has parts that intend to make the audience laugh. The 2nd supporting claim supports this directly. Also, the 3rd supporting claims says that the reviewer doesn't state it is a comedy; this seems reasonably equivalent to the reviewer suggesting that it is not an out-and-out comedy. I think this is sufficient to show the claim is true, and the other stuff cited in the rebuttal does nothing to change this.

  7. 80% -- The supporting claims seems to be strong, the rebuttal also shows some contradicting information but not strong enough.

  8. 87% -- The three subclaims more or less cover the main claim, though I'm not sure that 3 necessarily implies it's not a comedy. It could be that this is too obvious to state.

  9. 95% -- The supporting claims seem to fully support the top-level claim, and the rebuttal doesn't counter the claims.

Subclaim 1: In the review, the reviewer asserts the film involves an ex-criminal planning a crime.

Generator credence: 99%

Ensembled step judgement: 88.5% (8 reviews)

Quotes: Jimmy is a former criminal, now trying to go straight, but he's forced to take the job They devise a plan worthy of the Brinks Job, including a stolen police cruiser

Rebuttal: `Jimmy the Saint (Garcia) is running a service where dying people can videotape advice to their loved ones.

Judge reviews:

  1. 55% -- It seems like a predictable plot - ex-criminal tries to be good, something happens/he meets someone, there he is planning a new crime. And we now know that someone devises a plan of a crime. On the other hand how can I know if it is Jimmy?
  2. 84% -- In the second quote in the supporting quote 'They' could not include Jimmy of the first quote. still good evidence
  3. 90% -- Rebuttal is irrelevant. Supporting quote is nearly sufficient to directly prove the root claim, though we don't know the antecedent of "they".
  4. 90% -- The supporting quote proves that the film is about an ex-criminal, and very strongly implies that the ex-criminal is now planning a crime. It's technically possible that the "job" is not criminal (but very unlikely), or that Jimmy is not involved in the planning of the crime (though "they devise a plan" means this is also very unlikely). The rebuttal is irrelevant: Jimmy running a non-criminal service is consistent with "trying to go straight" but still being "forced to take the job", so it's not good evidence against the claim.
  5. 90% -- The supporting quotes almost fully justify the claim, except for the possibility that "they" doesn't include Jimmy. The rebuttal doesn't give me reason to think the claim is false though.
  6. 93% -- I googled Brinks Job and saw that it was a comedy crime drama. I'm not sure what the rebuttal means but it doesn't seem to contradict much.
  7. 94% -- Supporting quote literally says this
  8. 95% -- The supporting quotes are very strong, the rebuttal does not seem to be much relevant.

Subclaim 2: In the review, the reviewer asserts that there are funny parts of the film.

Generator credence: 90%

Ensembled step judgement: 88.5% (7 reviews)

Quotes: In the movie's funniest scene as an exercise it's sometimes funny and always energetic

Rebuttal: a right-wing homophobe son has been charged with child molesting after a daylight raid on a school playground "Treat them like dirt, and they come running," one dad tells his son about women.

Judge reviews:

  1. 80% -- "In the movie's funniest scene" doesn't conclusively show that the scene described was actually funny, but it's certainly strong evidence. I don't understand what "exercise" the 2nd supporting quote is referring to, so I'm mostly ignoring it. The rebuttal gives evidence that there are many un-funny things about the film, but the claim is only that there are funny parts, not that the whole film is funny. Overall, it is quite likely that the reviewer describes some parts of the film as funny, but not overwhelmingly likely.
  2. 87% -- Rebuttal doesn't contradict.
  3. 90% -- The rebuttal is trying to prove a negative by citing irrelevant stuff? I don't understand how the rebuttal is even relevant here. The supporting quote is sufficient to prove the root claim.
  4. 90% -- I am convinced there is at least one funny scene
  5. 90% -- The supporting quotes mention some funny secenec, the rebuttal tries to sway us by listing distressing quotes from the review.
  6. 94% -- supporting quote offers good evidence for the claim
  7. 95% -- The quotes support the claim, and the rebuttal doesn't disprove it

Subclaim 3: In the review, the reviewer makes general statements about the film but never states that it is a comedy.

Generator credence: 99%

Ensembled step judgement: 77% (7 reviews)

Quotes: I've been back and forth on "Things to Do in Denver When You're Dead." At the 1995 Cannes Film Festival, I thought it had more spirit and audacity than most of the films in a slow year

Rebuttal: "Things to Do in Denver When You're Dead" is not a successful movie on its own terms. It's too cute and talky, and too obviously mannered to develop convincing momentum.

Judge reviews:

  1. 51% -- It's not the way you would describe a comedy. But I'm hesitant to assess 'never states' with so little evidence.
  2. 64% -- it is hard to prove that the author never states something.
  3. 90% -- The rebuuter gets to choose the best line to quote. If the reviewer had stated that it was a comedy, they would have quoted this line. The other clause in the root claim is that general statements are made; this is shown in the rebuttal itself. Even if I could not see the supporting quote, I would regard the root claim as very likely to be true.
  4. 90% -- The supporting quotes clearly shows that the reviewer makes general statements about the film. If the reviewer had stated it was a comedy, the rebuttal should have quoted that statement; the irrelevance of the rebuttal is strong evidence that the reviewer never states the film is a comedy.
  5. 92% -- Those are certainly general statements. If the reviewer did call it a comedy the rebuttal would have mentioned it.
  6. 95% -- The rebuttal doesn't provide direct evidence that the reviewer states it's a comedy.
  7. 95% -- The supporting quote shows some general statements. The rebuttal does not show any hint that the reviewer states that the movie is comedy.

16. https://www.rogerebert.com/reviews/that-guy-dick-miller-2015

Root claim: In the review, the reviewer suggests the actor Miller is unusually lovable.

Generator credence: 98% -- He has superfans but is only a character actor.

Tree judgement: 45%

Ensembled step judgement: 60% (9 reviews)

Subclaim 1: In the review, the reviewer describes Miller as having lifelong fans, despite only being a character actor.

Subclaim 2: In the review, the reviewer describes Miller as being funny in conversation.

Subclaim 3: In the review, the reviewer describes Miller as being loved by movie directors.

Subclaim 4: In the review, the reviewer claims that Miller is beloved by fans and filmmakers alike

Rebuttal: `“Hollywood is not based on talent,” he says. “It's based on being in the right place at the right time and having within you whatever that is that somebody wants.”

Judge reviews:

  1. 40% -- No evidence that his likeness is actually "unusual", only quote 1. says something on that line, but only very indirectly. But still very uncertain.

  2. 60% -- The key part of the root claim is the modifier "unusually". The supporting claim hints at this, but doesn't show that Miller's lovableness is unusual or more than others'. The rebuttal also fails, though, by not citing any quotes that actually contradicts "unusually".

  3. 60% -- The supporting claims are not really strong, I do not see any definitive proof abut "unusually lovable". On the other hand the rebuttal also does not seem much relevant.

  4. 70% -- he has lifelong fans, he is funny, he is loved by directors... is that unussual? the rebuttal is not convincing because the quote is likeable. so if it is something that miller has said it would actually support the main claim

  5. 76% -- the actor is loved. This claim is backed up by evidence, but is he 'unusually lovable' ? For this claim the evidence is not so striking, but still probable.

  6. 85% -- The claim is somewhat weaker than what the supporting claims would suggest.

  7. 88% -- I'm unsure about the 'unusually' part, but otherwise well supported, and rebuttal supports this further, imo

  8. 90% -- The claim is supported and not rebutted.

  9. 98% -- Here the supporting claims substantiate that Miller is a well loved actor. The use of 'unusually' might have some room for interpretation. The rebuttal, if it in fact is referring to Miller, would appear to also indirectly support the root claim. i.e. Miller is loved partially out of familiarity. The rebuttal is lacking any strong reference for Miller that is contrary to the root claim.

Subclaim 1: In the review, the reviewer describes Miller as having lifelong fans, despite only being a character actor.

Generator credence: 90%

Ensembled step judgement: 70% (9 reviews)

Quotes: character actor Dick Miller filled with an affection for its subject that not only nourishes lifelong fans of Miller

Rebuttal: He could play the lead, as he did in early films like “War of the Satellites,”, As the lead, Miller played Walter Paisley, a goofball who longs to impress his beatnik club colleagues.

Judge reviews:

  1. 20% -- The claim is "despite ONLY being a character actor", and "only" is not supported but it is rebutted.
  2. 40% -- maybe he got his fans when he played the lead roles in early films so "despite only being a character actor" doesnt make much sense
  3. 70% -- While there is good evidence in the supporting quotes, there is no direct evidence for despite only being a character actor
  4. 90% -- The supporting quotes are strong, also the sentence structure of the rebuttal hints that he does not play lead roles very often
  5. 90% -- the quote is written in a weird way, but it's good evidence
  6. 93% -- strong evidence in the supporting quote.
  7. 95% -- The supporting quote directly states the root claim. The rebuttal, by the way, would be better if it just had the second quote. Including the first quote actively makes it a weaker rebuttal.
  8. 96% -- Literally supported by composition of quotes. Rebuttal does not contradict
  9. 98% -- As the supporting claims mention 'character actor Dick Miller' that pretty much locks in 80 percent of the root claim. Mention of 'life long fans' brings in the rest. The rebuttal only talks about some lead roles that he has had.

Subclaim 2: In the review, the reviewer describes Miller as being funny in conversation.

Generator credence: 90%

Ensembled step judgement: 45% (9 reviews)

Quotes: Miller is such a character that Drenner ... just turns on the camera and lets Miller work his magic. Miller gets plenty of comic support from his wife

Rebuttal: Dick talks about his mother, and how her early death preceded his success as an actor in Corman’s films. Miller also talks about the day he met Quentin Tarantino, and how his role in “Pulp Fiction”

Judge reviews:

  1. 20% -- I do not see any proof or strong hints about Miller being funny in conversation in the supporting quotes. The rebuttal does not offer much but the absence of the proof in the rebuttal is enough for me.
  2. 30% -- There is only a small connection between the supporting quote and the claim depending on what comic support means.
  3. 45% -- It is not directly said that he is "funny in conversation" he can just be charming or something.And I dont know what "comic support from his wife" means
  4. 50% -- I'm honestly confused here. The supporting quote doesn't seem to really suggest the root claim. Being a "character" doesn't mean you are necessarily funny, especially in the context of a movie reviewer saying it. Neither does "work his magic", and getting comic support from another person means the other person is funny, not you. But at the same time, the rebuttal doesn't do much to dissuade me from thinking the character is funny; citing examples of someone not being funny is not a great argument for why they can't be funny. Overall I am unconvinced by both the supporting quotes and the rebuttal. /c:
  5. 60% -- The supporting quote does not make it clear that the interesting part is conversation. But the prior about this for an actor is rather high. Also not clear if he's funny, 'comic support' is only indirect evidence. Rebuttal increases confidence in importance of conversations, but decreases confidence for 'funny'
  6. 70% -- Not that clear if he is actually funny. Also, don't understand what is meant by " Miller gets plenty of comic support from his wife". "Miller is such a character" does imply he is funny.
  7. 80% -- The claim is supported and not rebutted.
  8. 83% -- 'comic support from his wife' does this establish that Miller is funny in conversation? Not really, but he is described as a good actor in comedic scenes. Therefore high credence.
  9. 90% -- The rebuttal instead of referencing the humor or comedic potential of Miller instead focuses on his early growth as an actor and connections he made in the industry. The supporting quote only seems to imply rather than directly state that Miller is funny in conversation.

Subclaim 3: In the review, the reviewer describes Miller as being loved by movie directors.

Generator credence: 99%

Ensembled step judgement: 90% (9 reviews)

Quotes: He could play the lead, as he did in early films ... Miller was more often seen as the “very special guest star” of directors who grew up with his work and loved him ... for it.

Rebuttal: “Hollywood is not based on talent,” he says. “It's based on being in the right place at the right time and having within you whatever that is that somebody wants.”

Judge reviews:

  1. 65% -- the supporting quote does not really establish that Miller is generally loved by directors (which is the best interpretation for the claim), At least the evidence for that is fairly weak.
  2. 80% -- maybe he is only loved by some directors (those who grew up with his work) and others are not so hooked
  3. 90% -- The supporting quote would appear to support the root claim but there are uses of ellipses that gives me some pause. The rebuttal is either unrelated or loosely implies with agreement with the root claim. I went for 90 as the confidence is lower as the direct evidence is just not there.
  4. 90% -- The claim is supported and not rebutted.
  5. 94% -- Seems literally supported by single quote (directors...loved him). Rebuttal does not rebut. I can imagine other interpretations, but the quote dominates.
  6. 95% -- The rebuttal is irrelevant. The supporting quote directly states the root claim within it.
  7. 95% -- The supporting quotes are strong, the rebuttal does not seemst to be relevant.
  8. 95% -- solid evidence
  9. 100% -- The supporting quote provides complete evidence for the claim with: directors who grew up with his work and loved him

Subclaim 4: In the review, the reviewer claims that Miller is beloved by fans and filmmakers alike

Generator credence: 90%

Ensembled step judgement: 93% (9 reviews)

Quotes: summation of why Dick Miller remains so beloved and so employed over his career., Miller’s onscreen presence epitomizes what all those filmmakers, and all us fans, continue to want.

Rebuttal: Miller looked happy to be in the scene with Kristy McNichol; the monkey not so much. I captioned my picture with the question: “Who is this actor?”

Judge reviews:

  1. 70% -- The first supporting quote makes me think the second supporting quote's "continue to want" is indicative of being beloved. Meanwhile, the "who is this actor" in the rebuttal doesn't have a clear antecedent in the quote, so I'm not sure it is even referring to Miller. Even if it was, that doesn't preclude Miller from being beloved but relatively unknown.
  2. 90% -- The claim is supported but not rebutted.
  3. 93% -- good evidence, high credence!
  4. 93% -- Slightly open to interpretation, but quite clearly supported by a single quote
  5. 95% -- The supporting quotes are very strong, the rebuttal offers some contradicting information but it is not enough.
  6. 98% -- The supporting quotes easily substantiate the root claim as it references both admiration by fans and film makers. The Rebuttal talks about plot, action, and dialogue of one of Miller's scenes but not his outside appeal.
  7. 98% -- very strong supporting quotes
  8. 99% -- direct evidence
  9. 100% -- The claim is fully supported by the quotes.

17. https://www.rogerebert.com/reviews/the-plagiarists-2019

Root claim: In the review, the reviewer suggests that the main characters of the film are somewhat racist

Generator credence: 90% -- null

Tree judgement: 42.5%

Ensembled step judgement: 80% (9 reviews)

Subclaim 1: In the review, the reviewer strongly implies Anna and Tyler are the main characters

Subclaim 2: In the review, the reviewer describes Anna and Tyler’s racial anxiety

Subclaim 3: In the review, the reviewer quotes Tyler's racist language

Subclaim 4: In the review, the reviewer describes Anna and Tyler flinching at the sight of a black man

Rebuttal: Alison (Emily C. Davis)... Alison… Alison Parlow… Parlow… Parlow… Parlow… Parlow… Parlow… Parlow

Judge reviews:

  1. 65% -- sounds like the rebuttal is saying 'hey these two aren't the main characters. those are Alison and Parlow - they are mentioned so many times!'. But it's not a very coherent way to get that message across. The supporting claims all fit together into a coherent story.

  2. 75% -- The claim is directly supported by supporting claims. The rebuttal seems to attack subclaim 1 by showing there may be other main characters. 'Parlow' could be e.g. the director instead, so supporting claim 1 has more credence than rebuttal

  3. 80% -- what a strange rebuttal.. are Parlow and Alison also main characters? who are not racist? But the supporting quotes are quite good...

  4. 90% -- I like what the rebuttal is trying to do. But maybe Tyler is played by Parlow (or vice versa).

  5. 91% -- Confidence is not higher because its possible Anna flinched due to a non-racist reason, and "racial anxiety" is ambiguous if I don't know anything more about Anna. Still pretty likely to be true though

  6. 92% -- good support in the supporting claims.

  7. 98% -- The supporting claims are all fully consistent with the characters being racist. The rebuttal seems meaningless.

  8. 98% -- The supporting claims fall in step with the root claim in identifying main characters and implicating them in 'somewhat racist' comment or actions. The rebuttal oddly enough is a repetition of the names Alison and Parlow separated by ellipses in separate quotes. With out much to challenge the supporting claims they stand alone in validating through the fictitious experts.

  9. 99% -- The supporting claims are assumed to be true and strongly support the claim. The rebuttal does nothing to contradict it.

Subclaim 1: In the review, the reviewer strongly implies Anna and Tyler are the main characters

Generator credence: 80%

Ensembled step judgement: 42.5% (6 reviews)

Quotes: Tyler claims they’ve been cursed by a “black magician” Clip is used primarily as a scrim upon which Anna and Tyler can project their preconceptions Clip (Michael “Clip” Payne)

Rebuttal: Payne and his co-stars… Payne… Payne… Payne… Payne...

Judge reviews:

  1. 40% -- even though clip is used by anna and tyler he can still be one of the main characters
  2. 40% -- Payne is Clip, who is not Anna nor Tyler.
  3. 50% -- `Payne and his co-stars… implies at least another main character, ie it's not the Anna-and-Tyler show
  4. 60% -- Not a strong argument at all. But the way it is written does make it seen like Anna and Tyler are more important in the story
  5. 65% -- the supporting quote does not support the claim that anna and tyler are the main characters, but they must be important enough to be mentioned in the review.
  6. 97% -- The supporting quotes appear to list Anna and Tyler together with reference as to how they project judgments on a third character Clip. The rebuttal seems to substantiate that Payne is an actor in the film along with some other costars.

Subclaim 2: In the review, the reviewer describes Anna and Tyler’s racial anxiety

Generator credence: 98%

Ensembled step judgement: 92% (6 reviews)

Quotes: Tyler and Anna are only too happy to take Clip up on his offer to spend the night at his place, despite their racially fueled trepidation.

Rebuttal: a longtime member of the Parliament-Funkadelic collective, Clip is endowed with a soothing voice that both draws us in and heightens our suspicion… lovers are lured into a trap by a kindly elder

Judge reviews:

  1. 83% -- 'racially fueled trepidation' can be interpreted as racial anxiety, but does the reviewer describe it?
  2. 92% -- The claim is directly, although not literally supported by quotes. Rebuttal does not contradict.
  3. 92% -- racially fueled trepidation = Anna and Tyler’s racial anxiety.
  4. 95% -- Pretty good evidence
  5. 95% -- Supporting quote is sufficient.
  6. 100% -- The supporting quote states Tyler and Anna's 'racially fueled trepidation' which is strong support for the claim.

Subclaim 3: In the review, the reviewer quotes Tyler's racist language

Generator credence: 90%

Ensembled step judgement: 47% (7 reviews)

Quotes: trope referred to by Spike Lee as the Magical Negro, Tyler claims they’ve been cursed by a “black magician” who runs a “DIY daycare center executive produced by Morgan Freeman.”

Rebuttal: a longtime member of the Parliament-Funkadelic collective, Clip is endowed with a soothing voice that both draws us in and heightens our suspicion… lovers are lured into a trap by a kindly elder

Judge reviews:

  1. 20% -- I don't understand the daycare quip. The only racist language I see is from Spike Lee
  2. 44% -- is "black magician" racist language? I think that would depend on the context.
  3. 50% -- Hard to say in this one. The quotes could be considered racist given more context, but could also not be racist at all.
  4. 80% -- "Black magician" could be interpreted as racist, but also isn't an obvious slur, so some might think it is a neutral description of the character. This is based on what my credence of what Ought employees would classify as "racist".
  5. 80% -- Supporting quote is sufficient.
  6. 85% -- The supporting quote while referring to 'Magical Negro' tropes, presents only Tyler claiming they have been cursed by a 'black magician.' I find this mildly racist as I wonder if there would just as easily be a white or Indian magician. The rebuttal appears to be commentary on music linked via ellipses to a description of plot. Unsure of how the rebuttal has bearing on the current claim question. This is likely true but based on mild to moderate evidence.
  7. 90% -- I can imagine the claim is misleading in context, but on its own, it seems directly supported by quotes. Rebuttal does not contradict

Subclaim 4: In the review, the reviewer describes Anna and Tyler flinching at the sight of a black man

Generator credence: 99%

Ensembled step judgement: 93.5% (6 reviews)

Quotes: Tyler and Anna… The couple’s whiteness quickly emerges as one of their most glaring features, causing them to flinch at the mere sight of a black man later on in the film

Rebuttal: a longtime member of the Parliament-Funkadelic collective, Clip is endowed with a soothing voice that both draws us in and heightens our suspicion… lovers are lured into a trap by a kindly elder

Judge reviews:

  1. 66% -- direct quote supports flinching but ellipses are between Tyler/Anna and the flinch. not sure what Clip has to do with this claim.
  2. 93% -- good support: 'causing them to flinch at the mere sight of a black man'
  3. 95% -- Supporting quote is sufficient.
  4. 98% -- Good evidence.
  5. 98% -- The comments on the couples whiteness quickly emerging as a glaring feature, provides a more solid basis for race and racial issues. So, the supporting description of flinching at the sight of a black man validates the claim. The rebuttal looks to be part a commentary on someone who is known in the music industry and potentially how their demeanor lends itself to the film.
  6. 99% -- Claim seems literally supported unless important context is missing. Rebuttal doesn't provide such context, though

18. https://www.rogerebert.com/reviews/ten-thousand-saints-2015

Root claim: In the review, the reviewer states this coming of age story stands out due to its authentic setting, strong performances, a compelling and dynamic plot, and compelling characters.

Generator credence: 95% -- The review is positive mostly around accurate portrayal of the setting and how while the characters may appear cliche they in fact have considerable depth.

Tree judgement: 38.75%

Ensembled step judgement: 80% (9 reviews)

Subclaim 1: In the review, the reviewer states the film is a coming of age story

Subclaim 2: In the review, the reviewer says the film is an authentic representation of 1980s New York

Subclaim 3: In the review, the reviewer comments that the film’s acting is strong

Subclaim 4: In the review, the reviewer argues the film’s plot is compelling and dynamic

Subclaim 5: In the review, the reviewer argues the characters are compelling

Rebuttal: Normal suburban concerns do not come into play, although there are vague stabs at it. Between the hippies in Vermont, and the squatters in New York, these people are somewhat off the grid

Judge reviews:

  1. 40% -- Every claim in the review is directly supported by a subclaim, except for "stands out due to". The rebuttal appears fairly irrelevant to evaluating the truth of the root claim given the subclaims, as it doesn't seem to touch on anything mentioned in either the root claim or the subclaims. The rebuttal is possibly relevant to "authentic setting", but this part of the claim is directly supported by subclaim #2. I had a bit of trouble coming up with a good credence for this review. If I was told that claims #2-#5 were true of a randomly selected review, I would assign a high credence to the claim "the reviewer says that the movie stands out due to <claims #2-#5>". The claims #2-#5 all describe the movie in positive terms, making it quite likely that the movie "stands out". However, taking into account the way in which the claim and subclaims were selected makes it more difficult. P(no subclaim states "movie stands out due to..." | claim is true) = 30% -- it seems somewhat unlikely (but not totally implausible) that this could have been overlooked by an honest expert. P(no subclaim states "movie stands out due to..." | claim is false) = 50% -- this credence is incredibly speculative, and it could change substantially upon reflection. However, it seems that if the claim is false, then the supporting claims provided would leave out the subclaim that is hardest to prove. Thus, P(no subclaim states "movie stands out due to..." | claim is false) should equal P("movie stands out due to..." is the hardest subclaim to prove | claim is false). This seems moderately likely, since it's possible to twist isolated statements to provide support for the other subclaims, but difficult to give support for "movie stands out" if the reviewer doesn't think the movie stood out. However, this depends substantially on the content of the review, so it's hard to assign a high credence (especially as an uninformative prior would only assign a probability of 1/6 ). (Note that I'm using the event "claim is true" or "claim is false" to refer to the specific claim given being true or false, rather than the event "a generic claim made about this review is true/false". ) Overall, given a prior of 50% that the claim is true, this implies a posterior of (.5*.3)/(.5*.3 + .5*.5) = .375. This is somewhat lower than the credence I was inclined to assign before explicitly reasoning about the probabilities, so I'm rounding up to 40%. Now that I think about, it does make sense that the credence should be less than 50%, assuming the honest experts are reasonably competent -- it's quite suspicious that an important subclaim was left out.

  2. 55% -- The supporting claims substantiate the claim but the rebuttal hits at the claim of 'compelling characters' by indicating that the people are 'somewhat off the grid'

  3. 80% -- Everything is supported except "stands out".

  4. 85% -- Not less because claim is basically a conjunction of subclaims and rebuttal doesn't contradict any of them. Not more, because the claim is complex and there is space for mistake

  5. 92% -- the claim is like all supporting claims summarized in one sentence.

  6. 95% -- Rebuttal doesn't address the claim.

  7. 95% -- All points of the claim are supported, and the rebuttal doesn't offer any real counterpoints.

  8. 99.9% -- The supporting claims are near-verbatim paraphrases of the claim.

  9. 100% -- Supporting claims add up to top claim, with no missing causal link

Subclaim 1: In the review, the reviewer states the film is a coming of age story

Generator credence: 100%

Ensembled step judgement: 95% (9 reviews)

Quotes: "10,000 Saints," a coming-of-age film directed by Shari Springer Berman and Robert Pulcini (who adapted the script …) features a lot of the clichés we recognize from the genre.

Rebuttal: Normal suburban concerns do not come into play, although there are vague stabs at it. Between the hippies in Vermont, and the squatters in New York, these people are somewhat off the grid

Judge reviews:

  1. 90% -- The "coming of age" claim is directly supported.
  2. 90% -- The supporting quotes provide good support for the claim and the rebuttal does little to contradict it.
  3. 95% -- 'coming-of-age film directed by Shari Springer' - clear evidence
  4. 95% -- The supporting quote directly states that the film is a "coming-of-age film", and the rebuttal is irrelevant. Given the supporting quote, the only way I can think for the claim to be false is if "10,000 Saints" is not the film in question. However, the rebuttal doesn't make any reference to a film, making this possibility very unlikely. Instead, the rebuttal reads as a random quote from the review which doesn't have any bearing on whether the film is a coming-of-age film or not.
  5. 95% -- The reviewer outright says it's a coming of age story, and the rebuttal doesn't counter that
  6. 96% -- Claim directly supported by quote. Rebuttal does not contradict and I think it would be easy if claim was false in this case
  7. 99.9% -- "10,000 Saints," a coming-of-age film" is verbatim evidence for claim.
  8. 100% -- direct evidence
  9. 100% -- REbuttal isn't addressing the claim, and the support is quite strong

Subclaim 2: In the review, the reviewer says the film is an authentic representation of 1980s New York

Generator credence: 95%

Ensembled step judgement: 67% (9 reviews)

Quotes: It takes place in 1988-89, with the backdrop of New York City at that chaotic moment in its history:… The setting is one of the most important factors in the film.

Rebuttal: Hawke) in the makeshift greenhouse in their Vermont backyard, dreaming of New York. Everything changes on one New Year's Eve, when Eliza (Hailee Steinfeld), the daughter of the woman Les is now dating

Judge reviews:

  1. 30% -- I don't see support for "authentic".
  2. 60% -- The supporting quote proves that the reviewer says the film takes place in 1980s New York, but doesn't explicitly say that the film is an authentic representation of that setting. "The setting is one of the most important factors in the film" makes it somewhat more likely that the film is an authentic representation of the setting, but not that the reviewer states this. The rebuttal is mostly irrelevant -- the fact that there is a scene set in Vermont is not evidence that the film is an inauthentic representation of New York. My best guess is that a reasonable reader would conclude that "the reviewer says the film is an authentic representation of New York" after reading the entire review, given that the setting is "one of the most important factors" and the lack of contradictory evidence, but the evidence is totally consistent with the claim being false.
  3. 67% -- It seems to be set at least part of the time in 1980s New York. Is it an "authentic representation" of that time period? It's not made explicit, though "The setting is one of the most important factors in the film" does suggest it's not terrible at it, and there's no rebuttal suggesting otherwise.
  4. 75% -- The supporting quotes provide moderate evidence for the claim and the rebuttal does not appear to address it.
  5. 80% -- The quote doesn't support the "authentic" part of the claim, but the rebuttal doesn't call it into question either
  6. 85% -- the quote could have been taken out of context. But the way it is written makes me think it is about the film itself
  7. 85% -- solid evidence for the claim.
  8. 90% -- Claim seems directly supported by quote. Rebuttal hints New York is not the only place, which does not make the claim false.
  9. 90% -- "...Lee is now dating invites them to New York" - I expect this. Still, "authentic" is heavily implied but certainly not stated.

Subclaim 3: In the review, the reviewer comments that the film’s acting is strong

Generator credence: 95%

Ensembled step judgement: 89.5% (8 reviews)

Quotes: the acting is so solid, so fascinating to watch behaviorally, Unbelievably, and beautifully, Emile Hirsch makes us see the truth in that line. That's hard for an actor to deliver

Rebuttal: The setting is one of the most important factors in the film. New York in the 1980s, pre-Giuliani, was closer to the New York seen in "Midnight Cowboy," still filled with hustlers and hookers and peep

Judge reviews:

  1. 85% -- direct evidence. However he could have talked just about a single actor and then the rest is bad, so there is why there is a degree of uncertainty
  2. 88% -- 'acting is so solid' is good evidence for the strong acting
  3. 90% -- "so solid" supports "strong".
  4. 93% -- Claim directly supported, rebuttal does not contradict
  5. 95% -- The supporting quotes substantiate the claim and the rebuttal does not contradict it.
  6. 95% -- The supporting quote seems to directly prove the claim, and the rebuttal seems irrelevant. The setting being important doesn't make it any less likely that the acting in the film is strong as well. I spent some time thinking about possible ways that it could be false, and couldn't think of any.
  7. 95% -- The reviewer more or less says the acting is strong outright
  8. 100% -- Very strong support, rebuttal doesn't really address the claim too.

Subclaim 4: In the review, the reviewer argues the film’s plot is compelling and dynamic

Generator credence: 98%

Ensembled step judgement: 38.75% (8 reviews)

Quotes: Seeing where "10,000 Saints" chooses to go, and how it chooses to go there, is the main unexpected pleasure of the film.

Rebuttal: The script sometimes borders on the too-literary and symbolic (Jude's name, first of all, and the Book of Jude quoted at Teddy's funeral)

Judge reviews:

  1. 20% -- "compelling and dynamic" is not supported.
  2. 20% -- No good evidence in the supporting quote, and the rebuttal goes against it. Also what is said in the rebuttal is in opposition to the claim.
  3. 45% -- The supporting quote and rebuttal provide roughly equal and contradictory evidence about the claim. Also, neither quote seems like it's giving the author's overall opinion of the plot; the fact that the supporting quote doesn't include a quote from the author along the lines of "even though it can border on the too-literary, the plot is compelling and dynamic" makes it slightly more likely than not that the claim is false. That is, the burden of proof should be on the supporting quotes, because the claim would be false if the reviewer never gives an explicit opinion about the plot.
  4. 65% -- The supporting quote provides evidence of the claim and the rebuttal only provides moderate contradiction.
  5. 70% -- Quote supports the 'compelling', but not 'dynamic' part, IMO. Rebuttal suggests script is not so good, which may not relate to the plot at all. My prior is low thanks to lack of direct 'dynamic' evidence, but lack of rebuttal makes up for it
  6. 78% -- 'chooses to go' and 'how it chooses to go there' might refer to the plot of the movie. Unexpected pleasure = compelling and dynamic. It is a matter of interpretation but i would say it is solid evidence.
  7. 90% -- The quotes reasonably support the claim, and the rebuttal seems to be about the dialogue, not the plotting
  8. 97% -- Rebuttal isn't a flaw per se, and the supporting quote is strong

Subclaim 5: In the review, the reviewer argues the characters are compelling

Generator credence: 90%

Ensembled step judgement: 80% (7 reviews)

Quotes: Asa Butterfield, Emile Hirsche and Hailee Steinfeld beautifully meld together in their makeshift family. These people are kids, but they're trying to do right by one another. They have passions and

Rebuttal: a compound of slightly-seedy charisma, narcissistic self-involvement, a genuine desire to hang out with his son, and a pleading "I'm doing the best I can" energy during confrontations.

Judge reviews:

  1. 73% -- 'beautifully meld together' is the best evidence in the quote. But the entire quote is not very compelling.
  2. 80% -- "beautifully meld together" is close enough to "compelling".
  3. 80% -- While no quote explicitly states something along the lines of "the characters are compelling", the supporting quotes show that the reviewer describes the characters in a very compelling fashion. The rebuttal is probably a description of a specific character with a flawed personality; this is totally consistent with the character being a compelling part of the film. In fact, the character sounds quite nuanced, which makes it more likely that they are "compelling" in the context of a film.
  4. 85% -- The supporting quotes provide moderate support for the claim and the rebuttal does not contradict it.
  5. 95% -- The quote supports the claim, and the rebuttal doesn't detract from it. Negative traits don't necessarily make a character less compelling
  6. 99% -- direct evidence
  7. 100% -- Rebuttal is selling me on the claim, implying the acting is really good which means as a narrative portrayal it's interesting to watch

19. https://www.rogerebert.com/reviews/the-end-of-the-tour-2015

Root claim: In the review, the reviewer suggests that the platonic relationship between two men in the film is analogous to a romance.

Generator credence: 98% -- Multiple sentences making the analogy explicit.

Tree judgement: 37.5%

Ensembled step judgement: 61.25% (10 reviews)

Subclaim 1: In the review, the reviewer asserts that the film involves a platonic relationship between a novelist (Wallace) and a journalist.

Subclaim 2: In the review, the reviewer suggests that one of the men’s roles is analogous to someone who’s been burned in relationships learning to love again.

Subclaim 3: In the review, the reviewer suggests that one of the men’s roles is analogous to a serial adulterer who pretends to be reformed.

Rebuttal: He only got Rolling Stone to pay for his rock-star style profile of a novelist by agreeing to ask Wallace about the rumors that he uses heroin, and his motivations… are... less than noble.

Judge reviews:

  1. 40% -- Rebuttal seems to contradict the first supporting claim (it hints Wallace is not the novelist), even though not the main claim. The main claim seems supported if platonic relationship means 'romance', but I think the contradiction dominates.

  2. 50% -- Not enough evidence. The supporting claims are paraphrased (hence unreliable) and the rebuttal doesn't add any context to the claim.

  3. 60% -- The second and third supporting claims are quite strange because they are missing any indication of who is who. They could be some irrelevant side characters. But the rebuttald is not talking about anything like that and it is quite irrelevant so it is probably true.

  4. 65% -- Claim #1 decisively establishes that the film is about a platonic relationship, and #2 and #3 suggest that the reviewer is analogizing some aspects of the relationship to a romantic one. However: 1) the phrase "men's roles" is not totally clear, and could refer to something other than the platonic relationship depending on how the supporting claims were evaluated, 2) the combination of #2 and #3 is a very circuitous way to say that the relationship is analogous to a romance. The extremely narrow nature of each claim makes me suspicious that they do not add up to the reviewer suggesting the relationship is analogous to a romance in context. However, it's somewhat hard to imagine what context would falsify the claim given #2 and #3, so I still assign a 65% credence. The rebuttal doesn't seem very relevant, but if the reviewer just never compares the relationship to a romance, it would be difficult to prove it. I guess the rebuttal makes the relationship seem less romantic by implying that there are ignoble motivations (if the rebuttal quote is talking aotu the same platonic relationship as the claim).

  5. 65% -- Supporting claims seem sufficient at first. The rebuttal isn't very strong, so going with the supporting claims.

  6. 70% -- The claims support the the base claim. The rebuttal actually strengthens it, but establishing the gender of the novelist and journalist.

  7. 70% -- Very unsure. "Analogous" is very broad in definition. I guess it is easy for anything to be analogous to a romance, and it is not stated if the reviewer uses the word analogous, or if this is a interpretation of the text.

  8. 75% -- The supporting claims seem like the movie could be a romance. it is unclear though who are the main characters and who is in love with whom. But the rebuttal is kind of weak so I guess that is not the case and it is not uncommon that one of the characters in a romance acts badly (which rebuttal mentions)

  9. 80% -- The claim is supported but not rebutted.

  10. 90% -- The supporting claims indicate that there's a platonic relationship between two men, and compares their roles to romantic relationships. (It's possible, though unlikely, though only one of the man's roles is being compare, and these analogies are incidental to the platonic relationship).

Subclaim 1: In the review, the reviewer asserts that the film involves a platonic relationship between a novelist (Wallace) and a journalist.

Generator credence: 98%

Ensembled step judgement: 37.5% (8 reviews)

Quotes: account of the week-and-a-half ... writer ... Lipsky spent following ... Wallace Wallace ... likes Lipsky ... though he's observant enough … realize that Lipsky cannot have a real friendship with him

Rebuttal: He only got Rolling Stone to pay for his rock-star style profile of a novelist by agreeing to ask Wallace about the rumors that he uses heroin, and his motivations… are... less than noble.

Judge reviews:

  1. 10% -- Rebuttal suggests Wallace is not the novelist. Quotes omit suspiciously much context
  2. 30% -- The quote actually suggests that they can't have a friendship. The rebuttal points out that the motivations are less than noble.
  3. 40% -- no evidence there is a relationship at all (platonic or otherwise), but the fact that the absence of friendship is mentioned at all is a bit of evidence as well.
  4. 58% -- The supporting quotes are so difficult to parse with all the ellipses. I don't feel like I can just trust ellipses that are used this much, especially when it separates the subject of a sentence from its predicate. But the rebuttal is of no use at all, so I'm ultimately going with the supporting quotes.
  5. 60% -- The supporting quotes are missing a lot of text but the rebuttal does not try to use that so it is probably true but it is not really strong proof. The rebuttal does not seemse to be much relevant.
  6. 70% -- There seems to be interactions between Wallace and Lipsky; Unsure if Lipsky is the journalist in question.
  7. 70% -- I think "cannot have a real friendship" still qualifies as "platonic relationship".
  8. 85% -- The claim seems very likely to be literally true: the two men clearly have some kind of relationship (if only a manipulative business relationship), and given the quotes it seems very unlikely that it was a romantic relationship. If it weren't platonic it should've been easy for the rebuttal to provide direct evidence that the relationship was romantic and/or sexual. However, it's possible that a reasonable observer wouldn't call the claim "true" in context (e.g. it would be somewhat weird to describe business adversaries as having a "platonic relationship" even if technically true under some definitions of "relationship").

Subclaim 2: In the review, the reviewer suggests that one of the men’s roles is analogous to someone who’s been burned in relationships learning to love again.

Generator credence: 98%

Ensembled step judgement: 66.25% (8 reviews)

Quotes: somebody who's been burned over and over It's the story of a man learning to trust again (in a love story, it would be "to love again") while worrying that he's going to get burned one more time

Rebuttal: one of cinema's finest explorations of an incredibly specific dynamic—that of the cultural giant and the reporter who fantasizes about one day being as great as his subject, and in the same field

Judge reviews:

  1. 25% -- The supporting quote seems to contradict, suggesting its about 'trusting again' not 'loving again'. Rebuttal supports that further.
  2. 55% -- The only connection to learning to love again is the parenthetical in the supporting quote, but this isn't sufficient to establish that the reviewer is trying to draw an analogy between the man in question and a lover that's been burned in relationships. The rebuttal also makes it clear that the relationship is not particularly similar to a romantic one; overall, the claim is quite marginal and I can't decide if it's more likely to be true or false (in full context). The critical point is why the reviewer included the parenthetical about "in a love story..."; if that was part of a broader analogy then the claim would be clearly correct, and if it's not then it's somewhat difficult to understand why it was included.
  3. 70% -- "It's the story of a man learning to trust again (in a love story, it would be "to love again")"---seems like a good paraprase of the main claim
  4. 90% -- The supporting quote is compelling.
  5. 90% -- The claim is modest ("suggests" and "analogous") and is adequately supported.
  6. 90% -- Supporting quote explicitly says claim. There is some wiggle room so it isn't 100
  7. 90% -- Supporting quote is sufficient.
  8. 98% -- solid evidence

Subclaim 3: In the review, the reviewer suggests that one of the men’s roles is analogous to a serial adulterer who pretends to be reformed.

Generator credence: 98%

Ensembled step judgement: 80% (8 reviews)

Quotes: If this were a romantic drama, Lipsky might be … a serial cheater who wants everyone to think he's reformed and can be monogamous even though he's constitutionally incapable of that

Rebuttal: one of cinema's finest explorations of an incredibly specific dynamic—that of the cultural giant and the reporter who fantasizes about one day being as great as his subject, and in the same field

Judge reviews:

  1. 80% -- The rebuttal suggest the movie is not about romance/sex, but the claim uses the word 'analogous', and analogy seems to be supported by the quote
  2. 80% -- The supporting quote certainly appears to be an analogy between Lipsky and a serial adulterer; it's not 100% clear if Lipsky is one of the men in the claim, though. "the role is analogous to a serial adulterer" is also a somewhat vague claim, and I can imagine situations where it would not be true in full context, even given the supporting quote. The rebuttal doesn't seem very relevant.
  3. 80% -- Contains a gap that could be exploited. The rebuttal didn't exploit it and therefore it seems likely to be true. The Quote supports the claim.
  4. 80% -- Supporting quote is sufficient.
  5. 90% -- The claim is supported but not rebutted.
  6. 90% -- The supporting quote is compelling
  7. 95% -- There is missing some text in the supporting quote but the rebuttal does not try to use it so I suppose the sentence is true and therefore strong proof for the claim. The rebuttal does not seem to be much relevant.
  8. 100% -- direct evidence

20. https://www.rogerebert.com/reviews/femme-fatale-2002

Root claim: In the review, the reviewer suggests the film is artistically adventurous, and one reason for this is because it subverts the expectations.

Generator credence: 90% -- Many examples in text of surprising features of the movie and references to other artistically adventurous movies.

Tree judgement: 36.25%

Ensembled step judgement: 36.25% (8 reviews)

Subclaim 1: In the review, the reviewer suggests the film subverts expectations about a heist plot.

Subclaim 2: In the review, the reviewer asserts the film surprises viewers by using subtitles despite the actors not saying anything.

Subclaim 3: In the review, the reviewer describes parts of the movie that subvert expectations as being positive

Subclaim 4: In the review, the reviewer states a positive opinion of the filmmaking of this movie

Rebuttal: The movie is very light on dialogue, and many of the words that are spoken come across as if the characters are imitating movie actors followed other directors into a sheepish retreat from nudity

Judge reviews:

  1. 10% -- "positive opinion" does not support "artistically adventurous".

  2. 34% -- Supporting claims support only the second part ('subverts'), nothing supports 'artistically adventurous'. I expect this to be a bit hard to rebut, but 'followed other directors' suggests it may not be so adventurous

  3. 37% -- The supporting claims indicate that the film does "subvert expectations" and there are some unusual elements. However, there's not a lot to suggest "artistically adventurous". Also, I'm used to claims being more convincing than this, and given half of them are false despite most of them seeming more convincing, I think is probably false.

  4. 60% -- what is well supported? that it subverts the expecations. it is probably artistically adventurous (I am not sure if eveything done adventurously(meaning involving new ways or methods) in the movie would be considered as "artistically adventurous" but I guess so - "artistically" is probably added to make it clear the movie is not "an adventure")- there are subtitles even if the actors are not saying anything, "many of the words that are spoken come across as if the characters are imitating movie actors"- is it on purpose though?-probably it is the reviewer is supposed to have positive opinioin of the filmmaking. . I am not sure what to think about this claim "followed other directors into a sheepish retreat from nudity" that seems to contradict the adventurousness of the filmaker a bit

  5. 70% -- supporting claims are well-structured, but \followed other directors' gives me pause

  6. 80% -- The supporting claims are somehow strong but the rebuttal also offers a little of contradicting information.

  7. 80% -- The reviewer implies, but I don't see a subclaim about suggesting. Rebuttal doesn't move me very much.

  8. 86% -- the causal claim (it's good because it subverts expectations) is only reinforced a little by 3. and maybe 4. The other components are well substantiated

Subclaim 1: In the review, the reviewer suggests the film subverts expectations about a heist plot.

Generator credence: 98%

Ensembled step judgement: 90% (5 reviews)

Quotes: movie opens with a $10 million ... theft Her team includes the usual crew of heist-movie types ... But the movie announces its originality when none of these characters perform as they expect to

Rebuttal: No, I have not given away too much. The fact is, I have given away less than nothing, as you will fully appreciate after seeing the film.

Judge reviews:

  1. 85% -- the 'heist plot' part is indirectly supported and not disputed by rebuttal. the 'subverts' part seems supported by both quote and rebuttal
  2. 90% -- The claim is supported and not rebutted.
  3. 90% -- The supporting quotes are somehow strong, but there is no proof that the last sentence "But the movie announces its originality when none of these characters perform as they expect to" is connected with the heist. On the other hand, the rebuttal does not offer much.
  4. 91% -- good evidence for the claim
  5. 100% -- Rebuttal feels like reverse psychology to get viewers interested. Its implications suggest a satisfying subversion.

Subclaim 2: In the review, the reviewer asserts the film surprises viewers by using subtitles despite the actors not saying anything.

Generator credence: 99%

Ensembled step judgement: 80% (4 reviews)

Quotes: seen the movie twice ... plays differently the second time ... on the second viewing did I spot the sly moment when the subtitles supply ... dialogue--but the lips of the actors are not moving

Rebuttal: ``

Judge reviews:

  1. 80% -- The claim is supported.
  2. 80% -- quote supports the claim, but some context that may be important is missing. The rebuttal does not provide it, though
  3. 87% -- a lot of ellipses in the supporting quote reduce the probability slightly. But still good evidence.
  4. 95% -- There is a lot of missing text in the supporting quote but the rebuttal is missing.

Subclaim 3: In the review, the reviewer describes parts of the movie that subvert expectations as being positive

Generator credence: 90%

Ensembled step judgement: 90% (6 reviews)

Quotes: But the movie announces its originality when none of these characters perform as they expect to The movie tricks us, but not unfairly I haven't had as much fun second-guessing a movie since...

Rebuttal: The movie is very light on dialogue, and many of the words that are spoken come across as if the characters are imitating movie actors… standard thriller dialogue The movie tricks us

Judge reviews:

  1. 80% -- but not unfairly is a rebuttal too the rebuttal.
  2. 90% -- "fun" supports "positive", and is not rebutted.
  3. 90% -- Claim seems supported by quotes and rebuttal does not directly contradict
  4. 95% -- originality and fun are positive.
  5. 95% -- The supporting quotes are very strong, the rebuttal offers some information but it is not enough.
  6. 100% -- When a supporting quote has a fuller quote involving a snippet used in rebuttal, I simply demote the rebuttal to "too cheeky, adorable but not useful" category. Like a toddler. That "fun second-guessing" quote is very strong IMO.

Subclaim 4: In the review, the reviewer states a positive opinion of the filmmaking of this movie

Generator credence: 80%

Ensembled step judgement: 90% (5 reviews)

Quotes: Brian De Palma's "Femme Fatale" is a sexy thriller... This is pure filmmaking, elegant and slippery. an exercise in superb style and craftsmanship De Palma deserves more honor as a director.

Rebuttal: The movie is very light on dialogue, and many of the words that are spoken come across as if the characters are imitating movie actors followed other directors into a sheepish retreat from nudity

Judge reviews:

  1. 75% -- Supporting quotes are fairly strong, rebuttal's first quote is moderate evidence against. Second quote of rebuttal is unconvincing (saying he no longer uses nudity isn't much of a criticism).
  2. 90% -- The claim is well supported, and rebutted.
  3. 90% -- The supporting quotes contain a lot of positive praise. The rebuttal offers some negative, but not much.
  4. 93% -- pure, elegant, superb craftsmanship, honor as a director. Are positive statements.
  5. 95% -- Claim supported by quotes, rebuttal does not contradict

21. https://www.rogerebert.com/reviews/oasis-supersonic-2016

Root claim: In the review, the reviewer suggests the film is a conventional documentary about the rise of a rock band.

Generator credence: 95% -- No claim otherwise, and has standard documentary tropes.

Tree judgement: 35.75%

Ensembled step judgement: 35.75% (8 reviews)

Subclaim 1: In the review, the reviewer calls the film a documentary on the rise of the band Oasis and doesn’t suggest the film is unconventional

Subclaim 2: In the review, the reviewer says Liam and Noel are Oasis members

Subclaim 3: In the review, the reviewer suggests the film uses archival footage with voiceovers by Liam and Noel

Subclaim 4: In the review, the reviewer says the film uses footage of a song being performed for the first time

Rebuttal: It’s difficult to explain to people who weren’t there just how big an event this was nationwide. There’s no equivalent that I’ve seen stateside since. Oasis wasn’t just a musical phenomenon

Judge reviews:

  1. 10% -- "doesn't suggest the film is unconventional" doesn't support "suggests the film is a conventional documentary".

  2. 20% -- This workspace is quite confusing for me overall. Only supporting claim #1 seems relevant to evaluating the root claim; in particular, I don't understand why #2 and #3 mention "Liam and Noel", who don't show up in the root claim. Claim #1 confirms that the film is a documentary about a band, but doesn't mention whether the band is a rock band. Also, "doesn't suggest the film is unconventional" is not good evidence for "suggests the fillm is conventional": if the reviewer never comments on whether the film is conventional, then the root claim would be false IMHO. Overall, I don't see any good reason to think the claim is true; the only reason my credence isn't lower is the possibility that subclaims #2-#4 somehow provide support for the main claim for a reason that I don't understand. The rebuttal isn't very relevant, which makes it somewhat more likely that Oasis is a rock band. If Oasis were a different kind of band, it should have been easy to find a quote showing that. However, if the review never takes a stand on whether the film is conventional, then the claim would be false, but there would be no quote in the review that directly rebuts the claim.

  3. 41% -- '...and doesn’t suggest the film is unconventional' does not mean that he suggest the opposite. Other than that there is no evidence for the conventionality claim. Also, the rebuttal offers a argument for the contrary.

  4. 45% -- The evidence doesn't state the reviewer suggested the film was unconventional. The supporting claims do not reach the point to support the overarching claim.

  5. 60% -- the main claim says that the movie is conventional. to me it suggest that it is mediocre/normal/unoriginal. I think the movie can be something in beetween.

  6. 85% -- The supporting claims provide evidence for rock band, rise, documentary.

  7. 95% -- The rebuttal is trying to argue against "rise", but the first supporting claim states that directly.

  8. 96% -- That rebuttal is the most conventional music documentary quote I could have imagined

Subclaim 1: In the review, the reviewer calls the film a documentary on the rise of the band Oasis and doesn’t suggest the film is unconventional

Generator credence: 99%

Ensembled step judgement: 52.25% (8 reviews)

Quotes: Supersonic,” the new documentary about Oasis, captures this sense of instant superstardom, such as in moments in which the band is stunned at how many people know the words to ... songs at concerts

Rebuttal: And then “Oasis: Supersonic” kind of just ends… doesn’t get very far in the overall history of the band... someone completely unfamiliar with Oasis might think they broke up in the mid-‘90s.

Judge reviews:

  1. 20% -- The rebuttal is convincing. The supporting quote doesn't provide enough evidence for the rise of the band Oasis.
  2. 50% -- It is very unclear if this claim is true or not. The rebuttal does not disprove the claim anymore that the supporting quote proves it.
  3. 53% -- it's a bit unconventional to stop in the middle of a biopic when there's more content
  4. 59% -- it is hard to find evidence for a negative claim (that something is not suggested in the review)
  5. 60% -- The supporting quotes clearly show that the film is a documentary about the rise of a band called Oasis. However, the claim also says that "the reviewer doesn't suggest that the film is unconventional", which establishes a universal quantifier: the claim can only be true if no sentence suggests that the film is unconventional. Therefore, the strength of the rebuttal is much more important for establishing the truth of the claim. In particular, the root claim is more likely to be true than not if quote in the rebuttal is less strong than you would expect if the reviewer did in fact think the film was unconventional. The rebuttal provides very weak evidence that the reviewer suggests that the film is unconventional (i.e. by implying a more conventional film would have gotten farther in the overall history of the band). Even though the quote is quite weak, it's also very possible that the review overall suggests that the film is unconventional, without any short quote decisively showing that. Overall, the weakness of the strongest quote showing that the film is unconventional is moderate but not decisive evidence that the claim is true.
  6. 80% -- The supporting quotes with its mention of 'instant superstardom' is suggestive of a 'rise of the band' documentary. The rebuttal with its critique of the band ending in the 90s, while true, does still include a rise of the band from its start so that is in support of the claim. The question of the documentary being conventional or unconventional feels largely unaddressed by either of the quotes or rebuttal.
  7. 85% -- The rebuttal is arguing against the root claim's "doesn't suggest the film is unconventional". But to do this, all you'd need to do is quote where the reviewer suggests it is unconventional. The rebuttal doesn't do this well at all.
  8. 90% -- The claim is supported but not rebutted.

Subclaim 2: In the review, the reviewer says Liam and Noel are Oasis members

Generator credence: 98%

Ensembled step judgement: 78.75% (8 reviews)

Quotes: album called (What’s the Story) Morning Glory? by Oasis the film rests on the same thing Oasis did—the odd dynamic between Liam and Noel Gallagher Noel would give Liam the lyrics

Rebuttal: Noel became a singer merely because Liam would leave the stage almost an hour early and someone had to finish out the concert... almost-breakups and some of the shuffling of the rest of the band.

Judge reviews:

  1. 75% -- the review seems not to say that, but the supporting quote suggest they are members.
  2. 75% -- well it strongly implies that
  3. 80% -- The 2nd supporting quote, as well as the rebuttal (!), make it quite likely that Liam and Noel are in fact members of Oasis. However, the claim is that the reviewer says that Liam and Noel are Oasis members. It would be quite strange if the reviewer neglected to state this, but it's also quite strange that it's not directly stated in any of the supporting quotes.
  4. 90% -- Even the rebuttal supports the claim.
  5. 90% -- The rebuttal actually supports the root claim here.
  6. 100% -- The supporting quotes and the rebuttal jointly establish Liam and Noel as members of the band. I know Liam was singing from the supporting quotes and that Noel also was singing due to what was said in the rebuttal.
  7. 100% -- The supporting quote provides complete evidence for the claim.
  8. 100% -- The quote explicitly states this claim.

Subclaim 3: In the review, the reviewer suggests the film uses archival footage with voiceovers by Liam and Noel

Generator credence: 98%

Ensembled step judgement: 91.25% (8 reviews)

Quotes: We don’t even see Liam or Noel, who do their interviews in voiceover over archival footage

Rebuttal: Noel became a singer merely because Liam would leave the stage almost an hour early and someone had to finish out the concert... almost-breakups and some of the shuffling of the rest of the band.

Judge reviews:

  1. 55% -- The supporting quotes support the premise.
  2. 80% -- The claim is supported but not rebutted.
  3. 95% -- I don't understand how the rebuttal is relevant, and the claim almost exactly restates the supporting quote. I still assign a 5% chance ot the claim being false because I'm not confident I can enumerate and rule out all the possible ways the claim could be false yet.
  4. 95% -- Supporting quotes are sufficient.
  5. 96% -- supporting quote says what the claim claims-
  6. 99% -- rebuttal is non sequiter
  7. 100% -- The supporting quotes are nearly verbatim the contents of the claim.
  8. 100% -- The supporting quote provides complete evidence for the claim.

Subclaim 4: In the review, the reviewer says the film uses footage of a song being performed for the first time

Generator credence: 99%

Ensembled step judgement: 50% (9 reviews)

Quotes: They have a recording of “Champagne Supernova” which looks like it’s the first time that Liam is singing it off a notepad,

Rebuttal: Noel became a singer merely because Liam would leave the stage almost an hour early and someone had to finish out the concert... almost-breakups and some of the shuffling of the rest of the band.

Judge reviews:

  1. 10% -- "first time that Liam" doesn't support "performed for the first time".
  2. 45% -- The reviewer says it looks like the film was sung for the first time, not that it was sung for the first time.
  3. 50% -- (I had to re-read the definition of "true" in the instructions, as well as relevant slack comments, but I still wasn't sure so I asked this question: https://ought-inc.slack.com/archives/CMWRRHBC0/p1576513315001300"). I have no idea if the supporting quote means the claim is "true", because I'm still not sure what definition of true should be used, so I'll have to assign 50% confidence (in particular "looks like the first time" vs "is teh first time" and "singing off a notepad" vs "performed"). The rebuttal doesn't seem relevant.
  4. 60% -- Rebuttal doesn't rebut, but singing off a notepad doesn't necessarily imply it's the first performance
  5. 76% -- 'looks like(!)it’s the first time that Liam is singing it off a notepad' offers only medium evidence for the claim
  6. 95% -- The supporting claim provides very good but not complete evidence for the claim. He might have used that notepad before in earlier performances.
  7. 95% -- Rebuttal is irrelevant.
  8. 98% -- the claim is very well supported by the supporting quotes
  9. 100% -- The supporting quote offers a compelling example of what the claim posits. With left over character budget to spend, I am lead to believe this quote is the origin of the claim. The rebuttal offers little to the contrary.

22. https://www.rogerebert.com/reviews/the-arrangement-1969

Root claim: In the review, the reviewer argues that actress Faye Dunaway is the one true bright spot of the movie that escapes the laborious and dull repetition of the script

Generator credence: 90% -- Reviewer complains about the repetitive plot, and its obviousness quite often while praising the lead members of the cast, but only Faye Dunaway remains scratch free in the end.

Tree judgement: 30%

Ensembled step judgement: 30% (9 reviews)

Subclaim 1: In the review, the reviewer says the plot is repetitive and dull.

Subclaim 2: In the review, the reviewer praises the skill of lead actor Kirk Douglas while noting even his skills succumbed to Kazan’s script.

Subclaim 3: In the review, the reviewer praises Faye Dunaway’s performance.

Subclaim 4: In the review, the reviewer implies that Faye Dunaway is the best part of the movie

Rebuttal: does draw nourishment from the remarkable performances of Kirk Douglas and Faye Dunaway. They're the best things in the film, achieving effects even when Kazan's script is at its most uncomfortable.

Judge reviews:

  1. 5% -- It seems that Faye is not the one true bright spot.

  2. 15% -- The rebuttal provides a contradiction to the claim.

  3. 30% -- The rebuttal quote indicates that Faye's AND Kirk's performances are the best things in the film. The supporting claims are paraphrased while the rebuttal is a direct unbroken quote. Hence the claim seems less likely.

  4. 30% -- How can Faye be the one bright spot in the movie when Kirk's skills were also praised?

  5. 30% -- The rebuttal is extremely strong.

  6. 45% -- It seems like Kirk Douglas also did well despite the script, so it's not true that Faye is the only bright spot.

  7. 70% -- The most likely source for this to be false is the second part of the second claim, that while Kirk Douglas did well he succumbed to the script. This is what the Rebuttal draws attention to.

  8. 80% -- There is a strong conflict between the second supporting claim and the rebuttal but I can imagine that in the next sentence, the reviewer will add more information. Other supporting claims strongly supports the claim.

  9. 95% -- The rebuttal is challenging the claim strongly but 2 defends it. The other claims make it more likely that the claim is true.

Subclaim 1: In the review, the reviewer says the plot is repetitive and dull.

Generator credence: 99%

Ensembled step judgement: 52.5% (7 reviews)

Quotes: By the time he's analyzed the cause of his crisis...we've been through it ourselves a time or two, "The Arrangement" is one of those long, ponderous, star-filled "serious" films

Rebuttal: But Faye Dunaway interacts well and sensitively with Douglas, and the scenes involving their affair form a separate story (almost apart from the confusion of the main event) that is absorbing and real

Judge reviews:

  1. 20% -- The rebuttal (an unbroken quote) contradicts the claim
  2. 40% -- Supporting quotes don't really speak to "dull", andthey barely speak to "repetitive". Meanwhile the rebuttal shows the film is at last partly not dull.
  3. 65% -- While the quote supports the dull claim the support for the repetitive claim seems like it was manipulated somewhat. The rebuttal though actually reinforces the dull claim.
  4. 70% -- a time or two sure implies repetitive film. yet does not say so outright
  5. 70% -- The supporting quotes are not as strong as I would like to, only the first part "`By the time he's analyzed the cause of his crisis...we've been through it ourselves a time or two" is somehow supporting the claim, the other part also supports the claim but it is not really conclusive and I can imagine that it could mean different things depending on the context. The rebuttal looks relevant but if you concentrate on the sentence structure it looks like it follows some criticism.
  6. 75% -- The quotes are unclear, but the rebuttal helps clarify things. However, since the Faye and Kirk scenes are a separate storyline which is good, maybe that counts in the script's favour.
  7. 85% -- Pretty good evidence, but might not be true with more context

Subclaim 2: In the review, the reviewer praises the skill of lead actor Kirk Douglas while noting even his skills succumbed to Kazan’s script.

Generator credence: 99%

Ensembled step judgement: 70% (6 reviews)

Quotes: this isn't a "Kirk Douglas performance" but a piece of acting, Even Douglas is not quite able to pull off several scenes of hysterics and wall climbing

Rebuttal: it does draw nourishment from the remarkable performances of Kirk Douglas and Faye Dunaway. They're the best things in the film, achieving effects even when Kazan's script is at its most uncomfortable.

Judge reviews:

  1. 20% -- The rebuttal claims that Kirk's performance achieves effects even when the script is at its most uncomfortable
  2. 70% -- Skills can succumb in some scenes and also pull through in other scenes. This means the rebuttal does not fully contradict the root claim.
  3. 70% -- The quote gives an example of Kirk failing to overcome the bad script, but the summary seems more positive.
  4. 80% -- This shows that Kirk failed to do his best at acting in this film.
  5. 90% -- Even though the rebuttal is quite strong, I can imagine that it is following in the text just after supporting quotes.
  6. 100% -- Direct evidence

Subclaim 3: In the review, the reviewer praises Faye Dunaway’s performance.

Generator credence: 99%

Ensembled step judgement: 95.5% (6 reviews)

Quotes: Kirk Douglas and Faye Dunaway, however, move ... with so much self-confidence that we become involved with their characters Douglas' best scenes are with Miss Dunaway just grateful for Faye Dunaway

Rebuttal: keeps even her emptiness in reserve. But Faye Dunaway interacts well and sensitively with Douglas, and the scenes involving their affair form a separate story (almost apart from the confusion of the

Judge reviews:

  1. 60% -- Of the supporting quotes, only the last one really helps to support the root claim. But the rebuttal doesn't give more context to that last supporting quote showing that it is misleading, therefore I think that it is not misleading.
  2. 95% -- The supporting quotes are quite strong, the rebuttal actually does not offer much.
  3. 97% -- possible that quotes were taken out of context, but unlikely
  4. 97% -- The rebuttal also supports the claim.
  5. 100% -- The quotes and rebuttal support the claim.
  6. 100% -- Douglas' best scenes are with Miss Dunaway is explicit praise

Subclaim 4: In the review, the reviewer implies that Faye Dunaway is the best part of the movie

Generator credence: 90%

Ensembled step judgement: 30% (5 reviews)

Quotes: movie finally pounds at our moral sense so much that our moral sense wearies and becomes defensive, and we are finally just grateful for Faye Dunaway's beauty, and the way she doesn't raise her voice

Rebuttal: it does draw nourishment from the remarkable performances of Kirk Douglas and Faye Dunaway. They're the best things in the film, achieving effects even when Kazan's script is at its most uncomfortable.

Judge reviews:

  1. 15% -- Not the best part alone, it seems
  2. 30% -- The rebuttal makes it clear that the reviewer thinks both are the best things.
  3. 40% -- The claim is less likely because of omission. It intentionally omits the information that Kirk and Faye are the best aspects of the film.
  4. 50% -- The rebuttal implies that there may be a second part of the movie equal in skill as Faye.
  5. 87% -- It's certainly true that the reviewer praises Faye, but they also praise Kirk, so it's unclear to what extent the reviewer considers Faye the best part.

23. https://www.rogerebert.com/reviews/i-dont-feel-at-home-in-this-world-anymore-2017

Root claim: In the review, the reviewer argues that the film is good but not exceptional.

Generator credence: 95% -- Fits the start rating and also particular quotes.

Tree judgement: 30%

Ensembled step judgement: 30% (9 reviews)

Subclaim 1: In the review, the reviewer asserts that readers should see the movie for its focus on eccentric people.

Subclaim 2: In the review, the review highlights the excellent performance of an actor.

Subclaim 3: In the review, the reviewer suggests that there are multiple ways in which the film could have been better.

Rebuttal: The movie never escalates beyond a high simmer, though, and once we get to the inevitable (if welcome and satisfying) climax, you may start to tally up all the missed opportunities.

Judge reviews:

  1. 15% -- I don't see any evidence the film is good.

  2. 20% -- The rebuttal is more convincing than the supporting claims.

  3. 30% -- The supporting claims are not talking if the movie is good or not.

  4. 60% -- Both the supporting claims and rebuttal indicate the author gives the movie a mixed review. Does it rise to the level of "good"? I think so "readers should see the movie", "excellent performance of an actor", "welcome and satisfying climax". Still, "good but not exceptional" perhaps suggests a somewhat higher standard than this.

  5. 70% -- Claim #1 somewhat strongly suggests that the reviewer found the film good overall, and claim #2 supports that; #3 proves that the film was not exceptional. The rebuttal appears to argue that the claim is wrong because the film isn't good (rather than that the claim is wrong because the film is actually exceptional). This means that claims #1 and #2 are most relevant; overall I don't think the quote in the rebuttal is strong enough to outweigh #1 and #2, though it does introduce some uncertainty.

  6. 80% -- The rebuttal makes it sound like the reviewer was a bit too disappointed in the film to call it good.

  7. 85% -- The claim does not appear to be contradicted by the rebuttal and is potentially supported by it.

  8. 89% -- 3. establishes the 'not exceptional' part of the claim; 1 and 2 establish the rest.

  9. 90% -- Supporting claims and rebuttal validates the claim

Subclaim 1: In the review, the reviewer asserts that readers should see the movie for its focus on eccentric people.

Generator credence: 95%

Ensembled step judgement: 90% (6 reviews)

Quotes: The film is worth seeing for its interest in eccentric but realistic people, in particular Ruth, who's played with great intelligence and exactness by Lynskey

Rebuttal: The movie never escalates beyond a high simmer, though, and once we get to the inevitable (if welcome and satisfying) climax, you may start to tally up all the missed opportunities.

Judge reviews:

  1. 90% -- "worth seeing" supports "should see", and is not rebutted.
  2. 90% -- Strong supporting quotes and a rebuttal that doesn't challenge the claim
  3. 90% -- The supporting quote seems like very strong evidence, and the rebuttal seems pretty irrelevant, though it's possible that the difference between "eccentric" and "eccentric but realistic" makes the claim false in context (quite unlikley though I think)
  4. 95% -- The supporting quote is quite strong, the rebuttal does not offer contradicting information.
  5. 99% -- very good evidence in the supporting quote.
  6. 99% -- literal quote from text

Subclaim 2: In the review, the review highlights the excellent performance of an actor.

Generator credence: 98%

Ensembled step judgement: 89% (7 reviews)

Quotes: played with great intelligence and exactness by Lynskey ... it's a treat to see her front-and-center here ... She's ... a terrific audience surrogate. When she snarls or snaps, I wanted to cheer

Rebuttal: The movie never escalates beyond a high simmer, though, and once we get to the inevitable (if welcome and satisfying) climax, you may start to tally up all the missed opportunities.

Judge reviews:

  1. 80% -- The supporting quotes validate the claim. The rebuttal is commenting on the plot and effect of the movie, not the acting.
  2. 88% -- that does sound like a good performance.
  3. 90% -- The claim is supported but not rebutted.
  4. 90% -- The reviewer clearly thinks that an actor gave an excellent performance; the main uncertainty is whether "highlighted" is a fair adjective (it probably is).
  5. 90% -- The supporting quote is quite strong, even though some text is missing. The rebuttal does not offer contradicting information.
  6. 92% -- good evidence in the supporting quotes
  7. 95% -- Rebuttal is offtopic, quote supports the claim

Subclaim 3: In the review, the reviewer suggests that there are multiple ways in which the film could have been better.

Generator credence: 98%

Ensembled step judgement: 86.25% (6 reviews)

Quotes: `The movie never escalates beyond a high simmer, though, and once we get to the inevitable (if welcome and satisfying) climax, you may start to tally up all the missed opportunities.

Rebuttal: The film is worth seeing for its interest in eccentric but realistic people, in particular Ruth, who's played with great intelligence and exactness by Lynskey.

Judge reviews:

  1. 83% -- medium/good support in the supporting quote. And the rebuttal gives no evidence for the contrary
  2. 85% -- implying it'd be better if it reached beyond a simmer or didn't miss the opportunities
  3. 90% -- The claim is well supported, but not rebutted.
  4. 90% -- Strong supporting quotes
  5. 93% -- I can't think of a way that the claim could be false given the supporting quote, and the lack of a persuasive rebuttal. The fact that the film is "worth seeing" in the eyes of the reviewer is consistent with the film making mistakes.
  6. 95% -- The supporting quote is very strong.

24. https://www.rogerebert.com/reviews/barcelona-1994

Root claim: In the review, the reviewer argues that the movie explores life for young, upper-middle-class men and that this demographic has been neglected by filmmakers.

Generator credence: 92% -- Upper-middle class is suggested but not clearcut.

Tree judgement: 20%

Ensembled step judgement: 35% (9 reviews)

Subclaim 1: In the review, the reviewer suggests the main characters are upper-middle class.

Subclaim 2: In the review, the reviewer describes the main characters as young men.

Subclaim 3: In the review, the reviewer argues that young, white-collar, upper-middle-class men are neglected by filmmakers.

Rebuttal: Fred Boynton (Chris Eigeman) is a lieutenant junior grade in the Navy The presence of these Americans in Spain - one a capitalist, the other a militarist - makes them a target for left-wing groups

Judge reviews:

  1. 20% -- No proof suggesting young, upper-middle-class men are being neglected by filmmakers. If the reviewer made that statement, there would be more supporting quotes.

  2. 30% -- are these even talking about the same movies? Is the military considered white collar work? Probably not

  3. 35% -- military personnel(as characters mentioned in the rebuttal) is not a white-collar job, which is one clue (but they can of course be side characters). What is true about young, white-collar, upper-middle-class men doesnt have to be true about young, upper-middle-class men (but it would work vice versa)

  4. 35% -- The rebuttal provides evidence that the demographic of the characters in the movie are not uniform, making the claim less likely to be true

  5. 80% -- No support for the idea that the movie 'explores life'(which I interpret as normal life) for these people (as opposed to being say, a WW2 film), but the rebuttal doesn't quite address the claim.

  6. 82% -- The claim is logical conjunction of supporting claims. Rebuttal unclear, but seems to dispute the 'upper-middle class' part. capitalist imo supports 'upper-middle-class', militarist could go either way, so overall my credence is decreased but not much.

  7. 95% -- How is that rebuttal supposed to work?

  8. 98% -- The supporting claims support the claim near-verbatim. The rebuttals are short and mostly irrelevant (except possibly the assumption that a lieutenant in the Navy might not be upper-middle-class, but this is very weak evidence).

  9. 100% -- supporting claims are a decomposition of top claim

Subclaim 1: In the review, the reviewer suggests the main characters are upper-middle class.

Generator credence: 90%

Ensembled step judgement: 55% (5 reviews)

Quotes: His subjects are smart ... white-collar workers … In ... his first film, he showed this generation in its embryo stage, as Park Avenue preppies

Rebuttal: Fred Boynton (Chris Eigeman) is a lieutenant junior grade in the Navy The presence of these Americans in Spain - one a capitalist, the other a militarist - makes them a target for left-wing groups

Judge reviews:

  1. 45% -- Very unsure. It's possible they are, but the evidence is very discontinuous.
  2. 55% -- I don't even know if the rebuttal is about the same movie?? his "subjects" sounds more like documentary than "main characters"
  3. 70% -- Too many ellipses in the supporting quote. Military personnel apparently doesn't count as "white collar". But that's just one character. Also, being a target for left-wing groups could easily be evidence for the claim, no?
  4. 70% -- I assume white-collar and preppies implies upper-middle class, but the rebuttal is confusing because it sounds like a totally different film.
  5. 80% -- There seems to be multiple claims supporting the middle class nature of the main characters.

Subclaim 2: In the review, the reviewer describes the main characters as young men.

Generator credence: 99%

Ensembled step judgement: 85.25% (8 reviews)

Quotes: His subjects are smart, intense but somewhat naive white-collar workers, young men I'd hardly … seen young WASPs earning a living

Rebuttal: It appears at first to be about the casual lives of young men trying to launch their careers, but eventually… it reveals darker depths.

Judge reviews:

  1. 70% -- The supporting quote suggests there are young men, although it doesn't indicate much about whether they are main characters. The rebuttal seems to be trying to say it's not about young men, but I think it's reasonable that the emphasis is on the second half of the sentence, i.e. it is not about the launching of careers. I'd expect some more evidence if there were is another obvious main character
  2. 80% -- pretty good evidence, but not too direct. There are young characters, but it is not clear if they are the main characters.
  3. 87% -- If 'his subjects' mean main characters, then the quote directly supports the claim, unless there's important context missing before or after the quote. If that was the case, I would expect to see it in rebuttal, but it only adds evidence that 'young men' are the main characters. The 'reveals darker depths' does not contradict.
  4. 89% -- from the rebuttal and the supporting quotes it seems the characters are indeed young men, but what if the story just starts with them as young? then later in the review he would describe them in other terms too
  5. 90% -- Both supporting claim and rebuttal supports the claim.
  6. 90% -- It seems like the rebuttal is negating the casual part, not the young men part.
  7. 100% -- Main characters are young men, according to both supporting and rebutting quotes. But the darkness lays dormant.
  8. 100% -- The rebuttal also supports the claim; the "darker depths" are referencing the career launch rather than the fact that the main characters are WASPs

Subclaim 3: In the review, the reviewer argues that young, white-collar, upper-middle-class men are neglected by filmmakers.

Generator credence: 99%

Ensembled step judgement: 20% (7 reviews)

Quotes: If there is one part of American society that American movies are usually not interested in, it is the … white collar society of business and management.

Rebuttal: In "Metropolitan" (1990), his first film, he showed this generation in its embryo stage, as Park Avenue preppies. The movie develops like a Woody Allen autobiographical picture

Judge reviews:

  1. 7% -- This quote is very unbelievable a priori already, and the supporting quote is not convincing enough, as it shows a discontinuity that looks malicious
  2. 20% -- The gap in the sentence seem to indicate that the quotes are taken out of context.
  3. 20% -- the 'white collar' part is literally supported unless important context is missing. The 'young' and 'upper-middle-class men' parts are not directly supported. The rebuttal shows that there was at least one previous film about 'white-collar' by the same director, which I find as a much weaker evidence compared to the supporting quote. Overall my decision is dominated by the missing evidence for 'young' 'u-m-c men'.
  4. 60% -- The quote supports the claim, but the rebuttal points out that Woody Allen also made films possibly about similar people.
  5. 70% -- The ellipsis in the supporting quote is suspicious. I'm not familiar with the cultural references in the rebuttal, so I won't make too certain a prediction.
  6. 75% -- Supporting quote would be strong support if contiguous, but ellipse could have removed crucial context. However, the first quote in the rebuttal, if anything, supports the claim. The second quote in the rebuttal is mild evidence against (Woody Allen is not generally thought of as being "white collar"), but is not inconsistent with the claim being true.
  7. 100% -- good supporting quote! I guess I messed up on my last answer

25. https://www.rogerebert.com/reviews/home-2010

Root claim: In the review, the reviewer suggests that the movie is something of a high-brow art film.

Generator credence: 90% -- It’s French, with a mysterious premise and art-film actors.

Tree judgement: null%

Ensembled step judgement: 51.25% (10 reviews)

Subclaim 1: In the review, the reviewer notes that the film is French

Subclaim 2: In the review, the reviewer notes that the bizarre premise of the film is never explained, improving the film

Subclaim 3: In the review, the reviewer argues that an actor in the movie intrigues by subtly but deeply assuming her character

Subclaim 4: In the review, the reviewer argues that that the movie depicts deep currents of family dynamics

Rebuttal: The family seems ordinary enough, if not quite conventional. The parents snuggle, the boy plays, the sister in her early 20s sunbathes and smokes in the front yard,

Judge reviews:

  1. 30% -- the supporting quotes dont give an impression of high-brow art film (maybe the second one). the rebuttal is not extremely convincing but gives some relevant info

  2. 40% -- Not all french productions are highly intellectual artworks. Not every bizarre premise results in art.

  3. 50% -- Neither the supporting claims not rebuttal is validating the claim

  4. 55% -- The rebuttal uses "seems", which might indicate that the reviewer later shows why this isn't the case. Meanwhile, the supporting claims all subtly suggest the root claim. I'm not fully convinced, but it does seem more likely than not.

  5. 70% -- The supporting claims establish that the movie has many properties associated with high-brow art films, but doesn't directly prove that the reviewer states this. I'm not sure how to interpret "reviewer suggests", or how high a burden of proof to expect for the supporting claims. My best guess is that the claim is still true if the entirety of the review conveys the impression that the film is a high brow art film, even if the reviewer never explictly states this. If so, then the claim is fairly likely to be true given the supporting claims. The rebuttal seems mostly irrelevant: it needs to prove that the reviewer does not suggest that the movie is a high brow art film (even though the film has many characteristics of such a film). Instead, it just establishes that a family in the film is ordinary?

  6. 75% -- 'Bizzare' 'French' films do have prior of being 'high-brow art' :). There is no direct support, but given claim 4, rebuttal does not really contradict

  7. 80% -- no quotes but supporting claims seem high-brow, rebuttal isn't that compelling

  8. 85% -- The supporting claims while possibly included in a definition of sub-claims for the root claim, appear inadequate to fully support the root claim. I feel " ...notes that the film is an art film" and or "...notes that the film is high-brow" if included in the supporting claims would make credence more solid. The rebuttal offers some hint to the unusual in 'the family seems ordinary enough' as that implies there is some reveal later of just how bizarre or different. Missing 'art film' here is an important anchor as I can also say high brow science fiction or family drama.

  9. 88% -- 'high-brow' is not mentioned in the supporting claims, but the description fits.

  10. 95% -- Rebuttal doesn't work, supporting claim 3 also doesn't work. Supporting claims 1 & 2 are salient, and 4 is somewhat minor but applicable.

Subclaim 1: In the review, the reviewer notes that the film is French

Generator credence: 99%

Ensembled step judgement: 95% (9 reviews)

Quotes: There are two questions never answered in the French film “Home.” How did this family come to live here? And why does the mother fiercely refuse to leave, even after a four-lane freeway opens in her

Rebuttal: The family seems ordinary enough, if not quite conventional. The parents snuggle, the boy plays, the sister in her early 20s sunbathes and smokes in the front yard,

Judge reviews:

  1. 88% -- The supporting quote strongly suggest the root claim.
  2. 95% -- The only way how the claim could be false if "Home" wasn't the reviewed film. The rebuttal does not contradict this, though.
  3. 95% -- The supporting quote explicitly says "French film". The main way for the claim to be false is if the film "Home" is not the main subject of the review. However, the rebuttal provides no evidence that the review is about a different movie. The rebuttal seems irrelevant, as it never mentions the name, language, or country of origin of the movie being reviewed.
  4. 96% -- 'French film “Home.” ' good evidence.
  5. 100% -- Rebuttal is like, "how can a conventional family be French?! French families can't be conventional!" xD
  6. 100% -- Quote says it is French.
  7. 100% -- The supporting quotes saying 'in the French film "Home."' validates this claim.
  8. 100% -- The supporting quote clearly says French film "Home"
  9. 100% -- The supporting quote provides complete evidence for the claim.

Subclaim 2: In the review, the reviewer notes that the bizarre premise of the film is never explained, improving the film

Generator credence: 95%

Ensembled step judgement: 69.25% (2 reviews)

Quotes: two questions never answered … why does the mother fiercely refuse to leave, even after a four-lane freeway opens in her front yard? Both are more satisfactory remaining as questions

Rebuttal: opening of the highway was not a surprise for them. Maybe they got the house cheap because it was coming., would not make sense in many households, but in this one, it represents a certain continuity

Judge reviews:

  1. 59% -- what is the 'bizarre premise'? The two questions in the supporting quote? maybe, but as such it offers only weak evidence for the claim.
  2. 100% -- Rebuttal almost supports the claim?

Subclaim 3: In the review, the reviewer argues that an actor in the movie intrigues by subtly but deeply assuming her character

Generator credence: 85%

Ensembled step judgement: 87% (9 reviews)

Quotes: Isabelle Huppert, you know since forever. Usually looking fundamentally the same, always assuming a new character from the inside out. Intriguing us. There's thought in that face, but it's inscrutable

Rebuttal: Marthe the mother (Isabelle Huppert) does the laundry. (“Today is whites day.”) There's horseplay in the bathtub, which the family seems to share rather freely.

Judge reviews:

  1. 70% -- The supporting quote seems sufficient.
  2. 80% -- The claim appears to be very well supported by the quotes.
  3. 87% -- good evidence in the supporting quote: 'assuming a new character from the inside out.' --> assuming character. 'There's thought in that face, but it's inscrutable' -> deep and subtle
  4. 89% -- the claim is well suppported. "from inside out" can be paraphrased as "deeply", "There's thought in that face, but it's inscrutable`" can be paraprased as "subtly"
  5. 90% -- Claim not literally, but clearly enough supported by quotes. Rebuttal does not contradict
  6. 90% -- The supporting quote seems to be strong evidence for the claim: 1) "intriguing us" is directly supported by the quote 2) "subtly assuming her character" is strongly supported by "usually looking fundamentally the same" as well as "inside out" 3) "but deeply" is strongly supported by "always assuming a new character from the inside out. Intriguing us. There's thought in that face, but it's inscrutable" The rebuttal provides a quote about the same actor mentioned in the supporting quotes, but doesn't seem to disprove any of the claims. The actress' role as described in the quote is rather mundane, but this is entirely consistent with the claim.
  7. 95% -- Rebuttal doesn't work at all. Supporting quote doesn't quite prove "subtly" part, but implies it well.
  8. 95% -- Rebuttal doesn't speak against the quote
  9. 100% -- The supporting quotes noting "assuming a new character from the inside out...inscrutable' validates the claim. The rebuttal describes some action of the mother character but lacks any challenges to the acting capacity or approach of this actor.

Subclaim 4: In the review, the reviewer argues that that the movie depicts deep currents of family dynamics

Generator credence: 95%

Ensembled step judgement: 80% (9 reviews)

Quotes: What happens would not make sense in many households, but in this one, it represents a certain continuity, and confirms deep currents we sensed almost from the first.

Rebuttal: family seems ordinary enough, if not quite conventional. The parents snuggle, the boy plays, the sister in her early 20s sunbathes and smokes in the front yard, the teenage daughter wears mostly

Judge reviews:

  1. 70% -- The claim is probably true but should be more concise
  2. 80% -- Claim validated by strong supporting quotes
  3. 80% -- The supporting quote is sufficient.
  4. 80% -- "represents... deep currents" is quite convincing... and it is said about the particular household in the movie - so it probably can be translated as "deep currents of family dynamics"
  5. 85% -- no real rebuttal and solid evidence
  6. 90% -- The supporting quote explictly states that the movie has "deep currents" in a matter that concerns "this household", which seems like extremely strong support for the claim. The rebuttal appears to argue that the family is ordinary, but this is perfectly consistent with the family having deep interpersonal dynamics.
  7. 95% -- The word 'dynamics' could be open to interpretation, but otherwise the claim is quite literally supported. Rebuttal does not contradict (esp. with the word 'seems')
  8. 96% -- Before deep currents are seen, that family should look ordinary. Supporting quote is very strong.
  9. 100% -- The supporting quote offers validation of the claim with use of the same descriptive language. The rebuttal offers some tangential support in suggesting the family appears normal but in reality is unique.

26. https://www.rogerebert.com/reviews/jawbreaker-1999

Root claim: In the review, the reviewer explains that while the film is intended to be a comedy, it is actually deeply disturbing

Generator credence: 3% -- Reviewer doesn’t say this, instead just seems to have the attitude that it’s a bad comedy

Tree judgement: 75%

Ensembled step judgement: 91% (11 reviews)

Subclaim 1: In the review, the reviewer explains that the film is supposed to be a comedy

Subclaim 2: In the review, the reviewer explains that the film is deeply disturbing

Rebuttal: The movie is a slick production of a lame script, the cast founder in amateur-night dialogue and a plot that desperately stretches and still barely struggles to the 91-minute mark

Judge reviews:

  1. 51% -- The rebuttal provides weak contradiction of the claim in stating that it is a "lame script", which is unlikely to be deeply disturbing but otherwise does not contradict the claim.

  2. 90% -- The supporting quotes are very storng, the rebuttal does not offer much.

  3. 90% -- The claim is well supported, and not really rebutted.

  4. 92% -- the supporting claims establish both parts of the claim.

  5. 92% -- Claims 1 & 2 are very strong.

  6. 95% -- The supporting claims directly imply the root claim, and the rebuttal claim doesn't rebut.

  7. 95% -- The supporting claims both are well focused to the root claim. I would look to the rebuttal to disprove commenting on the movie being a comedy or its being disturbing. Instead the rebuttal mainly focuses on the script, dialogue, and plot.

  8. 98% -- the main claim seems to be very well supported and the rebutall is weak - it doesnt seem to be related to the main claim that much

  9. 99% -- Supporting claims are near-verbatim evidence for the claim. The rebuttals are fully consistent with the claim being true.

  10. 100% -- Rebuttal is not relevant to claim (it is not mutually exclusive from the claim), the supporting claims do however indicate the claim is true.

  11. 100% -- coherent decomposition

Subclaim 1: In the review, the reviewer explains that the film is supposed to be a comedy

Generator credence: 90%

Ensembled step judgement: 80% (3 reviews)

Quotes: I knew high school comedies were desperate for new ideas, but "Jawbreaker'' is the first one I've seen where the bad girl is stoned with corsages.

Rebuttal: corpse back to Liz's bed, where they fake a rape, her mouth, taping it shut and locking her in a car trunk for a ride to a restaurant where they plan a birthday breakfast. Liz chokes to death.

Judge reviews:

  1. 80% -- Rebuttal does not adequately rebut claim of "high school comedies" because dark thematic elements (rape, death etc.) can be included in a comedy
  2. 80% -- The claim is supported, and not rebutted well.
  3. 98% -- The supporting quote does directly name 'comedies', lending some verification to the root claim. It would be nice to have another quote but at 148 this is adequate. The rebuttal seems to contradict a traditional situational comedy but would be okay for a dark or 'supposed to be' derivative of a comedy.

Subclaim 2: In the review, the reviewer explains that the film is deeply disturbing

Generator credence: 3%

Ensembled step judgement: 75% (8 reviews)

Quotes: carry the corpse back to Liz's bed, where they fake a rape scene., sticking a jawbreaker into her mouth, taping it shut and locking her in a car trunk for a ride...Liz chokes to death.

Rebuttal: The movie is a slick production of a lame script, the cast founder in amateur-night dialogue and a plot that desperately stretches and still barely struggles to the 91-minute mark

Judge reviews:

  1. 10% -- The supporting quotes don't say that the reviewer was disturbed.
  2. 75% -- the quotes are disturbing.
  3. 75% -- The supporting quotes provide evidence of the claim.
  4. 85% -- The rebuttal doesnt contradict the main claim, the supporting quotes sound indeed disturbing
  5. 90% -- The supporting quote is quite strong, the rebuttal does not offer much...
  6. 90% -- Rebuttal does not address the very convincing quote supporting the claim. A film can have a lame script, be amateurish and still be disturbing
  7. 98% -- I feel most would consider the supporting quote content of corpse(s), fake a rape scene, and chokes to death compelling towards 'deeply disturbing.' The Rebuttal lacks talking about the movie's emotional impact and instead comments on script and dialogue.
  8. 100% -- indeed I am disturbed

27. https://www.rogerebert.com/reviews/salaam-bombay-1988

Root claim: In the review, the reviewer argues that the movie presents an overly romantic picture of the life of street children in Bombay, and this leads the movie to troublingly suggest that street children are better left on the street

Generator credence: 2% -- Reviewer doesn’t state a connection between “romantic picture of life on the street” and “street children are better left on the street”

Tree judgement: 75%

Ensembled step judgement: 75% (9 reviews)

Subclaim 1: In the review, the reviewer says the movie is based on the lives of street children of Bombay

Subclaim 2: In the review, the reviewer explains the romanticising effect of color photography

Subclaim 3: In the review, the reviewer argues photography romanticises the society portrayed in the movie

Subclaim 4: In the review, the reviewer says the movie gave them the troubling impression that children fare better on the streets

Rebuttal: One of the questions asked, but not answered, by the film is, what should be done about these children? `the movie is about children doing the best they can for themselves.

Judge reviews:

  1. 17% -- I don't see evidence that their life is depicted as actually good and so question whether it's romanticizing for real

  2. 70% -- The second and third supporting quotes sounds quite strange to me, the first and last are quite strongly supporting the claim. the rebuttal does not contradict much.

  3. 75% -- the supporting claims are very good but so is the rebuttal. If the problem with the film is that it might suggest children have it better on the streets (than where? factory? home?)- how should it also rise a question "what should be done about these children"?

  4. 75% -- The supporting claims do make the claim somewhat likely to be true.

  5. 75% -- The supporting claims completely cover the claim. (The claim is about the review - the rebuttal suggests the movie may be different.)

  6. 80% -- The rebuttal does not seem very convincing.

  7. 89% -- The first three supporting claim give good evidence for the claims first part. The 4. supporting claim gives good evidence for the claims last part.

  8. 95% -- The supporting claims show that the movie is based on "street children in Bombay", and that it gives "the troubling impression that children fare better on the streets". Presenting "an overly romantic picture" is implied by saying that " photography romanticises the society". The rebuttal is still consistent with the claims being true.

  9. 99% -- The supporting claims fall in line with what is needed to substantiate the claim so I would be looking to the rebuttal for something to refute. The rebuttal in noting what the film does not do, propose what needs to be done with the street children, appears to me to fall in line with the reviewer feeling the situation is romanticized. Similarly, 'the movie is about children doing the best they can for themselves' portrays a sentiment of an observer documentary over an encouraging action type of documentary.

Subclaim 1: In the review, the reviewer says the movie is based on the lives of street children of Bombay

Generator credence: 95%

Ensembled step judgement: 91% (5 reviews)

Quotes: The filmmakers gathered a group of the street children of Bombay and talked with them… Out of these interviews emerged a screenplay that was a composite of several lives.

Rebuttal: One of the subplots of the film involves the relationship between a drug dealer and the prostitute who is his common-law wife.

Judge reviews:

  1. 60% -- drug dealers, sex workers, and spouses are typically not children. By highlighting their inclusion the Rebuttal shows the movie is not just about street children. However it could still be based on the lives of street children - just not exclusively
  2. 91% -- good evidence in the supporting quote
  3. 95% -- The supporting quote is quite strong, the rebuttal does not offer really contradicting information.
  4. 100% -- The supporting quote easily substantiates that the film is about street children of Bombay. There is an ellipses used and about 30 unused character budget but the location of the ellipses seems to be for abbreviation. The rebuttal notes a sub-plot that might exist alongside the street children stories or be a part of the collection. .
  5. 100% -- The supporting quote provides complete evidence for the claim.

Subclaim 2: In the review, the reviewer explains the romanticising effect of color photography

Generator credence: 95%

Ensembled step judgement: 94.25% (6 reviews)

Quotes: a well-known truism of filmmaking that color photography tends to make locations look better than they are; we lose the smells and the suffering and see the bright colors and the sunlight.

Rebuttal: tended to romanticize it somewhat. And yet there are moments that remain raw and painful, as Chaipau drops his street-smart facade for a second...

Judge reviews:

  1. 90% -- The claim is probably true due to the supporting quote.
  2. 94% -- supporting quote gives good evidence for the claim. The Rebuttal says that the movie is not overly romanticizing, but that does not tackle the claim.
  3. 95% -- yep the review sounds like a photography 101 text!
  4. 95% -- The supporting quote is very strong, the rebuttal is not really relevant.
  5. 95% -- The supporting quote establishes the claim with certainty.
  6. 100% -- The supporting quote is continuous, on topic, and rather verbatim validation of the claim. The rebuttal appears to be an example of the claim provided in the film review.

Subclaim 3: In the review, the reviewer argues photography romanticises the society portrayed in the movie

Generator credence: 95%

Ensembled step judgement: 89% (5 reviews)

Quotes: That happens here, I think; the very act of photographing this society probably has tended to romanticize it

Rebuttal: tended to romanticize it somewhat. And yet there are moments that remain raw and painful, as Chaipau drops his street-smart facade for a second...

Judge reviews:

  1. 70% -- The claim is much stronger than the the statement in the supporting quote.
  2. 89% -- there is good evidence that photography romanticizes at least somewhat. But no overly.
  3. 90% -- The supporting quote is very strong, the rebuttal tries to show contrast to it but it is not really relevant.
  4. 95% -- The supporting quote is sufficient to validate the claim with 'tended to romanticize it'. The rebuttal adds a little to the supporting quote with 'romanticize it somewhat.' Even though the rebuttal appears to be counter to the claim noting 'raw and painful' moments that exists along side of and not exclusive of the romanticized elements.
  5. 96% -- by saying 'I think' the reviewer publically endorses the point ie argues it

Subclaim 4: In the review, the reviewer says the movie gave them the troubling impression that children fare better on the streets

Generator credence: 90%

Ensembled step judgement: 80% (5 reviews)

Quotes: one of the underlying beliefs of "Salaam Bombay!": That the street life, however hard, is preferable we are left with the troubling impression that… children seem to fare better on the streets

Rebuttal: are rounded up by the police and herded into a large institution that combines the worst features of an orphanage and a prison, but that doesn't seem to be the answer

Judge reviews:

  1. 65% -- The implications of the rebuttal are substantial, and effect how the claim itself is evaluated.
  2. 80% -- The quote seem to support the claim to a large part.
  3. 90% -- The supporting quote is quite strong, the rebuttal actually also support it a bit.
  4. 98% -- The verbal meaning in the supporting quotes appears to validate the claim. I was worried that the ellipses used instead of continuing that sentence is just a stand alone comment made elsewhere. That is functioning as a prepositional phrase and it looks as if the direct object of that has been swapped out. Strangely, here the rebuttal noting that being rounded up by the police into an orphanage/prison serves to sufficiently co-validate the claim.
  5. 98% -- heck this gave me 'the troubling impression that children fare better on the streets'

28. https://www.rogerebert.com/reviews/the-east-2013

Root claim: In the review, the reviewer describes the film as featuring high-school students whose political activism stems from issues with their parents.

Generator credence: 10% -- He reviewer does not assert that they are highschool students and circumstantial evidence suggests otherwise.

Tree judgement: 60%

Ensembled step judgement: 65% (9 reviews)

Subclaim 1: In the review, the reviewer asserts that one character’s motivation for activism is issues with her father.

Subclaim 2: In the review, the reviewer suggests that the movie portrays activism in high-school students as a response to parental issues.

Rebuttal: team medic Doc, Toby Kebbell's Doc, a Stanford-trained former aid worker who suffers from Parkinson's-like symptoms., de facto leader, Benji, Jane and Benji's mutual attraction

Judge reviews:

  1. 30% -- The claim and supporting claim 2 seems to be written by the same person, with no quotes backing it up, making it suspicious. The rebuttal seems to indicate someone older, perhaps countering the claim about the film featuring high-schools students. The claim seems to be an exaggeration of sub-claim 1.

  2. 60% -- The supporting claims do make the claim appear more likely true.

  3. 65% -- The supporting claims substantiate the claim and the rebuttal does little to contradict it.

  4. 75% -- The supporting claims help the claim. The Rebuttal seems off topic.

  5. 80% -- The supporting claims don't fully support "featuring" in the root claim, but because the rebuttal doesn't call attention to this, I'm preferencing the supporting claims' view.

  6. 90% -- Rebuttal isn't strong, it's trying to counter high school part with the mention of one Stanford graduate. Overall claim does follow from the subclaims, with some leeway.

  7. 99% -- supporting claims are Extremely Strong. Rebuttal reads like a barrel of monkeys picked phrases at random

  8. 99% -- The supporting quote (especially #2) are near-verbatim support for the claim.

  9. 100% -- supporting claims fully suport claim

Subclaim 1: In the review, the reviewer asserts that one character’s motivation for activism is issues with her father.

Generator credence: 95%

Ensembled step judgement: 75% (7 reviews)

Quotes: members of the group … zealot-like Izzie activists are … rich kids angry at their parents By rooting … Izzie's struggle for clean water in daddy issues

Rebuttal: Jane's boss (Patricia Clarkson) assigns her to find and monitor The East, a reclusive anarchist cell whose symbolic stunts have attracted media attention to corporate wrongdoing.

Judge reviews:

  1. 55% -- There seems to be some information present in the quotes to successfully support the claim, but there is curcial context missing
  2. 75% -- Too much discontinuity makes me uncertain that it was manipulated, but what is in it is still pretty good confidence-wise
  3. 75% -- Rebuttal seems unrelated, supporting quote is suspicious with those ellipsises
  4. 80% -- The quotes support the claim, only 80% because of the gaps in quotes. It is unclear how the rebuttal counters the claim, except the anarchist cell addresses corporate wrongdoing while Izzie advocates for clean water, so they quotes might be taken out of context.
  5. 80% -- It's hard to prove a negative. I'm not sure how the rebutter could quote anything that would deny the root claim if it isn't in the text at all. So the onus is on the supporting quote to show me definitively that it is in the text. All it would take to make this a 95% certainty is to give me the full unbroken last quote in the supporting quotes section. As it is, I'll still give it 75%. But I feel wary of the supporting quote because it doesn't give the full quote. I would think that is hiding something. But then the rebutter also doesn't expand the quote, which makes me think that the supporting quote isn't misleading after all.
  6. 85% -- The claim is that there exists one student who is and activist with daddy issues. The quote supports this.
  7. 90% -- 2 fairly strong quotes and a rebuttal that doesn't rebut the main claim

Subclaim 2: In the review, the reviewer suggests that the movie portrays activism in high-school students as a response to parental issues.

Generator credence: 10%

Ensembled step judgement: 60% (5 reviews)

Quotes: falling back on conservative clichés about activism … the idea that activists are all just rich kids angry at their parents daddy issues movie about slumming brats

Rebuttal: team medic Doc, Toby Kebbell's Doc, a Stanford-trained former aid worker who suffers from Parkinson's-like symptoms., de facto leader, Benji, Jane and Benji's mutual attraction

Judge reviews:

  1. 30% -- The supporting quotes makes it unlikely, that the claim is true, because the statement seems to be out of context.
  2. 60% -- quote "the idea that activists are all just rich kids angry at their parents" actually makes me think the claim is wrong. But maybe that is just the reviewer criticizing the way the movie portrays it. Very unclear.
  3. 80% -- What's the point of this rebuttal? I don't get it. That supporting quote is cut in suspicious places, so I don't want to take it at face value. I'm moved towards giving higher credence by the lack of strength in the rebuttal quotes.
  4. 80% -- The rebuttal just isn't relevant here. Without a good rebuttal, even weakly supporting quotes are convincing.
  5. 85% -- The rebuttal seems off topic. There are holes in the supporting quote that could be exploited but the rebuttal doesn't seem to take advantage of them.

29. https://www.rogerebert.com/reviews/abandon-2002

Root claim: In the review, the reviewer argues that the film is artistically ambitious in its choice of ending but that the ending does not work well.

Generator credence: 5% -- It’s not ambitious but ending is bad.

Tree judgement: 60%

Ensembled step judgement: 60% (9 reviews)

Subclaim 1: In the review, the reviewer suggests that the ending does not work well.

Subclaim 2: In the review, the reviewer suggests that a standard movie convention is reversed by the ending of the film.

Subclaim 3: In the review, the reviewer suggests the ending of the film is absurd and hard to make sense of.

Rebuttal: the movie is a thriller, and so we must watch as the human elements and the intelligence, which have absorbed and entertained us, are ground up in the requirements of the Shocking Climax. Too bad.

Judge reviews:

  1. 20% -- I interpret 'Shocking Climax' in the rebuttal as a standard movie ending for thriller, which contradicts subclaim 2 and main claim. It's a bit unclear, though

  2. 45% -- "the ending does not work well" appears well-supported. "standard movie convention is reversed" is weakly suggestive of 'artistically ambitious'. The rebuttal seems critical but consistent with the second half of the claim. Overall, in the bottom half of convincing claims.

  3. 60% -- The term "art film" means something specific. To say it is "artistically ambitious" could mean that specific thing, or it could just mean the normal way of saying it. If the latter, then the supporting claims are sufficient. If the former, then the rebuttal is pretty strong. I think the latter is likelier to be true.

  4. 80% -- Uncertainty is in "artistically ambitious". 2 goes in that direction, but is not too direct.

  5. 85% -- If these points are true the premise is true. The rebuttal did not disprove any point.

  6. 87% -- sounds like the writers worked really hard at making something bad. rebuttal is a quote that seems to support Claim.

  7. 90% -- The supporting claims are good evidence for the main claim: 2 suggests the choice of ending is "artistically ambitious" by reversing a standard movie convention; and 1&3 imply the ending "does not work well". The rebuttal seems to suggest the ending is bad and also unoriginal (typical for a thriller). However, the statement in such a short quote could be reversed in the following sentence, especially given the way it's phrased ("we must"). This is also hinted at by the "Too bad" at the end - I can imagine the next sentence being something like "The director has totally different plans for us". Assuming the supporting claims have been deemed likely even after reading the rebuttal, the evidence for the claim is quite strong overall - but "1 is not a probability".

  8. 100% -- Rebuttal is giving a bit more detail to the supporting claims, bolstering how it doesn't work. It's trying to suggest "artistically ambitious" is false, but thrillers have to end in a climax. That attempt would indeed be intended as shocking, if it's artistically ambitious.

  9. 100% -- The supporting claims fall in line with what would be needed to support the claim. The rebuttal is in fact a continuous quote that strongly validates #1.

Subclaim 1: In the review, the reviewer suggests that the ending does not work well.

Generator credence: 99%

Ensembled step judgement: 85% (7 reviews)

Quotes: "Abandon" is a moody, effective thriller for about 80 percent of the way … If you walk out before the ending, you'll think it's better than it is. never steps wrong until the final scenes

Rebuttal: Maybe this is the ending the movie needed, but it seems so arbitrary as it materializes out of thin air. Or maybe I'm still being unfair. Maybe it doesn't come from thin air…

Judge reviews:

  1. 73% -- Quotes support the claim quite strongly. Rebuttal suggests otherwise, but it seems to related to whether it 'materializes out of thin air' rather than whether it works well.
  2. 80% -- The reviewer states quite clearly that the ending does not work well in the supporting quotes, by saying that the ending is much worse than the rest of the film. The rebuttal suggests the reviewer is actually torn about this by showing their doubting their previous opinion aloud. This lowers my credence, but not by much, because of the following caveat: the quotes contain no positive appraisal of the ending, which would be much stronger evidence. This suggests that, for all their doubting, the reviewer never quite abandons their initial (negative) position.
  3. 90% -- Even the rebuttal supports the root claim.
  4. 98% -- Solid evidence
  5. 98% -- If you walk out before the ending, you'll think it's better than it is is a damniung review of the ending
  6. 100% -- Rebuttal's "maybe not" isn't very convincing. It feels like following sentence is along the lines of "but I can't see how it could make sense" or something like that.
  7. 100% -- Both the supporting quotes and the rebuttal are in support of the claim. Both mention the ending and the negative impact.

Subclaim 2: In the review, the reviewer suggests that a standard movie convention is reversed by the ending of the film.

Generator credence: 60%

Ensembled step judgement: 60% (7 reviews)

Quotes: Maybe this is the ending the movie needed … the Law of Economy ... no movie introduces a character unnecessarily ... rule doesn't precisely apply here, but it's relevant in a reverse sort of way.

Rebuttal: But the movie is a thriller, and so we must watch as the human elements and the intelligence, which have absorbed and entertained us, are ground up in the requirements of the Shocking Climax. Students of Ebert's Bigger Little Movie Glossary will be familiar with the Law of Economy of Characters, which states that no movie introduces a character unnecessarily, so that the apparently superfluous character is the one to keep an eye on.

Judge reviews:

  1. 22% -- The rebuttal does a good job of showing the ending may be conventional 'Shocking Climax' and that 'rule doesn't apply here' part is related to 'Law of Economy of Characters' rather than the ending.
  2. 50% -- Can't quite make heads or tails of this. I'd have to see the film. Does the rule apply? Is the ending good? I don't know.
  3. 70% -- I'm not entirely convinced by the claim, but I'm tentatively in agreement.
  4. 85% -- The rebuttal makes me uncertain in this one, and the discontinuities in the quote do not help. But the part rule doesn't precisely apply here, but it's relevant in a reverse sort of way. is quite good evidence
  5. 85% -- Supporting quote is sufficient.
  6. 100% -- This rather interesting pairing of supporting quotes and rebuttal is in and outside of the quote rules. The supporting quotes would justify and validate the claim, even though it has used three ellipses in a continuous sentence or quote of 195 characters. The rebuttal is outside of that rule with over 400 characters and the rebuttal contains and fills in much of the ellipses content of the supporting quote. Assuming the order of sections of quotes rule was followed I have two earlier fragments in any order, followed by a large text block, and then finishing with a last continuous quote. There really is quite a bit offered to substantiate the claim.
  7. 100% -- The Rebuttal reinforced the claim.

Subclaim 3: In the review, the reviewer suggests the ending of the film is absurd and hard to make sense of.

Generator credence: 95%

Ensembled step judgement: 83% (9 reviews)

Quotes: Maybe a rational ending with a reasonable explanation would have seemed boring. Maybe this is the ending the movie needed, but it seems so arbitrary as it materializes out of thin air.

Rebuttal: Or maybe I'm still being unfair. Maybe it doesn't come from thin air... no movie introduces a character unnecessarily, so that the apparently superfluous character is the one to keep an eye on

Judge reviews:

  1. 58% -- The supporting quotes suggest the ending is not 'rational' with a 'reasonable explanation'. The rebuttal suggests the reviewer is in two minds about this, however. It does seem like the reviewer leans more towards it being true
  2. 60% -- The supporting quote supports the claim, although I find the choice of words in a bit strange; I think "unexpected" would be a better characterization of the supporting quote than "hard to make sense of". On the other hand, the rebuttal shows convincingly that the reviewer has doubts about what's been said in the supporting quote (I'm assuming both quotes are related because the expression "out of thin air" features in both and I find it unlikely the expression has been used more than once in the review referring to different things). Since the claim uses the verb "suggests", I'm inclined to give it higher credence, because even if the reviewer does not state decisively, fully convinced, the end is absurd, it's nonetheless claimed somewhere in the review. But, since the truth seems to be more nuanced than the claim, I am only weakly in favor of it.
  3. 83% -- From both sets of quotes it seems reviewer suggests both the ending is absurd and not, but that would make the main claim rather true
  4. 90% -- The part of the rebuttal's quote after "no movie.." has a context about law of economy of characters. If I were to include that I'd say 100. I might've overcorrected at 90, but I'll go with that.
  5. 90% -- Explicitly states the claim. The 10 is the uncertainty of the reviewer himself making it. There is a chance he completely reverts the claim in subsequent paragraphs.
  6. 90% -- Supporting quote is sufficient.
  7. 97% -- Even if the rebuttal shows a change of perception from the author, the top claim is still true.
  8. 99% -- The supporting quote is continuous and rather descriptive in sustantiating a nonsensical ending. 'arbitrary' 'out of thin air' The rebuttal reveals the reviewers doubts but I feel the tone here is more "benefit of the doubt" instead of "I doubt it."
  9. 99% -- with the maybe... maybe framing, they suggest both. So yes Claim is true.

30. https://www.rogerebert.com/reviews/cyborg-1989

Root claim: In the review, the reviewer argues the movie is a surrealistic mashup of different genres.

Generator credence: 1% -- It’s just a conventional genre film.

Tree judgement: 60%

Ensembled step judgement: 70% (9 reviews)

Subclaim 1: In the review, the reviewer claims the movie has comedy elements.

Subclaim 2: In the review, the reviewer claims the movie has science fiction elements.

Subclaim 3: In the review, the reviewer claims the movie has horror elements.

Subclaim 4: In the review, the reviewer claims the movie breaks the fourth wall, highlighting the fact that it’s an artificial fiction.

Rebuttal: Movies like this... do not work when they… let us see the reality, which is that several strangely garbed actors feel vaguely embarrassed while wearing bizarre costumes and reciting unspeakable lines.

Judge reviews:

  1. 15% -- No mention of the word "surrealistic"

  2. 70% -- The supporting claims partially prove the claim but do not provide enough evidence for [50]surrealistic

  3. 70% -- Supporting claims are sufficient.

  4. 85% -- that is quite a few genres!

  5. 86% -- very good support in the supporting claims. Only the 'surrealistic' part of the claim has only medium evidence in 4.

  6. 89% -- good supporting quotes, "surrealistic" is defined as " with ideas and images mixed together in a strange way" which fits

  7. 90% -- The supporting quotes are quite strong, I had to google what means "breaks the fourth wall" and it seems that it supports the surrealistic part of the claim. Also, the rebuttal is not really offering any counter-evidence. Actually the last sentence supports the surrealistic part of the claim.

  8. 90% -- Given the first three supporting claims, I have very high credence it has different genres. "A surrealistic mashup" has less evidence, although "breaking the fourth wall" is perhaps slightly surreal. The rebuttal, if anything, increases my credence in the claim.

  9. 92% -- My credence is reliant on the "experts" already reviewing the supporting claims. The rebuttal lacks providing any reference to genre but does propose that the film 'do not work' and that the actors are in embarrassing and bizarre costumes. So the rebuttal serves to tangentially validate the 'surreal mashup' part of the claim.

Subclaim 1: In the review, the reviewer claims the movie has comedy elements.

Generator credence: 1%

Ensembled step judgement: 68.75% (6 reviews)

Quotes: I do remember laughing heartily at that point, about 30 seconds into the movie. Cyborg" is one of the funniest examples of this category

Rebuttal: "After the plague, things really got bad." I do remember laughing... that several strangely garbed actors feel vaguely embarrassed while wearing bizarre costumes and reciting unspeakable lines.

Judge reviews:

  1. 40% -- The supporting quotes provide evidence that the movie provokes laughter but not for [49] comedy elements
  2. 65% -- Having a funny line does not mean it has "comedy elements". A movie with the comedy genre has tropes that push humor throughout; it's not just a bunch of repeated jokes. But saying that the movie is a funny example does seem to push the idea that the root claim is true.
  3. 80% -- author laughing and 'funniest example of this category' are medium/strong evidence for the claim
  4. 95% -- I would say 'laughing heartily' is highly suggestive of 'comedy elements' along with 'bizarre costumes.'
  5. 96% -- even the Rebuttal remembers laughing!
  6. 100% -- direct evidence

Subclaim 2: In the review, the reviewer claims the movie has science fiction elements.

Generator credence: 99%

Ensembled step judgement: 94% (7 reviews)

Quotes: Few genres amuse me more than post-apocalyptic fantasies about supermen fighting for survival The plot of "Cyborg" is simplicity itself. The movie's heroine ... is half-woman, half-robot

Rebuttal: Once we know the central players, the movie turns into a sadomasochistic passion play, in which the village tries out varieties of unspeakable tortures on the hero, including crucifixion

Judge reviews:

  1. 90% -- Supporting quote is sufficient.
  2. 93% -- 'half-woman, half-robot' in the quote is enough to establish the 'science fiction element' claim.
  3. 95% -- the supporting quote is quite strong, the rebuttal does not offer contradicting information.
  4. 95% -- it is called Cyborg. That alone tells me it is sci-fi
  5. 99% -- direct evidence (small chance it is metaphorical)
  6. 100% -- The supporting quote's mention of 'post-apocalyptic fantasies' and 'half-woman, half-robot' is enough to substantiate 'science fiction elements.'
  7. 100% -- The supporting quotes seem to support the claim completely.

Subclaim 3: In the review, the reviewer claims the movie has horror elements.

Generator credence: 30%

Ensembled step judgement: 60% (7 reviews)

Quotes: the movie turns into a sadomasochistic passion play, in which the village tries out varieties of unspeakable tortures on the hero, including crucifixion

Rebuttal: Movies like this... do not work when they… let us see the reality, which is that several strangely garbed actors feel vaguely embarrassed while wearing bizarre costumes and reciting unspeakable lines.

Judge reviews:

  1. 55% -- [98]varieties of unspeakable tortures do not fully satisfy the category of [49]horror elements
  2. 60% -- I can see why someone might call what is in the supporting quotes "horror elements". But it's not clear to me that they necessarily are.
  3. 60% -- it sounds to me like "horror elements"
  4. 92% -- good support in the quotes.
  5. 95% -- that sounds pretty horrifying. Not sure what the Rebuttal is on about.
  6. 95% -- Seems like horror elements are present
  7. 100% -- The supporting quote's mention of 'sadomasochistic' and 'varieties of unspeakable tortures' would qualify for horror elements.

Subclaim 4: In the review, the reviewer claims the movie breaks the fourth wall, highlighting the fact that it’s an artificial fiction.

Generator credence: 1%

Ensembled step judgement: 69.5% (6 reviews)

Quotes: The movie takes place in a future world in which all civilization has been reduced to a few phony movie sets let us see the reality, which is that several strangely garbed actors feel .. embarrassed

Rebuttal: Movies like this... do not work when they… let us see the reality, which is that several strangely garbed actors feel vaguely embarrassed while wearing bizarre costumes and reciting unspeakable lines.

Judge reviews:

  1. 40% -- There is little evidence in the supporting quotes for [35]the movie breakes the fourth wall
  2. 66% -- medium support for the 'breaking of the fourth wall' claim
  3. 80% -- It's a weird statement which makes it inherently harder to evaluate. But seems correct
  4. 90% -- Supporting quote is sufficient.
  5. 92% -- sounds like they do break the 5th wall. Jury's out on whether the work is successul as a whole.
  6. 100% -- The supporting quotes substantiates the claim with 'future world..reduced to a few phony movie sets...' The rebuttal offers a continuation of one of the quotes containing and ellipses in the supporting quotes. The rebuttal over adding context to the supporting quotes, does not challenge anything.

31. https://www.rogerebert.com/reviews/do-not-resist-2016

Root claim: In the review, the reviewer describes the film as a documentary that mixes real footage with depictions of troubling hypothetical scenarios.

Generator credence: 2% -- no mention of hypotheticals

Tree judgement: 58.75%

Ensembled step judgement: 70% (9 reviews)

Subclaim 1: In the review, the reviewer describes the film as depicting a hypothetical where the US police are heavily militarized as in fascist states.

Subclaim 2: In the review, the reviewer describes the film as a documentary that includes real footage.

Rebuttal: And yet here we are in a city council meeting, watching councilmembers weigh in on whether to accept a gift of a BearCat armored vehicle. This is but one of the many surreal images

Judge reviews:

  1. 30% -- The rebuttal supports the claim of heavy militarisation, but the quote does not indicate a hypothetical scenario. The supporting claims are not quotations, hence I believe the rebuttal is more reliable.

  2. 50% -- Is the lie that it's a straight documentary, or that it's all real footage? I have questions

  3. 70% -- Rebuttal isn't really helping me decide. Are the hypothetical scenarios just stated? Or filmed with actors? Depiction implies some filming.

  4. 85% -- The subclaims together support the main claim, and the rebuttal doesn't contradict it. I'm still not sure whether the film is a documentary, or primarily fiction.

  5. 88% -- Not more because such claim would be hard to rebut. But supporting claims do directly support the overall claim

  6. 90% -- If the claims are true the premise is true. The 10 is if the experts errored

  7. 95% -- the supporting quotes are very good and I would say that the rebuttal almost supports the main claim...

  8. 95% -- sufficient supporting claims.

  9. 100% -- Supportin Claims are decomposition of top claim

Subclaim 1: In the review, the reviewer describes the film as depicting a hypothetical where the US police are heavily militarized as in fascist states.

Generator credence: 2%

Ensembled step judgement: 58.75% (6 reviews)

Quotes: many surreal images The council ... says yes to the monstrosity, just as seemingly every municipality in the United States eventually says yes to gifts of … tanks … mines fascist mentality

Rebuttal: And yet here we are in a city council meeting, watching councilmembers weigh in on whether to accept a gift of a BearCat armored vehicle. This is but one of the many surreal images

Judge reviews:

  1. 55% -- Rebuttal is helping the claim, or trying to hinder by implying it's not just a hypothetical scenario? Come to think of it, supporting quote also has an implication along these lines.
  2. 55% -- "heavily militarized" ca be argued for by each police department accepting an armored vehicle, I think.
  3. 70% -- All evidences point to increased militarisation
  4. 70% -- Very uncertain. Weird that the rebuttal seems to go in favor of the claim. The supporting quote is not that good.
  5. 80% -- It seems quite likely that 'every municipality eventually says yes' is hypothethical even if tanks are out of context. In addition, the rebuttal does not attack this directly
  6. 80% -- The quote supports the claim, though I'm not sure whether the city acquiring weapons directly translates to militarizing the police. The part about 'gifts' makes it seem like this is not just about a fascist state but something even weirder. So I don't think this is a straight 1984 type hypothetical.

Subclaim 2: In the review, the reviewer describes the film as a documentary that includes real footage.

Generator credence: 99%

Ensembled step judgement: 61.25% (6 reviews)

Quotes: A major downside of this documentary, And in one of the film's most appalling scenes, from a Senate committee meeting in 2014, the Deputy Undersecretary of Acquisition for the Defense Department

Rebuttal: many surreal images The council ... says yes to the monstrosity, just as seemingly every municipality in the United States eventually says yes to gifts of … tanks … mines fascist mentality

Judge reviews:

  1. 20% -- No good evidence that footage is real. Only "Senate committee meeting in 2014", which maintains a certain degree of uncertainty.
  2. 55% -- Unknown if this is real footage of the Senate committee meeting.
  3. 80% -- sufficient supporting quote
  4. 89% -- The quotes support both parts of the claim, though the documentary quote might not refer to the film under review.
  5. 90% -- Documentaries tend to include real footage. Supporting quote indicates documentary and inclusion of real footage
  6. 100% -- Rebuttal doesn't really address the claim, and supporting quotes are strong

32. https://www.rogerebert.com/reviews/to-live-1994

Root claim: In the review, the reviewer expresses that they are glad that the movie’s political statement was distributed globally despite the movie being banned in China

Generator credence: 5% -- null

Tree judgement: 57.5%

Ensembled step judgement: 86% (10 reviews)

Subclaim 1: In the review, the reviewer emphasizes the movie’s political statements about the interpretation of Chinese history

Subclaim 2: In the review, the reviewer emphasizes the movie’s criticism of Chinese Communism

Subclaim 3: In the review, the reviewer describes the movie’s politically charged reception

Subclaim 4: In the review, the reviewer expresses excitement that the movie is spreading despite being banned in China

Rebuttal: It is a big, strong, energetic film, made by a filmmaker whose vision takes in four decades of his nation's history, and who stands apart from all the political currents, and sees that ordinary people

Judge reviews:

  1. 20% -- The supporting claims don't say anything about "globally".

  2. 40% -- The supporting claims don't validate the claim that the movie's political statement was distributed globally

  3. 85% -- The claim has 2 main parts: the reviewer expresses that... A) The are glad the movie's political statement is distributed globally B) The movie was banned in China Supporting claim #1 is good evidence for part of A (that the reviewer finds the movie's political statement important). Supporting claim #4 is very strong evidence for both parts A and B: it proves that the movie is banned in China. Combined with supporting claim #1, it is also very strong evidence for part A. Supporting claims #2 and #3 don't seem to add much given #1 and #4, but they provide additional circumstantial evidence that the movie is political and might be banned in China. The only part of the rebuttal that seems relevant to the root claim is "who stands apart from all the political currents". This could be interpreted as evidence that the movie is not a political statement; however, the quote is ambigouous, and that interpretation is directly contradicted by supporting claim #1. Overall, the claim seems highly likely to be true. The main way the claim could be false is if the the reviewer expresses excitement that the movie is spreading for reasons unrelated to the movie's political statement.

  4. 89% -- assuming excitement is positive

  5. 90% -- Even though the filmmaker stands apart from all the policical currents his movie can have a political message. And the supporting claims are quite convincing (but I am not sure about "globally" - the movie is spreading but where to is unclear)

  6. 90% -- The supporting claims collectively support the claims. although not explictly connecting the reviewer's gladness to the political statement, it seems quite likely. The rebuttal isn't inconsistent with the claim being true - "stands apart from all the political currents" could mean in China (which would support the claim).

  7. 95% -- The rebuttal seems to try to hint that the director is not political but I think that the claim can be true even if the author is not trying to be political. Also, the supporting claims are very strong.

  8. 99% -- All of the supporting claims are strong evidence for the claim.

  9. 100% -- The rebuttal isn't good, the supporting claims directly deal with the claim

  10. 100% -- The supporting claims appear to back all the sub-claims in the root claim and the rebuttal appears to be a direct continuous quote that validates the film is political in nature.

Subclaim 1: In the review, the reviewer emphasizes the movie’s political statements about the interpretation of Chinese history

Generator credence: 3%

Ensembled step judgement: 70% (7 reviews)

Quotes: Jolly murals of Mao Tse Tung appear on the courtyard walls, and then fade, a starving doctor, jailed by the Red Guards, cannot assist at a crucial time because he has gorged himself

Rebuttal: It is a big, strong, energetic film, made by a filmmaker whose vision takes in four decades of his nation's history, and who stands apart from all the political currents, and sees that ordinary people

Judge reviews:

  1. 20% -- I don't see any support for "interpretation".
  2. 55% -- The supporting quotes show that the movie has political statements, but not that the reviewer is emphasizing them. The rebuttal doesn't seem very relevant; it provides weak evidence that the movie is not political ("stands apart from all the political currents")
  3. 85% -- The supporting quotes do reference what appears to be Chinese history. Granted 'Chinese history' does not verbatim appear in the quote and I would be relying on my worldly knowledge of 'Mao Tse Tung' and 'the Red Guards' to make the connection to China over another country. The rebuttal serves to validate the political nature of the movie.
  4. 90% -- murals of Mao Tse Tung ... on the courtyard and a starving doctor, jailed by the Red Guards are both highly controversial political issues about Chinese history and therefore support the claim.
  5. 90% -- The supporting quote is very strong. The rebuttal does not offer much relevant information.
  6. 90% -- it is written so as to sound political
  7. 100% -- Very strong supporting quotes validate the claim, rebuttal supports the claim as well

Subclaim 2: In the review, the reviewer emphasizes the movie’s criticism of Chinese Communism

Generator credence: 3%

Ensembled step judgement: 57.5% (8 reviews)

Quotes: Life is very hard. But they survive under the new communist, ordinary human lives conducted under terrifying conditions, when one's fate could hinge on a chance remark or an instant political edict

Rebuttal: It is a big, strong, energetic film, made by a filmmaker whose vision takes in four decades of his nation's history, and who stands apart from all the political currents, and sees that ordinary people

Judge reviews:

  1. 10% -- I don't see support for "emphasizes", and the rebuttal undermines "criticism".
  2. 50% -- The supporting quotes and the rebuttal appear in support of the claim. The key aspect missing for me is mention of Chinese communism specifically. There are other communist countries that may fit this description. So, I would have to give it a credence equal to chance.
  3. 60% -- it is not clear if it is about China (china or chinese is never mentioned) and communism is currently in vietnam, laos, cuba... but from the description "ordinary human lives conducted under terrifying conditions, when one's fate could hinge on a chance remark or an instant political edict`" it sounds like it (but that might be only my impression because through the media I hear more often about china than the other countries), I think that the rebuttal doesnt contradict the main claim, and that the description of lives of ordinary people critsizes and blames "the new communist"
  4. 70% -- The supporting quotes are somewhat ambiguous, in a way that doesn't seem consistent with the reviewer emphasizing the movie's criticism of Chinese Communism. The first supporting quote is evidence that characters in the movie struggle under communism, but not that reviewer or the movie are criticising communism. The second supporting quote is clearly a criticism of some sort of society/political system, but it doesn't show that the movie is criticizing Chinese Communism in particular. For example, the quote could be about the regime that preceded the "new communist" regime mentioned in the first supporting quote. The quotes given in rebuttal is weak evidence against the claim, particularly the "stands apart from all the political currents". This suggests that the reviewer is not emphasizing the movie's criticism of any political system, presumably including Chinese Communism. However, the rebuttal is much weaker than I would expect if the claim were false (so it actually raises my credence in the claim somewhat). In particular, it appears that the 2nd supporting quote is indeed about Chinese Commuism (or the rebuttal should have shown otherwise), making the claim more likely. Overall, the claim seems more likely than not to be true, but it's somewhat strange that the supporting quotes aren't more explicit, given that the reviewer supposedly "emphasizes" the movie's criticism of Communism.
  5. 90% -- Strong supporting quotes
  6. 90% -- The quote seems highly coherent with the claim.
  7. 90% -- The supporting quote is very strong, the rebuttal does not offer much.
  8. 95% -- life is called hard under Communism. political edicts are 'terrifying'. This is a political criticism.

Subclaim 3: In the review, the reviewer describes the movie’s politically charged reception

Generator credence: 99%

Ensembled step judgement: 95% (5 reviews)

Quotes: The honesty of "To Live" earned Zhang Yimou and Gong Li not only a two-year ban on further co-productions, but a ban on even speaking about their film.

Rebuttal: It is a big, strong, energetic film, made by a filmmaker whose vision takes in four decades of his nation's history, and who stands apart from all the political currents, and sees that ordinary people

Judge reviews:

  1. 90% -- The claim is well supported.
  2. 95% -- The rebuttal tries to sway me by citing the apolitical stance of a filmmaker but the supporting quote is very strong.
  3. 98% -- a ban on speaking? What a politically charged reaction to the film!
  4. 100% -- The quote highly support the claim by giving more specific information that are coherent with the claim.
  5. 100% -- The supporting quote is continuous and strong support for 'politically charged reception' as the apparent co-creators were banned for 2 years on any co-productions and banned 'on even speaking about their film.'

Subclaim 4: In the review, the reviewer expresses excitement that the movie is spreading despite being banned in China

Generator credence: 99%

Ensembled step judgement: 90% (6 reviews)

Quotes: ban on further co-productions, but a ban on even speaking about their film. But "To Live" has been made, it is playing all over the world, exciting to see these new films as they emerge from China

Rebuttal: people everywhere basically want what his heroine cries out for, a quiet life. It is exciting to see these new films as they emerge from China. They are history being written, celebrated, and mourned

Judge reviews:

  1. 75% -- Following the supporting quote, only interviews about the film and co-productions are banned in China. But it is nevertheless fairly safe to assume, that the film is also banned there.
  2. 90% -- The claim is supported and not rebutted.
  3. 90% -- Strong supporting quotes
  4. 95% -- The supporting quote is very strong. The rebuttal does not offer much relevant information.
  5. 98% -- exciting to see these new films as they emerge from China is a rephrase of Claim
  6. 100% -- The supporting quotes and the rebuttal work together to create longer continuous quote of greater context. There is adequate evidence to support the movie spreading despite its ban in China.

33. https://www.rogerebert.com/reviews/footloose-1984

Root claim: In the review, the reviewer suggests that the movie alludes to Puritan religion in America.

Generator credence: 4% -- There is no argument for this even if themes overlap somewhat.

Tree judgement: 52%

Ensembled step judgement: 52% (7 reviews)

Subclaim 1: In the review, the reviewer asserts that the movie is set in a US small town which has banned all dancing and the ban is led by a preacher.

Subclaim 2: In the review, the reviewer suggests that preacher learning that his daughter is not a virgin causes great consternation.

Subclaim 3: In the review, the reviewer suggests that the preacher’s response to people burning books at the library is an important plot point.

Rebuttal: `ban on rock 'n' roll and dancing. The ban is led by an uptight preacher named Shaw Moore (John Lithgow), who is still grieving because he lost a child in a car wreck five years ago.``

Judge reviews:

  1. 12% -- The supporting claims taken together do not imply that the review mentions Puritan religion at all, even though this should be an easy claim to prove if true. The most charitable interpretation is that the supporting claims each show that the movie is set in a town with elements common to Puritan religion, but this is unlikely to be the strongest evidence for the root claim if the root claim is actually true. Additionally, the rebuttal shows that the preacher died in a car wreck, meaning that the movie is set in a time period after the Puritan religion died out. Maybe I'm missing something, but the root claim seems pretty unlikely, even assuming the supporting claims are true.

  2. 45% -- The claims do not establish the religion in the small town.

  3. 59% -- I have some knowledge of Puritan religion. However, I am going to approach this without that knowledge. Puritan religion is a very specific mention in the claim and the supporting claims fail to mention it verbatim. I can substantiate religion by mention of 'preacher' in two of the supporting claims and 'in America' as #1 says 'US small town.' With out mention or definition of Puritan, the film could be mentioning any number of faith systems. The rebuttal with noting a car wreck also places the time frame of the movie well outside of the horse transportation era of early Puritans.

  4. 75% -- I'm not sure how to ascertain Puritan part of the claim, rest seems alright.

  5. 80% -- The supporting claims substantiate the main claim and the rebuttal does not directly contradict it and renders some support to the main claim.

  6. 90% -- The supporting claims clearly prove the main claim by showing the film takes place in America (#1) and that it deals with puritanism and religion (dancing ban lead by a preacher, loss of a woman's virginity causes consternation, book burning). The rebuttal does not disprove the claim (it even makes it stronger by showing the preacher and rock and roll ban are in fact featured in the review), and merely tries to deflect attention by pointing to the death of the preacher's son, which is irrelevant. 90% because "1 is not a probability".

  7. 92% -- I'm a bit confused why 'Puritan' is with capital P, but otherwise, given claims strength ('alludes'), it seems well supported by claims and rebuttal does not contradict

Subclaim 1: In the review, the reviewer asserts that the movie is set in a US small town which has banned all dancing and the ban is led by a preacher.

Generator credence: 99%

Ensembled step judgement: 57.5% (6 reviews)

Quotes: Bomont, which is somewhere in the Midwest old fuddies in Bomont have imposed a total ban on … dancing. The ban is led by an uptight preacher

Rebuttal: although I seriously doubt a town like this exists anywhere In the very next scene, the preacher is arguing against the book burners

Judge reviews:

  1. 10% -- The supporting quote has an ellipses placed just after a prepositional phrase so while it says 'dancing' after that it could easily be any other proper noun, noun, or verb. The rebuttal suggests instead the ban was on books and that the town mentioned may be fictional.
  2. 50% -- It is unclear if the ban is on dancing. There is a gap in the quote. The rebuttal contains hints that the gap could be book burners.
  3. 80% -- Rebuttal fails to address the claim itself, but "total ban on ... dancing" is a suspicious cut
  4. 90% -- The supporting quotes directly show that the movie is set in a town that has banned something, and that the ban is led by a preacher. If the ban were about something other than dancing, the rebuttal should have included proof that it was about something else. In fact, the rebuttal appears to be irrelevant, making the claim highly likely to be true.
  5. 92% -- The claim is literally supported by quotes unless there is important missing context to '... dancing'. The rebuttal does not show any, though
  6. 95% -- 'dancing. The ban is led by an uptight preacher' is near-verbatim evidence. 'Bomont, which is somewhere in the Midwest is strong but not explict evidence of 'US small town',

Subclaim 2: In the review, the reviewer suggests that preacher learning that his daughter is not a virgin causes great consternation.

Generator credence: 95%

Ensembled step judgement: 85% (5 reviews)

Quotes: preacher's daughter confronts her dad in church and announces she isn't a virgin … The preacher turns livid, starts to scream

Rebuttal: the preacher, who is played by Lithgow… error with the Lithgow character. It sets him up as an unyielding reactionary, and then lets him change his mind 180 degrees without a word of explanation.

Judge reviews:

  1. 85% -- The supporting quotes follows the claim as noting that one is not a virgin in church would seem an odd place to share that type of personal information. Similarly, the preacher turning livid would make sense especially if the church disapproved of premarital sex. I did notice that the quotes used 125 characters leaving 75 of unspent budget. The supporting quote does contain and ellipses so was there too much there to include the 75 characters, was the choice made out of convienience, or is there an attempt to change the meaning or emphasis. The rebuttal while noting the preacher character capacity to be a reactionary and also capacity to change his mind, fails to reference what the preacher reacted towards. So, this is more likely than not but less than 100 %
  2. 85% -- There is a gap in the quote that could be exploited. The rebuttal did not show the gap and therefore it is likely to be honest. This means that it is more likely true than false.
  3. 85% -- The supporting quote proves that the movie contains a preacher who learns that his daughter is not a virgin, and that something caused the preacher great consternation. Due to the ellipses between the 1st and 2nd quotes, I don't know for sure if the preacher "turns livid, starts to scream" because he learns that his daughter isn't a virgin, but it seems pretty likely. The rebuttal isn't very strong: it confirms that the preacher is an unyielding reactionary at some point (making it more likely he would "start to scream" after learning his daughter isn't a virgin). The rebuttal shows that the preacher changes his mind at some point, but provides no evidence that the change occurs before learning his daughter is a virgin. Overall, it's quite likely that the claim is true, even if the supporting quotes don't give ironclad proof.
  4. 92% -- The claim is quite literally supported. There could be some missing context, but rebuttal doesn't attempt to add it.
  5. 100% -- Rebuttal is too general to work

Subclaim 3: In the review, the reviewer suggests that the preacher’s response to people burning books at the library is an important plot point.

Generator credence: 80%

Ensembled step judgement: 65% (5 reviews)

Quotes: interrupted by news that they're burning books down at the library. In the very next scene, the preacher is arguing against the book burners

Rebuttal: the preacher, who is played by Lithgow… error with the Lithgow character. It sets him up as an unyielding reactionary, and then lets him change his mind 180 degrees without a word of explanation.

Judge reviews:

  1. 55% -- If I include other information I've seen about this movie, I'd give this 65%. Rebuttal isn't really addressing the claim, and supporting quote doesn't demonstrate importance of the event for the plot. I assume a completely plot irrelevant scene wouldn't be there with the mentioned content.
  2. 65% -- It's fairly hard to establish whether something is an important plot point or not in a short quote, but the supporting evidence makes it slightly more likely than not. The reviewer thought burning books was important enough to spend two sentences on, and apparently the book burning is relevant to two scenes in the movie. The rebuttal seems irrelevant -- the movie making an error with the preacher character isn't completely consistent with the preacher's response to the book burnings being an important plot point. Still, it's hard to be confident that the claim is true when the supporting quotes do not directly assert that the response is an important plot point. I'm giving a credence of 65% because I think a random sentence in a review that describes part of the plot is roughly 65% likely to be an important plot point.
  3. 70% -- The Rebuttal hints at tensions around book burning that resulted in a flip of the preacher. This actually strengthens the plot point claim. Nothing is explicit though.
  4. 80% -- The supporting quote establishes that book burning occurs and that the preacher responds to it but fails to include any reviewer commentary that it is 'an important plot point.' The rebuttal offers up that Lithgow plays the preacher and that his character made an error in going 180 degrees without explanation. This rebuttal would lend more credence if I could substantiate it was in reference to the book burning and not another occurrence in the film. So the claim is likely to be true but less than 100 %
  5. 85% -- The quotes support the book burning and response as happening, but isn't explicit that it's important. However, it being mentioned is suggestive of that. The rebuttal seems mostly irrelevant

34. https://www.rogerebert.com/reviews/serial-1980

Root claim: In the review, the suggests describes the movie as a comedy that parodies science fiction and horror movies and satirizes society.

Generator credence: 2% -- No science fiction references. But is a comedy that satirizes.

Tree judgement: 51%

Ensembled step judgement: 51% (10 reviews)

Subclaim 1: In the review, the reviewer describes the movie as a comedy

Subclaim 2: In the review, the reviewer describes in the movie where the residents are a kind of zombie

Subclaim 3: In the review, the reviewer describes people in the movie as aliens who look and act like humans

Subclaim 4: In the review, the reviewer states that the movie was based on a satirical book

Subclaim 5: In the review, the reviewer describes the movie as featuring religious cults

Rebuttal: `Nobody in this movie are real enough for us to identify with; they're seen as such completely robotized victims of the Me Decade that they don't have enough freedom of choice to make their...``

Judge reviews:

  1. 20% -- The supporting claims don't attempt to validate the claim (except sc1) and aren't very convincing in itself.

  2. 35% -- 'describes people in the movie as aliens' is an unusually weak support for 'science fiction'. It could be just an analogy and I would expect more evidence for such simple claim. Rebuttal also points in this direction

  3. 48% -- I am not sure about "parodies science fiction and horror"- it is not mentioned anywhere in the supporting quotes. And if it was a parody would the reviewer be dissapointed that we cant identify with anyone?

  4. 60% -- no good evidence for "satirizes society". But there is evidence for the overall weirdness of the claim that made me think it was wrong

  5. 65% -- No information about science fiction present. Also no evidence for parodying science fiction.

  6. 70% -- The first and fourth supporting claims are very strong, the second and third not so. "the residents are a kind of zombie" could also mean that they are unresponsive and not interact much with the main character. You could probably find a similar explanation for the third claim. The fifth supporting claim does not seem to be much relevant to the claim without more context. The rebuttal does offer somehow relevant information but it still not strong enough to be sure that the claim is false, esppecial because there are 5 subclaims so there should be enoug ways how to disapprove the main claim.

  7. 70% -- The supporting claims substantiate the main claim and the rebuttal does not directly contradict it.

  8. 70% -- Although the claim is very vague, the supporting claims seem to generally support it

  9. 75% -- The claim can be divided into 4 parts: the reviewer suggests describes the movie as... A) a comedy B) a parody of science fiction movies C) a parody of horror movies D) satirizing society Supporting claim #1 directly proves A. Supporting claim #2 provides some support for C (zombies are common in horror movies, but this doesn't establish that the movie is parodying horror movies). Supporting claim #3 provides some support for B (again, aliens are common in SF movies but this doesn't show that the movie is a SF parody). Supporting claim #4 provides some support for B, C, and D by making it more likely that the movie is a parody and/or satire (though it doesn't explicitly show that the movie satirizes society). Supporting claim #5 provides weak support for D, as religious cults are a feature of society that is often parodied. The main weak points in the claim are: 1) "the movie is parodying SF/horror movies" 2) "the movie is satirizing society" The rebuttal doesn't seem relevant to the claim. Overall, the claim seems likely but not certain to be true conditional on the supporting claims being true.

  10. 98% -- The supporting claims do stack up well to substantiate the root claim. The rebuttal describes the quality of the character development but doesn't talk much about the film topic.

Subclaim 1: In the review, the reviewer describes the movie as a comedy

Generator credence: 98%

Ensembled step judgement: 95% (9 reviews)

Quotes: It is not a very good movie; it's one of those comedies where the occasional genuine laughter is punctuated by long periods of silence.

Rebuttal: Nobody in this movie are real enough for us to identify with; … robotized victims of the Me Decade that they don't have enough freedom of choice to make their behavior meaningful - or funny

Judge reviews:

  1. 55% -- The author does describe the movie as a comedy, although it doesn't seem very effective in terms of being comical or entertaining
  2. 80% -- The supporting quote states that the movie is a comedy but is badly done (which makes it still a comedy) and the rebuttal quote reiterates this point.
  3. 95% -- Supporting quote explicitly shows the movie being called a comedy.
  4. 95% -- Claim seems well supported by quotes, rebuttal doesn't contradict.
  5. 95% -- The supporting quote explicitly describes the movie as a comedy. The rebuttal is consistent with the movie being a comedy, just not a good comedy. As the supporting evidence admits that it's "not a very good movie", the rebuttal doesn't reduce my credence in the claim at all. The main way the claim could be wrong is if the supporting quote is describing a different movie, but if that's the case then the rebuttal should have provided evidence to that effect.
  6. 99% -- "it's one of those comedies" - it is a comedy
  7. 100% -- The supporting quotes identify the movie as a comedy verbatim. The rebuttal only talks about how believable the characters are.
  8. 100% -- direct evidence
  9. 100% -- The quote fully supports the claim by stating: it's one of those comedies

Subclaim 2: In the review, the reviewer describes in the movie where the residents are a kind of zombie

Generator credence: 30%

Ensembled step judgement: 60% (7 reviews)

Quotes: Marin is across the bay from San Francisco, and … is inhabited by a race of upper-middle-class zombies mesmerized by the social trends

Rebuttal: `Nobody in this movie are real enough for us to identify with; they're seen as such completely robotized victims of the Me Decade that they don't have enough freedom of choice to make their...``

Judge reviews:

  1. 40% -- From the quote, the claim seem true, but misleading. According to my understanding of the rules, I interpret this as rather false
  2. 45% -- Perhaps 'zombies' in the quote is a figure of speech, not literal?
  3. 75% -- Seems somewhat true due to inhabited by a race of upper-middle-class zombies`
  4. 80% -- The supporting quote is strong evidence for the claim; the main way I can think of for the claim to be false is if "upper-middle-class zombies" is merely metaphor or hyperbole. The rebuttal provides mediocre evidence for this: if the author feels that "nobody in the movie is real enough to identify with", then it's more likely that they would describe some non-zombie characters as metaphorical "zombies". However, the inclusion of a specific region that's inhabited by "zombies" makes it fairly likely that the supporting quote is referring to literal zombies.
  5. 90% -- is inhabited by a race of upper-middle-class zombies - very good evidence and the rebuttal has nothing to offer
  6. 95% -- The supporting quotes are credence for the root claim in a literal way but perhaps loosely in a figurative sense. "A kind of Zombie" I am interpreting to mean here to include the figurative sense. The rebuttal would reinforce this figurative interpretation.
  7. 99% -- direct evidence

Subclaim 3: In the review, the reviewer describes people in the movie as aliens who look and act like humans

Generator credence: 10%

Ensembled step judgement: 76.5% (8 reviews)

Quotes: It shows us an alien life form with a superficial resemblance to the human being. These creatures ride bicycles to the ferry to conserve fuel.

Rebuttal: `Nobody in this movie are real enough for us to identify with; they're seen as such completely robotized victims of the Me Decade that they don't have enough freedom of choice to make their...``

Judge reviews:

  1. 20% -- The use of the term 'aliens' are a figure of speech
  2. 51% -- These creatures ride bicycles to the ferry to conserve fuel- that is just one example, otherwise they might behave unlike humans
  3. 85% -- The claim seems to be true due to the fact, that it is supported by the spporting quotes.
  4. 87% -- The supporting quote is moderately strong evidence for the claim. My main doubt is whether "superficial resemblance to a human being" is sufficient to describe the aliens as "looking and acting" like humans. I don't think that the difference is very important from the perspective of someone deciding whether to see the movie, nor do I think it would be important for most top-level claims. Thus, I think the claim is quite likely to be true. "These creatures ride bicycles to the ferry to conserve fuel" is further weak evidence of "acting like humans" -- this seems like something that humans would plausibly do, but also like something aliens would plausibly do (i.e. it's not distinctively human). The rebuttal seems irrelevant to the claim. "Nobody in this movie are real enough for us to identify with" might be a weak argument that the aliens don't "act like humans", but I think that the two phrases have completely different meanings (I'm not sure if this was the intended point of the rebuttal quote or not).
  5. 90% -- The supporting quote is strong and in the case, the claim is a metaphor for something, the rebuttal does not show it clearly...
  6. 90% -- I can't think of any other interpretation of 'It shows us an alien life form'
  7. 100% -- The supporting quotes provide literal support for the root claim. The rebuttal talks about the level of character development.
  8. 100% -- direct evidence

Subclaim 4: In the review, the reviewer states that the movie was based on a satirical book

Generator credence: 99%

Ensembled step judgement: 90% (9 reviews)

Quotes: "Serial" first appeared as a California best seller by Cyra McFadden, who satirized life in Marin as if it were a continuing soap opera. And now we have the movie

Rebuttal: Everybody out here is debating whether the movie is accurate or not." That anyone could argue whether "Serial" is accurate is mind-boggling.

Judge reviews:

  1. 85% -- The supporting quote substantiates the claim and the rebuttal does not address it.
  2. 90% -- Claim seems well supported by quote, rebuttal doesn't contradict.
  3. 90% -- The supporting quote decisively shows that "Serial" is a satire, and suggests that the movie being reviewed is based on Serial ("and now we have the movie"). The rebuttal actually seems like additional evidence for the claim: by linking "Serial" with "the movie", it makes it much more likely that the movie being reviewed was in fact based on the book "Serial". Apart from that, the rebuttal appears to be irrelevant -- the accuracy of "Serial" doesn't affect whether it's a satire.
  4. 93% -- "best seller" probably means it was a book and "satirized life" suggest it was satirical
  5. 95% -- Rebuttal is not really a rebuttal? Supporting quote gives strong evidence.
  6. 97% -- the link between the book and the movie is not totally certain, but very probable (why else would someone mention the book in the review?)
  7. 99% -- The quote clearly indicates, that the movie was based on a satirical book.
  8. 100% -- The supporting quotes identify "serial" as a satirical movie based upon a book. The rebuttal only mentions debating accuracy which would appear to support the root claim as it would make sense to be confused about believing a satire is accurate.
  9. 100% -- Strong supporting quote. The rebuttal validates the claim further

Subclaim 5: In the review, the reviewer describes the movie as featuring religious cults

Generator credence: 99%

Ensembled step judgement: 95% (9 reviews)

Quotes: Their kids join obscure religious cults days of the flower children, with incantations about the earth and the sky a bike gang raid on a cult headquarters

Rebuttal: ``

Judge reviews:

  1. 70% -- Strong supporting quotes, non-existent rebuttal
  2. 85% -- The supporting quotes substantiate the claim and there is nothing in the rebuttal.
  3. 95% -- The supporting quotes are fairly strong evidence that the movie features religious cults, and the lack of any rebuttal quotes is further confirmation
  4. 99% -- The supporting quote is strong and the rebuttal is missing...
  5. 99% -- Claim seems clearly supported, no rebuttal
  6. 100% -- The supporting claims verbatim substantiate the root claim. The rebuttal here is a blank pointer ???
  7. 100% -- ok evidence, but no rebuttal
  8. 100% -- There is complete evidence to be found in the supporting quotes.
  9. 100% -- very good evidence, no rebuttal

35. https://www.rogerebert.com/reviews/bad-boys-1995

Root claim: In the review, the reviewer argues that the plot leaves the audience wishing for answers to unanswered questions.

Generator credence: 5% -- The reviewer asks some questions about the plot, but answers some of them, and the rest are rhetorical, intended to convey that the plot makes no sense.

Tree judgement: 50%

Ensembled step judgement: 60% (9 reviews)

Subclaim 1: In the review, the reviewer argues that the audience wishes they knew more about what was going on between the characters.

Subclaim 2: In the review, the reviewer asks questions about things left unanswered in the movie

Rebuttal: The answer, of course, is that the photos are there to support the vacuum-brained dialogue. The climax is ideal for those with attention-span deficit. it's out of those Idiot Plots

Judge reviews:

  1. 10% -- "reviewer asks questions" does not support "audience wishing for answers".

  2. 30% -- The rebuttal is stronger than the supporting claims

  3. 60% -- The rebuttal is really strong. But the 1st supporting claim directly contradicts it, and since we're assuming a team of highly skilled reviewers verified the supporting claim, I must trust that they know that the quote in the rebuttal must be out of context.

  4. 60% -- The rebuttal may support that some questions are unanswered (climax...attention-span deficit), but decreases my credence audience wishes answers (Idiot Plots). The first supporting claim may be related to questions during the plot, rather than questions about film overall.

  5. 70% -- it seems well supported. I am not sure if "audience wishes they new more about what was going on between the characters" equals to "the plot leaves the audience wishing for answers to unanswered questions" but it probably does

  6. 75% -- The supporting claims are both quite strong evidence for the claim, given they are presumed to have high credence. The rebuttal does seem to suggest that the movie does assist somewhat in helping parse the plot, but that doesn't mean it was necessaro;y sufficient.

  7. 80% -- The supporting quotes are quite strong, the rebuttal does not offer much relevant.

  8. 93% -- 1. being correct would give ~88% credence. It's almost the same than the top claim. 2. adds about 5% more credence. Uncertainty is that the rebuttal makes it seem as if some of these statements might have been said in an ironic manner

  9. 96% -- reviewer literally says things are left unanswered

Subclaim 1: In the review, the reviewer argues that the audience wishes they knew more about what was going on between the characters.

Generator credence: 40%

Ensembled step judgement: 50% (8 reviews)

Quotes: There are also a lot of curious interludes in which Lawrence and Smith do verbal riffs…finishing each other's sentences…We wish we knew them well enough to figure out what they think they're saying.

Rebuttal: each other's sentences or not bothering to complete thoughts at all, to show a kind of easy familiarity, I guess. We are glad they know each other so well. We wish vacuum-brained dialogue

Judge reviews:

  1. 10% -- "we wish" does not support "the audience wishes".
  2. 20% -- The rebuttal contradicts the claim by providing more context to the supporting quote.
  3. 60% -- The supporting quote directly states what is in the claim. But the connotation of the claim is strongly contradicted by the rebuttal. I think that the rebuttal would be more useful to help someone decide if they want to watch the movie, but that's not the question at hand -- instead, I need to say how likely it is that the claim is true. So I'm going with the supporting quote. It directly states the claim, even if it may be misleading to someone deciding whether to watch the film.
  4. 70% -- Both the support quotes and rebuttal support the idea that two characters "finish each other's sentences". They have an "easy familiarity", but the audience will find it hard to "figure out what they think they're saying". The former is to some degree a description of "what is going on" between the characters, but the latter seems stronger.
  5. 83% -- "We wish we knew them well enough to figure out what they think they're saying." - pretty convincing
  6. 85% -- Quotes and rebuttal complement. After reading both, I'm not sure if the claim is misleading overall, but on its own it seems almost literally true
  7. 95% -- Solid evidence
  8. 97% -- sounds like a really annoying dialogue option for the audience.

Subclaim 2: In the review, the reviewer asks questions about things left unanswered in the movie

Generator credence: 75%

Ensembled step judgement: 90% (5 reviews)

Quotes: For example, why is Lawrence's apartment filled with photographs of Smith? Is Lawrence gay? Whom do they make these movies for?

Rebuttal: The answer, of course, is that the photos are there to support the vacuum-brained dialogue. The climax is ideal for those with attention-span deficit. it's out of those Idiot Plots

Judge reviews:

  1. 60% -- The supporting quotes list multiple questions that are left unanswered in the movie, so the claim is successfully supported. This is true even though the rebuttal makes it clear that the claim would be misleading to people who are deciding whether to watch the film. But I won't put a strong confidence level here because I can see how someone else might want to say that the claim is false because it gives the wrong connotation.
  2. 90% -- The supporting quotes show the reviewer asking questions.
  3. 90% -- The 'reviewer ask questions' is literally supported. It's unclear if the questions are really unanswered in the movie. The rebuttal increases my credence because it seems to show the questions are unanswered in the movie, and the reviewer only finds answers on the meta-movie level
  4. 99% -- my god, they do ask questions. What sass.
  5. 100% -- direct evidence

36. https://www.rogerebert.com/reviews/red-sorghum-1989

Root claim: In the review, the reviewer explains that "Red Sorghum" tells the tale of a widow in WW2 China who leads her workers against the invading Japanese.

Generator credence: 9% -- Nothing about the widow is mentioned after the Japanese invasion. Before that she’s said to take charge, so it’s kind of vague. This kind of individualistic movie isn’t likely at all in China, I think.

Tree judgement: 40%

Ensembled step judgement: 40% (9 reviews)

Subclaim 1: In the review, the reviewer explains that the widow takes charge of her husband’s winery after he dies

Subclaim 2: In the review, the reviewer says the events in the movie takes a drastic turn asWW2 starts and Japan invades

Subclaim 3: In the review, the reviewer says the US audience would be more interested in the melodrama than the workers’ resistance against the Japanese

Rebuttal: Japanese troops invade the area, flattening the fields with forced labor from the vineyard. The workers revolt against the Japanese, and after their uprising is crushed, the Japanese order two of the

Judge reviews:

  1. 10% -- No evidence that the workers are her workers

  2. 20% -- Taking charge of the winery is not the same as leading the workers.

  3. 40% -- There does appear to be a widow whose workers revolt against the invading Japanese. There's no indication that the widow leads it, however, and the rebuttal makes me think it may be relatively minor part of the movie (but maybe she leads a later revolt?). Supporting claim #3 indicates there's a melodrama involved - possibly the widow is primarily involved with something other than the worker's resistance. However, if this is the case, why didn't the rebuttal include it?

  4. 40% -- The rebuttal supports parts of the claim, (though it's not clear the widow actually leads the workers), but the quote about melodrama makes it sound like there's a second subplot in the movie. Actually the rebuttal seems like the revolt is not a very large part of the story. On reconsidering I think that if the movie was actually primarily about the uprising the reviewer would have phrased things differently.

  5. 40% -- The main parts of the claim are that the reviewer explains: A) the movie is set in WW2 china B) the movie tells the tale of a widow with workers C) the workers resist the invading Japanese D) The widow leads the workers' resistance Supporting claim #1 is strong evidence that there is a widow who owns a business in the movie (though it's unclear if the widow is a main character) Supporting claim #2 is strong evidence that the movie is set in Asia during WW2 (though not China specifically) Supporting claim #3 is strong evidence that the workers resist the Japanese Overall, Part C is clearly supported, parts A and B are partially supported, and part D is unsupported. The rebuttal appears to be mostly irrelevant, but it may be weak evidence against the claim by suggesting that the workers' revolt was unrelated to the widow. The main problem with the claim is the lack of support for part D; I think this is slightly more likely to be malicious than to be an honest oversight, but I'm very uncertain. If I learned that the 3 supporting claims were true of a random review (and weren't adversarially selected), I would be fairly confident that part D is true, so my credence largely depends on my beliefs about the meta-game. However, I don't know much about the meta-game yet. :/

  6. 79% -- the 'leads her workers' part is only indirectly supported. Rebuttal does not contradict

  7. 80% -- Except for the part that mentions that the widow is the leading figure of the uprising, the claim is well supported

  8. 88% -- Supporting claims are quite exhaustive. Rebuttal upholds Claim.

  9. 100% -- the rebuttal proves the claim

Subclaim 1: In the review, the reviewer explains that the widow takes charge of her husband’s winery after he dies

Generator credence: 99%

Ensembled step judgement: 94.25% (6 reviews)

Quotes: pre-arranged marriage with a much older man. The good news was that he owned a vineyard. The bad news was that he had leprosy., The leper dies, and the young widow takes over control of the winery

Rebuttal: He now is an alcoholic, who gets drunk one night and urinates into a vat of the wine. But somehow his urine serves as a catalyst, and the vintage tastes better than ever before.

Judge reviews:

  1. 90% -- The claim is supported but not rebutted.
  2. 94% -- Claim seems well supported, rebuttal does not contradict
  3. 95% -- the supporting quote: The leper dies, and the young widow takes over control of the winery`is pretty much the paraphrase of the main claim
  4. 95% -- Rebuttal doesn't contradict, the quote supports the claim well.
  5. 95% -- The supporting quote directly states that the widow tkes charge of her husband's winery. The rebuttal quote appears to be irrelevant.
  6. 100% -- direct evidence

Subclaim 2: In the review, the reviewer says the events in the movie takes a drastic turn asWW2 starts and Japan invades

Generator credence: 99%

Ensembled step judgement: 82.5% (6 reviews)

Quotes: Then "Red Sorghum" takes an unexpected turn into realism, as World War II begins and Japanese troops invade the area Japanese order two of the local people skinned alive in front of the others

Rebuttal: escorts fights off the assailant, and then slips away into the fields - only to accost her the next day in a raid of his own. But she is grateful to him for having saved her life, and they make love.

Judge reviews:

  1. 55% -- While, it's true the Japanese invading changes the tone of the movie, it doesn't seem like drastic turn is completely justified.
  2. 80% -- The claim is not rebutted.
  3. 90% -- The supporting quote explicitly states that the movie takes an "unexpected turn", as WW2 starts and Japan invades. The only way I can see for the claim to be false is if there's good reason to think that the turn isn't "drastic". However, I'd say that an "unexpected turn" that results in people being skinned alive could reasonably be described as "drastic". The rebuttal seems fairly irrelevant -- maybe it's trying to argue that the turn wasn't drastic, but it's not very strong evidence.
  4. 97% -- Claim clearly supported by single quote, rebuttal does not contradict
  5. 99% -- the claim is fully supported by supporting quotes (expecially the first one)
  6. 100% -- direct evidence

Subclaim 3: In the review, the reviewer says the US audience would be more interested in the melodrama than the workers’ resistance against the Japanese

Generator credence: 95%

Ensembled step judgement: 80% (7 reviews)

Quotes: "Red Sorghum" perhaps can be read ... as a hymn in praise of the way the workers resisted the Japanese invaders. Western audiences probably are going to be more interested in the melodrama

Rebuttal: "Red Sorghum" perhaps can be read as a parable of China's development

Judge reviews:

  1. 60% -- The rebuttal seems like strong evidence that the ellipses are misleading but I don't see how that affects the strength of the claim. The supporting quotes clearly show that the US audience would be "more interested than the melodrama". However, it's possible that the review compares the melodrama to something other than "the workers' resistance against the Japanese" (in which case the rebuttal would become more relevant). If so, the claim could easily be false. For example, it seems plausible that the review is saying "Western audiences would be more interested in the melodrama of the workers' resistance than in [some other theme of the movie, like the parable of China's development].
  2. 80% -- the main claim says "US audience" and the supporting quote says "western audiences". it is correct, but somewhat less precise, I wonder if that might be the catch?
  3. 80% -- The claim is supported but not rebutted.
  4. 95% -- The rebuttal doesn't affect the relevant part of the quote.
  5. 98% -- Western audiences probably are going to be more interested in the melodrama`
  6. 99% -- Clearly supported by a single quote
  7. 100% -- direct evidence

37. https://www.rogerebert.com/reviews/eat-that-question-frank-zappa-in-his-own-words-2016

Root claim: In the review, the reviewer discusses musicians Frank Zappa and Prince and praises the testimony of Prince during the PMRC hearings.

Generator credence: 1% -- he praises Zappa not Prince.

Tree judgement: 36.25%

Ensembled step judgement: 99% (9 reviews)

Subclaim 1: In the review, the reviewer discusses musician Frank Zappa.

Subclaim 2: In the review, the reviewer discusses musician Prince.

Subclaim 3: In the review, the reviewer praises the testimony of Prince during the PMRC hearings.

Rebuttal: ``

Judge reviews:

  1. 97% -- the supporting claims completely entail the claim. Thus very high credence.

  2. 99% -- Well supported, no rebuttal

  3. 99% -- The supporting quote is very strong and the rebuttal is missing.

  4. 99% -- The subclaims match almost verbatim to the claim, and the rebuttal is blank

  5. 99.9% -- The supporting claims support the main claim verbatim, and there is no rebuttal

  6. 100% -- there's no rebuttal

  7. 100% -- The supporting claims match exactly to sub-claims I would pull out of the claim statement. The rebuttal is left empty so there is a lack of anything to challenge the fictitious expert reviewers.

  8. 100% -- No rebuttal

  9. 100% -- The supporting claims provide complete evidence for the claim when taken in total

Subclaim 1: In the review, the reviewer discusses musician Frank Zappa.

Generator credence: 99%

Ensembled step judgement: 99% (8 reviews)

Quotes: Frank Zappa’s impact on 20th century music, both rock and classical, still hasn’t been entirely reckoned with

Rebuttal: ``

Judge reviews:

  1. 99% -- the supporting quote does what the claim claims.
  2. 99% -- The supporting quote is very strong and the rebuttal is missing.
  3. 99% -- Based on the supporting quotes, the review clearly discusses Frank Zappa. Given that Zappa had a huge impact on music, he was probably but not certainly a musician (he could also be e.g. a producer). The blank rebuttal makes me more confident that Zappa was indeed a musician. (Note that I'm assuming that a blank review is tantamount to saying "there is no evidence contradicting the claim". I'm still not sure if this is a reasonable assumption or not, though it doesn't affect my credence substantially in this case). Actually, I believe that I'm allowed to look up whether Zappa was a musician -- it's likely that this is context that other participants would know. My credence in the claim before looking this up is 90%. Google confirms that Frank Zappa is a musician, so my credence is now 99%.
  4. 100% -- No rebuttal. Then again, the claim is so general it's kind of unfair
  5. 100% -- The supporting quote clearly references Frank Zappa and references his impact on the music scene. The rebuttal is empty of characters so there is lack of anything to offer a challenge.
  6. 100% -- Well supported, no rebuttal
  7. 100% -- No rebuttal
  8. 100% -- The supporting claim fully proves the claim.

Subclaim 2: In the review, the reviewer discusses musician Prince.

Generator credence: 99%

Ensembled step judgement: 36.25% (8 reviews)

Quotes: In a couple of weeks two albums of previously unheard music will see official release. His vault is as legendary to fans of his work as Prince’s vault is to fans of Prince. I’m a fan of both

Rebuttal: “Eat That Question: Frank Zappa in His Own Words,” an archival-footage documentary

Judge reviews:

  1. 15% -- So the supporting quote references Prince as a comparison which while suggestive of Prince being included in the review it is not direct evidence of such. The rebuttal shows the tile of the film reviewed "Eat that Question: Frank Zappa in His One Words" so it is possible that Prince might not be in the picture at all.
  2. 25% -- Clearly Prince is mentioned in the review, but it appears that the documentary is about Frank Zappa. It's ambiguous, but I think most people would intepret the claim as to mean the review is primarily about Prince, which appears to be false. (Or at least, include a lengthy discussion of Prince, when it may just be a one-off mention).
  3. 40% -- The supporting quote only supports Prince beeing mentioned, not really beeing discussed.
  4. 89% -- Is the quote enough to justify the claim that the reviewer discusses Prince? Does the first part of the supporting claim also refer to Prince? I guess, but these questions reduce the credence slightly.
  5. 96% -- The review clearly discusses Prince. It looks like the review only mentions Prince as an aside, and the rebuttal seems to show that the main subject of the review is a film about Zappa, but that's still consistent with the claim made. I know that Prince is a musician, so I'm quite confident that this claim is true. The most likely way for the claim to be false is if the Prince mentioned is someone other than the musician, but that's quite unlikely since this Prince's work is being compared to "two albums of previously unheard music".
  6. 99% -- The review really mentions Prince name, I think that it is impossible to rebut.
  7. 100% -- The rebuttal doesn't address and therefore doesn't disprove the claim
  8. 100% -- Reviewer certainly discueses Zappa

Subclaim 3: In the review, the reviewer praises the testimony of Prince during the PMRC hearings.

Generator credence: 1%

Ensembled step judgement: 43.75% (8 reviews)

Quotes: Prince … the man who once instructed keyboardist Don Preston to play “Louie, Louie” on the Royal Albert Hall pipe organ. And on the other hand, his indignant common sense during the PMRC hearings

Rebuttal: “Eat That Question: Frank Zappa in His Own Words,” an archival-footage documentary… Zappa… Zappa… Zappa…. Zappa… Zappa… Zappa… Zappa… Zappa… the man who once instructed keyboardist Don Preston

Judge reviews:

  1. 5% -- The supporting quote has an ellipses right after the proper noun Prince. This means we can now pair the subject Prince with any other verb that follows it. This appears to be a redirect. The rebuttal, somewhat comically, appears in addition to sharing the film title, demonstrates within quote rules how many times Frank Zappa as a proper noun appears.
  2. 25% -- The claim has two main parts: A) The reviewer praised the testimony of someone during the PMRC hearings B) Prince was testifying at a PMRC hearing (I'm breaking it down like that because the ellipses in the supporting quotes make me suspect that Prince was not the person testifying at the hearing) Part A is weakly supported by the supporting quotes: "indignant common sense" sounds like a positive description, but the reviewer never directly praises the testimony. Part B is strongly supported, conditional on the ellipses not leaving out important context. However, I'm somewhat suspicious, because I expect claiming "X did Y" when in reality Z did Y seems like it would be a common malicious expert strategy. Furthermore, the phrasing of the supporting quotes is a bit odd, if Prince really was the person testifying. Also, looking up "Prince" per instructions reveals that he is a musician, and I wouldn't expect a musician to be testifying at a hearing (though I don't know what a PMRC hearing is). The rebuttal is fairly strong evidence that Prince was not the person testifying at the PMRC hearings. The rebuttal establishes that the subject of the review is probably a documentary on Frank Zappa; this means that the body of the supporting quote is probably referring to Zappa rather than Prince. It's somewhat odd that the rebuttal doesn't include the full context around "Prince..." if Prince is not the person testifying in the PMRC hearings, but the claim is still likely to be false.
  3. 50% -- The supporting quote, we have no clue if the last sentence is talking about Prince because the first sentence is cut off. I don't know what the rebuttal was trying to demonstrate.
  4. 59% -- Both quote and rebuttal omit context that would show who the PMRC hearing is about. The part about 'praise is also unclear to me. Both quote and rebuttal are not very strong, but rebuttal seems weaker
  5. 80% -- There is missing text in the supporting quote, but the rebuttal does not offer some plausible contradicting information.
  6. 90% -- The supporting quote believably support the claim.
  7. 95% -- direct evidence in the supporting quote.
  8. 95% -- Zappa Zappa, Zappa?

38. https://www.rogerebert.com/reviews/in-my-country-2005

Root claim: In the review, the reviewer argues that the movie centers on an interracial romantic affair in South Africa during apartheid.

Generator credence: 1% -- It’s after apartheid

Tree judgement: 33.75%

Ensembled step judgement: 33.75% (10 reviews)

Subclaim 1: In the review, the reviewer describes the film as centered on an affair between the two leads in South Africa.

Subclaim 2: In the review, the reviewer describes the leads as a black man and a white South African woman.

Subclaim 3: In the review, the reviewer describes the restrictions on interratial relationships during apartheid in South Africa

Rebuttal: But white rule gave way… The Truth and Reconciliation Commission, the inspiration of... leaders in the new society… John Boorman's "In My Country" is set at the time of the commission's hearings

Judge reviews:

  1. 10% -- All parts are directly supported by supporting claims except for 'during apartheid': 'describes restrictions...during apartheid' does not necessarily mean the film is during apartheid. The rebuttal suggests the film takes place during hearings of 'Truth and Reconcilitation Commission' which I know was established after apartheid. I find that as a much stronger evidence than supporting claim 3.

  2. 25% -- There are 4 main parts to the root claim: A: There is an interracial romantic affair in the movie B: The affair occurs in South Africa C: This affair is central to the movie D: The affair occurs during apartheid Supporting claim #1 is strong evidence for parts B and C Supporting claim #2 is strong evidence for part A (when combined with claim #1) Supporting claim #3 is evidence for part D Overall, the main way the claim could be false given the supporting claims is if the central romantic affair didn't occur during apartheid. I'm not sure if that would count as "clearly false" though, making it somewhat less likely. The rebuttal is evidence that the movie isn't set during apartheid. It claims that the movie is instead set during the time of the commission's hearings, presumably referring to the Truth and Reconciliation Commission. The remainder of the rebuttal quotes seem to establish that the Truth & Reconciliation Commission was established after apartheid (which is confirmed by a Google search using the procedure in the instructions). Overall, this makes the claim fairly likely to be false.

  3. 30% -- I don't know whether "at the time of the commission's hearings" is "during apartheid" .

  4. 45% -- The first two supporting claims almost unambiguously support the first part of the main claim. The only uncertainty is the affair was during apartheid. The rebuttal suggests it might have after apartheid ended. It is presumably set during "The Truth and Reconciliation Commission". Whether that was before or after apartheid ended I couldn't be sure, but it sounds like something that happened either as it was ending, or shortly afterwards (probably the latter). Another factor is the I think a movie about an interracial affair during apartheid (when there were restrictions) would be considered more dramatic than one afterward.

  5. 50% -- The rebuttal seems to indicate that the movie is not centered on the affair, but might be misleading due to the contractions.

  6. 60% -- The first two supporting claims are strong but the last one can be misleading, there is no proof that the movie is set in the apartheid period. On the other hand, the rebuttal shows strong counter-evidence that the movie is set at the time after apartheid. But of course, the big part of the movie can be happening as a flash back.

  7. 70% -- The supporting claims mostly establish the claim. The rebuttal suggests the movie might be at the end of that time period indicated in the claim, but nothing more conclusive.

  8. 95% -- the supporting claims provides good evidence for the claim.

  9. 97% -- The supporting claims are complete in covering all the sub-claims in the root claim and how they interrelate. The rebuttal lacks any significant challenge as is lacks mention of affairs and relationships. The rebuttal quotes are four loosely connected phrases which also carries less credence.

  10. 99% -- supporting claims 1 2 & 3 fully support the claim

Subclaim 1: In the review, the reviewer describes the film as centered on an affair between the two leads in South Africa.

Generator credence: 99%

Ensembled step judgement: 80% (6 reviews)

Quotes: In the final decades of apartheid in South Africa stars Samuel L. Jackson as Langston Whitfield, … and Binoche as Anna Malan, a white Afrikaaner The affair between Whitfield and Mala

Rebuttal: In the final decades of apartheid in South Africa, few observers thought power would change hands in the country without a bloody war. But white rule gave way peacefully to the Nelson Mandela...

Judge reviews:

  1. 65% -- The supporting quotes are strong evidence that the film is centered around an affair between the the two leads, and that one of them is a white Afrikaaner, but doesn't explicitly show that the movie is set in South Africa. The only supporting quote that mentions South Africa apparently corresponds to a general discouse about post-apartheid South African history, rather than proof that the show is set there. The fact that a main character is Afrikaaner is evidence that the show is set in South Africa, but not decisive proof (for example, the Afrikaaner could have left South Africa post-apartheid). I find it a bit confusing that there was no short quote in the review explicitly establishing where the movie was set (or at least no such quote appears in the support or rebuttal sections), but that isn't strong evidence in either direction. Overall, it's moderately more likely than not that a movie with an Afrikaaner lead is set in South Africa. The "in the final decades of apartheid..." quote doesn't add additional evidence that the movie is set in South Africa, since any movie with an Afrikaaner character is almost going to mention South Africa in some manner.
  2. 80% -- The claim is modest and the rebuttal is not convincing.
  3. 80% -- The rebuttal counters a different claim than the one being made.
  4. 93% -- The supporting quotes piece together the lead actors, the relationship, and the racial difference. I would hope for a supporting quote that was more continuous but at about 187 characters the separations seem justifiable. The Rebuttal talks about the setting of the film but not the leads. It is remotely plausible that the film is more a documentary about South Africa but not by much.
  5. 95% -- both agree it's in South Africa. Supporting gives further evidence it is about an affair.
  6. 95% -- Claim directly supported by quotes, rebuttal does not contradict

Subclaim 2: In the review, the reviewer describes the leads as a black man and a white South African woman.

Generator credence: 99%

Ensembled step judgement: 80% (5 reviews)

Quotes: stars Samuel L. Jackson as Langston Whitfield, … and Binoche as Anna Malan, a white Afrikaaner irony in an Afrikaaner convincing an African American that Mandela's new South Africa...

Rebuttal: `

Judge reviews:

  1. 45% -- The supporting quote never directly proves that the reviewer describes Samuel L. Jackson as a black man. The second supporting quote makes it clear that an African American is involved in some capacity, but not that the lead is African American. If the reviewer really did describe Samuel L. Jackson as black, it seems like it should have been very easy to include that in the supporting quotes, which makes me think the claim isn't true. On the other hand, that seems like a pointless thing to lie about, since it would be so easily disproven, so there's a reasonable chance that the honest expert simply neglected to include the proof, or that the quote describing Jackson as black is too long to include for some reason. Overall, I'm pretty confused/uncertain, and the blank rebuttal doesn't help.
  2. 80% -- There is no rebuttal. (It has a starting (punctuation) ' but no ending (punctuation)'.) This appears to be an error.
  3. 90% -- The claim is well supported.
  4. 95% -- The supporting quotes substantiate the mixed race relationship between the leads. The rebuttal here seems like a typo as the field exists but it is only populated by [```].
  5. 99% -- Claim directly supported, no rebuttal

Subclaim 3: In the review, the reviewer describes the restrictions on interratial relationships during apartheid in South Africa

Generator credence: 98%

Ensembled step judgement: 76% (6 reviews)

Quotes: Although apartheid imposed criminal penalties for interracial sex under its "Immorality Act," that does not necessarily mean that interracial sex has to be in the foreground of a movie

Rebuttal: The Truth and Reconciliation Commission, the inspiration of Mandela... that does not necessarily mean that interracial sex has to be in the foreground of a movie about Truth and Reconciliation

Judge reviews:

  1. 50% -- neither of these quotes seem to directly address the question
  2. 70% -- The quotes suggest the claim is true (and that the reviewer isn't necessarily happy that this is the case).
  3. 94% -- The supporting quotes are evidence for the root claim. The rebuttal states that interracial sex was not in the foreground but that doesn't necessarily negate the discussion of interracial marriage.
  4. 95% -- The supporting quote seems like an explicit example of the reviewer describing the restrictions on interracial relationships during apartheid in South Africa, and the rebuttal seems irrelevant. Unless the claim's use of "interratial" rather than "interracial" is important, or I'm missing something else important, I don't see how the claim could be false.
  5. 97% -- in the supporting quote the reviewer describes the restrictions on interracial relationships.
  6. 98% -- The claim is directly supported by quote. Rebuttal shows the movie does not take place during apartheid, but that does not contradict the claim.

39. https://www.rogerebert.com/reviews/tale-of-tales-2016

Root claim: In the review, the reviewer suggests that some of the film’s content is offensive.

Generator credence: 10% -- Never says offensive, just “nasty” and “dismal”. Unlikely that the reviewer would think this offensive.

Tree judgement: 32.5%

Ensembled step judgement: 75% (9 reviews)

Subclaim 1: In the review, the reviewer asserts that stories behind the film are gory and erotic but intended for children.

Subclaim 2: In the review, the reviewer suggests the film’s depiction of an old woman is degrading.

Subclaim 3: In the review, the reviewer suggests that events in the movie are morally depraved.

Rebuttal: splendid visuals that pay homage to a great tradition of the gorgeously ornate and sublimely grotesque in Western art... As much candy as the movie encourages the eyes to gorge on

Judge reviews:

  1. 60% -- Everything is offensive to someone. But does the reviewer suggest that this film in particular is? I can see where all the supporting claims are true yet the root claim is false. But if that were the case, the rebutter would be pointing this out directly, I think. The rebutter isn't, so my confidence in the supporting claims' view is increased.

  2. 70% -- The supporting claims substantiate the claim and the rebuttal does not clearly contradict it as they can both be true in a review.

  3. 75% -- The supporting claims make it likely, that the claim is true but do not provide full evidence.

  4. 80% -- The claims show that the film could be offensive for some people (and the claim uses word 'suggests' which seems accurate). If it were false, I would expect better rebuttal

  5. 80% -- I think it's reasonable to infer that the reviewer is suggesting that the film's content is offensive based on supporting claims #2 and #3. Most people would find "morally depraved" events offensive, as well as a "degrading" depiction of an old woman. Supporting claim #1 doesn't necessarily show that the reviewer suggests that the film's content is offensive, but it's certainly compatible with that claim. The rebuttal isn't relevant AFAICT: it might be weak evidence against the supporting claims, but it's not evidence that the root claim is false given that the supporting claims are true. My main uncertainty is whether the reviewer has to literally use the word "offensive" for the root claim to be true, or whether the claim is true as long as the reviewer makes it clear that they believe the film to have content that would make it offensive to any reasonable viewer.

  6. 90% -- These claims if true, would almost certainly prove the general claim, and the rebuttal is weak. The claims also seem easy to verify, so I trust the reviewers of these claims.

  7. 95% -- The supporting claims are very strong, the rebuttal does not seems relevant.

  8. 99% -- Solid support. And top claim says "some"

  9. 100% -- The supporting claims give examples of offensive content. The rebuttal talks about the 'splendid' visual but lack commentary on the morality of the content.

Subclaim 1: In the review, the reviewer asserts that stories behind the film are gory and erotic but intended for children.

Generator credence: 90%

Ensembled step judgement: 90% (9 reviews)

Quotes: ``Basile’s stories for children were a lot gorier, grimmer, and suffused with eroticism, or fear of eroticism, than contemporary ... film takes three of Basile’s stories and visualizes them`

Rebuttal: Garrone goes back to a 17th-Century writer from Naples, Giambattista Basile, the father of the fairy tale... As was the custom back in the day, Basile’s stories for children

Judge reviews:

  1. 40% -- The supporting quote is talking about the original book, not about the movie. The rebuttal is not really relevant, it is very hard to say if it is true or not.
  2. 80% -- The rebuttal actually supports the general claim.
  3. 90% -- The rebuttal appears to support the claim rather than to contradict it.
  4. 90% -- The supporting quotes directly show that the film is based off Basile's stories, and that the much gorier & more erotic than contemporary ... (presumably the phrase after the ellipsis is something like "fairy tales"). My main uncertainty is whether "more erotic & gory than contemporary stories" is equivalent to "gory and erotic", but I think it's quite likely. The rebuttal doesn't seem very relevant: the stories being normal for the time doesn't disprove the claim that they are erotic and gory.
  5. 95% -- The claim seems very accurate given the quote. Rebuttal does not contradict.
  6. 95% -- The rebuttal supports the root claim.
  7. 98% -- The supporting quotes appear to link the gory and erotic with children stories under the author Basile. The rebuttal gives some background information and the last part of the quote here is completed by part of the supporting quotes.
  8. 100% -- direct evidence
  9. 100% -- The supporting claim do give complete evidence for all of the statements in the claim.

Subclaim 2: In the review, the reviewer suggests the film’s depiction of an old woman is degrading.

Generator credence: 60%

Ensembled step judgement: 32.5% (7 reviews)

Quotes: an old woman in relatively wretched physical shape glues down her sagging breasts to her flank what they used to call a crone

Rebuttal: splendid visuals that pay homage to a great tradition of the gorgeously ornate and sublimely grotesque in Western art... As much candy as the movie encourages the eyes to gorge on

Judge reviews:

  1. 25% -- The supporting quotes don't directly say that the depiction is degrading, nor do the descriptions seem intrinsically degrading (making it less likely that the overall review implies that the depiction is degrading). Furthermore, the rebuttal provides strong evidence that the reviewer sees at least some of the depictions in the movie as "splendid" and "georgeous", making it less likely that the reviewer found the depiction of the old woman degrading. Overall, the supporting quotes aren't sufficient to prove the claim, and it seems relatively unlikely to be true in the full context of the review.
  2. 30% -- I am not sure if describing old people as not really physically fit can be considered degrading. I do not consider supporting quotes as really strong.
  3. 35% -- I find it degrading, but the reviewer doesn't seem to suggest it actually is.
  4. 60% -- The supporting quotes suggest heavy imagery but do not directly support the claim.
  5. 70% -- Different people find different things degrading. But I think we can safely say that the reviewer would find this degrading.
  6. 75% -- The quote does not have to mean the depiction is degrading in some contexts. The rebuttal does not offer such context, though, and in isolation the claim seems supported.
  7. 90% -- The supporting quotes are in line with what the claim suggests. However, there is only the films depiction of an old woman and not the reviewer's accompanying critique of how the woman is portrayed. The rebuttal speaks to the quality of the visuals but lacks any commentary on how women are portrayed.

Subclaim 3: In the review, the reviewer suggests that events in the movie are morally depraved.

Generator credence: 90%

Ensembled step judgement: 35% (9 reviews)

Quotes: Garrone’s film … visualizes them with no stinting on the nasty bits dismal doings has to watch helplessly as her husband slaughters any parties who attempt to come to her aid

Rebuttal: a princess who is married off to an ogre has to watch helplessly as her husband no stinting on the nasty bits. The bloody heart of an ugly sea creature is eaten raw

Judge reviews:

  1. 10% -- The supporting quote does not offer much and the rebuttal actually shows as it tries to mislead us. I personally do not consider eating fresh meat from animals in movies as "morally depraved".
  2. 10% -- Nothing in the supporting quotes suggest that the reviewer says that the events are morally depraved; if the claim were true, it should have been very easy to find a quote saying as much. Phrasing such as "no stinting on the nasty bits" suggests to me that the reviewer doesn't see the "nasty bits" as depraved. I don't entirely understand how the rebuttal quotes are supposed to rebut the claim, but if the reviewer simply never states that the events are depraved, it's hard to find a short quote that proves this.
  3. 35% -- The rebuttal clearly shows that the quotes are taken out of context
  4. 50% -- As Yudkowsky's The Sword of Good shows so beautifully, just because the nasty things are done to a monster does make them any less morally depraved. So the rebuttal is not at all convincing. But would the reviewer know this? The root claim isn't about whether such-and-such is morally depraved, but whether the reviewer is suggesting this. I'm unclear on what the reviewer might believe, so I find judging the truth of this root claim to be especially difficult.
  5. 70% -- Only the last of supporting quote seems to depict 'morally depraved' event. It may not be an overall assessment in some contexts, but rebuttal does not offer such context and supports the claim even more instead
  6. 75% -- The supporting claims do suggest very heavy imagery.
  7. 80% -- The rebuttal appears to support the claim rather than contradict it, in addition to substantiation by the supporting quotes (the possible rebuttal is that an ogre slaughtering parties is not morally depraved compared to a human doing so, which might be the case and reduces the credence to that extent)
  8. 80% -- Evidence isn't super direct, but "morally depraved" can mean a lot of things.
  9. 93% -- The supporting quotes do portray many gory details of the film, including portraying cruelty. I have some pause around pure gore and what is more a moral question. Usually, 'moral' comes up more frequently when a film is trying to make a statement or has complex drama. Here, my inclination is to see the gore as part of just shock value. In any case, the rebuttal aligns with the examples given in the supporting quotes. So, credence yes but not 100.

40. https://www.rogerebert.com/reviews/gloomy-sunday-2003

Root claim: In the review, the reviewer argues the movie has limited distribution in the US because of its controversial subject matter.

Generator credence: 0% -- Limited because its a foreign movie.

Tree judgement: 30%

Ensembled step judgement: 37.25% (8 reviews)

Subclaim 1: In the review, the reviewer describes the movie as only opening in a single US theater.

Subclaim 2: In the review, the reviewer argues the movie positively portrays the officer in charge of the final solution in Hungary

Subclaim 3: In the review, the reviewer argues that the film makes light of Nazism, by focusing attention mostly on a conventional love story.

Rebuttal: In New Zealand, it ran for more than a year and became a local phenomenon The movie, which has been fanciful and romantic, now descends into tragedy and betrayal. a touching presence and reality

Judge reviews:

  1. 35% -- Supporting claims don't show any causal link between limited distribution and controversy. Rebuttal makes it seem that it isn't that controversial to begin with. But still a lot of uncertainty

  2. 35% -- 'limited distribution' part is supported, but the supporting claims do not show that 'controversial subject matter' is the reason. From both supporting claims and rebuttal, it doesn't even seem that controversial.

  3. 38% -- "positively portrays the officer in charge of the final solution in Hungary" and "makes light of Nazism" does seem like it'd be controversial, and "opening in a single US theater' does support a limited distribution there. However, it might be for other factors other than controversy (e.g. just not promoted). Supporting claims are also usually stronger than this, and I've answered a lot of these as true when only half are supposed to be. However, I'd expect the rebuttals to be stronger.

  4. 65% -- The "because" in the root claim is not directly addressed in any of the supporting claims. But the rebuttal doesn't show that it's wrong, and it seems reasonable to infer the "because" from the supporting claims. If the rebuttal had focused on info about the US distribution, I'd be more likely to take the rebuttal as good evidence.

  5. 75% -- the supporting claim support both parts of the claim (the movie has limited distribution and it has a controversial subject), but not the causal claim (that a is because of b)

  6. 90% -- Popular in New Zealand doesn't contradict controversial in US. I'm fairly certain supporting claims are triggering tidbits for a US audience.

  7. 90% -- There are two parts to the claim: the reviewer argues that... A) the movie has limited distribution in the US B) A) is because of the movie's controversial subject matter. Supporting claim #1 is very strong evidence for part A. Supporting claim #2 is strong evidence for part B (by showing that the movie is likely to be controversial). Supporting claim #3 is also strong evidence for part B, by giving more reason to think that the movie has controversial subject matter. The supporting claims do not explicitly establish a link between the movie's controversial subject matter and its limited distribution. In order to disprove the claim, the rebuttal needs to show that the movie had limited distribution in the US for reasons other than controversial subject matter. However it does not do this: the movie being successful in NZ is irrelevant to why it wasn't distributed in the US, and claims about the nature of the movie are similarly irrelevant. If the claim were false, it should have been easy for the rebuttal to prove it (just include a quote with the real reason that the movie only opened in a single US theater).

  8. 95% -- Supporting claim one verifies limited US distribution. Supporting claims 2 & 3 appear to substantiate "controversial subject matter" which is indirectly substantiated by positive appraisals of Nazism and genocide or 'final solution in Hungary.' The rebuttal notes success in New Zealand but not the US. The last portions of the rebuttal appear to be commenting on the emotional flow of the plot and its impact over the dynamics of the movie's release.

Subclaim 1: In the review, the reviewer describes the movie as only opening in a single US theater.

Generator credence: 95%

Ensembled step judgement: 76.25% (6 reviews)

Quotes: The movie opens today at the Wilmette Theatre. So far as I can tell, this is its first American theatrical booking...In New Zealand, it ran for more than a year and became a local phenomenon

Rebuttal: The movie will play for a week or two and disappear from the Wilmette and, for all I know, from North America. Maybe not. Maybe it will play for 80 weeks`

Judge reviews:

  1. 35% -- The claim is that the movie only opens in a single US theater, but I have no idea if the supporting quote is good evidence for the claim because I don't know what the definition of opening in a theater is. If the movie is first shown at the Wilmette Theatre but later shown at other theaters, is it "only opening in a single US theater" or not? For the purposes of this workspace, I will assume that the answer is "it is not" in order to avoid confusion. In particular "the movie only opened in a single US theater" makes it sound to me as if the movie is very unpopular, rather than that the movie was shown in one particular theater first then later being played in other theaters. Given this assumption, the supporting quote seems like very weak evidence for the claim. The supporting quote is totally consistent with the movie eventually being shown at more than one theater, and indeed its popularity in New Zealand is evidence that the movie may be popular in the US as well. The rebuttal is further evidence that the claim is false, because it explicitly says that the movie may "play for 80 weeks". Overall, the claim may end up being true, but it may also end up being false, depending on the definition of "opening" and things that happen after the review was written. I have no idea how to assign a credence in this situation, but I do think the claim is more likely to be false than true.
  2. 70% -- "first" appearances not the same as ONLY appearance. Writer themselves not confident: "so far as I can tell", "for all I know". Still - they clearly did check, so I kinda trust their data.
  3. 95% -- Supporting quote's "first" claim wasn't convincing, rebuttal's quote sold me on the claim though. "Maybe not" doesn't save it from strongly insuniating it's playing in just one theater.
  4. 98% -- Both supporting quote and rebuttal together indicate that it is opening in one US theater. The only ambiguity is that "so far as I can tell", but given the claim is about what the reviewer describes (as opposed to the global truth), I don't think this lends too much uncertainty to the claim.
  5. 100% -- direct evidence
  6. 100% -- The supporting quotes and the rebuttal appear to validate the root claim. The supporting quote would be adequate on its own

Subclaim 2: In the review, the reviewer argues the movie positively portrays the officer in charge of the final solution in Hungary

Generator credence: 1%

Ensembled step judgement: 30% (5 reviews)

Quotes: A regular customer is none other than Hans Wieck, now in charge of the Hungarian final solution, and he gives Laszlo an exemption; his beef rolls are a contribution to the war effort

Rebuttal: Wieck, too, is said to be based on a historical figure, a Nazi named Kurt Becher who held a similar job in Budapest.

Judge reviews:

  1. 15% -- The supporting quotes show the Wieck character giving an exemption which while it may be a positive action in the film it does not necessarily follow that the film as a whole is portraying this character in a positive light. I would go 50 if that was the only evidence. As the rebuttal notes that the Wieck character is based upon a similar historical figure, I believe it is more likely that the film was trying to be historically accurate or show depth of character over forcing a positive portrayal.
  2. 30% -- The supporting quote establishes that a character inthe movie (Wieck) is in charge of the Hungarian final solution. However, it provides no evidence of the reviewer arguing that the movie portrays Wieck positively. Wieck giving Lazlo an exemption may or may not count as a positive portrayal, but it's not evidence that the reviewer says that the movie portrays Wieck positively. The rebuttal seems mostly irrelevant. This doesn't raise my credence in the claim, because it's impossible to prove that the reviewer doesn't say something with only a short quote. OTOH, I would have expected a much stronger supporting quote if the claim were true.
  3. 50% -- I'm not sure whether the example in the supporting quote is necessarily a positive portrayal. Awful people also do small niceties, especially those that make them feel more powerful.
  4. 62% -- no good support that the reviewer argues that there is a positive portrayal of the officer.
  5. 80% -- Not enough information to say that the portrayal is positive

Subclaim 3: In the review, the reviewer argues that the film makes light of Nazism, by focusing attention mostly on a conventional love story.

Generator credence: 3%

Ensembled step judgement: 60% (6 reviews)

Quotes: romantic melodrama set in Budapest before and during the Holocaust, and that is not, you will agree, an ideal time to set a love story epilogue ... gratuitous ... we could have done without it

Rebuttal: Then there is an epilogue, which is gratuitous and overlong; we could have done without it. now descends into tragedy and betrayal. The carefree days of romance and denial are over

Judge reviews:

  1. 25% -- Both the supporting quotes and the rebuttal mention love and romance so that aspect of the root claim is easily substantiated. I am given pause as to how to address "makes light of Nazism." The supporting quote talks about how the Holocaust is an odd setting for a love story but the following quote to further substantial it being made light of is full of ellipses. The Rebuttal has a longer second quote that insinuates consequences from being in the Holocaust setting via vis "days of romance and denial are over". So I am thinking this is more likely to be false.
  2. 60% -- Evidence is not clear at all, and could be argued to portray the opposite of the claim
  3. 60% -- it does sound like an odd placement for a love story. The rebuttal raises an interesting point that perhaps the Holocaust angle is dealt with in the long second part of the film
  4. 63% -- the supporting quote offers only weak evidence for the claim. It does not really support; 'the reviewer argues that the film makes light of Nazism'
  5. 70% -- There are two parts to the claim: A) "makes light of Nazism" and B) "focuses attention on a conventional love story". The supporting quotes provide strong support for both. "romantic melodrama set in Budapest" is strong support for B, and "that is not, you will agree, an ideal time to set a love story" is strong support for A. That is, it's reasonable to infer that the reviewer feels that the Holocaust is a bad time for a love story because this has the effect of making light of Nazism. The rebuttal provides some evidence that the film has darker moments rather than continuously making light of Nazism. However, the rebuttal provides no explicit statement of the author's opinion to contradict the supporting quote, so I think the author's overall opinion is indeed that the film makes light of Nazism.
  6. 80% -- Tragedy and betreyal can easily follow a love story. First quote in rebuttal, which fills the gaps for the last supporting quote is a strange inclusion, it just shows supporting quote is being honest with ellipsis usage. However, "makes light of" is a very subjective take. Personally, I think that's an OK thing to do after so many decades. It's alright to move on, having learned some lessons. So I'm downgrading this from 90 to 80, because as a potential watcher this subclaim wouldn't dissuade me. Love story aspect would :P

41. https://www.rogerebert.com/reviews/still-alice-2014

Root claim: In the review, the reviewer argues that the excellent portrayal of Alzheimer’s by lead actress Julianne Moore and writer/director Glatzer' personal experience of ALS combine to make a genuinely amazing movie that shines bright with hope.

Generator credence: 5% -- The review isn’t putting forth effusive praise, or claiming that it’s great. It’s just heartwarming.

Tree judgement: 28.75%

Ensembled step judgement: 28.75% (10 reviews)

Subclaim 1: In the review, the reviewer notes that the lead character Alice gets diagnosed with Alzheimer’s.

Subclaim 2: In the review, the reviewer remarks that Julianne Moore has an exceptional performance.

Subclaim 3: In the review, the reviewer remarks about Glatzer’s ALS diagnosis.

Subclaim 4: In the review, the reviewer implies Glatzer’s struggles and hopes are reflected in the movie.

Rebuttal: they also don’t tell this story with much nuance or artistry in adapting Lisa Genova’s novel., the sensation of watching a rather workmanlike production better suited to the small screen.

Judge reviews:

  1. 10% -- The supporting claims don't cover the "genuinely amazing movie that shines bright with hope" part of the main claim, and the rebuttal makes the movie seem less than amazing.

  2. 23% -- Some subclaims not supported: that Moore is the lead, that Glatzer has personal experience, 'bright' with hope, 'genuinely amazing'. Rebuttal disputes the 'genuinely amazing' part

  3. 25% -- Most of the claim seems true, except "genuinely amazing movie" is not supported in the claims, and the rebuttal suggests some criticisms I would not expect a "genuinely amazing movie" to have.

  4. 40% -- The supporting claims validates the facts in the claim. The supporting claims do not support the claim that it is an amazing movie, which the rebuttal challenges successfully.

  5. 40% -- The supporting claims do substantiate the claim but the rebuttal quote contradicts some of the ideas that render the claim less likely

  6. 90% -- The supporting claims address all the criteria that I set that would be needed for the claim to be true. The Rebuttal is off topic. Therefore this statement is almost certainly true.

  7. 91% -- the claim summarizes the supporting claims well.

  8. 97% -- The supporting claims are a well organize representation of the sub-claims, neatly placing the topic, lead actress, performance, director, and the movie's impact. A great portion of the credence is lent to the fictional prior 'expert reviewers.' The first part of the rebuttal suggests the film is a book adaptation and is critical of how the story was adapted to film. While this is a critical comment it lacks a strong challenge to the contents of the root claim. The rebuttal also has a critical comment about the feel of the production which may exist alongside great acting performances and mentions of the directors ALS diagnosis.

  9. 100% -- All the claims support the original claim very well.

  10. 100% -- The rebuttal is good, but the supporting claims perfectly support the claim

Subclaim 1: In the review, the reviewer notes that the lead character Alice gets diagnosed with Alzheimer’s.

Generator credence: 100%

Ensembled step judgement: 92.5% (7 reviews)

Quotes: Dr. Alice Howland, an esteemed linguistics professor at Columbia University who finds she’s suffering from early-onset Alzheimer’s disease

Rebuttal: Alice has everything. It’s her 50th birthday, and she’s celebrating at a chic New York restaurant with her adoring family. Her husband (Alec Baldwin), who’s also an academic, toasts her as the most

Judge reviews:

  1. 86% -- In the quote someone gets diagnosed with Alzheimer’s, but is only referenced with 'she'. That means a slight reduction in credence, but it's still high
  2. 90% -- Strong supporting quote. The rebuttal seems to be intentionally misleading.
  3. 95% -- The quote explicitly states that Alice has Alzheimer's. The only way this isn't true is that if she is not the lead character. The rebuttal fails to establish this fact.
  4. 95% -- The supporting quote substantiates the claim and the rebuttal does not contradict it.
  5. 100% -- Claim very clearly supported, rebuttal does not contradict
  6. 100% -- The rebuttal doesn't address the claim
  7. 100% -- The supporting quote outright states that Alice gets diagnosed with Alzheimer's.

Subclaim 2: In the review, the reviewer remarks that Julianne Moore has an exceptional performance.

Generator credence: 100%

Ensembled step judgement: 90% (8 reviews)

Quotes: Julianne Moore elevates “Still Alice” such a smart, clever and instinctive actress that she never hits a false note finds unexpected avenues into her character

Rebuttal: Moore plays it small for the most part, conveying fear with her eyes or slight shifts in the tone of her voice

Judge reviews:

  1. 85% -- The first quote is sufficient to the support the claim. The Rebuttal does plant some doubt.
  2. 90% -- Strong supporting quotes validate the claim. Parts of the rebuttal also supports central claim.
  3. 90% -- Claim seems well supported and rebuttal does not directly contradict
  4. 90% -- The supporting quotes evidence the claim and the rebuttal is potentially compatible with the claim.
  5. 94% -- 'smart, clever and instinctive actress' = exceptional performance
  6. 95% -- Julianne Moore elevates “Still Alice”, is the strongest evidence for validating the claim. As Julianne is mentioned in the claim by name she also appears in the quotes associated with the movie name. While the remaining supporting quotes lack a proper noun connection they appear good tangential support for the claim. The rebuttal has description about Moore's acting methodology that may either descriptive or critical.
  7. 95% -- The quote supports the claims, and the rebuttal merely describes an aspect of her performance, nothing about quality
  8. 100% -- The rebuttal details Moore's acting technique, but doesn't say that the reviewer finds it detracts from her performance

Subclaim 3: In the review, the reviewer remarks about Glatzer’s ALS diagnosis.

Generator credence: 100%

Ensembled step judgement: 94.5% (7 reviews)

Quotes: Adding to the poignancy is the fact that Glatzer was diagnosed with ALS in 2011

Rebuttal: Co-directors and writers Richard Glatzer and Wash Westmoreland don’t shy away from the steady and terrifying way the disease can take hold of a person and strip away her ability to communicate and

Judge reviews:

  1. 90% -- Claim fully supported by supporting quote
  2. 90% -- The supporting quotes substantiate the claim and the rebuttal does not contradict it.
  3. 99% -- clear evidence in the supporting quote.
  4. 100% -- The quote clearly states the claim
  5. 100% -- Claim is very clearly supported
  6. 100% -- The rebuttal just isn't as related to the claim as the supporting quote is
  7. 100% -- The reviewer remarks about her diagnosis in the quote, and the rebuttal doesn't counter it.

Subclaim 4: In the review, the reviewer implies Glatzer’s struggles and hopes are reflected in the movie.

Generator credence: 95%

Ensembled step judgement: 70% (7 reviews)

Quotes: he is all too familiar with the struggle of remaining creative and vital. The fact that this is a personal story, earnestly told and filled with hope, ultimately is what shines through with great clarity

Rebuttal: Surely, he is all too familiar

Judge reviews:

  1. 40% -- There is some support for the claim in the supporting quotes, but this does not extend to confirming whose struggles are reflected; the rebuttal seems to indicate that the struggles might be referring to one that isn't actually reflected in the movie, which makes it less likely.
  2. 60% -- Strong supporting quote. The gender of the director isn't a barrier to his struggles and hopes being reflected in the movie.
  3. 80% -- Claim is well supported except the connection with 'Glatzer'. Rebuttal does not dispute this, though, and I would expect it to do so if this was false
  4. 80% -- The supporting quotes aren't great, I thought Glatzer was a woman? That being said, the rebuttal doesn't really do anything
  5. 88% -- solid evidence in the supporting quote.
  6. 90% -- It explicitly states that the story is a personal story shining forth with hope. The only way it isn't true is if it isn't Glatzer that it is about. The Rebuttal is off topic.
  7. 95% -- The quotes seem to support the claim, and the rebuttal doesn't detract from it

42. https://www.rogerebert.com/reviews/janis-1975

Root claim: In the review, the reviewer discusses the relation between Joplin and Franklin and praises Franklin’s intensity in performance. .

Generator credence: 1% -- It’s Joplin’s energy

Tree judgement: 27.5%

Ensembled step judgement: 83.25% (10 reviews)

Subclaim 1: In the review, the reviewer discusses the relation between Joplin and Franklin.

Subclaim 2: In the review, the reviewer praises Franklin’s intensity in performance.

Rebuttal: Janis Joplin tells an interviewer in "Janis,"... Howard Alk's "Janis," a documentary… portrait of Janis Joplin that emerges in the film… Janis talking to interviewers in the film

Judge reviews:

  1. 65% -- The supporting claims substantiate the claim and the rebuttal does not directly contradict it.

  2. 70% -- Considering the supporting claims alone, I would give 95% or higher credence as the sub-claims follow the composition of the claim. The rebuttal shows that the documentary 'Janis' includes her in interviews and posits that a portrat of Janis Joplin emerges from the film. From the rebuttal I get a leaning towards the film being a documentary mostly about Janis and not entirely about comparisons to Franklin, although that may still be a part of it. In considering the expert review of the supporting claims I can say this is likely but much less than 100%

  3. 81% -- Claim is a logical conjuction of supporting claims rebuttal does not really contradict, so my credence is based on priors for supporting claims

  4. 90% -- The supporting claims fully imply the main claim, which is just a combination/restatement of those. The rebuttal is irrelevant, so I assume there is no strong evidence against the claim. 90% because "1 is not a probability".

  5. 95% -- Knowing rebuttal is just cherry picked snippets mean that its refusal to involve Franklin isn't any evidence.

  6. 98% -- very good evidence in the supporting claims

  7. 99% -- The supporting quote is very strong, the rebuttal does not offer contradicting information.

  8. 99% -- supporting claims 1 & 2 are strong together

  9. 99% -- The two supporting claims seem to directly imply the root claim (in fact, they correspond to the root claim almost verbatim). I don't really understand how the rebuttal is relevant, but it would be very hard for any rebuttal to prove that the claim was false given that these suppporting claims are true.

  10. 100% -- the claim and the supporting quotes are identical

Subclaim 1: In the review, the reviewer discusses the relation between Joplin and Franklin.

Generator credence: 40%

Ensembled step judgement: 33.75% (8 reviews)

Quotes: Joplin … began to sing largely by chance … she speaks with admiration of Otis Redding and Aretha Franklin and says she's not that good

Rebuttal: And she speaks with admiration of Otis Redding and Aretha Franklin and says she's not that good: "But I've got good strength, and I'm gonna hang in there."

Judge reviews:

  1. 15% -- What is the relation between Joplin and Franklin? The wording compels me to expect some personal relation, but admiration isn't sufficient for that.
  2. 30% -- "relationship between Joplin and Franklin" - there should be also a description of Aretha feelings about her if it is said like that. Is it even a description of a relationsip if she talks just about her(aretha's) work?
  3. 35% -- From the supporting quotes and the rebuttal I can gather that Joplin talks about her admiration of Franklin. However, Joplin talking about admiration is not the reviewer comparing Joplin's music to Franklin or how Joplin and Franklin are similar or different. The comparison seems limited to Joplin's in interview comments. So this become much less likely to be true. The first part of the supporting quotes sort of hangs out there in space as 'Joplin...' can be connected to almost anything.
  4. 40% -- Even though the claim is not strong 'discusses...relation', the evidence seems even weaker than I expected - Joplin only mentions 'Franklin'. If the claim was true, I would expect more evidence, this seems rather far-fetched. Rebuttal does not really contribute.
  5. 70% -- The supporting quote shows the reviewer discusses Joplin's view of her talent in comparison to that of Franklin. The rebuttal largely agrees so I won't discuss it further. This could be understood as the "relation" between them in an abstract sense (relation between two concepts). However, my original interpretation of the main claim (which I think is the most fair given that we're talking about people) was that Joplin and Franklin had some personal relation. For this reason, I think the main claim is somewhat misleading. This lowers my credence in it, but only slightly.
  6. 70% -- "relation between Joplin and Franklin" is quite vague; I think the supporting quotes are literally consistent with the claim, but the claim implies a stronger relationship between Joplin and Franklin than is actually the case. I don't see how the rebuttal is relevant. I've spent a bit of time thinking about how to assess this claim and I'm still not sure, so I'll go with a 70% credence that it's true
  7. 90% -- The supporting quote is quite strong even though there is some missing text, the rebuttal does not offer contradicting information.
  8. 99% -- the quote discusses the relationship between the two

Subclaim 2: In the review, the reviewer praises Franklin’s intensity in performance.

Generator credence: 1%

Ensembled step judgement: 27.5% (6 reviews)

Quotes: Franklin …The energy in which she sang them is astonishing. The cameras are on top of her, recording an intensity that could be sensed, perhaps, but not quite seen far out in the audience

Rebuttal: Janis Joplin tells an interviewer in "Janis,"... Howard Alk's "Janis," a documentary… portrait of Janis Joplin that emerges in the film… Janis talking to interviewers in the film

Judge reviews:

  1. 2% -- The supporting quote has Franklin, a proper noun, with an ellipses immediately following. In this format any action or description in the review can be attached to a single Franklin reference. So, Franklin shows up but I lack any context around it. If the intensity of Franklin's performance was so noteworthy then I would expect Franklin attached directly to some sort of verb action in the review.
  2. 10% -- I don't want to believe the supporting quote, and I also don't view lack of evidence in rebuttals to be enough to move my credence. What's the "..." in supporting quote? Why does that quote involve so many words after its point about intensity is made while omitting whatever is in between "Frankin" and "The energy"? Also, "T" instead of "t" means some sentence ended in that void. Very cheeky.
  3. 80% -- The fact that the supporting quote is broken and that the main character in the film seems to be Janis Joplin, not Aretha Franklin, make me suspicious of the claim. However, if the broken quote were misleading, the rebuttal could give the full quote to disprove it. The fact that it doesn't makes me more confident in the claim being true.
  4. 85% -- Quote directly supports the claim, unless important context in 'Franklin...The' is missing. If that was relevant, I would expect to find it in the rebuttal, though.
  5. 90% -- There is missing text in the supporting qoute but the rebuttal is not really strong.
  6. 99% -- Aretha can sing. Rebuttal is nonsensical.

43. https://www.rogerebert.com/reviews/the-heiresses-2019

Root claim: In the review, the reviewer argues than an affair with a younger woman is part of what causes a transformation in a main character.

Generator credence: 10% -- The reviewer is careful to not say they have an affair (maybe to avoid spoilers), just that one has a crush on the other.

Tree judgement: 21.25%

Ensembled step judgement: 84% (8 reviews)

Subclaim 1: In the review, the reviewer suggests that a female main character (Chela) has an affair with a younger woman (Angy).

Subclaim 2: In the review, the reviewer argues that the main character (Chela played by Brun) has a personal transformation.

Subclaim 3: In the review, the reviewer suggests that the main character’s (Chela) relationship with a younger woman (Angy) is part of what causes the transformation.

Rebuttal: ``

Judge reviews:

  1. 50% -- Unclear if it is the transformation that causes the affair or vice versa. The rebuttal didn't provide any context

  2. 66% -- Oh geez if the movie is about discovering the main character gay that very well could be a transformation

  3. 90% -- The supporting claims substantiate the main claim and there is no rebuttal.

  4. 95% -- The supporting claims are sufficient to prove the claim. The rebuttal is nonexistent.

  5. 97% -- very good evidence from the supporting claims.

  6. 99% -- rebuttal has nothing to say and the suppporting claims give almost no space for doubt

  7. 99% -- The supporting claims are very strong and there is no rebuttal.

  8. 100% -- The supportings claims fully support the claim when taken in total

Subclaim 1: In the review, the reviewer suggests that a female main character (Chela) has an affair with a younger woman (Angy).

Generator credence: 50%

Ensembled step judgement: 47.5% (6 reviews)

Quotes: Chela … have been together for 30 years Chela … so drawn to … sensuous Angy she's like a high school girl with a crush Angy, a much younger woman, operates in a world of sexual possibility

Rebuttal: While this may sound like another entry in the "middle-aged lady gets her groove back" genre, "The Heiresses" is up to something a little bit different, a little bit darker The plot is minimal.

Judge reviews:

  1. 33% -- I don't see quoted evidence of an affair. Just of an attraction
  2. 40% -- The supporting quotes do not support the claim, that Chela has an affair with angy.
  3. 70% -- The supporting quotes strongly suggest the root claim, but don't clarify if the affair actually happened, or if Chela is the real subject of the first quote. But the rebutter doesn't contradict these low-confidence parts, so I give the benefit of the doubt to the supporting quotes.
  4. 80% -- The supporting quotes are compelling
  5. 85% -- The supporting quotes substantiate the claim, but contain contractions that might be misleading. The rebuttal does not directly respond to the claim.
  6. 90% -- Some text is missing from the supporting quotes but they are quite strong anyway. The rebuttal does not offer much.

Subclaim 2: In the review, the reviewer argues that the main character (Chela played by Brun) has a personal transformation.

Generator credence: 99%

Ensembled step judgement: 82.5% (6 reviews)

Quotes: tension comes from watching Brun's slow transformation Chela is so weighted down at the start of the film that the lightening of her mood is wonderful to see but it's also alarming Chela (Ana Brun)

Rebuttal: "The Heiresses," a Paraguayan drama written and directed by Marcelo Martinessi, could be called a quiet movie. There aren't too many earth-shaking moments. The plot is minimal.

Judge reviews:

  1. 80% -- The supporting quotes almost completely support the root claim. It doesn't prove she's a main character, nor necessarily that the parentheses are supposed to say that the actor's name is in parentheses, but since the rebuttal doesn't attack either of these weak points, my credence remains high for the truth of the claim.
  2. 80% -- The supporting quote does present good evidence for a change in personality lightening of her mood of the main character.
  3. 90% -- The supporting quotes validate the claim
  4. 91% -- 'Brun's slow transformation' is good evidence
  5. 95% -- the rebuttal says nothing important and the supporting quotes are very good.It is obvious from the quotes that chela is of the main characters
  6. 95% -- The supporting quotes are really strong, the rebuttal does not offer much.

Subclaim 3: In the review, the reviewer suggests that the main character’s (Chela) relationship with a younger woman (Angy) is part of what causes the transformation.

Generator credence: 50%

Ensembled step judgement: 21.25% (6 reviews)

Quotes: Chela … so drawn to the sensuous Angy she's like a high school girl with a crush, she misses visiting day at the prison Chela's … attraction to Angy … all of these things threaten to overwhelm her

Rebuttal: Chela's emotions, her hopes, her attraction to Angy, her loneliness ... all of these things threaten to overwhelm her quiet movie. There aren't too many earth-shaking moments. The plot is minimal.

Judge reviews:

  1. 12% -- I don't see how the character transforms. From what? Into what? There's no strong evidence for the claim.
  2. 20% -- There is no proof about any transformation in the supporting quotes and the rebuttal is quite strong.
  3. 25% -- There is no good evidence for a relationship between Chela and Angy in the supporting quote.
  4. 60% -- The first rebuttal quote is not strong; it seems to argue against something stronger than what the root claim is saying. The second quote in the rebuttal also doesn't say much. The failure of the rebutter to give anything better gives me more credence in the supporting quotes.
  5. 75% -- no direct evidence that the relation caused her transformation. Thus only weak to medium evidence for the claim.
  6. 80% -- Chela's attraction to Angy, is part of other factors that causes the transformation.

44. https://www.rogerebert.com/reviews/the-grand-seduction-2014

Root claim: In the review, the reviewer explains that the film is about a fading town brought together by learning to play cricket.

Generator credence: 3% -- Cricket is a small part of the story. They only learn to play cricket to attract the doctor, who they need, in order to attract the petroleum factory.

Tree judgement: 20%

Ensembled step judgement: 61.25% (8 reviews)

Subclaim 1: In the review, the reviewer compares the film to another film about a fading town brought together by a common cause.

Subclaim 2: In the review, the reviewer describes how the main character gets the town interested in cricket

Subclaim 3: In the review, the reviewer explains that the town people are slowly brought closer by playing cricket together.

Rebuttal: The scheme grows more complex by the day. There are aspects of illegal surveillance (the guest house where the doctor is staying is bugged), mass roleplaying (residents fake interest in cricket

Judge reviews:

  1. 30% -- The rebuttal directly contradicts the supporting claim that the town gets interested in cricket

  2. 35% -- "Residents fake interest in cricket... and end up actually liking it", "residents fake interest in cricket... and forfeit their Indian citizenship". Really, this could go both ways, but rebuttal is stronger. Sorry India.

  3. 70% -- There are 3 parts to the claim: A) The reviewer explains that the film is about a fading town B) (the reviewer explains) that the town is brought together C) (the reviewer explains) that the town is brought together by cricket Supporting claim #1 is very weak evidence for parts A and B, but doesn't directly establish either. Supporting claim #2 is strong evidence that the film is related to cricket, which is moderate support for part C. Supporting claim #3 is strong evidence for parts B and C when combined with #2: #3 shows that playing cricket brings the town closer together, and #2 provides moderately strong evidence that this is a major theme in the film (because the main character is the one who starts the interest in cricket). Given the supporting claims, the most likely way for the claim to be false is if the film isn't about a fading town itself, but merely compared to a different film about a fading town in the review. It's also possible (but somewhat unlikely) that "the townspeople are brought together by playing cricket" is true but tangential to the main plot. If so, I'd interpret the claim as false from the perspective of someone deciding whether to watch the movie. The rebuttal makes it somewhat more likely that "the town is brought closer together by playing cricket" is not a central part of the plot, as the cricket-playing is apparently part of a complex scheme involving mass role-playing. However, the rebuttal quote and supporting claims #2 and #3 could be referring to different things: supporting claim #2 suggests that the town is genuinely interested in cricket, not faking it, and I don't see how faking interest in cricket would bring the town closer together (per claim #3). Thus, it's perfectly possible that the cricket playing becomes an important part of the plot for reasons unrelated to the complex scheme. Overall, the rebuttal weakly reduces my credence in the claim: if nothing else, the claim completely ignoring the "complex scheme" in the rebuttal makes it less likely that the claim is a clearly true statement of what the film is about. My main uncertainty is how the lack of support for "fading town" affects the likelihood that the claim is true. I suspect the answer is "not much", because an honest expert could easily neglect to include a subclaim explicitly supporting "fading town", whereas a malicious expert could simply omit "fading town" from the claim if they were unable to support it. This logic wouldn't hold if "fading town" were the only false part of the claim, but then I doubt the claim would qualify as "clearly false" as stipulated in the instructions.

  4. 75% -- The supporting claims all seem to strongly support the claim, however the rebuttal (talking about "a scheme") makes me suspect that the film's true purpose is about something else. However, I would think if the film really was primarily about something else, the rebutter could have elaborated a little more on the 'scheme'.

  5. 80% -- The supporting quotes are really strong, the rebuttal too, but the claim is not if the town likes cricket but that they learn it together.

  6. 82% -- so the 2 theories are 'heartwarming sport film' or 'creepy Truman Show spy shenanigans'? Supporting claims 1 2 and 3 are all also true of a sneaky spy town pretending to play cricket... but my priors on such a film are much lower than a straight sport flick

  7. 85% -- Maybe the filme is not about learning to play cricket, even if it is a main point of the film.

  8. 90% -- The claim is supported but not rebutted.

Subclaim 1: In the review, the reviewer compares the film to another film about a fading town brought together by a common cause.

Generator credence: 75%

Ensembled step judgement: 70% (7 reviews)

Quotes: "The Grand Seduction," about a Newfoundland fishing town mounting a wild scheme to draw a petroleum factory, is a film in this vein.

Rebuttal: film in this vein. It's a remake of "Seducing Doctor Lewis," a French Canadian movie I haven't seen but that I'm told is pretty much the same. The excellent Canadian actor and filmmaker

Judge reviews:

  1. 60% -- The review does compare the film to another similar film. It is unclear if it is about a fading town brought together by a common cause.
  2. 65% -- unsure if they actually are "brought together"
  3. 75% -- "town mounting a wild scheme to draw a petroleum factory", this is the common cause I think. "Brought together" part isn't supported though.
  4. 75 % -- The full quote appears to be "The Grand Seduction," about a Newfoundland fishing town mounting a wild scheme to draw a petroleum factory, is a film in this vein. It's a remake of "Seducing Doctor Lewis," a French Canadian movie I haven't seen but that I'm told is pretty much the same. The excellent Canadian actor and filmmaker..." If "The Grand Seduction" is the film in question, then the claim seems pretty likely to be true, and I'm not sure why the quote provided in the rebuttal is evidence against it. The reviewer is clearly comparing "The Grand Seduction" to "Seducing Doctor Lewis". As the review states that "The Grand Seduction" is based on "Seducing Doctor Lewis", the latter must also be about "a Newfoundland fishing town mounting a wild scheme to draw a petroleum factory". Thus, both films are clearly about a brought together by a common cause, and I think it's reasonable to infer that a town embarking on "a wild scheme to draw a petroleum factory" is a fading town. I spent a few minutes trying to think of ways the rebuttal quote is evidence against the claim, but I can't think of any. The possibility that I'm missing something important reduces my credence somewhat. It could be argued that saying "film X is a remake of film Y" isn't "comparing" the two films, but that seems like a fairly pedantic and unlikely interpretation.
  5. 80% -- The claim is supported but not rebutted.
  6. 80% -- Sounds very plausible, the rebuttal is not really strong.
  7. 95% -- yes they sure do compare it

Subclaim 2: In the review, the reviewer describes how the main character gets the town interested in cricket

Generator credence: 4%

Ensembled step judgement: 20% (7 reviews)

Quotes: the main character, Murray, Murray…presents the fundamentals of cricket to his fellow townspeople, enlisting a seamstress to throw together cricket uniforms from available swatch of white

Rebuttal: The scheme grows more complex by the day. There are aspects of illegal surveillance (the guest house where the doctor is staying is bugged), mass roleplaying (residents fake interest in cricket

Judge reviews:

  1. 20% -- "gets the town interested" is not supported by "presents the fundamentals" and is rebutted by "fake interest".
  2. 20% -- supporting quote is explained as part of mass roleplaying, so I'm more in favor of rebuttal
  3. 20% -- The supporting quotes establish that the main character introduces cricket to his townspeople, but not that he succeeds in getting them interested. The rebuttal provides moderately strong evidence that Murray does not genuinely interest the townspeople in cricket: "residents fake interest in cricket". It's possible that these quotes are referring to two different things, but cricket is a relatively uncommon topic, so I think it's more likely that "residents fake interest in cricket" is referring to the same time and place as "Murray presents the fundamentals of cricket".
  4. 65% -- not clear that people actually get interested
  5. 70% -- The supporting quote is strong. The rebuttal is also strong, but I can imagine a movie where they first fake interest to become really interested later.
  6. 80% -- The rebuttal doesn't counter the claim
  7. 90% -- Sounds plausible, and the rebuttal doesn't contradict Claim.

Subclaim 3: In the review, the reviewer explains that the town people are slowly brought closer by playing cricket together.

Generator credence: 9%

Ensembled step judgement: 27.5% (4 reviews)

Quotes: all join forces…enlisting a seamstress to throw together cricket uniforms from… tablecloths and curtains, and stage a "championship game" atop a rocky cliff reviving a nearly dead town

Rebuttal: The scheme grows more complex by the day. There are aspects of illegal surveillance (the guest house where the doctor is staying is bugged), mass roleplaying (residents fake interest in cricket

Judge reviews:

  1. 20% -- This supporting quote is almost good. But not quite. If only "revive" part was in the same quote as the cricket game.
  2. 30% -- "reviving a nearly dead town" can mean multiple things. Unsure in this one
  3. 80% -- There is a lot of missing text in the supporting quote. The rebuttal is strong, but the claim is not if the town likes cricket but that brought closer together by it.
  4. 90% -- "fake interest" doesn't rebut "playing cricket together".

45. https://www.rogerebert.com/reviews/something-in-the-air-2013

Root claim: In the review, the reviewer states that the film depicts the civil unrest that occurred in France in the summer of 1968.

Generator credence: 2% -- While the film is about French society in the aftermath of May 1968, it is explicitly stated that most of the story (about Gilles) occurs in 1971.

Tree judgement: 17.75%

Ensembled step judgement: 17.75% (10 reviews)

Subclaim 1: In the review, the reviewer explains that the film’s French title, “After May”, references the events of 1968 in France.

Subclaim 2: In the review, the reviewer explains that the summer of 1968 was a time of unrest in France.

Subclaim 3: In the review, the reviewer describes how the film follows the experiences of young revolutionaries.

Rebuttal: (Clément Métayer) as he wanders impulsively after romance, adventure, artistic ambitions and revolutionary politics, from the suburbs of Paris to Italy and back again, during the summer of 1971.

Judge reviews:

  1. 10% -- It seems that the film doesn't actually takes place in 1968, but in 1971, after the events of the civil unrest.

  2. 10% -- The quote in the rebuttal provides sufficient contradiction to the claim by implying the movie is set in 1971 rather than 1968.

  3. 17% -- given the supporting claims and rebuttal, it seems much more plausible that the film is about young revolutionaries at some time other than 1968 (potentially 1971); claim #1 doesn't contradict this, as the film could have a title referencing the events of 1968 for many reasons. It seems very likely that stronger supporting claims could have been chosen had the film been about 1968. Also, maybe "After May" means "after the events of 1968" (i.e. in 1971)? That would still make claim #1 true I think, but would clearly make the overall claim false. However, it's possible that the supporting claims were just chosen poorly, and the rebuttal is technically compatible with the root claim.

  4. 20% -- It refers to the 1968 events, but the rebuttal suggests the film was set in 1971, and this is consistent with it being influenced by ("After May") the events of but not set in 1968. Although it's possible that the rebuttal is referring to something later in the film, the wording doesn't suggest this.

  5. 50% -- I am unsure as to which argument is stronger. They both have flaws.

  6. 65% -- The supporting claims clearly outline the root claim. However, the rebuttal while containing supporting descriptions to the claim contains reference to 1971, a different date. The rebuttal also appears to describe the main character and the main plot action. Absent 'expert reviewers' for supporting claims I would say this is false or perhaps 39 credence. I will give this more than that as I am considering that the rebuttal might be part of an extended timeline in the film.

  7. 70% -- The film could be about a different set of revolutionaries but is alluding to the french events. The Rebuttal does reference another year which is this loop hole in support. However, it is still more likely to be true.

  8. 92% -- support claims entail the claim.

  9. 99% -- I am convinced that this depicts the civil unrest in France 1968

  10. 100% -- I think the supporting claims are quote strong. the rebuttal kind of proves the claim too by mentioning "revolutionary politics"

Subclaim 1: In the review, the reviewer explains that the film’s French title, “After May”, references the events of 1968 in France.

Generator credence: 98%

Ensembled step judgement: 95% (8 reviews)

Quotes: The French title…is "Apre mai," which translates into English as "After May." And everyone, or at least everyone of a certain age, knows that "May" means one thing in French: May 1968.

Rebuttal: May 1968. "Bliss was it in that dawn to be alive," wrote Wordsworth of an earlier revolution in France, "But to be young was very heaven!" If that isn't an actual on-screen epigraph in Assayas' movie

Judge reviews:

  1. 92% -- The supporting quote seems to directly support the claim, and the rebuttal seems pretty irrelevant. I don't see how quotes from Wordsworth abotu earlier revolutions would be evidence against the claim. I'm a bit nervous about going higher than 90% when I don't have a good sense of possible failure modes, though.
  2. 95% -- The supporting quote gives clear evidence that it references May 68.
  3. 95% -- A title can reference multiple things, so the supporting quote is sufficient.
  4. 99% -- supporting quotes are clear facts. rebuttal is demented raving.
  5. 100% -- The quote explicitly answers the claim.
  6. 100% -- direct evidence
  7. 100% -- The rebuttal is not effective
  8. 100% -- The supporting quotes connect the "After May" reference to 1968 France. The rebuttal offers an extension of the quote provided in the supporting quotes.

Subclaim 2: In the review, the reviewer explains that the summer of 1968 was a time of unrest in France.

Generator credence: 97%

Ensembled step judgement: 85% (9 reviews)

Quotes: "May" means one thing in French: May 1968…a tumultuous and transformative period in modern French history: student protests, demonstrations…culminating in a general strike that shut down the nation

Rebuttal: And everyone, or at least everyone of a certain age, knows that "May" means one thing in French: May 1968. "Bliss was it in that dawn to be alive," wrote Wordsworth of an earlier revolution in France

Judge reviews:

  1. 51% -- The supporting quotes have gaps in it that could be exploited. The tumultuous time could be a different year. The rebuttal though fails to point though.
  2. 70% -- Somewhat unclear, but revolution is always a time of unrest, even if it is "bliss".
  3. 85% -- The supporting quote directly states that May 1968 was a time of unrest in France, and the rebuttal is irrelevant. My only hesitation is whether "May 1968 was a time of unrest" implies that "summer of 1968 was a time of unrest" ; it seems very likely but not certain.
  4. 90% -- Supporting quotes strongly support claim, although "May" is late spring in France so it isn't totally clear that the summer was full of unrest
  5. 93% -- supporting quotes offers good evidence for the claim.
  6. 98% -- what's the rebuttal getting at?
  7. 99% -- direct evidence
  8. 100% -- the rebuttal is not effective
  9. 100% -- Both supporting quotes and the rebuttal are in support of the claim. The rebuttal is continuous and serves to add detail to an ellipses in the supporting quotes. The ellipses appears necessary as 198 of the character budget was used.

Subclaim 3: In the review, the reviewer describes how the film follows the experiences of young revolutionaries.

Generator credence: 40%

Ensembled step judgement: 42.75% (10 reviews)

Quotes: The movie is populated with fresh-faced would-be high school revolutionaries…facing a dizzying array of new experiences…issues of life-and-death importance swirling around them

Rebuttal: would-be high school revolutionaries whose precociousness is mingled with a comically endearing naïveté. These kids are poised on the cusp of "real life," facing a dizzying array of new experiences

Judge reviews:

  1. 30% -- "would-be" implies that they are NOT, in fact, revolutionaries. The supporting quote argues against the root claim.
  2. 40% -- The supporting claim and the rebuttal together provide an extended quote; while the movie clearly has young would-be revolutionaries, the claim states that they are 'young revolutionaries'; further, the claim states that the film follows their experiences, but that is not evidenced in the quotes.
  3. 40% -- "would-be high school revolutionaries" are not necessarily actual revolutionaries! However, the rebuttal isn't very compelling: if the movie were really about e.g. unruly highschoolers it should've been easier to find a direct quote saying so. Also, "issues of life-and-death importance swirling around" makes it much more likely that the context is in fact a revolution or something similar (and high schoolers can certainly participate in revolutions). Overall, it's unclear if the claim would be described as true by a reasonable reader with full context; for example, I think it would be fair to describe some of the HK protestors as "fresh-faced would-be high school revolutionaries" as well as "young revolutionaries". The two terms aren't incompatible.
  4. 51% -- The film depicts would be revolutionaries, no revolutionaries and the rebuttal points this out.
  5. 70% -- Neither the quote nor rebuttal disagree very much, the key issue is whether "would-be high school revolutionaries" = "revolutionaries". I'd say yes, but I could see an argument for saying they're not "real" revolutionaries.
  6. 85% -- good evidence
  7. 86% -- high school revolutionaries are young, yes. and every revolutionary is at some point a would-be revolutionary.
  8. 100% -- direct evidence
  9. 100% -- I think the rebuttal also proves the claim.
  10. 100% -- While the supporting quotes contains some omissions, the rebuttal fills some of that context in as there is textual overlap. All quotes reference experiences of youth and revolution. r

46. https://www.rogerebert.com/reviews/seventh-son-2015

Root claim: In the review, the reviewer describes the movie as a zany comedy.

Generator credence: 1% -- It’s sometimes very unrealistic but not in a comedic way.

Tree judgement: 15%

Ensembled step judgement: 33.75% (10 reviews)

Subclaim 1: In the review, the reviewer compares the movie to a Mel Brooks comedy movie from 1970.

Subclaim 2: In the review, the reviewer argues the movie is a successor to comedy film “The Big Lebowski”.

Subclaim 3: In the review, the reviewer argues that the male lead is a zany character.

Rebuttal: the Sergei-Bodrov-directed “Seventh Son,” a medieval fantasy tale with a strong Young Adult stress in its plotting… The YA aspect... the titular seventh son… “Seventh Son… Saturday afternoon matinee

Judge reviews:

  1. 15% -- Doesn't seem like the supporting claims are enough to say that the movie itself is zany

  2. 20% -- Zany lead + comedy = zany comedy feels like a stretch to me, but not too much. rebuttal seems more believable in its statement of genre

  3. 30% -- I'm so used to supporting claims that are near-verbatim evidence of the original claim, that I'm assuming that one that isn't is probably false. However, it is heavily implied to be a "comedy" with the male lead being "zany", and the rebuttals don't provide much evidence against this. But it is primarily the fact this seems weaker than many other claims I've seen so far makes me think it's more likely to be false.

  4. 45% -- zany character doesnt have to mean it is a zany comedy, but "The big Lebowski" could probably be called a zany comedy

  5. 68% -- The supporting claims substantiate the claim, but the quotes in the rebuttal cast some doubt as to their veracity.

  6. 70% -- The claims support the 'comedy' part, but not directly that it's a 'zany' comedy, even though they point somewhat in that direction. I can imagine both HE and ME choosing these claims. Rebuttal seems irrelevant.

  7. 80% -- There is nothing which indicates "Seventh Son" is the move in question. Also supporting claims are more convincing

  8. 85% -- The supporting claims are complete in supporting the root claim as they compare examples of comedies and identify the lead as zany. However, the direct quote rebuttal seems out of step with this with its 'a medieval fantasy tale' and other notions focusing on the young adult aspect. Its possible to have a comedy in a medieval fantasy tale but I question then why wouldn't that aspect take more priority in phrasing. So, in believing the Supporting Claims to be true, I would gather the root claim true with some doubts.

  9. 90% -- The claim is well supported and the rebuttal is not convincing.

  10. 100% -- The rebuttal is confusing/too broken up and doesn't address the claim

Subclaim 1: In the review, the reviewer compares the movie to a Mel Brooks comedy movie from 1970.

Generator credence: 1%

Ensembled step judgement: 17.5% (10 reviews)

Quotes: Hope for the best, expect the worst.” Mel Brooks popularized that adage in a funny song he co-wrote for one of his most amusing films, the relatively obscure 1970 “The Twelve Chairs.

Rebuttal: “The Twelve Chairs.” It’s a useful sentiment in general, but, for the film reviewer, something that applies most particularly in the first two months of the new year, which, at least as far as

Judge reviews:

  1. 5% -- The movie isn't being compared, Mel Brooks seems to be just mentioned.
  2. 10% -- "compares the movie" is not supported.
  3. 15% -- The quote doesn't compare the movie being reviewed to a Mel Brooks movie, but instead an adage from a song from that movie. I think it's a stretch to call that a comparison to the movie. The rebuttal suggests it's an adage the reviewer is keeping in mind as advice, not because the movies are comparable.
  4. 25% -- Quote does not show reviewer meant comparison. Not more because I think a true claim would be chosen with better supporting quotes. Not less because rebuttal hints, but not directly contradicts.
  5. 30% -- Rebuttal indicates the supporting quotes are being used out of context (about the film industry not just the movie in question), however I'm not certain in this analysis
  6. 30% -- neither the claim or the rebuttal make a strong case, which favours it being false
  7. 30% -- the reviewer probably mentions the movie for the adage "Hope for the best, expect the worst.” he probably wasnt comparing the two movies
  8. 70% -- I don't really know, don't even know what the film being reviewed is. I chose the supporting quotes, because the fact that they prove the Mel Brooke's movie is mentioned, proves there is a comparison. That comparison may only exist between the adage and the movie being reviewed, but the Rebuttal doesn't prove that sufficiently. I think this claim is strategic because it puts a higher burden of proof on the rebuttal.
  9. 75% -- There seems to be some support for the claim in the supporting quote and little relation to the claim in the rebutal.
  10. 85% -- The supporting quotes appear to substantiate the root claim in that it mentions the reference in "Twelve Chairs." While this is a strong reference it still is rather loose to be to extend a reference or a similar title to the movie as a whole. The rebuttal appears to be a continuation of the supporting quotes but doesn't provide much additional context.

Subclaim 2: In the review, the reviewer argues the movie is a successor to comedy film “The Big Lebowski”.

Generator credence: 30%

Ensembled step judgement: 28.75% (4 reviews)

Quotes: every other film critic ... is going to make some kind of “I didn’t think the sequel to ‘Big Lebowski’ would be THIS weird” joke a pleasure to see The Dude and Maude in a clinch once more

Rebuttal: the adult leads of the picture are Jeff Bridges and Julianne Moore. Bridges plays a righteous witch-hunting “spook”... Moore plays a near-omnipotent witch

Judge reviews:

  1. 10% -- "every other film critic" excludes "the reviewer".
  2. 35% -- The quote “I didn’t think the sequel to ‘Big Lebowski’ would be THIS weird” joke` can be interpreted as being ironic. As in, the movie is not actually a successor, but because of its actors, people would say that as a joke. But this could be true, and the movie still be considered a bit of a spiritual successor to it.
  3. 80% -- I don't know the actors mentioned. I'm guessing they are playing The Dude and Maude. Rebuttal quote and supporting quote about weirdness have a synergistic verisimilitude going on (that's some chunky phrasing!). Support is clearer than the rebuttal.
  4. 100% -- The supporting quote directly addresses the claim

Subclaim 3: In the review, the reviewer argues that the male lead is a zany character.

Generator credence: 95%

Ensembled step judgement: 15% (5 reviews)

Quotes: Bridges’ ... is more like his “True Grit” Rooster Coburn crossed with Richard Harris in “Camelot,” ... In other words, it’s a truly eccentric performance

Rebuttal: Bridges plays a righteous witch-hunting “spook”… Bridges’ mumble-mouthed, grizzled hero is more like his

Judge reviews:

  1. 10% -- Eccentric is not the same as zany.
  2. 15% -- The supporting quote has an ellipses right after the proper noun suggesting to me a very shaky connection to what follows it. If the lead character is indeed zany then producing a quote that is continuous would be very straight forward. This is likely false as the rebuttal does have the presumed male lead 'Bridges' linked with more relevant adjectives.
  3. 35% -- The quote seems to be trying to mislead, and the rebuttal is convincing against the claim.
  4. 95% -- Eccentric --> zany is an agreeable inference IMO, rebuttal's "is more like his" choice cut makes it seem so very dishonest too.
  5. 100% -- the supporting quotation is very strong!

47. https://www.rogerebert.com/reviews/great-movie-fanny-and-alexander-1983

Root claim: In the review, the reviewer finds the supernatural elements in the movie to be merely a pretty distraction from the core theme.

Generator credence: 5% -- The reviewer suggests at several points that the supernatural elements are fundamental in creating a dream-like story arc for the movie, though he does also find them pretty.

Tree judgement: 12.5%

Ensembled step judgement: 40% (9 reviews)

Subclaim 1: In the review, the reviewer finds the supernatural elements of the movie to be distracting

Subclaim 2: In the review, the reviewer finds the supernatural elements of the movie pretty.

Rebuttal: and Alexander" is above all the story of what Alexander understands is really happening. If magic is real, if ghosts can walk, so be it. Bergman has often allowed the supernatural into his films

Judge reviews:

  1. 20% -- "merely" is not supported.

  2. 25% -- Supporting claims don't take into account "from the core theme". The rebuttal seems to support that, it seems like the supernatural is part of the core theme.

  3. 40% -- The reviewer finds the supernatural elements a) distracting and b) pretty, but not necessarily a "pretty distraction from the core theme" (though this wouldn't be an unreasonable assumption). The rebuttal suggests the core theme might actually be about the supernatural.

  4. 50% -- Supporting claims support all but the 'from the core theme' part. On the other hand, rebuttal suggests that it may be part of the core theme. I see neither true nor false more likely

  5. 75% -- Supporting claims are sufficient.

  6. 90% -- Rebuttal seems to imply supernatural elements may be imaginations of Alexander. This supports the distraction interpretation.

  7. 90% -- Both supporting claims taken together support the claim somewhat. But they do not strictly prove it because of missing information about the core theme.

  8. 93% -- pretty and distracting is established in the supporting claims. That's almost the whole content of the claim.

  9. 98% -- The rebuttal here verifies the existence of supernatural in the film but lacks any commentary on how it related to a core theme in the movie. The supporting claims then stand to validate the root claim.

Subclaim 1: In the review, the reviewer finds the supernatural elements of the movie to be distracting

Generator credence: 15%

Ensembled step judgement: 12.5% (7 reviews)

Quotes: There are fairy-tale elements here, but "Fanny and Alexander" is above all the story of what Alexander understands is really happening

Rebuttal: and Alexander" is above all the story of what Alexander understands is really happening. If magic is real, if ghosts can walk, so be it. Bergman has often allowed the supernatural into his films

Judge reviews:

  1. 5% -- I'm not seeing the distraction angle at all.
  2. 10% -- "distracting" is not supported at all.
  3. 15% -- not well supported at all, and the context also goes against it.
  4. 25% -- The supporting quotes and the rebuttal are from the same block of text in the review and both substantiate the use of supernatural elements in the film. There is a lack of reviewer commentary to say that those supernatural elements are distracting in any way.
  5. 40% -- [11]fairy-tale elements are not in this sense [38]supernatural elements
  6. 50% -- Both show 'supernatural elements' are there. Quote shows it's not the main thing, but that does not need to mean distracting. Rabuttal may show that 'fairy-tail' elements are distracting, but not necessarily all 'supernatural' elemeents, so I'm really unsure.
  7. 70% -- "Distracting" is not fully supported by the supporting quotes, but since the rebuttal also argues for this, I'm convinced.

Subclaim 2: In the review, the reviewer finds the supernatural elements of the movie pretty.

Generator credence: 95%

Ensembled step judgement: 34% (7 reviews)

Quotes: Rarely have I felt so strongly during a movie that my mind had been shifted into a different kind of reality. `The movie is astonishingly beautiful.’

Rebuttal: The movie is astonishingly beautiful. The cinematography is by Bergman's longtime collaborator Sven Nykvist, who surrounds the Ekdahls with color and warmth, and bleeds all of the life out of the

Judge reviews:

  1. 10% -- The movie is beautiful, not the supernatural elements.
  2. 30% -- There is no strong evidence present of supernatural elements in the supporting quote
  3. 38% -- Neither quote nor rebuttal really show there are supernatural elements, and this could be hard to disprove with a rebuttal quote. The claim that it's supernatural elements that's beautiful is also unclear
  4. 60% -- The first supported quote is sufficient, even though the second supporting quote is well contradicted by the rebuttal.
  5. 65% -- Rebuttal would need to either challenge on "pretty" or on "supernatural". It's agreeing on pretty, and "Ekdahls" resembles a word made up for some sort of supernatural creature. But supporting quotes don't have any information about supernatural entities portrayed in the movie.
  6. 100% -- The supporting quote is verbatim support for the movie having beautiful visuals. The rebuttal is a continuation of quotes in the supporting quotes and adds additional detail.
  7. 100% -- direct evidence

48. https://www.rogerebert.com/reviews/twice-upon-a-yesterday-1999

Root claim: In the review, the reviewer argues that a major theme of the movie is the mistreatment of women in society.

Generator credence: 3% -- Reviewer believes film is actually somewhat sexist.

Tree judgement: 10%

Ensembled step judgement: 22.5% (11 reviews)

Subclaim 1: In the review, the reviewer suggests that the male lead character is unsympathetic to reflect men’s position in society.

Subclaim 2: In the review, the reviewer compares the movie to other films with a strong female lead.

Rebuttal: Are women simply a plot convenience for the hero? I ask even though the film was directed by a woman, Maria Ripoll. She should have known better `It's a tedious contrivance about a messy drunk``

Judge reviews:

  1. 10% -- The supporting claims are not convincing.

  2. 10% -- Rebuttal effectively contrasts with supporting claims. Shows they are showing how the author is upset about the lack of focus on mistreatment of women within the film by the director. The opposite of the claim.

  3. 20% -- The claim does not have support in supporting claim, I find no evidence for 'mistreatment'. Rebuttal also hints in this direction, 'women...plot convenience for the hero' seems somewhat more compatible with a film that's not primarily about mistreatment

  4. 25% -- The supporting claims do not directly provide evidence of the claim; having an unsympathetic male lead does not automatically make the mistreatment of women a major theme in the movie. The rebuttal counters the claim.

  5. 25% -- These are much weaker supporting claims than I'm used to in this experiment. The rebuttal gives moderate evidence that the major themes are about other things.

  6. 30% -- the supporting claims dont say much about mistreatement of women. the rebuttal is convincing

  7. 35% -- I don't understand what supporting claim #1 is supposed to mean. My best guess is that it's saying "the reviewer suggests that the male lead character is portrayed as unsympathetic in order to reflect how men in society are too powerful/otherwise flawed", but I'm highly unsure. I understand supporting claim #2, but I don't see how it's relevant to the claim. If I understand supporting claim #1 correctly, it is weak evidence for the claim insofar as it means that the movie is discussing gender roles and the flaws that men can have. However, there are many ways for the supporting claim #1 to be true and the overall claim to be false (e.g. the male lead is unsympathetic for reasons unrelated to mistreatment of women). If the claim were true, I would expect the expert to choose stronger supporting claims. The rebuttal is reasonably strong evidence that the claim is false conditional on the supporting claims being true. In particular, "the film was directed by a woman...but she should have known better" seems to imply that the movie is not sufficiently cognizant of the mistreatment of women in society. At the very least, the quote in the rebuttal is somewhat inconsistent with mistreatment of women being a major theme in the movie. "a tedious contrivance about a messy drunk" is also weak evidence against the claim, because it suggests that the movie is about something other than the mistreatment of women in society. My credence in "the claim given the supporting claims" is only as high as it is because I'm not sure if I interpreted supporting claim #1 correctly. It's quite possible that supporting claim #1, if interpreted correctly, strongly supports the claim and defuses the force of the rebuttal.

  8. 44% -- the supporting claims do not support the claim.

  9. 50% -- Two claims reasonably but not decisively support root claim. First quote potentially undermines it but it Maria Ripoll quote may not be referencing the same movie. This is consistent with supporting Claim #2. So these two pieces of evidence offset each other and I stay at 50

  10. 50% -- The supporting claims and rebuttal neither confirms nor denies the claim

  11. 70% -- I'm not sure how the first quote in the rebuttal is supposed to rebut the claim. From the supporting claims, I can accept mistreatment of women to be a theme but I'm not convinced it's a major theme.

Subclaim 1: In the review, the reviewer suggests that the male lead character is unsympathetic to reflect men’s position in society.

Generator credence: 5%

Ensembled step judgement: 10% (8 reviews)

Quotes: Victor is a bore who's smashed most of the time, and if the great wheels of the universe revolve to give him a second chance, is there no hope for the Sylvias?

Rebuttal: Are women simply a plot convenience for the hero? I ask even though the film was directed by a woman, Maria Ripoll. She should have known better It's a tedious contrivance about a messy drunk

Judge reviews:

  1. 5% -- I'm having immense difficulty parsing the supporting quote. Is "the Sylvias" a moniker for Victor? And how does this quote demonstrate the truth of the claim? If this is the best evidence, it's as if there's no evidence.
  2. 10% -- The claim is about the intentions of the director, and is not supported.
  3. 10% -- I don't know what 'Sylvias' means here. In any case, I don't find any evidence for 'reflection' or 'men's position in society' in general.
  4. 20% -- Supporting quote has limited relevance to the claim.
  5. 20% -- I don't fully understand what the claim means. "the male lead character is unsympathetic" is fairly straightforward, but I don't know what "to reflect men's position in society" means. The supporting quotes give good evidence for a male character in the movie being unsympathetic, but don't establish that he's a lead character. I don't know if the supporting quotes establish that the lead character is unsympathetic to reflect men's position in society, but I doubt it, because there is no mention of "men's position in society" in the quotes. The rebuttal also suggests that "reflect men's position in society" is false, by suggesting that the film is unlikely to be reflecting on society in general. "Are women simply a plot convenience for the hero?" suggests that the film doesn't explore gender roles in society very deeply, and "tedious contrivance about a messy drunk" is also inconsistent with an exploration of gender and society. However, the rebuttal doesn't focus on whether Victor is a lead character (so I assume he is). My main uncertainty is around the meaning of "unsympathetic to reflect men's position in society", but the claim seems fairly unlikely to be true.
  6. 30% -- the male lead probably is unsympathetic but I dont know about "to reflect men’s position in society"- "the great wheels of the universe" - it probably refers to something bigger than our society
  7. 36% -- i cannot see any connection of the supporting quotes to the claim.
  8. 50% -- I don't think this quote really supports the claim. Victor sounds unsympathetic but idk if he's the male lead. Maybe the "sylvias" suggests some broader generalization of Victor and the universe but not enough to say conclusively that the reviewer thinks Victor represents men. Other notes: These pairings of quotes and claims seem suboptimal. It seems strange that the rebuttal stays the same from previous workspace to this one. I wish I could have line breaks!

Subclaim 2: In the review, the reviewer compares the movie to other films with a strong female lead.

Generator credence: 5%

Ensembled step judgement: 20% (9 reviews)

Quotes: Gwyneth Paltrow, in "Sliding Doors," was snared in ... romance and adultery. The new German film "Run Lola Run" plays the same 20 minutes three different ways

Rebuttal: several movies about alternatives in time... Divergent timelines figured in "Groundhog Day," with Bill Murray... Gwyneth Paltrow… was snared in alternate time lines, romance and adultery.

Judge reviews:

  1. 5% -- The claim doesn't appear to be blatantly false, but it smells wrong. Rebuttal's inclusion of Bill Murray with the context of temporal shnenigans suggest the implication that female lead comparisons isn't a main topic in the review.
  2. 10% -- The rebuttal provides context to indicate that the supporting quotes are misleading.
  3. 20% -- The supporting quote was taken out of context (as revealed by the rebuttal) - and countering the claim
  4. 20% -- I honestly have no idea how to interpret the different movies mentioned. It seems like the rebuttal is making the argument that the reviewer is comparing movies about alternatives in time rather than alternatives with strong female leads, but I'm not sure, and I'm also not even sure which movie is being reviewed. At the end of the day, no evidence is provided that the reviewer mentions two different movies with strong female leads (which is a direct implication of the claim), so the claim is less likely to be true. It seems like it should have been easy to establish "this movie has a strong female lead" and "the reviewer compares this movie to another movie that also has a strong female lead" (even with the tight character limit), so the lack of evidence for this is a particularly bad sign. If I had to guess, I'd say that the main topic of the review is "Run Lola Run", which is being compared to "Sliding Doors" and "Groundhog Day" because all 3 movies have divergent timelines. There is evidence that "Sliding Doors" has a female lead, but no evidence that "Run Lola Run" has a female lead, or that female leads are relevant to the comparison. This decreases my credence in the claim further. The most likely way that the claim is true is if the honest expert simply neglected to show that the movie being reviewed had a female lead (perhaps because they thought it was obvious?). This seems unlikely but still possible -- ~20% overall chance that the claim is true?
  5. 30% -- it seems it is more about alternative in time than a strong female lead
  6. 30% -- It seems that the claim is not literally false - there is comparison with two other films with (probably strong) female leads, but from the rebuttal, it is clearly misleading (the comparison is really about alternative timelines). I choose to interpret this as the claim being false.
  7. 46% -- the supporting quote mentions one female lead. But do evidence that the author compares the movie to others with female lead.
  8. 51% -- I'm assuming Sliding Doors is different movie and Gwyneth Paltrow was the lead. But have no decisive evidence to suggest this. Other notes: These quotes (supporting & rebuttal) basically read incomprehensible to me. I have no idea what to make of the ellipses.
  9. 90% -- The rebuttal does not undermine the claim. The reviewer could compare to both "strong female lead" and "alternate time lines".

49. https://www.rogerebert.com/reviews/atlantis-the-lost-empire-2001

Root claim: In the review, the reviewer suggests argues that a major weak point of the movie is how much it derives from other movies.

Generator credence: 7% -- The film gets a very positive review (3.5 stars) and the derivative aspects of the film are not highlighted anywhere as a problem.

Tree judgement: 10%

Ensembled step judgement: 10% (9 reviews)

Subclaim 1: In the review, the reviewer describes the movie Atlantis as taking a lot from old novels and even more from a particular anime

Subclaim 2: In the review, the reviewer describes Disney exploiting the popularity of anime

Subclaim 3: In the review, the reviewer describes a character as typical for the genre and as borrowing from Citizen Kane

Subclaim 4: In the review, the reviewer argues that the movie borrows from “The Little Mermaid”

Rebuttal: spectacular closing sequence embraces the classic energy of the comic book style. You especially see that in the movie's spectacular closing sequence `rousing in an old pulp science fiction sort of way``

Judge reviews:

  1. 4% -- No evidence that this is a weak point. Uncertainty comes from word "exploiting" in supporting claim 2.

  2. 5% -- The supporting claims are not supporting the claim about "a major weak point", on the other hand the rebuttal is very strong this time.

  3. 10% -- "major weak point" is not supported.

  4. 20% -- Clear it derives from other movies, however the rebuttal argues pretty convincingly that not a bad thing. None of the supporting quotes argue the copies are a weak point of the movie.

  5. 20% -- The is no evidence in the supporting claims that proves the borrowings to be a weak point of the movie.

  6. 25% -- Claims show that the movie 'derives' from others, but except for 'exploiting', it does not show it is a 'major weak point'. Rebuttal does a good job of showing it is more likely the other way round

  7. 35% -- The key missing ingredient is "major weak point". None of the supporting quotes come close to this and the rebuttal points out that the reviewer may entirely think otherwise.

  8. 60% -- Supporting claims strongly support the movie deriving from other movies. Whether this is a "weak point" is not established. Though I'd guess if the reviewer thought these derivations were neutral or positive, the rebutter would have selected a quote specifically showing this.

  9. 71% -- Does not directly state that this is a weak point. But probably implies it?

Subclaim 1: In the review, the reviewer describes the movie Atlantis as taking a lot from old novels and even more from a particular anime

Generator credence: 98%

Ensembled step judgement: 86.25% (8 reviews)

Quotes: so does "Atlantis" spring from the old Edgar Rice Burroughs novels ... (There is also discussion ... how it springs even more directly from a 1989 … anime named "Nadia: the Secret of Blue Water”

Rebuttal: ` (There is also discussion on the Web about how it springs even more directly from a 1989 Japanese anime named "Nadia: the Secret of Blue Water.")

Judge reviews:

  1. 10% -- "discussion on the web" rebuts "the reviewer describes".
  2. 60% -- The rebuttal is not strong.
  3. 95% -- Claim directly supported by quotes, unless important context is omitted. Rebuttal shows that omission of context in quotes was honest more than anything else.
  4. 95% -- The supporting quotes are very compelin also the rebuttal somehow supports one part of the claim.
  5. 98% -- Here it looks like both the supporting quotes and the rebuttal are in support of the root claim.
  6. 98% -- Yep, that's what it says unless I'm reading that wrong
  7. 100% -- direct evidence
  8. 100% -- The claim seems to be true due to the information at hand.

Subclaim 2: In the review, the reviewer describes Disney exploiting the popularity of anime

Generator credence: 55%

Ensembled step judgement: 30% (9 reviews)

Quotes: Disney seems to be testing the visual and story style of anime--those action-jammed animated Japanese movies that occupy shelves in every video store, meaning someone must be renting them.

Rebuttal: stands by itself as one of the great animated action sequences the movie was directed by Gary Trousdale and Kirk Wise, who made the wonderful "Beauty and the Beast" for Disney.

Judge reviews:

  1. 10% -- "seems to be testing" does not support "exploiting".
  2. 20% -- Testing visual style of anime != exploiting; rebuttal indicates some care for the genre.
  3. 30% -- The claim is exaggerating. There is no information of exploitation present in the information at hand.
  4. 40% -- testing is different from exploiting
  5. 80% -- Sufficient supporting quotes.
  6. 90% -- The supporting quote seems to be strong, the rebuttal is not realy relevant.
  7. 94% -- Claim directly supported by a single quote. Rebuttal does not really contradict
  8. 100% -- direct evidence
  9. 100% -- The supporting quote is continuous and directly verifies the contents of the root claim. The rebuttal doesn't reference Disney's choices as a production company.

Subclaim 3: In the review, the reviewer describes a character as typical for the genre and as borrowing from Citizen Kane

Generator credence: 98%

Ensembled step judgement: 70% (9 reviews)

Quotes: All stories like this require a rich, reclusive billionaire to finance an expedition to the lost corners of the earth, and "Atlantis" has ... who lives Citizen Kane-style

Rebuttal: You will note among this crew no dancing teacups, even though the movie was directed by Gary Trousdale and Kirk Wise, who made the wonderful "Beauty and the Beast" for Disney.

Judge reviews:

  1. 40% -- Kane-style is not quite the same as borrowed from Kane
  2. 55% -- "who lives citizen kane style" is not the same as saying it borrows from citizen kane. You can be compared to a thing that you don't borrow from. I'm not sure how the rebutter could point this out with a quote, though.
  3. 70% -- Supporting quote strongly supports claim, however rebuttal does poke some holes. Rating is not lower because rebuttal could be a comment about the directors style and not the movie itself.
  4. 90% -- The claim is supported and not rebutted.
  5. 90% -- The supporting quotes are omitting some important part of the text but the rebuttal is not trying to exploit that so it is probably not important. The rebuttal does not seem to be relevant.
  6. 95% -- the supporting quote isn't clear cut, but the rebuttal isn't strong either
  7. 95% -- 'All stories like this require...' is strong evidence for the first part of the claim, 'who lives Citizen Kane-style' is fairly strong evidence for the second half
  8. 96% -- Claim literally supported by quotes unless important context is missing. In this case it's unlikely because of the connection billionaire - Citizen Kane-style. Rebuttal shows other inspirations, but does not really contradicts.
  9. 98% -- The supporting quotes reveals the character 'type' borrowed from many similar films and references 'Citizen Kane' directly. The rebuttal only references the directors.

Subclaim 4: In the review, the reviewer argues that the movie borrows from “The Little Mermaid”

Generator credence: 95%

Ensembled step judgement: 91.25% (10 reviews)

Quotes: The submerged land is ruled by her father the King ... who wants to banish the outsiders, but Kida has eyes for Milo in a subplot owing more than a little to "The Little Mermaid."

Rebuttal: `You will note among this crew no dancing teacups, even though the movie was directed by Gary Trousdale and Kirk Wise, who made the wonderful "Beauty and the Beast" for Disney.``

Judge reviews:

  1. 70% -- Sufficient supporting quote.
  2. 90% -- Claim is pretty much a rewording of supporting quotes.
  3. 90% -- "owing more than a little" supports "borrows from".
  4. 95% -- The supporting quote is very compeling, the rebuttal does not seems to be much relevant...
  5. 96% -- Claim quite literally supported by quotes. Rebuttal tries to show there may be other inspirations, but that does not really contradict the claim
  6. 99% -- direct evidence
  7. 99% -- a subplot owing more than a little to "The Little Mermaid."`
  8. 99% -- The quote specifically mentions "The Little Mermaid" as strong influence for the film.
  9. 99% -- 'subplot owing more than a little to "The Little Mermaid' is near verbatim-evidence for claim
  10. 100% -- The supporting quotes verbatim substantiate the root claim. The rebuttal speaks loosely about what is typical of the directors.

50. https://www.rogerebert.com/reviews/melinda-and-melinda-2005

Root claim: In the review, the reviewer argues that the film is flawed because it emphasizes ideas over making us empathize with the characters.

Generator credence: 10% -- The reviewer thinks it’s not actually a flaw.

Tree judgement: 10%

Ensembled step judgement: 20% (9 reviews)

Subclaim 1: In the review, the reviewer describes the film as centered on the idea that movies are fictional and not real.

Subclaim 2: In the review, the reviewer argues that the film is flawed because we can’t empathize with the characters.

Subclaim 3: In the review, the reviewer suggests that the film emphasizes that the main characters are not real but that the filmmaker (Allen) is.

Rebuttal: "Melinda and Melinda" is a movie about the symbiosis of the filmmaker and the audience...He shows us how he does it and how we do it. In its complexity and wit, this is one of his best recent films.

Judge reviews:

  1. 20% -- The supporting claims don't support "emphasizes ideas over making us empathize".

  2. 20% -- No causal link between "emphasizes ideas" and "flawed"

  3. 20% -- The reviewer seems to appreciate the filmmaker's style from the rebuttal quote

  4. 35% -- The supporting claims fall within the scope of the root claim. The rebuttal however, in suggesting a symbiosis of filmmaker and audience, stands at odds with #1 and #3. Symbiosis of the film maker and audience is strongly suggestive of either empathetic characters or just empathy as a conceptual presence in films.

  5. 63% -- Given SC #2, 'the reviewer argues that the film is flawed' is true, and an inability to empathise with the characters is part of this. Whether or not it is 'because it emphasizes ideas' is in question. The rebuttal suggests it has 'complexity and wit' and is 'one of his best recent films', so the reviewer doesn't seem averse to these 'ideas'. Yet it still seems hard to separate the two - it is the idea that "the main characters are not real" that is presumably causing us to not have empathy with them.

  6. 65% -- #3 is not particularly strong evidence for the claim, as far as I can tell. Taken together, #1 and #2 suggest the claim is true but do not directly imply it -- the reviewer could believe that the inability to empathize with the characters is a flaw, but is unrelated to an emphasis on ideas. For example, maybe the characters are just unlikeable; #3 makes this interpretation somewhat less likely by suggesting a potential link between the movie emphasizing an idea (that movies are fictional), and being unable to empathize with the characters. However, when considering that these claims were optimized to maximize support for the root claim, I find it odd that no claim directly links "centered on ideas" and "difficult to empathize with characters". The rebuttal isn't a knock down argument, but it's difficult to prove that the text doesn't say something. The fact that the overall review is so positive about the movie suggests that the reviewer sees subclaim #2 as a relatively minor problem, which is slightly less likely if it's related to the central conceit of the film

  7. 84% -- the supporting quotes offer good evidence: 1. flawed 2. due to lacking empathy 3. deals with abstract ideas. But the evidence is weak for 1. is because of 3.

  8. 90% -- A movie can be great and flawed at the same time.

  9. 95% -- The rebuttal is arguing against something slightly different from the root claim. The movie can be the best and still be flawed.

Subclaim 1: In the review, the reviewer describes the film as centered on the idea that movies are fictional and not real.

Generator credence: 99%

Ensembled step judgement: 75% (8 reviews)

Quotes: More than any other film that comes to mind … “Melinda" says, clearly and without compromise, that movies are only movies … made up of thin air, the characters are not real

Rebuttal: Woody Allen's "Melinda and Melinda" begins with friends having dinner in a Manhattan restaurant. One of the friends is played by Wallace Shawn, who (Allen's audiences will know) has had a famous

Judge reviews:

  1. 50% -- "centered on" isn't really supported or rebutted.
  2. 60% -- Not sure if the film is centered in that idea. It says that... but is it centered in that?
  3. 80% -- The supporting claim validates the claim
  4. 85% -- good support for the claim. Only the ellipses in the supporting quotes reduce the probability a little
  5. 87.5% -- The supporting quote is strong evidence that the movie does say that "movies are fictional and not real", but there's a small chance that it's incorrect to describe the movie as "centered" on this idea in the full context of the review. I don't see how the rebuttal is relevant.
  6. 94% -- The supporting quotes appear to substantiate the claim in that there is direct statements noting movies are fictional not real entities. I did notice the 172 word count and the use of ellipses. Here the use of '...' seems to be abbreviation over attempts to mislead. The rebuttal contains unrelated plot information.
  7. 95% -- Sufficient supporting quote.
  8. 100% -- Rebuttal isn't addressing the claim while supporting quote is strong

Subclaim 2: In the review, the reviewer argues that the film is flawed because we can’t empathize with the characters.

Generator credence: 10%

Ensembled step judgement: 10% (8 reviews)

Quotes: “Melinda" fails the standards ... because it doesn't deliver a direct emotional charge. It doesn't leave us happy or sad for the characters, or ... which characters we were supposed to care about

Rebuttal: care about. That however is not Allen's failure, but his purpose. More than any other film that comes to mind, "Melinda and Melinda" says, clearly and without compromise, that movies are only movies

Judge reviews:

  1. 5% -- Here the rebuttal with its continuation of quotes in the supporting quotes block, appears to invalidate the claim. "That however is not Allen's failure, but his purpose."
  2. 10% -- Rebuttal provides context to the supporting quote. The lack of empathize-ability with the characters is the purpose of the movie, not it's flaw, according to the reviewer.
  3. 10% -- At a surface reading, claim is true. Taken as advice for watching it, it's false.
  4. 12% -- No evidence that the film is "flawed"
  5. 15% -- The claim "we can't empathize with the characters" seems well-supported, but "the film is flawed" seems very unlikely to be true. The only support for "the film is flawed" in the supporting quote is "fails the standards". However, the ellipse immediately following is very suspicious, and combined with the rebuttal quote, I don't think that the reviewer sees "failing the standard" (whichever standard it is) as a flaw.
  6. 20% -- "fails the standards" is not the same as "is flawed because".
  7. 90% -- Rebuttal argues for the root claim. "However" indicates that the reviewer pointed out a flaw.
  8. 92% -- the claim is a good characterization of the supporting quotes.

Subclaim 3: In the review, the reviewer suggests that the film emphasizes that the main characters are not real but that the filmmaker (Allen) is.

Generator credence: 95%

Ensembled step judgement: 80% (9 reviews)

Quotes: At the end of "Melinda and Melinda," we realize that neither Melinda nor Melinda is real, but Woody Allen certainly is.

Rebuttal: Woody Allen's "Melinda and Melinda" begins with friends having dinner in a Manhattan restaurant. One of the friends is played by Wallace Shawn, who (Allen's audiences will know) has had a famous

Judge reviews:

  1. 20% -- "we realize that" is not the same as "the film empathizes that".
  2. 40% -- The supporting quote is misrepresented here. The quote is the reviewers interpretation of the film, not what the film emphasises.
  3. 80% -- The supporting quote is decent evidence, and the rebuttal is irrelevant, but it's not 100% clear that the film "emphasizes" that the main characters are not real (rather than simply allowing the audience to realize that at the end). If the quote/claim pair were picked randomly my credence would be higher, but the fact that the quote was selected as the strongst evidence for the claim makes me slightly more skeptical.
  4. 80% -- Supporting quote is sufficient.
  5. 90% -- Direct evidence, but sounds fishy somehow, would want more context
  6. 90% -- Casting choice is irrelevant IMO, to how the characters are depicted.
  7. 93% -- very good support in the supporting quote
  8. 97% -- In a movie called 'Melinda and Melinda', presumably two of the main characters are called 'Melinda', and the supporting quote suggests they're not real and Allen is. Although 'Wallace Shawn' is (presumably) a male name, and they weren't specified in the supporting quote, so perhaps that could be a caveat (but probably not).
  9. 98% -- The supporting quotes appear to almost verbatim reflect the contents of the claim in a continuous quote. The rebuttal contains some expository information but lacks comments on the real and the fictional.

51. https://www.rogerebert.com/reviews/the-men-who-stare-at-goats-2009

Root claim: In the review, the reviewer suggests that in the movie people with Jedi powers (from the Star Wars universe) are on Earth.

Generator credence: 1% -- In the movie, people (maybe) have paranormal powers that are similar to Jedi powers but not the same.

Tree judgement: 10%

Ensembled step judgement: 20% (9 reviews)

Subclaim 1: In the review, the reviewer asserts that people in the film can spy at a distance and kill animals using mental powers.

Subclaim 2: In the review, the reviewer describes a character in the film as training to be a Jedi.

Subclaim 3: In the review, the reviewer describes someone in the film as explaining what Jedis are to another character (who hasn’t seen Star Wars).

Rebuttal: acidhead Vietnam veteran named Bill Django `faced with the fact that the movie is based on the 2004 book… doesn't matter if the book is truthful. It doesn't claim the paranormal powers are real.``

Judge reviews:

  1. 5% -- Rebuttal is quite strong in that the powers aren't real. Also if the powers were real it would need to be a proper Star Wars movie, as I don't think other people would have the rights (I don't think this is info contamination, as this is a extremely well known movie, but it is a sort of meta-reasoning that might not be wanted in the experiment).

  2. 15% -- All the supporting claims feel like they have another explanation than Jedis. Rebuttal is somewhat convincing.

  3. 20% -- No quotations supporting claim or subclaims.

  4. 40% -- Let's break down the top claim: what's the evidence for the characters being on Earth? Supporting claim #3 talks about characters who haven't seen Star Wars. This suggests the film takes place in our Universe, where Star Wars is a fictional franchise. Furthermore, the rebuttal mentions a Vietnam veteran, so I'm quite convinced the characters are here on Earth. Now, do these characters have Jedi powers? The supporting claims only establish that 1. some characters have certain (not necessarily "Jedi") mental powers like telepathy and 2. a character is training to be a Jedi. The rebuttal claims one of these characters is an "acidhead" (which suggests these paranormal powers are perhaps a hallucination) and that the book the film is based on does not claim these powers are real. This does not contradict supporting claims #2 and #3, which deal only with what the characters in the film believe, but it contradicts supporting claim #1 because this one claims the reviewer themself asserts these mental powers are real. Therefore, I think the claim hinges on supporting claim #1 being true because the other two only talk about the characters' beliefs. This claim seems to be contradicted by the rebuttal, especially the second quote, but it's lacking in context so I'm very uncertain about this. Overall, I am leaning weakly towards the claim being false.

  5. 58% -- 'spy at a distance and kill animals using mental powers' is not enough to be titled jedi powers (i think). supporting quote 2. und 3. also offer no real support.

  6. 70% -- The supporting quote never specifies it takes place on Earth. But then the rebuttal mentions Vietnam, which makes me trust that the claim is true more strongly.

  7. 75% -- (NB: I don't know anything about Star Wars or Jedi powers. I looked up "Jedi powers" using the procedure in the instructions, but didn't get any useful information about whether "spy at a distance" and "kill animals using mental powers" are typical "Jedi powers"). There are two parts to the claim: A. The movie has characters with Jedi powers (from the Star Wars universe) B. These characters are on Earth. Neither of these parts are directly shown by the supporting claims. Supporting claim #1 could be evidence that A is true (depending on what "Jedi powers" are), but it cannot be decisive evidence for A because there are many possible non-Jedi explanations for characters having these powers. Supporting claim #2 is stronger evidence for A, but is also not decisive (for example, "Jedi training" could be used as a metaphor for the kind of training the character is doing, without literally meaning the character has Jedi powers). Supporting claim #3 is not evidence for any part of the root claim, as far as I can tell. Part B of the root claim is not supported by any supporting claim; normally "the movie would be set on Earth" would be a reasonable assumption, but not for a movie when characters have powers from Star Wars. The rebuttal provides reasonable evidence for part B: if the movie has a Vietnam vet character, then it's probably set on Earth (though this "Bill Django" could also be the author of the book the movie is based on, or related in some other way to the movie without being a character). Apart from that, I don't see how the rebuttal is relevant. It could be interpreted as evidence that people in the movie don't actually have Jedi powers (but e.g. incorrectly believe they do due to being acidheads), but this possibility is ruled out by supporting claim #1.The rebuttal is unsuccessful because it fails to show that the characters have mental powers for reasons other than being a Jedi. Overall, the root claim is quite likely to be true given the supporting claims and lack of a credible rebuttal, but there are still ways that it could be false. In particular, it's reasonably likely on priors that the mentions of "Jedis" are merely an analogy (there are many non-Jedi reasons someone can have mental powers). Also, even if the characters aren't literally Jedi, it might be hard for the rebuttal to prove this with a short quote.

  8. 77% -- Claim support presence of 'Jedi' and paranormal 'powers', not necessarily a connection. My prior is lower because spying at distance are not really Jedi powers. However, if the claim was false in this, I would expect rebuttal to focus on more context for Jedi rather than avoiding it

  9. 87% -- The sporting claims appear to cover most sub-claims of the root claim with exception to relating the listed powers as specifically Jedi powers and that there is a relation of this unfolding on earth as opposed to some other setting earth like or not. The rebuttal offers that the movie is based upon a book lending some credence towards this movie as science fiction. In relying on the supporting claims I can lean towards this being true with some open question as to the actual context of earth.

Subclaim 1: In the review, the reviewer asserts that people in the film can spy at a distance and kill animals using mental powers.

Generator credence: 50%

Ensembled step judgement: 10% (9 reviews)

Quotes: they could spy at a distance penetrate enemy lines in spirit, not in body shows Wilton videos of a goat and a hamster killed by brain power

Rebuttal: In theory, they could spy at a distance… In theory, mind power can... movie is based on the 2004 book… doesn't matter if the book is truthful. It doesn't claim the paranormal powers are real.

Judge reviews:

  1. 3% -- The powers aren't really real. There's a chance this is some sort of surreal movie though.
  2. 10% -- I have the inkling suspicion that this movie takes place in a mental hospital or something.
  3. 10% -- The rebuttal shows the supporting quotes are misleading by omitting the crucial "in theory", which suggests the powers aren't real. It further explains the book the film is based on does not claim the paranormal powers are real. 10% because "0 is not a probability".
  4. 20% -- Taken alone, the supporting quotes are reasonably strong evidence for the claim, but the rebuttal is quite decisive. Adding "in theory" to the quote "they could spy at a distance" substantially changes the meaning, and makes it quite unlikely that the reviewer is actually asserting that people at in the film can spy at a distance. Given this, it seems quite likely that the quote "in theory, mind power can" in the rebuttal directly precedes "penetrate enemy lines in spirit, not in body" from the supporting quotes. It's also unlikely that "shows Wilton videos of a goat and a hamster killed by brain power" is evidence that the reviewer is asserting that people in the film can kill animals with mental powers (rather than e.g. saying that one character showed another a hoax video of killing a goat with mental powers). The final two quotes in the rebuttal weaken the claim further by explicitly saying that a book "doesn't claim paranormal powers are real". It's likely but not certain that the book in question is the "2004 book" that the movie is based on, implying that the movie doesn't claim that the paranormal powers are real. Overall, if the movie actually had characters with paranormal powers, there should have been much stronger supporting quotes. The most likely way for the claim to be true is if the movie was based on the 2004 book, but embellished the book by claiming the paranormal powers were real (unlike the book). In this case, the rebuttal quotes would be irrelevant as they apply to the book rather than the movie, though I'd still expect stronger supporting quotes.
  5. 30% -- Rebuttal provided stronger evidence than the supporting quotes
  6. 40% -- Claim is literally supported by quotes, but rebuttal shows interesting context which subverts this. My best interpretation is: someone in the film asserts people have paranormal power, but the film itself does not.
  7. 43% -- a video shown where animals a killed by 'brain power' is not enough to establish that people can kill with mental powers. Also the rebuttal offers good counterevidence ('It doesn't claim the paranormal powers are real')
  8. 80% -- Supporting quote is sufficient.
  9. 94% -- The supporting quotes check off mention of spying at a distance, spiritual powers, and killing animals with mental powers. There is about 55 more character budget available in the supporting quotes which makes use of the ellipses mildly concerning. The rebuttal appears to add additional credence as it also references these claims and cites that most of the film is based upon a 2004 book.

Subclaim 2: In the review, the reviewer describes a character in the film as training to be a Jedi.

Generator credence: 95%

Ensembled step judgement: 42% (9 reviews)

Quotes: The movie flashes between the recent Middle East and events 20 years earlier, when Cassady trained under Django to become, he explains, a Jedi Warrior.

Rebuttal: acidhead Vietnam veteran named Bill Django `faced with the fact that the movie is based on the 2004 book… doesn't matter if the book is truthful. It doesn't claim the paranormal powers are real.acidhead Vietnam veteran named Bill Django (Jeff Bridges), who sold the Army the notion of fighting men who could transcend physical limitations

Judge reviews:

  1. 30% -- The supporting quote proves that one character in the movie claims that he trained to be a Jedi Warrior, but not that the reviewer describes the character as training to be a Jedi in reality. Given the rebuttal, the most likely interpretation is that Cassady trained under a man (Django) who incorrectly claimed to have Jedi powers. I'm ~80% sure that this or a similar interpretation is true. My main uncertainty here is whether said interpretation would makes the claim true or false; my best guess is that it makes the claim false, because the reviewer doesn't claim that the character is literally training as a Jedi (nor is the character training as a Jedi in reality). However, it's also plausible that Cassady believed he was training to be a Jedi, and that that's sufficient to make the claim true.
  2. 35% -- Something is wrong here, in a sneaky way. From all the context I've seen about this movie, I'd rate this 5.
  3. 42% -- in the supporting quote not the reviewer describes it as jedi training but it seems like an indirect quote from the character. So the claim invokes (probably) untruth associations.
  4. 80% -- The supporting quote validates the claim that a character in the film is training to be a Jedi
  5. 90% -- The supporting quote shows the character named Cassady undergoes training as a Jedi. The rebuttal points to the fact that this training is fake since the teacher is an "acidhead" and, at the meta level, because the movie is based on a book that does not claim paranormal powers are real. I am very confident in my view of what's going on in the film, but I understand why assigning a truth value to this claim can be tricky, since the training is not real (as it would be in a real Star Wars film). Nonetheless I am going to give it a 90%, because, taken literally, the claim only says the character is training to be a Jedi - and this is true regardless of whether the training turned out to be a scam or not.
  6. 90% -- Claim is literally supported by a single quote. It may be misleading in not making a distinction between 'Jedy' and 'Jedi Warrior' in a different meaning, but imo that does not make the claim false.
  7. 95% -- The supporting quotes covers the content of the claim in a continuous one sentence quote. The rebuttal mentions a Django character that was the films character for training Jedis. This adds additional credence to the supporting quotes.
  8. 95% -- Supporting quote is sufficient.
  9. 100% -- Not a real jedi. Unsure on how I should score this. Either 0 or 100

Subclaim 3: In the review, the reviewer describes someone in the film as explaining what Jedis are to another character (who hasn’t seen Star Wars).

Generator credence: 95%

Ensembled step judgement: 76.5% (8 reviews)

Quotes: He explains his theory of Jedi Warriordom to Wilton, who has apparently never seen "Star Wars" episodes I, II and III.

Rebuttal: Wilton (Ewan McGregor) He explains his theory of Jedi Warriordom to Wilton, who has apparently never seen "Star Wars" episodes I, II and III. Little joke.

Judge reviews:

  1. 55% -- It's apparently clear from the supporting quote that a character in the film is describing a "theory of Jedi Warriordom" to someone who hasn't seen Star Wars. It's unclear if there are important contextual differences between explaining a "theory of Jedi Warriordom" and "explaining what Jedis are", and it's also unclear what "Little joke" from the end of the rebuttal means. If the character is explaining what Jedis are as a joke, I believe that would still make the claim true. If the reviewer is joking about the character explaining Jedis, then that would make the claim false. Neither of those hypotheticals seem very likely though, so I have no idea what "little joke" actually means. Overall, the supporting quote seems to make it more likely than not that the claim is true, but I'm unsure about too many things to assign a high credence.
  2. 60% -- The claim is supported by supporting quotes and rebuttal
  3. 82% -- 'his theory of Jedi Warriordom' =/ explaining what jedis are in star wars (that's what the claim suggests), but it is close enough that the claim can be true with a sensible interpretation
  4. 85% -- "Wilton's Jedi Warriordom" and "what Jedis are" could have some major differences between them.
  5. 90% -- The supporting quote and the rebuttal tell the same story: someone explains his theory about Jedis to somebody else, who has not seen Star Wars. This fully proves the claim. 90% because "1 is not a probability".
  6. 98% -- Claim literally supported by quote. Rebuttal seems intentionally misleading by making the claim lighter with 'joke', but doesn't really contradict
  7. 100% -- direct evidence
  8. 100% -- The supporting quote and rebuttal are nearly identical and both nearly verbatim substantiate the claim. 'explains his theory of jedi warriordom'
You can’t perform that action at this time.