clean template parameters of less than 50 words (#9)

pyproject.toml

@@ -73,7 +73,7 @@ license = "MIT"
 name = "copypatrol-backend"
 readme = "README.md"
 repository = "https://github.com/JJMC89/copypatrol-backend"
-version = "2024.4.3"
+version = "2024.4.10"
 
 [tool.poetry.dependencies]
 python = ">=3.11,<4"

src/copypatrol_backend/check_diff.py

@@ -56,6 +56,19 @@ def clean_wikitext(text: str, /, *, site: APISite) -> str:
     for link in wikicode.ifilter_external_links():
         with suppress(ValueError):
             wikicode.replace(link, link.title or "")
+    # remove template parameter values of less than 50 words
+    for tpl in reversed(wikicode.filter_templates()):
+        keep_params = []
+        for param in tpl.params:
+            if len(param.value.strip_code().strip().split(" ")) >= 50:
+                keep_params.append(param)
+        if len(keep_params) == 0:
+            with suppress(ValueError):
+                wikicode.remove(tpl)
+        else:
+            for param in keep_params:
+                with suppress(ValueError):
+                    tpl.remove(param)
     # remove references of less than 50 words
     for tag in wikicode.ifilter_tags():
         if tag.tag.lower() not in ("ref", "references"):

testing/fixtures/Basil_Lee_Whitener-1173291523-cleaned.txt

@@ -1,5 +1,3 @@
-American politician
-Basil Lee Whitener BasilLeeWhitener.jpg Member of theU.S. House of Representativesfrom North Carolina January 3, 1957 January 3, 1969 Woodrow W. Jones Jim Broyhill 11th District (1957-1963) 10th District (1963-1969) Democratic Party 1915 05 14 York County, South Carolina 1989 03 20 1915 05 14 Gastonia, North Carolina Rutherford College University of South Carolina Duke University Law School
 Basil Lee Whitener (May 14, 1915 – March 20, 1989) was a Democratic U.S. Representative from North Carolina between 1957 and 1969.
 
 Whitener was born in York County, South Carolina on May 14, 1915, and was educated in the public schools of Gaston County, North Carolina. He graduated from Lowell High School in 1931 and from Rutherford College in 1933, attending the University of South Carolina from 1933 to 1935 and graduating from Duke University Law School in 1937. He was admitted to the North Carolina bar in 1937 and commenced practice of law in Gastonia, North Carolina.
@@ -8,17 +6,10 @@ In 1941 Whitener was elected to the North Carolina House of Representatives and
 
 Whitener was elected as a Democrat to the Eighty-fifth and to the five succeeding Congresses (January 3, 1957 – January 3, 1969); he was an unsuccessful candidate for reelection in 1968 to the Ninety-first Congress and an unsuccessful candidate for election in 1970 to the Ninety-second Congress. He resumed the practice of law.
 
-In 1966, Whitener unsuccessfully introduced an amendment to a bill to make Title VI of the Civil Rights Act of 1964 inoperative.2008 Perlstein Rick Rick Perlstein Nixonland: The Rise of a President and the Fracturing of America Chapter 6, p. 134 live 2023-09-01 Scribner On August 9, the twelfth day of debate in the full House on the civil rights bill, a North Carolina congressman by the apt name of Basil Whitener introduced an amendment to moot Title VI of the 1964 Civil Rights Act outright (Whitener had earlier whined of an amendment offering relief for Negroes injured or intimidated while voting, ).
+In 1966, Whitener unsuccessfully introduced an amendment to a bill to make Title VI of the Civil Rights Act of 1964 inoperative.
 
 Whitener was a resident of Gastonia, North Carolina until his death there on March 20, 1989.
 
 References
 
 External links
-Biography
-W000410
-8059913 Basil Lee Whitener
-
-us-hs
-North Carolina 11 Woodrow W. Jones Roy A. Taylor January 3, 1957 – January 3, 1969
-North Carolina 10 Charles R. Jonas Jim Broyhill January 3, 1963 – January 3, 1969

testing/fixtures/Kommet,_ihr_Hirten-1126962296-added.txt

@@ -16,4 +16,4 @@ Angels and shepherds together we go
 Seeking the Savior from all earthly woe;
 While angels, winging, His praise are singing,
 Heaven's echoes ringing, peace on earth bringing,
-Good will to men.left
+Good will to men.

testing/fixtures/Kommet,_ihr_Hirten-1126962296-cleaned.txt

@@ -1,15 +1,14 @@
-German Christmas carol Carl Riedel portrait.jpg Carl Riedel, who wrote the text by Carl Riedel German Bohemian 1870
 (Come, ye shepherds) is a German Christmas carol from Bohemia which was derived from a Czech carol, . It reflects elements from the nativity story, the annunciation to the shepherds, their walk to the manger and their Adoration, inviting to follow their example. The first line, , addresses shepherds, men, and women.
 
 History
-The text of was derived from a Czech carol, .34 The German text was written as a free translation from the Czech original by Carl Riedel in Leipzig.34 It was first published under the title de Die Engel und die Hirten no (The Angels and the Shepherds) in 1870 in his collection de Altböhmische Gesänge für gemischten Chor (Old-Bohemian chants for mixed choir).34 The song is in the tradition of shepherd songs, derived from the Annunciation to the shepherds and the Adoration of the Shepherds from the Gospel of Luke (Luke 2:8-20).34 Riedel's version is suited for a broad public.
+The text of was derived from a Czech carol, . The German text was written as a free translation from the Czech original by Carl Riedel in Leipzig. It was first published under the title (The Angels and the Shepherds) in 1870 in his collection (Old-Bohemian chants for mixed choir). The song is in the tradition of shepherd songs, derived from the Annunciation to the shepherds and the Adoration of the Shepherds from the Gospel of Luke (Luke 2:8-20). Riedel's version is suited for a broad public.
 
-The song is still popular. It is included in the Protestant hymnal Evangelisches Gesangbuch as EG 48, and in some regional sections of the Catholic hymnal Gotteslob of 1975 and its second edition, the Gotteslob of 2013.34
+The song is still popular. It is included in the Protestant hymnal Evangelisches Gesangbuch as EG 48, and in some regional sections of the Catholic hymnal Gotteslob of 1975 and its second edition, the Gotteslob of 2013.
 
 In English, the song became known as , translated by Mari Ruef Hofer in 1912.
 
 Text
-The Czech song (Bringing the news) has five stanzas. Riedel wrote a version in three stanzas of five lines each, rhyming in pairs with a concluding short last line. The beginning is (Come, you shepherds, men and women).34
+The Czech song (Bringing the news) has five stanzas. Riedel wrote a version in three stanzas of five lines each, rhyming in pairs with a concluding short last line. The beginning is (Come, you shepherds, men and women).
 
 Kommet, ihr Hirten, ihr Männer und Fraun,
 Kommet, das liebliche Kindlein zu schaun,
@@ -44,10 +43,10 @@ Angels and shepherds together we go
 Seeking the Savior from all earthly woe;
 While angels, winging, His praise are singing,
 Heaven's echoes ringing, peace on earth bringing,
-Good will to men.left
+Good will to men.
 
 Melody
-In 1847, the melody was first published in the collection cs Katolicky kancionál in Olmütz. Manuscripts are not dated but seem to stem from the first half of the 19th century. It is a bordun melody similar to folk music melodies.
+In 1847, the melody was first published in the collection in Olmütz. Manuscripts are not dated but seem to stem from the first half of the 19th century. It is a bordun melody similar to folk music melodies.
 
 See also
 List of Christmas carols