Switch branches/tags
Nothing to show
Find file
Fetching contributors…
Cannot retrieve contributors at this time
1877 lines (1692 sloc) 210 KB
description: The downwards deletionism spiral discourages contribution and is how Wikipedia will die.
tags: Wikipedia, experiments, sociology, Haskell
A perennial lure of technology is its promise to let us do things that we couldn't do before, and in ways we wouldn't before.
An example here would be Wikipedia and wikis in general: by lowering the 'cost' of changing a page, and using software that makes undoing most vandalism far easier than doing it, the participation goes through the roof. It's not the technology itself that really matters, but how easy and comfortable it is to contribute[^hill]. I've often thought that if the 'barriers to entry' were charted against 'contributed effort', one would see an [exponentially inverse](!Wikipedia "Exponential decay") relation. An entire essay could likely be written on how the Wikipedia community put up small barriers - each individually reasonable, and not too onerous even in the aggregate - of referencing, of banning anonymous page creation, etc. led to the first sustained drop in contributors and contribution. The effect is nonlinear.
[^hill]: [Benjamin Mako Hill](!Wikipedia) has been [investigating]( why Wikipedia, out of 8 comparable attempts to write an online encyclopedia, succeeded; his conclusion seems to be that Wikipedia succeeded by focusing on developing content and making contribution easy. ["The contribution conundrum: Why did Wikipedia succeed while other encyclopedias failed?"](
> "One answer, which seems obvious only in retrospect: Wikipedia attracted contributors because it was built around a familiar product — the encyclopedia. Encyclopedias aren't just artifacts; they're also epistemic frames. They employ a particular — and, yet, universal — approach to organizing information. Prior to Wikipedia, online encyclopedias tried to do what we tend to think is a good thing when it comes to the web: challenging old metaphors, exploding analog traditions, inventing entirely new forms...Another intriguing finding: Wikipedia focused on substantive content development instead of technology. Wikipedia was the only project in the entire sample, Hill noted, that didn't build its own technology. (It was, in fact, generally seen as technologically unsophisticated by other encyclopedias' founders, who saw themselves more as technologists than as content providers.) [GNUpedia](!Wikipedia "GNE (encyclopedia)"), for example, had several people dedicated to building its infrastructure, but none devoted to building its articles. It was all very *if you build it, they will come*...There are two other key contributors to Wikipedia's success with attracting contributors, Hill's research suggests: *Wikipedia offered low transaction costs to participation, and it de-emphasized the social ownership of content*. Editing Wikipedia is easy, and instant, and virtually commitment-free. "You can come along and do a drive-by edit and never make a contribution again," Hill pointed out. And the fact that it's difficult to tell who wrote an article, or who edited it — rather than discouraging contribution, as you might assume — actually encouraged contributions, Hill found. "Low textual ownership resulted in more collaboration," he put it. And that could well be because Wikipedia's authorless structure lowers the pressure some might feel to contribute something stellar. The pull of reputation can discourage contributions even as it can also encourage them. So Wikipedia "took advantage of marginal contributions," Hill noted — a sentence here, a graf there — which, added up, turned into articles. Which, added up, turned into an encyclopedia."
# New regimes
The best rule of thumb here is perhaps the one cited by [Stewart Brand](!Wikipedia) in _[The Clock of the Long Now](!Wikipedia)_:
> "According to a rule of thumb among engineers, any tenfold quantitative change _is_ a qualitative change[^hamming], a fundamentally new situation rather than a simple extrapolation."
[^hamming]: From "Digital Filters II" in _The Art of Doing Science and Engineering_, [Richard W. Hamming](!Wikipedia) 1997:
> "This is exactly the same mistake which was made endlessly by people in the early days of computers. I was told repeatedly, until I was sick of hearing it, computers were nothing more than large, fast desk calculators. "Anything you can do by a machine you can do by hand.", so they said. This simply ignores the speed, accuracy, reliability, and lower costs of the machines vs. humans. Typically a single order of magnitude change (a factor of 10) produces fundamentally new effects, and computers are many, many times faster than hand computations. Those who claimed there was no essential difference never made any significant contributions to the development of computers. Those who did make significant contributions viewed computers as something new to be studied on their own merits and not as merely more of the same old desk calculators, perhaps souped up a bit."
Clear as mud, eh? Let's try more quotes, then:
> "The human longing for freedom of information is a terrible and wonderful thing. It delineates a pivotal difference between mental emancipation and slavery. It has launched protests, rebellions, and revolutions. Thousands have devoted their lives to it, thousands of others have even died for it. And it can be stopped dead in its tracks by requiring people to search for "how to set up proxy" before viewing their anti-government website.
> I was reminded of this recently by Eliezer's [Less Wrong Progress Report]( He mentioned how surprised he was that so many people were posting so much stuff on [Less Wrong](, when very few people had ever taken advantage of [Overcoming Bias](' policy of accepting contributions if you emailed them to a moderator and the moderator approved. Apparently all us folk brimming with ideas for posts didn't want to deal with the aggravation."^[Yvain, ["Beware Trivial Inconveniences"]( The connection to Wikipedia [is obvious](]
> "We examine open access articles from three journals at the University of Georgia School of Law and confirm that legal scholarship freely available via open access improves an article's research impact. Open access legal scholarship – which today appears to account for almost half of the output of law faculties – can expect to receive 50% more citations than non-open access writings of similar age from the same venue."[^law]
[^law]: Abstract of["Citation Advantage of Open Access Legal Scholarship"](, Donovan & Watson 2011. Are legal scholars *lazy*? Are law libraries *ill-funded*? Do legal scholars have little incentive to write well-researched papers? And yet, making papers a little easier to access results in a dramatic difference in citation.
[Swan 2010]( surveyed a 31 studies and found 27 showing benefits to OA. For example, benefits to [open access](!Wikipedia) were found in [biology]( by [Gunther Eysenbach](!Wikipedia), and [Steve Lawrence](!Wikipedia "Steve Lawrence (computer scientist)") found similar results for [computer science]( articles online or offline:
> "The mean number of citations to offline articles is 2.74, and the mean number of citations to online articles is 7.03, or 2.6 times greater than the number for offline articles. These numbers mask variations over time -- in particular, older articles have more citations on average, and older articles are less likely to be online. When considering articles within each year, and averaging across all years from 1990 to 2000, we find that online articles are cited 4.5 times more often than offline articles.
> We also analyzed differences within each publication venue, where multiple years for the same conference are considered as separate venues. We computed the percentage increase in the average number of citations to online articles compared to offline articles. When offline articles were more highly cited, we used the negative of the percentage increase for offline articles. For example, if the average number of citations for offline articles is 2, and the average for online articles is 4, the percentage increase would be 100%. For the opposite situation, the percentage increase would be -100%. Figure 2 shows the results. Averaging the percentage increase across 1,494 venues containing at least five offline and five online articles results in an average of 336% more citations to online articles compared to offline articles published in the same venue [the first, second (median), and third quartiles of the distribution are 58%, 158%, and 361%]."
On the other hand, one [economics]( study showed no benefit, [Craig et al 2007]( found no benefit in one physics subfield:
> "Three non-exclusive postulates have been proposed to account for the observed citation differences between OA and non-OA articles: an open access postulate, a selection bias postulate, and an early view postulate. The most rigorous study to date (in condensed matter physics) showed that, after controlling for the early view postulate, the remaining difference in citation counts between OA and non-OA articles is explained by the selection bias postulate. No evidence was found to support the OA postulate per se; i.e. article OA status alone has little or no effect on citations. Further studies using a similarly rigorous approach are required to determine the generality of this finding."
> "There are tools to just say, "Give me your social security number, give me your address and your mother's maiden name, and we send you a physical piece of paper and you sign it and send it back to us." By the time that's all accomplished, you are a very safe user. But by then you are also not a user, because for every step you have to take, the dropoff rate is probably 30 percent. If you take ten steps, and each time you lose one-third of the users, you'll have no users by the time you're done with the fourth step."^[[Max Levchin](!Wikipedia), PayPal co-founder; pg 11, [_Founders at Work_](]
> "For example, usability theory holds that if you make a task 10% easier, you double the number of people that can accomplish it. I've always felt that if you can make it 10% easier to fill in a [bug report](!Wikipedia), you'll get twice as many bug reports. (When I removed two questions from the [Joel On Software](!Wikipedia) signup page, the rate of new signups went up dramatically)."^[_Joel on Software_, ["FogBugz"](]
> "Think of these barriers as an obstacle course that people have to run before you can count them as your customers. If you start out with a field of 1000 runners, about half of them will trip on the tires; half of the survivors won't be strong enough to jump the wall; half of *those* survivors will fall off the rope ladder into the mud, and so on, until only 1 or 2 people actually overcome all the hurdles. With 8 or 9 barriers, *everybody* will have one non-negotiable deal killer....By incessant pounding on eliminating barriers, [Microsoft] slowly pried some market share away from Lotus."^[_Joel on Software_, ["Strategy Letter III: Let Me Go Back!"](]
> "The vast majority of raters were previously only readers of Wikipedia. Of the registered users that rated an article, 66% had no prior editing activity. For these registered users, rating an article represents their first participatory activity on Wikipedia. These initial results show that we are starting to engage these users beyond just passive reading, and they seem to like it...Once users have successfully submitted a rating, a randomly selected subset of them are shown an invitation to edit the page. Of the users that were invited to edit, 17% attempted to edit the page. 15% of those ended up successfully completing an edit. These results strongly suggest that a feedback tool could successfully convert passive readers into active contributors of Wikipedia. A rich text editor could make this path to editing even more promising."^[["Rate this Page" is Coming to the English Wikipedia](, WMF blog]
# Toeing the precipice
It may take only a few restrictions before one has inched far enough the 'barriers' axis that the 'contributions' does in fact fall by tenfold. One sees Wikipedia slowly adding restrictions:
- 2005: we [ban anonymous page creation](!Wikipedia "Wikipedia:Wikipedia Signpost/2005-12-05/Page creation restrictions");
- 2006: anonymous users must [solve CAPTCHAs]( if they wish to add URLs;
- 2007: use `{{fact}}` templates institutionalized, and tougher referencing guidelines;
- 2008: harsher [AfDs](!Wikipedia "Wikipedia:Articles for deletion") mean a banner year for [deletionists]( such as [User:TTN](!Wikipedia);
- 2009: [flagged revisions](!Wikipedia "Wikipedia:Flagged revisions") on some wikis, in some areas of English Wikipedia. The end of live changes, [prior restraint](!Wikipedia) on publication.
- 201?: [new editors](!Wikipedia "Wikipedia:Village pump (miscellaneous)/Archive 31#We need more New Page Patrollers") [banned](!Wikipedia "Wikipedia:Village pump (proposals)/Proposal to require [autoconfirmed](!Wikipedia "Wikipedia:User access levels#Autoconfirmed users") status in order to create articles") from article creation (for their own good, of course)
Each of these steps seems harmless enough, perhaps, because we can't see the things which do *not* happen as a result (this is a version of [Frédéric Bastiat]('s [fallacy of the invisible](!Wikipedia "Parable of the broken window")). The legalistic motto "that which is not explicitly permitted is forbidden" has the virtue of being easy to apply, at least.
Few objected to the banning of anonymous page creation by [Jimbo Wales](!Wikipedia) during the [Seigenthaler incident](!Wikipedia), and most of those were unprincipled ones. The objector was all for a tougher War on Drugs - er, I mean Terror, or was that Vandalism? (maybe Poverty) - but they didn't want to be stampeded into it by some bad PR. Too, few objected to [CAPTCHA](!Wikipedia)s: 'take that you scumbag spammers!' The ironic thing is, as a fraction of edits, vandalism shrunk from [2003-2008]( (remaining roughly similar since) and similarly, users specializing in vandal fighting and their workload of edits [have shrunk](; graphing new contributions by size, one finds that for both registered and anonymous users, [the apogee was 2007]( and vandalism has been decreasing ever since. (A more ambiguous statistic is the reduced number of actions by [new page patrollers](
## Falling
> 'Who alive can say, \
> "Thou art no Poet -- may'st not tell thy dreams?" \
> Since every man whose soul is not a clod \
> Hath visions, and would speak, if he had loved, \
> And been well nurtured in his mother tongue.'^[[John Keats](!Wikipedia); _[The Fall of Hyperion: A Dream](!Wikipedia)_ I 11-5]
But by 2007 the water had become hot enough to be felt by devotees of modern fiction (that is, anime & manga franchises, video games, novels, etc.), and even the great Jimbo [could not expect](!Wikipedia "Wikipedia:Articles for deletion/Mzoli's Meats") to see his articles go un-AfD'd.
But who really cares about what some nerds like? What matters is [Notability](!Wikipedia "Wikipedia:Notability") with a capital N, and the fact that our feelings were hurt by some [Wikigroaning](! After all, clearly the proper way to respond to the observation that [Lightsaber combat]( was longer than [Sabre](!Wikipedia "Sabre") is to delete its contents and have people read the short, scrawny - but _serious_! - [Lightsaber](!Wikipedia "Lightsaber") article instead.
If it doesn't appear in Encarta or Encyclopedia Britannica, or isn't treated at the same (proportional) length, then it must go!
## By the numbers
> "Imagine a world in which every single person on the planet is given free access to the sum of all human knowledge. That's what we're doing."^[[Jimmy Wales](, 2004]
Deleting based on notability, fiction articles in particular, doesn't merely ill-serve our readers (who are numerous; note how many of Wikipedia's [most popular pages]( are fiction-related, both now and [in 2007]( or [2011](!Wikipedia "Wikipedia:Wikipedia Signpost/2011-09-19/Popular pages"), or how many Internet searches lead to Wikipedia for cultural content[^aus]), but it also damages the community.
[^aus]: ["The search queries that took Australian Internet users to Wikipedia"](, Waller 2011:
> "This exploratory study analyses the content of the search queries that led Australian Internet users from a search engine to a Wikipedia entry. The study used transaction logs from Hitwise that matched search queries with data on the lifestyle of the searcher. A total sample of 1760 search terms, stratified by search term frequency and lifestyle, was drawn....The results of the study suggest that Wikipedia is used more for lighter topics than for those of a more academic or serious nature. Significant differences among the various lifestyle segments were observed in the use of Wikipedia for queries on popular culture, cultural practice and science."
We can see it indirectly in the global statistics. The analyses ([2007](!Wikipedia "User:Dragons flight/Log analysis"), [2008](!Wikipedia "Wikipedia:Wikipedia_Signpost/2009-01-03/Editing stats")) show it. We are seeing fewer new editors, few new articles, fewer new images; less of everything, basically, except tedium.
Worse, it's not that the growth of Wikipedia has stopped accelerating in important metrics. The rate of increase has in some cases not merely stopped increasing, but started dropping!
> "...the size of the active editing community of the English Wikipedia peaked in early 2007 and has declined somewhat since then. Like Wikipedia's article count, the number of active editors grew exponentially during the early years of the project. The article creation rate (which is tracked at [Wikipedia:Size of Wikipedia](!Wikipedia)) peaked around August 2006 at about 2400 net new articles per day and has fallen since then, to around under 1400 in recent months. [The graph is mirrored at Andrew Lih's ["Wikipedia Plateau?"](]
> ...[User:MBisanz](!Wikipedia) has charted the number of new accounts registered per month, which tells a very similar story: March 2007 recorded the largest number of new accounts, and the rate of new account creation has fallen significantly since then. Declines in activity have also been noted, and fretted about, at [Wikipedia:Requests for adminship](!Wikipedia)...."
This been noted in multiple sources, such as Felipe Ortega's thesis, ["_Wikipedia: A Quantitative Analysis_"](
> "So far, our empirical analysis of the top ten Wikipedias has revealed that the stabilization of the number of contributions from logged authors in Wikipedia during 2007 has influenced the evolution of the project, breaking down the steady growing rate of previous years...
> Unfortunately, this results raise several important concerns for the Wikipedia project. Though we do not have empirical data from 2008, the change in the trend of births and deaths [new & inactive editors] will clearly decrease the number of available logged authors in all language versions, thus cutting out the capacity of the project to effectively undertake revisions and improve contents. Even more serious is the slightly decreasing trend that is starting to appear in the monthly number of births of most versions. The rate of deaths, on the contrary, does not seem to leave its ascending tendency. Evaluating the results for 2008 will be a key aspect to validate the hypothesis that this trend has changed indeed, and that the Wikipedia project needs to put in practice more aggressive measures to attract new users, if they do not want to see the monthly effort decrease in due course, as a result of the lack of human authors."^[pg 136, "4.4 Demographic Analysis of the Wikipedia Community"]
Ortega notes indications that this is a pathology unique to En:
> "In the first place, we note the remarkable difference between the English and the German language versions. The first one presents one of the worst survival curves in this series, along with the Portuguese Wikipedia, whereas the German version shows the best results until approximately 800 days. From that point on, the Japanese language version is the best one. In fact, the German, French, Japanese and Polish Wikipedias exhibits some of the best survival curves in the set, and only the English version clearly deviates from this general trend. The most probable explanation for this difference, taking into account that we are considering only logged authors in this analysis, is that the English Wikipedia receives too contributions from too many casual users, who never come back again after performing just a few revisions."^[_ibid_. pg 137]
Erik Moeller of the WMF tried to [wave away the results]( by pointing out that "The number of people writing Wikipedia peaked about two and a half years ago, declined slightly for a brief period, and has remained stable since then", but he also shoots himself in the foot by pointing out that the number of articles keeps growing. That is not a sustainable disparity. Worse, as the original writers leave, their articles become [legacy code](!Wikipedia) - on which later editors must engage in [archaeology](!Wikipedia "Software archaeology"), trying to retrieve the original references or understand why something was omitted, or must simply remove content because they do not understand the larger context or are ignorant. (I have had considerable difficulty answering some straightforward questions about errors in articles I researched and wrote entirely on my own; how well could a later editor have handled the questions?)
The numbers have been depressing ever since, from the 2010 [informal](!Wikipedia "User:Mr.Z-man/newusers") & [Foundation]( [study]([^Kaggle] on editor demographics to [2011 article contributions](!Wikipedia "File:EnwikipediagrowthGom.PNG"); the _WSJ_'s statistician Carl Bialik [wrote in September 2011]( that "the number of editors is dwindling. Just 35,844 registered editors made five or more edits in June, down 34% from the March 2007 peak. Just a small share of Wikipedia editors—about 3%—account for 85% of the site's activity, a potential problem, since participation by these heavy users has fallen even more sharply."
[^Kaggle]: The Kaggle [background information]( on the "Wikipedia's Participation Challenge" includes an interesting extract from the WMF report:
> "Between 2005 and 2007, newbies started having real trouble successfully joining the Wikimedia community. Before 2005 in the English Wikipedia, nearly 40% of new editors would still be active a year after their first edit. After 2007, only about 12-15% of new editors were still active a year after their first edit. Post-2007, lots of people were still trying to become Wikipedia editors. What had changed, though, is that they were increasingly failing to integrate into the Wikipedia community, and failing increasingly quickly."
Only in 2010 and 2011 has the Foundation seemed to wake up and see what the numbers were saying all along; while Wales [says]( some of the right things like "A lot of editorial guidelines...are impenetrable to new users", he also back-handedly dismisses it - "We are not replenishing our ranks. It is not a crisis, but I consider it to be important." By December 2011, Sue Gardner seems to reflect a more realistic view in the WMF, calling it the "holy-shit slide"[^Gardner] (see below for the graph). It is almost unintentionally hilarious to look at the proposed solutions; one amounts to [restoring early Wikipedia culture & practices](!Wikipedia "Wikipedia:Wiki Guides/New pages") in private sandboxes, protected from the regulars & their guidelines. Band-aids like [Wikilove]( or [article rating buttons]( are not getting at the core of the problem; a community does not live on high-quality rating tools ([Everything2](!Wikipedia)) or die on poor ones (YouTube). The Foundation/developers sometimes do the right thing, like [striking down](!Wikipedia "Wikipedia:Wikipedia Signpost/2011-09-26/News and notes#Foundation overrules community consensus on autoconfirmation trial") an English Wikipedia 'consensus' to restrict article creation even further, but will it be enough? To quote Carl Bialik again:
> Adding more editors "is one of our top priorities for the year," says Howie Fung, senior product manager for the Wikimedia Foundation, which aims to increase the number of editors across all languages of Wikipedia to 95,000 from 81,450 by June of next year.
[^Gardner]: From the 19 December 2011 ["The Gardner interview"](!Wikipedia "Wikipedia:Wikipedia Signpost/2012-01-02/Interview"):
> Much of the interview concerned the issues she raised in a [landmark address]( in November to the board of Wikimedia UK, in which she said the slide showing a graph of declining editor retention (below) is what the Foundation calls “the holy-shit slide". This is a huge, "really really bad" problem, she told Wikimedia UK, and is worst on the English and German Wikipedias.
> A prominent issue on the English Wikipedia is whether attempts to achieve high quality in articles – and perceptions that this is entangled with unfriendly treatment of newbies by the community – are associated with low rates of attracting and retaining new editors. Although Gardner believes that high quality and attracting new editors are both critical goals, her view is that quality has not been the problem, although she didn't define exactly what article quality is. What we didn't know in 2007, she said, was that “quality was doing fine, whereas participation was in serious trouble. The English Wikipedia was at the tail end of a significant drop in the retention of new editors: people were giving up the editing process more quickly than ever before.
>> 'Participation matters because it drives quality. People come and go naturally, and that means we need to continually bring in and successfully orient new people. If we don't, the community will shrink over time and quality will suffer. That's why participation is our top priority right now.'
> ...Deletions and reversions might be distasteful to new editors, but how can we, for instance, maintain strict standards about biographies of living people (BLP) without reverting problematic edits and deleting inappropriate articles? Gardner rejected the premise:
>> 'I don't believe that quality and openness are inherently opposed to each other. Openness is what enables and motivates people to show up in the first place. It also means we'll get some bad faith contributors and some who don't have the basic competence to contribute well. But that's a reasonable price to pay for the overall effectiveness of an open system, and it doesn't invalidate the basic premise of Wikipedia: that openness will lead to quality.'
> ...While staking the Foundation's claim to the more technical side of the equation, Gardner doesn't shrink from providing advice on how we can fix the cultural problem.
>> 'If you look at new editors' talk pages, they can be pretty depressing – they're often an uninterrupted stream of warnings and criticisms. Experienced editors put those warnings there because they want to make Wikipedia better: their intent is good. But the overall effect, we know, is that the new editors get discouraged. They feel like they're making mistakes, that they're getting in trouble, people don't want their help. And so they leave, and who can blame them? We can mitigate some of that by toning down the intimidation factor of the warnings: making them simpler and friendlier. We can also help by adding some praise and thanks into the mix. When the Foundation surveys current editors, they tell us one of the things they enjoy most about editing Wikipedia is when someone they respect tells them they're doing a good job. Praise and thanks are powerful.
>> ...[Around the time of the [Seigenthaler](!Wikipedia "Wikipedia biography controversy") and [Essjay](!Wikipedia "Essjay controversy") controversies] Jimmy went to Wikimedia and said "quality … we need to do better", [and through the distortions of the ripple-effect in the projects] there was this moral panic created around quality … what Jimmy said gave a whole lot of people the license to be jerks. ... Folks are playing Wikipedia like it's a video game and their job is to kill vandals ... every now and again a nun or a tourist wanders in front of the AK47 and gets murdered ...'
> Many people have complained that Wikipedia patrollers and administrators have become insular and taken on a bunker mentality, driving new contributors away. Do you agree, and if so, how can this attitude be combated without alienating the current core contributors?
>> 'I wouldn't characterize it as bunker mentality at all. It's just a system that's currently optimized for combating bad edits, while being insufficiently concerned with the well-being of new editors who are, in good faith, trying to help the projects. That's understandable, because it's a lot easier to optimize for one thing (no bad edit should survive for very long) than for many things (good edits should be preserved and built upon, new editors should be welcomed and coached, etc.). So I don't think it's an attitudinal problem, but more an issue of focusing energy now on re-balancing to ensure our processes for patrolling edits, deleting content, etc. are also designed to be encouraging and supportive of new people.'
> How can a culture that has a heavy status quo bias be changed? How can the community be persuaded to become less risk-averse?
>> 'My hope is that the community will become less risk-averse as the Foundation makes successful, useful interventions. I believe the Vector usability improvements are generally seen as successful, although they of course haven't gone far enough yet. Wikilove is a small feature, but it's been adopted by 13 Wikipedia language-versions, plus Commons. The article feedback tool is on the English Wikipedia and is currently being used in seven other projects. The new-editor feedback dashboard is live on the English and Dutch Wikipedias. New warning templates are being tested on the English and Portuguese Wikipedias. And the first opt-in user-facing prototype of the visual editor will be available within a few weeks. My hope is all this will create a virtuous circle: support for openness will begin to increase openness, which will begin to increase new editor retention, which will begin to relieve the workload of experienced editors, which will enable everyone to relax a little and allow for more experimentation and playfulness.
>> Regaining our sense of openness will be hard work: it flies in the face of some of our strongest and least healthy instincts as human beings. People find it difficult to assume good faith and to devolve power. We naturally put up walls and our brains fall into us-versus-them patterns. That's normal. But we need to resist it. The Wikimedia projects are a triumph of human achievement, and they're built on a belief that human beings are generally well-intentioned and want to help. We need to remember that and to behave consistently with it.'"
I am skeptical that Gardner's initiatives will change the curves (although they are not bad ideas); my general belief is that deleting pages, and the omnipresent threat of deletion, are far more harmful than complex markup. I should note that Gardner has [read and praised]( this essay, but also that much of this essay is based on my feelings and [may not generalize](
I [suspected]( that he and the WMF would fail. Remember, most measures are directed against *casual* users. Power users can navigate the endless processes, or call in powerful friends, or simply wait a few years^[The successful recreation of [Mzoli's](!Wikipedia) article and the endless deletion debates about Daniel Brandt (crowned in success for the deletionists) again come to mind.] The most powerful predictor of whether an editor will stop editing is... how much they are editing.[^ML-Contest] [User:Resident Mario](!Wikipedia) (joined 2008) points in his December 2011 essay ["Openness versus quality: why we're doing it wrong, and how to fix it"](!Wikipedia "Wikipedia:Wikipedia Signpost/2011-12-26/Opinion essay")[^Mario] to a dramatic graph of editor counts^[Gardner's [December UK address]( contained other graphs worth looking at.]:
![Active Wikipedians: Actual versus Strategy]( "Active Wikipedians: Actual versus Strategy; continued declines charted against original optimistic WMF projections")
[^ML-Contest]: In June 2011, [Kaggle](!Wikipedia) and the WMF [announced]( a ["Wikipedia's Participation Challenge"]( to develop a better statistical model for predicting editor retention; while the training data [was biased](, [the results]( are not too surprising: the single best predictor is the frequency of any edits prior to the cutoff. See 2nd place, [Ernest Shackleton]( or contestant [Keith T. Herring](
> "A randomly selected Wikipedia editor that has been active in the past year has approximately an 85 percent probability of being inactive (no new edits) in the next 5 months. The most informative features (w/r/t the features I considered) captured both the edit timing and volume of an editor. More specifically the exponentially weighted edit volume of a user (edit weight decreases exponentially with increased time between the edit and the end of the observation period) with a half-life of 80 days provided the most predictive capability among the 206 features included in the model.
> Other attributes of the edit history, such as uniqueness of articles, article creation, comment behavior, etc. provided some additional useful information, although roughly an order of magnitude or less than the edit timing and volume when measured as global impact across the full non-conditioned editor universe."
[^Mario]: I disagree with parts of Mario's essay; for example, his first example is wrong as there are countless articles to write from [the sister wikis](!Wikipedia "User:Piotrus/Wikipedia interwiki and specialized knowledge test"), and many specialist sources like _[The Encyclopedia of Science Fiction](!Wikipedia) have hundreds or thousands of entries that Wikipedia does not (I counted a dozen or so just linking to the articles [written by Jonathan Clements]( - eg. many of the biography redlinks in "[Seiun Award](!Wikipedia)" or "[Nihon SF Taisho Award](!Wikipedia)".) And every day, sites like the Anime News Network or _New York Times_ post dozens of reviews or other references that can be easily & profitably worked into articles - [but aren't](#the-editing-community-is-dead-who-killed-it).
[One comment](!Wikipedia "Wikipedia talk:Wikipedia Signpost/2011-12-26/Opinion essay") makes the good point that the theory of completeness would not predict any flatlining in the *smaller and less complete* wikis, yet we seem to observe a general flatlining.
However, his reason 2 is similar to my own theory about the Seigenthaler affair and the BLP reaction, and his reason 3 is my previous point about process & the fallacy of the invisible/broken window fallacy.
And it's casual users who matter. We lost the credentialed [experts](!Wikipedia "Wikipedia:Expert retention") years ago, if we ever had them. [Surveys]( asking why are almost otiose; they will do so if they are exceptional or if they are [managing PR]( around a discovery. But Wikipedia is not [Long Content](About#long-content); why *would* they contribute if they can get the traffic they desire just by [inserting]( [links](^[Which as links to credentialed sources will be uncontroversial and require little defense, vastly improving the ROI of editing Wikipedia. Wikipedia gets a great deal of traffic, and even highly obscure articles exert surprising influence; one can look at the traffic rates on specific pages with <>.]? Why would they build their intellectual houses on sand?[^Knuth] They get the best of both worlds - gaining traffic and avoiding the toxic deletionists.
[^Knuth]: [To quote]( the great computer scientist [Donald Knuth](!Wikipedia):
> "I think that Wikipedia's enormously successful, but it's so brittle, you know, if I was, if I spent a lot of time writing an article for the Wikipedia, and I wanted to make sure nobody screwed it up, I would have to check that article every day to make sure that it was still okay, and you know, after I've done that I want to move on and go on to other, other things in my life. With [TeX](!Wikipedia), I wanted stability especially urgently because people are depending on it to be a fixed point that they can build on, so in that respect, I differ from the [GNU public license](!Wikipedia "GNU General Public License#Versions")."
The GPL contains clauses that users of GPLed code may use the terms of later versions of the GPL, which may fix any legal vulnerabilities or exploits discovered. This is a common practice among copyleft licenses and in fact, the WMF itself cross-licensed the entire set of Wikipedias and other projects from the [GFDL](!Wikipedia) to [Creative Commons](!Wikipedia) as well based on a one-time provision GNU added at WMF's request.
And we can see this quite directly: when the general population of editors get solicited to contribute to AfD, their !votes are different from the AfD regulars, and in particular, when keep !voters spread the word about an AfD, their recruits are much more likely to !vote keep a well, while would-be deleters do their cause no favor with publicity[^bot-solicitation]. Can there be any more convincing proof that deletionism and its manifestations are a cancer on the Wikipedia corpus?
[^bot-solicitation]: ["The Effects of Group Composition on Decision Quality in a Social Production Community"](, Lam et al 2010, pg 7:
> "We also found that there have been two bots (computer programs that edit Wikipedia) — `BJBot` and `Jayden54Bot` — that automatically notified article editors about AfD discussions and recruited them to participate per the established policy. These bots performed AfD notifications for several months, and offer us an opportunity to study the effect of recruitment that is purely policy driven. We use a process like one described above to detect successful instances of bot-initiated recruitment: if a recruitment bot edited a user's talk page, and that user !voted in an AfD within two days, then we consider that user to have been recruited by the bot.
> Using the above processes, we identified 8,464 instances of successful recruiting. Table 2 shows a summary of who did the recruiting, and how their recruits !voted. We see large differences in !voting behavior, which suggests that there is bias in who people choose to recruit. (From these data we cannot tell whether the bias is an intentional effort to influence consensus, or the result of social network homophily [14].) Participants recruited by keep !voters were about four times less likely to support deletion as those recruited by delete !voters. The participants that bots recruited also appear unlikely to support deletion, which reflects the policy bias we observed earlier.
> To see what effect participant recruitment has on decision quality, we introduce four binary variables: `BotRecruit`, `NomRecruit`, `DeleteRecruit`, and `KeepRecruit`. These variables indicate whether a bot, the AfD nominator, a delete !voter, or a keep !voter successfully recruited somebody to the group, respectively.
> Looking back to table 1, we find that regardless of the decision, none of the first three variables has a statistically significant effect. On the other hand, when a keep !voter recruited someone to the discussion, we see a significant effect: delete decisions are more likely to be reversed. We offer two possible explanations: the first is that recruitment by keep !voters, biased as it may appear, is a sign of positive community interest, and suggests that the article should be kept. If the community decides otherwise and deletes the article, then decision quality suffers. An alternative explanation is that keep !voter recruitment is a sign of activism among those who prefer to keep the article. These proponents may be especially persistent in maintaining the article's existence in Wikipedia, even if it requires working to reverse a delete decision."
### The editing community is dead; who killed it?
Having discussed the broad trend of deletionism and problems with editors, let's look at one specific deletionist practice which has, as far as I know, never been examined before, despite being a classic deletionist practice and, like most deletionist practices, one that by the numbers turns out to badly misserve both editors and readers: the practice of *moving links from External Links to the Talk page*.
The reason for my interest in this minor deletionist practice is that I no longer edit as much as I used to, and so frequently when I find an excellent citation (article, review, interview etc.) I will often just copy it into the External Links section or (if I am feeling especially energetic) I will excerpt the important bits onto the article's Talk page. I realized that this constitutes what one might call a "[natural experiment](!Wikipedia)": I could go back and see how often the excerpts were copied by another editor into the article. This is better than just looking at "how often anime editors edit" or "how often anime articles are edited" because it is less related to outside events - perhaps anime news was simply boring over that period or perhaps some new bots or scripts were rolled out. Whereas if there are no anime editors who will edit *even when presented with gift-wrapped RSs* (links & excerpts specifically called out for their attention, and trivially copy-pasted into the article), then that's pretty convincing evidence that there is no longer a 'there' there - that the editors are no longer active.
On at least two articles ([Talk:Gurren Lagann#Interviews](!Wikipedia) & [Talk:Royal Space Force: The Wings of Honnêamise#Sources](!Wikipedia)), I have been strenuously opposed by editors who object to having more than a handful of links in the designated External Links section; they acknowledged the links were (mostly) all undoubted RSs and relevant to the article - but they refused to incorporate the links into the article. This is bad from every angle, yet few other editors were interested in helping me.
So I've begun going through my old mainspace Talk edits using `Special:Contributions`, starting all the way back in [April 2007]( (>4 years ago, more than enough time for editors to have made use of my gifts!), looking for cases where I've dumped such references. I compiled two lists, of [146 anime-related edits](#anime-edits), and [102 non-anime-related edits](#non-anime-edits).
Before going any further, it's worth asking - to avoid [hindsight bias](!Wikipedia) and post hoc rationalization - what you expect my results to be.
When asking yourself, remember that these edits, and a larger set of edit we'll soon examine, are *selected* edits; they are high-quality edits, ones where I thought the relevant article *must* cover it. They are not low-quality dumps of text or links by a passing anonymous editor or done out of idle amusement. What percentage would you expect to have been used after a week, enough time that most article-watchlisting editors will have seen the diff and had leisure to deal with task more complex than reverting vandalism?
50% doesn't seem like a bad starting point. How about after a year? Or two? Maybe 70% or 90%? After that, if it hasn't been dealt with, it's probably not ever going to be dealt with (even assuming the section hasn't been stuffed in an archive page). Hold onto your estimate.
Once the lists were compiled and weeded, I wrote [a Haskell program](#analysis-script) to do the analysis. The program loads the specified Talk page URLs and extracts all URLs from the Talk diff so it can check whether any of them were linked in the Article (which, incidentally, leads to false positives and an *over*estimation^[It was too hard to extract only the URL(s) being *added* by a diff, so the script simply extracts all URLs it can find in the diff part of the HTML; so if an editor made 4 edits adding URLs A, B, C, and D, and only A were added to the article, then the script would 4 times extract A-D, spot A in the article, and declare victory. This may account for KrebMarkt's increased success rate compared to my edits, because she is accustomed to piling up her suggested links in one tidy section.]).
The results for my edits when run on the two lists:
- anime: 146 edits, 11 were used, or <8%
- non-anime: 102 edits, 3 used, or <3%
For comparison, we can look at an editor who has devoted much of her time to finding references for anime articles - but made the colossal mistake of believing the EL partisans when they said external links should either be incorporated into article text or listed on the talk page. [User:KrebMarkt](!Wikipedia) has made perhaps thousands of such edits from impeccable RSs; it is possible that my own contributions are skewed downwards, say, by a congenital inability to select good references. Hence, looking at her reference-edits will provide a cross-check.
I compiled her most recent 1000 edits to the article talk space with a quick download: `elinks -dump '' '' | grep '&diff='`. Then I manually removed edits which were minor or did not seem to be her usual reference-edits, resulting in the following list of [958 edits](#krebmarkt) from December 2010 to December 2011. (KrebMarkt almost exclusively adds anime-related references, so I did not prepare a non-anime list.) The results:
- Of the 958 edits adding references, 36 were used in the article, or <4%
- Combining my anime & non-anime with KrebMarkt's edits, we have 1206 edits adding references, of which less than 50 were used in the article, or <4.15%
Besides it being surprising that KrebMarkt (not a particularly committed inclusionist, if she be an inclusionist at all) had a success rate half mine, <4.15% is shockingly low.
1156 ignored edits represents a staggering waste of editor-time^[I added a few links to Talk pages to time how long it took for a KrebMarkt-style edit: to go from the ANN page to a saved and reloaded page which I had checked by eye that the edit was correct was upwards of 30 seconds. >30 seconds times 958 edit is >479 minutes or >8 hours; my excerpting edits take at least 5 minutes to do, so those 248 edits represent >21 hours of work.]. This cannot be explained as our faults: we are both experienced editors (I began editing in 2004, and KrebMarkt in 2008), who know what good RSs are. And all of the edits contain good RSs. (The reader is invited to check edits and see for himself whether they are solid and valuable RSs, like reviews by the [Anime News Network](!Wikipedia).) That perhaps $\frac{1}{10}$ of our suggested references are included is due solely to the apathy or nonexistence of other editors. (If such a rate is a 'success', may the Almighty preserve us from a failure!)
Since that will not soon change for the better, this leads to one conclusion: *the idea that references hidden on Talk pages will one day be used is false*.
#### Tallying the Damage
##### _Ignoti, sed non occulti_
One might try to defend this wasteful practice by claiming that some editors and readers will go to the Talk page and there might notice and visit the deleted links. This could only ameliorate the problem slightly, but it's worth investigating just how rarely Talk pages are visited so we can explode this particular instance of the 'fallacy of the invisible'. How many of our readers actually look at the talk page as well? (Do a quick estimate, as before, so you can know if you were right or wrong, and by how much.) I know some writers writing articles on Wikipedia have mentioned or rhapsodized at length on the interest of the talk pages for articles, but they are rare birds and statistically irrelevant.
It might be enough simply to know how much traffic to talk pages there is period. I doubt editors make up much of Wikipedia's traffic, with the shriveling of the editing population, which never kept pace with the growth into a top 10/20 website, so that would give a good upper bound. It would seem to be very small; there's not a single Talk page in the top 1000 on <>. We can look at individual articles; [Talk:Anime](!Wikipedia) has 273 hits over [one month]( while the article [Anime](!Wikipedia) has [128,657 hits]( (a factor of 471); or [Talk:Barack Obama](!Wikipedia) with 1800 over [that month]( compared to [Barack Obama](!Wikipedia) with its [504,827 hits]( (a factor of 280).
The raw stats used by `` are [available]( for download, so we can look at *all* page hits, sum all article and all Talk hits and see what the ratio is for the entire English Wikipedia is on one day. (each file seems to be an hour of the day so I downloaded 24 and `gunzip`ped them all.) We do some quick shell scripting. To find the aggregate hits for just talk pages:
grep -e '^en Talk:' -e '^en talk:' pagecounts-* | cut -d ' ' -f 3 | paste -sd +|bc
To find aggregate hits for non-talk pages:
grep -e '^en ' pagecounts-* | grep -v -e '^en Talk:' -e '^en talk:' | cut -d ' ' -f 3 | paste -sd + | bc
The numbers look sane - 58,2771 for all talk page hits versus 2,0268,0742 for all non-talk page hits. A factor of 347 is pretty much around where I was expecting based on those previous 2 pages. The traffic data developer, Domas, says the statistics exclude API hits but includes logged-in editor hits, so we can safely say that anonymous users made far *fewer* than 58k page views that day and hence the true ratios are worse than our previous ratios of 471/280/347. To put the relative numbers into proper perspective, we can convert into percentages:
- If we take the absolutely most favorable ratio, Obama's at 280, and then further assume it was looked at by 0 logged-in users (yeah right), then that implies something posted on its talk page will be seen by <0.35% of interested readers ($(\frac{504827}{1800} \times 1.0) \times 100$).
- If we use the aggregate statistic and say, generously, that registered users make up only 90% of the page views, then something on the talk page will be seen by <*0.028*% of interested readers ($(\frac{202680742}{582771} \times 0.1) \times 100$).
Page views don't tell us the most interesting thing, how many people *would have* clicked on the link if it had been on the article and not the Talk page. It's impossible to answer this question in general, unfortunately, since Wikipedia does not track clicks. However, I have approximated the ratio for at least one article: the [n-back](!Wikipedia) article links to my [DNB FAQ](). There are a few dozen visitors each day from Wikipedia, Google Analytics tells me. What will happen if the link is removed to the Talk page? The article and general interest in n-back haven't changed - those variables are still the same. The same sort of people will be visiting the article and (not) visiting the Talk page. The visitor count will dramatically fall, probably to less than 1 a day. The link was in the article for perhaps half a year; I [shifted]( [it]( to the Talk page with a fake message praising the contents, to mimic how an editor might genuinely post the link on the Talk page (asking the forbearance & cooperation of my fellow editors in hidden comments). It is ought to be trivial and pointless - everyone should acknowledge, but it's still worth precommitting: [I]( [predict]( that Talk click-throughs will average <5% of Article click-throughs, and the difference between the 2 datasets will be statistically significant at p<0.05.
TODO: analysis - totals before/after, average per day before/after, export both data segments to CSV and run t-test on difference (how high will the _p_-value be? hopefully <0.01 since I'm sinking in 100 days)
##### The Forgotten Reader
More instructive is estimating how many readers have been deprived of the chance to use the references for just the subset of 1206 edits we have already looked at above. We can reuse `` with [a little more programming](; we will ask it how many hits/page-views, in total, there were in November 2011 of the 472 unique articles covered by those 1206 edits.
The total: *8,480,394*.
Extrapolating backwards to 2007/2008 is left as an exercise for the reader.
When we consider how false the idea that this practice serves the editor, and when we consider how many readers are ill-served, they suggest that the common practice of 'moving reference/link to the Talk page' be named for what it is: a subtle form of deletion.
It would be a service to our readers to end this practice entirely: if a link is good enough to be hidden on a Talk page (supposedly in the interests of incorporating it in the future, which we have seen is a empty promissory note), then it is good enough to put at the end of External Links or a Further Reading section, and the literally millions of affected readers will not be deprived of the chance to make use of them.
I fully expect to see this practice for years to come.
## No club that would have me
> 'Elaborate euphemisms may conceal your intent to kill, but behind any use of power over another the ultimate assumption remains: "I feed on your energy."'[^dune]
[^dune]: "Addenda to Orders in Council - The Emperor Paul Muad'dib", Frank Herbert's _[Dune Messiah](!Wikipedia)_
This result will come as no surprise to longtime [inclusionists]( The deletion process deletes [most articles](!Wikipedia "Wikipedia:AFD 100 days") which enter it, and has long been complained about by outsiders. Entire communities (such as the [web comics](!Wikipedia "Webcomic")^[See [Slashdot](!Wikipedia)'s ["Call For Halt To Wikipedia Webcomic Deletions"]( for an overview.] or [MUD](!Wikipedia "MUD") online communities^[["MUD history dissolving into the waters of time"](]) have been alienated by purges of articles - purges which not infrequently result in abuse of process, much [newbie biting](!Wikipedia "Wikipedia:Please_do_not_bite_the_newcomers"), and comical spectacles like AfD regulars (usually deletionists) insisting a given article is absolutely non-notable and experts in the relevant field demurring; a particularly good AfD may see statements of experts dismissed on speciously procedural grounds such as having been made in the expert's blog (and so failing [WP:RS](!Wikipedia), or perhaps simply being dismissed as [WP:OR](!Wikipedia)) and not a traditional medium (despite the accelerating abandonment of 'traditional' RSs by experts in many fields[^anime-publishing]). The trend has been clear. [Andrew Lih](!Wikipedia), who has been editing Wikipedia even longer than myself (since 2003) and who wrote [a book](!Wikipedia "The Wikipedia Revolution: How a bunch of nobodies created the world's greatest encyclopedia ") on Wikipedia, writes in ["Unwanted: New articles in Wikipedia"](
> 'It's incredible to me that the community in Wikipedia has come to this, that articles so obviously "keep" just a year ago, are being challenged and locked out. When I was active back on the mailing lists in 2004, I was a well known deletionist. "Wiki isn't paper, but it isn't an attic," I would say. Selectivity matters for a quality encyclopedia. But it's a whole different mood in 2007. Today, I'd be labeled a wild eyed inclusionist. I suspect most veteran Wikipedians would be labeled a bleeding heart inclusionist too. How did we raise a new generation of folks who want to wipe out so much, who would shoot first, and not ask questions whatsoever? [If Lih can write this in 2007, you can imagine how people who identified as inclusionists in 2004, such as myself or [The Cunctator](!Wikipedia "User:The Cunctator"), look to Wikipedians who recently joined.]
> It's as if there is a [Soup Nazi](!Wikipedia) culture now in Wikipedia. There are throngs of deletion happy users, like grumpy old gatekeepers, tossing out customers and articles if they don't comply to some new prickly hard-nosed standard. It used to be if an article was short, someone would add to it. If there was spam, someone would remove it. If facts were questionable, someone would research it. The beauty of Wikipedia was the human factor — reasonable people interacting and collaborating, building off each other's work. It was important to start stuff, even if it wasn't complete. Assume good faith, neutral point of view and if it's not right, `{{sofixit}}`. Things would grow.'
[^anime-publishing]: Anime and manga are particularly bad. The American and Japanese anime bubbles of the 2000s popped, and with them went a flood of magazines and books - the economic reality has set in that they are simply not sustainable in a modern environment, which of course is very useful to deletionists who want to apply rigid universal norms to articles sans any context. This leads to odd situations like experts self-publishing; from Brian Ruh's ANN column ["The Ghost with the Most"](
> "This time, though, instead of a fictional book about the supernatural I'm going to be examining a nonfiction book about Japanese ghosts – [Patrick Drazen's]( _A Gathering of Spirits: Japan's Ghost Story Tradition: From Folklore and Kabuki to Anime and Manga_, which was recently self-published through the [iUniverse](!Wikipedia) service. This is Drazen's second book; the first one, _Anime Explosion! The What? Why? & Wow! of Japanese Animation_, came out in 2002 from [Stone Bridge Press](!Wikipedia) and was an introduction to many of the genres and themes that can be found in anime.
> I think the switch from a commercial press to self-publication may indicate the direction English-language anime and manga scholarship may be heading in. A few years ago, when Japanese popular culture seemed like the Next Big Thing, there were more publishers that seemed like they were willing to take a chance on books about anime and manga. Unfortunately, as I know firsthand (and as I've heard from other authors, confirming that it's not just me) these books didn't sell nearly as well as anyone was hoping, which in turn meant that these publishers didn't want to take risks with additional books along these lines. After all, all publishers need to make money in one way or another to stay afloat. In the last few years, the majority of books on anime and manga have been published by university presses, perhaps most notably the University of Minnesota Press. But I already gushed about them in my last column, so I'll spare you from any additional public displays of affection.
> However, this puts books like Drazen's in an odd predicament. It's not really an academic book, since it lacks the references and theories something like that would entail, which means it's not a good candidate for a university press. However, since few popular presses have seen their books on anime and manga reflect positively on their bottom lines, there aren't many other options these days other than self-publishing. Of course, these days publishing a book on your own doesn't have nearly the same connotations it did decades ago, when vanity presses were the domain of those with more money (and ego) than sense. These days you can self-publish a quality product, get it up on Amazon for all to see, and (if you're savvy about these things) perhaps even make a tidy profit."
I was particularly depressed to read [in the comments]( things from administrators whose names I recognize due to their long tenure on Wikipedia, like [Llywrch](!Wikipedia "User:Llywrch") (joined 2002):
> "I'm sorry that you encountered that, Andrew — but not surprised. I had [my own encounter]( with the new generation of "quote policy, not reasoning" deletionists; I feel as if I encountered (to quote from the song) "the forces of evil from a bozo nightmare." No one — including me — looked good after that exchange. (I keep thinking that I should have said something different, but the surrealism of the situation multiplied with the square of my frustration kept me from my best.)"
Or [Stbalbach](
> "I'm a long time editor, since 2003, ranked in the top 300 by number of edits (most in article space). On May 11th 2007 I mostly gave up on Wikipedia – there is something wrong with the community, in particular people deleting content. I'd never seen anything like it prior to late 2006 and 2007. Further, the use of "nag tags" at the top of articles is out of hand. It's easier to nag and delete than it is to research and fix. Too many know-nothings who want to "help" have found a powerful niche by nagging and deleting without engaging in dialog and simply citing 3 letter rules. If a user is unwilling or incapable of working to improve an article they should not be placing nag tags or deleting content."
Also interesting is [Ta bu shi da yu]('s comment, inasmuch as Ta bu invented the infamous `{{fact}}`:
> "I have also seen this happening. It's incredible that those who are so incredibly stupid can get away with misusing the speedy deletion tag! As for DRV… don't make me laugh. It seems to be slanted to keep articles deleted. I can't agree more with your sentiments that if you know all the codes to WP:AFD, then you are a menace to Wikipedia."
Why is this culture changing? In part because article writing seems to get no more respect. A [review article](/docs/2011-oneill.pdf) summarizes the findings of Burke and Kraut 2008^[[‘Taking Up the Mop: Identifying Future Wikipedia Administrators'](, Moira Burke and Robert Kraut, in _Proceedings of the Conference on Human Factors in Computing Systems_, Florence, Italy, 5-10. April 2008, pp. 3441-6]:
> " is proving increasingly hard to become a Wikipedia administrator: 2,700 candidates were nominated between 2001 and 2008, with a success rate of 53 percent. The rate has dropped from 75.5 percent until 2005 to 42 percent in 2006 and 2007. Article contribution was not a strong predictor of success. The most successful candidates were those who edited the Wikipedia policy or project space; such an edit is worth ten article edits."
What sort of editor, with a universe of fascinating topics to write upon, would choose to spend most of his time on the policy namespace? What sort of editor would choose to stop writing articles?[^statements] Administrators with minimal experience in creating content - and much experience in destroying it and rewriting the rules to permit the destruction of even more. Is this not almost the opposite of what one wants? And imagine how the authors must feel! An article is not a trivial undertaking; sometime sit down, select a random subject, and try to write a well-organized, fluent, comprehensive, and accurate encyclopedia article on it. It's not as easy as it looks, and it's even harder to write a well-referenced and correctly formatted one. To have an article deleted is bad enough; I can't imagine any neophyte editors wanting to have anything to do with Wikipedia if an article of theirs got railroaded through AfD. It is easier to destroy than to create, and destruction is infectious. (In [a study]( of 3.3 years of the online SF game [_Pardus_](!Wikipedia "Pardus (browser game)"), players were found to 'pay it forward' when the subject of negative actions; the community was only saved from an epidemic of attacks by the high mortality & quitting rate of negative editors - I mean, negative players[^Pardus].)
[^Pardus]: From pg 5 of the [PDF]( (or see popular coverage in eg. [_Technology Review_](
> "Transition rates of actions of individuals show that positive actions strongly induces positive *reactions*. Negative behavior on the other hand has a high tendency of being repeated instead of being reciprocated, showing the ‘propulsive' nature of negative actions. However, if we consider only reactions to negative actions, we find that negative reactions are highly overrepresented. The probability of acting out negative actions is about 10 times higher if a person received a negative action at the previous timestep than if she received a positive action.
> ...The analysis of binary timeseries of players (good-bad) shows that the behavior of almost all players is ‘good' almost all the time. Negative actions are balanced to a large extent by good ones. Players with a high fraction of negative actions tend to have a significantly shorter life. This may be due to two reasons: First because they are hunted down by others and give up playing, second because they are unable to maintain a social life and quit the game because of loneliness or frustration. We interpret these findings as empirical evidence for self organization towards reciprocal, good conduct within a human society. Note that the game allows bad behavior in the same way as good behavior but the extent of punishment of bad behavior is freely decided by the players."
It's worth noting the distinction between 'reciprocation' and 'repeated'; otherwise this phenomenon might have an obvious explanation as a statistical artifact resulting from an ordinary game activity like 1-on-1 fights or duels.
[^statements]: From "Cultural Transformations in Wikipedia or ‘From Emancipation to Product Ideology': An Interview with Christian Stegbauer", collected in [_A Wikipedia Reader_](
> "Our 2006 research [Christian Stegbauer, ‘Wikipedia. Das Rätsel der Kooperation' (‘Wikipedia: the mystery behind the cooperation'), Wiesbaden: VS, 2009, p. 279 et seq.] compared content on user pages from their original starting date to the present. 13 We noticed a transformation from emancipation to product ideology among those who had reached leadership status, but not for ones less integrated. Typical statements from a user site's first days would be: ‘Wikipedia is a great idea'; ‘[a] never-ending encyclopedia created by many different authors'; ‘everyone should be able to exchange their knowledge for free'; ‘Wikipedia is like fulfilling a dream – a book in which everyone can write what they want'; ‘the Internet shouldn't be regarded as a goldmine'; ‘Making information available free of charge is an important task'; ‘the project's concept is fantastic'; ‘the idea behind Wikipedia is well worth supporting'.
> Six out of seven users who changed their ideological statements were core users, and five of these were administrators. Half of them deleted their opinion on emancipation ideology in the same instance they became administrators. In five out of nine cases, they expressed the product ideology, including remarks about ‘unreasonable' people damaging the project, about endless discussions that should not take place when energy should be invested in the articles instead, and about ‘difficult' people who are not welcome at Wikipedia. We also found phrasing such as ‘certain level of expertise is necessary for writing the articles' or that liberal processing is the reason behind low quality contributions."
Deleting articles and piling on policy after guideline after policy are both *directly* opposed to why Wikipedians contribute! When [surveyed in 2011](!Wikipedia "Wikipedia:Wikipedia Signpost/2011-06-13/News and notes#First results of editor survey: Wikipedians 90% male, 71% altruist"):
> 'The two most frequently selected reasons for continuing to edit Wikipedia were "I like the idea of volunteering to share knowledge" (71%) and "I believe that information should be freely available to everyone" (69%), followed by "I like to contribute to subject matters in which I have expertise" (63%) and "It's fun" (60%).'
And ironically, the more effort an editor pours into a topic and the longer & more detailed the article becomes, the more blind hatred it inspires in deletionists. If you look at AfDs for small articles or stubs, the deletionists seem positively lucid & rational; but make the article 50kB long, and watch the rhetoric fly. I call this the *[fancruft](!Wikipedia) effect*: deletionists are mentally allergic to information they do not care about or like.
If a deletionist sees an article on "Lightsaber combat"[^combat] and it's just a page long, then he has little problem with it. It may strike him as too big, but reasonable. But if the article dares to be comprehensive, if it is clearly the product of many hours' labor on the part of multiple editors, if there are touches like references and quotes - then [something is wrong on the Internet](, the very universe is out of joint that this article has been so well-developed when so many more deserving topics languish, it is a cosmic injustice. A dirty beggar is parading around acting like an emperor. The article *does not know its place*. It needs to be smacked down and hard. And who better than the deletionist?
[^combat]: I bring up the 'Lightsaber combat' article because I did substantial work referencing it before its wiki-deletion, but because it was redirected the original page history still survives. It is worthwhile comparing the [original page]( with its [replacement section](!Wikipedia "Lightsaber combat#Choreography") in the 'Lightsaber' article.
I am chuffed to note that the merge has resulted in *inferior* references! eg. the [Nick Gillard](!Wikipedia) quote in paragraph 2 is unsourced and has a `{{fact}}` template, but *was* referenced in the original. Further, that quote is trivially re-referenced (#3 hit in Google). My standards may be too high, but I can't help but think that it takes real incompetence to not only lose a reference, but be unable to re-find such an easily found quote.
What is the ultimate status-lowering action which one can do to an editor, short of actually banning or blocking them? Deleting their articles.
In a particular subject area, who is most likely to work on obscurer articles? The experts and high-value editors - they have the resources, they have the interest, they have the competency. Anyone
who grew up in America post-1980 can work on `[[Darth Vader]]`; many fewer can work on `[[Grand Admiral Thrawn]]`. Anyone can work on `[[Basho]]`; few can work on `[[Fujiwara no Teika]]`.
What has Wikipedia been most likely to delete in its shift deletionist over the years? Those obscurer articles.
The proof is in the pudding: all the high-value/status Star Wars editors have decamped for somewhere they are valued; all the high-value/status Star Trek editors, the Lost editors... the list goes on. They left for a community that respected them and their work more; these specific examples are striking because the editors had to *make* a community, but one should not suppose such departures are limited to fiction-related articles. There may be [evaporative cooling of the community]( but it's *not* towards the obsessive fans.
> "The greatest pleasure is to vanquish your enemies and chase them before you, to rob them of their wealth and see those dear to them bathed in tears, to ride their horses and clasp to your bosom their wives and daughters."^[Attributed to [Genghis Khan](!Wikipedia)]
Outsiders! I realize it might sound like a stretch that anyone enjoys the power of nominating articles, that being a deletionist could be a joyful role. You say you understand how administrators (with their ability to directly delete, to ban, to rollback etc.) could grow drunk on power, but how could AfD nominations lead to such a feeling?
But I know from personal experience that there is power exercised in nominating for deletion. Well do I know the dark arts of gaming the system: of the clever use of templates, of the process of deleting the article by carefully challenging and removing piece after piece, of invoking the appropriate guidelines and policies to demolish arguments and references.
I have seen the wails and groans in the edit summaries & comments of my opponents, and exulted in their defeat. It's very real, the temptation of exercising this power. It's easy to convince yourself that you are doing the right thing, and merely enforcing the policies/guidelines as the larger community set them down. (Were all my nominations just? No, but I have succeeded in fooling myself so well that I can no longer tell which ones truly did deserve deletion and which ones were deleted just because I disliked them or their authors.)
Who can say how many authors take it personally? The deletion process is inherently insulting: "Out of 2.5 million articles, yours stands out as sucking so badly that it is irredeemable and must be obliterated." And it is ultimately sad[^AFD] - life is short but must that be true of articles as well as men?
[^AFD]: The pathos has, at times, moved me to verse. To quote one of mine from [WP:HAIKU](!Wikipedia) (a homage to Basho's famous verse in _[The Narrow Road to Oku](!Wikipedia)_):
Summer AFD -
the sole remnant of many
editors' hard work.
It is not a coincidence that I put that haiku before the final haiku on the page - a haiku commenting on [editors who have abandoned or left the project](!Wikipedia "WP:MISSING"):
The summer grasses.
I edit my user page
One last time - really.
# What Is To Be Done?
Wikipedia was enabled by software. It enabled a community to form. This community did truly great work; it's often said Wikipedia is historic, but I think most people have lost sight of how historic Wikipedia is as it fades into the background of modern life; perhaps only scholars of the future have enough perspective on this leviathan, in the same way that Diderot's encyclopedia was - for all the controversy and banning - not given its full due at publication. (But how could it? Encyclopedias are more processes than finished works, and of no encyclopedia is this more true than Wikipedia.)
That community did great work, astonishing in breadth and depth, I said. But that community is also responsible for misusing the tools. If vandalism is easier to remove than create, then it will tend to disappear. But AfD is not vandalism. There are no technical fixes for deletionist editors. As long as most editors have weak views, are willing to stand by while 'nerdy' topics feel the ax, who think 'deletionists mostly get it correct', then the situation will not change.
Could deletion be a positive feedback cycle? Will the waves of deletion continue to encourage editors to leave, to not sign up, to let the deletionists continue their grisly work unopposed, until Wikipedia is a shell of what it was?
Like the cooling dwarf star left by a supernova - its lost brilliance traveling onwards to eternity.
# External links
- [Wikipedia:Wikipedia is failing](!Wikipedia)
# Talk page reference appendices
## Analysis script
The following Haskell program requires the Haskell base libraries and the [TagSoup]( library. The script is parallel. One can compile it like `ghc -threaded -rtsopts -O2 script.hs`; to run it one pipes in a list of newline-delimited Talk page edits, like `./script +RTS -N4 -RTS < urls.txt`^[One could also avoid compilation and run it much more slowly as `cat urls.txt | runhaskell script`.] which will then print out a summary like
Checked 1024 edits
112 were used
The following sections provide 3 lists of selected edits which one could input.
import Control.Concurrent (forkIO, newEmptyMVar, putMVar, takeMVar, MVar)
import Control.Monad (liftM, void)
import Data.List (elemIndices, intersect, isPrefixOf, nub, sort)
import Network.HTTP (getRequest, rspBody, simpleHTTP)
import Text.HTML.TagSoup (parseTags, Tag(TagOpen, TagText))
main :: IO ()
main = do args <- liftM lines getContents
results <- mapM parallel args
results' <- mapM takeMVar results
count <- liftM (length . filter id) $ sequence results'
putStrLn $ "Checked " ++ show (length args) ++ " edits"
putStrLn $ show count ++ " were used"
parallel :: String -> IO (MVar (IO Bool))
parallel s = do m <- newEmptyMVar
_ <- forkIO $ void (putMVar m (comparePages s))
return m
comparePages :: String -> IO Bool
comparePages url = do src <- liftM parseTags $ openURL url
let talkUrls = rmDupes $ concatMap urlsExtract $ text $ diff src
artcl <- liftM parseTags $ openURL (article url)
let articleUrls = uniq $ extractURLs artcl
return $ 0 /= length (talkUrls `intersect` articleUrls)
where uniq = nub . sort -- don't double-count in `intersect`!
-- throw out any URL appearing twice in a diff -
-- must be an old URL in both new and old pages!
-- only unique URLs could have been added.
rmDupes x = filter (\y -> length (elemIndices y x) <= 1) x
openURL :: String -> IO String
openURL u = do res <- simpleHTTP $ getRequest u
case res of
Left _ -> return ""
Right y -> return $ rspBody y
-- pull all text; hopefully, out of the diff-only part of the HTML page
text :: [Tag String] -> [String]
text src = [x | (TagText x) <- src]
urlsExtract :: String -> [String]
urlsExtract = filter (not . null) . map trimmer . words
{- crop a string down to the "http://" prefix, or return nothing
> trimmer "*" ~> ""
> trimmer "* []" ~> ""
> trimmer "* [ Foo]" ~> ""
> trimmer "# It will break!" ~> "" -}
trimmer :: String -> String
trimmer [] = []
trimmer y = fst $ break (\x -> x=='[' || x ==']' || x==' ') $ if "http://" `isPrefixOf` y
then y else trimmer (tail y)
-- ""
-- ~> ""
article :: String -> String
article url = "" ++ takeWhile (/= '&') (drop 47 url)
-- pull out all external links (but not local relative links)
extractURLs :: [Tag String] -> [String]
extractURLs arg = [x | TagOpen "a" atts <- arg,
(_,x) <- atts,
"http://" `isPrefixOf` x]
-- cut everything up to "<table class='diff diff-contentalign-left'>", then cut everything
-- after "<!-- diff cache key enwiki:diff:version:1.11a:oldid:150651404:newid:171214764 --> </table><hr class='diff-hr' />"
diff :: [Tag String] -> [Tag String]
diff = takeWhile ast' . dropWhile ast
where ast, ast' :: Tag String -> Bool
ast x = case x of
TagOpen "table" [("class","diff diff-contentalign-left")] -> False
_ -> True
ast' x = case x of
TagOpen "hr" [("class","diff-hr")] -> False
_ -> True
## `` script
This is compiled and run much the same way, minus the `-rtsopts -threaded` options (it is not parallel).
import Data.List (isInfixOf, nub, sort)
import Network.HTTP (getRequest, rspBody, simpleHTTP)
import Text.HTML.TagSoup (parseTags, Tag(TagText))
main :: IO ()
main = do stats <- fmap (nub . sort . map article . lines) getContents
srcs <- mapM openURL stats
print $ sum $ map total srcs
openURL :: String -> IO String
openURL u = do res <- simpleHTTP $ getRequest u
case res of
Left _ -> return ""
Right y -> return $ rspBody y
article :: String -> String
article url = "" ++ takeWhile (/= '&') (drop 47 url)
total :: String -> Int
total s = read (head $ text $ parseTags s) :: Int
-- target: TagText " has been viewed 215 times in 201111. "
text :: [Tag String] -> [String]
text src = map (takeWhile (/= ' ') . drop 17) [x | (TagText x) <- src, " has been viewed " `isInfixOf` x]
## Gwern
### Anime edits!_%28manga%29&diff=prev&oldid=340045477!&diff=prev&oldid=345223719!%29&diff=prev&oldid=346415893!&diff=prev&oldid=379162317
Recent edits, for use in future updates:
<!-- Note: the preceding edits were all transcriptions of an offline book; CANNOT use the script! -->
A different experiment: will links to solid RSs and even links to the [Encyclopedia of Science Fiction](!Wikipedia) be removed? So far [one was](!!&diff=next&oldid=480198773
### Non-anime edits,_Washington,_D.C.&diff=prev&oldid=372359688,_Inc.&diff=prev&oldid=345222854
Recent edits:
## KrebMarkt!&diff=prev&oldid=429277476!&diff=prev&oldid=426076777!!_Vanguard&diff=prev&oldid=426075822;Gate&diff=prev&oldid=426073857!&diff=prev&oldid=426071634!!&diff=prev&oldid=426066464!!&diff=prev&oldid=426065130!&diff=prev&oldid=424575219!!_Vanguard&diff=prev&oldid=424574711!&diff=prev&oldid=424574410;Gate&diff=prev&oldid=424573342!&diff=prev&oldid=424573127!&diff=prev&oldid=424571582!&diff=prev&oldid=424566170!!_Vanguard&diff=prev&oldid=423385691;Gate_(anime)&diff=prev&oldid=423384235!!&diff=prev&oldid=423383867!!_Japan&diff=prev&oldid=423381294!&diff=prev&oldid=423380220!&diff=prev&oldid=423375167!!_Vanguard&diff=prev&oldid=422177510!&diff=prev&oldid=422177455!&diff=prev&oldid=422175912!!&diff=prev&oldid=422162614!!_Vanguard&diff=prev&oldid=420996152!&diff=prev&oldid=420995611!&diff=prev&oldid=420995474!&diff=prev&oldid=420995359!!&diff=prev&oldid=420214758!&diff=prev&oldid=420214609!!_Vanguard&diff=prev&oldid=420214174!&diff=prev&oldid=420213150!_(manga)&diff=prev&oldid=420213028!&diff=prev&oldid=420032309!&diff=prev&oldid=418670078!!&diff=prev&oldid=418661272!&diff=prev&oldid=417675511!!_Vanguard&diff=prev&oldid=417674632!&diff=prev&oldid=417674324!&diff=prev&oldid=417672806!&diff=prev&oldid=417671610!&diff=prev&oldid=417477271!!_Vanguard&diff=prev&oldid=416254334!&diff=prev&oldid=416247280!&diff=prev&oldid=416246793,_Flowers&diff=prev&oldid=416245354!_Edo_Rocket&diff=prev&oldid=416222725!!&diff=prev&oldid=416221831!_Magister_Negi_Magi&diff=prev&oldid=416220380,_Zetsubou-Sensei&diff=prev&oldid=416219772!!&diff=prev&oldid=416218961!&diff=prev&oldid=416216260!!_Vanguard&diff=prev&oldid=415000867,_Flowers&diff=prev&oldid=415000113!&diff=prev&oldid=414994046!&diff=prev&oldid=414993419!!_Vanguard&diff=prev&oldid=413748451!&diff=prev&oldid=413744727!&diff=prev&oldid=413744426!!&diff=prev&oldid=412403029!!_Vanguard&diff=prev&oldid=412401694!&diff=prev&oldid=412399372,_Flowers&diff=prev&oldid=412398863!&diff=prev&oldid=412398648!!&diff=prev&oldid=412392406!!_Japan&diff=prev&oldid=411043049!&diff=prev&oldid=411041544,_Zetsubou-Sensei&diff=prev&oldid=411039833!&diff=prev&oldid=411038588!&diff=prev&oldid=411032749!&diff=prev&oldid=411031993!_Edo_Rocket&diff=prev&oldid=411031057!!&diff=prev&oldid=409627854!&diff=prev&oldid=409626444!&diff=prev&oldid=409626261!_Edo_Rocket&diff=prev&oldid=408258066!&diff=prev&oldid=408255424!&diff=prev&oldid=408254978!&diff=prev&oldid=408252121!!&diff=prev&oldid=408251165!&diff=prev&oldid=406896594,_Zetsubou-Sensei&diff=prev&oldid=406852813,_Flowers&diff=prev&oldid=405202413,_Flowers&diff=prev&oldid=405201519!_Magister_Negi_Magi&diff=prev&oldid=404869298!&diff=prev&oldid=404866875!&diff=prev&oldid=403146831