When scholarship meets Wikipedia

Maybe you have to be something of a fan of early jazz to know coronetist Bix Beiderbecke -- he's in the picture above, horn pointed at the camera. Beiderbecke was an influential performer in the late 1920s, but he died young -- in 1931, just 28 years old -- and he hasn't made it as far into the popular consciousness as, say, Dizzie Gillespie or Louis Armstrong. 

One who does know Beiderbecke is Brendan Wolfe. Wolfe not only named his litblog for the musician; he's at work on a book about him for Speck Press, which recently published Ted Goia's "The Birth (and Death) of the Cool." Wolfe, a longtime writer and editor, currently works as an encyclopedia editor.

So one day this summer, he decided he would update Beiderbecke's Wikipedia entry. "The entry I found was terrible," he writes. "Part of why I did this is because I love Wikipedia. I think it's a great resource, and I say that even though the online encyclopedia that employs me follows a different model."

All went well, as Wolfe poured his knowledge, scholarship and research into a new entry. As it stands now (after Wolfe's work, and possibly inclusive of others) Beiderbecke's Wikipedia entry lists more than 30 sources and includes 111 footnotes. This is a fairly high degree of research for the online encyclopedia; by contrast, Chick Webb, another major 1920s performer who died in the 1930s, has an entry with just 10 sources and no footnotes at all.

But there was a hitch. While Wolfe was working with others behind the scenes at Wikipedia, they suggested he submit it for "good article" status. He waited, very anxiously at first, to hear from a volunteer Wikipedia editor; months later, he did. And he was dismayed by what he heard. This was one note, which begins with the editor quoting Wolfe's writing:

"the second number was marred by alcohol consumed by the musicians, who included Tommy Dorsey on trombone and Beiderbecke's best friend, Don Murray, on clarinet". Does this implies that Tommy Dorsey and Don Murray contributed to the alcoholic mistaken in the 2d number? Tommy Dorsey and Don Murray contributed to Beiderbecke's life or work at last, except as members as the same band for a while? If these 2 musicians NOT contributed in at least of one of these aspects, they are irrevelant - just name dropping.

Wolfe writes, "I am not embarrassed to admit that this filled me with rage. As did this:" (again, the note begins with a quote from Wolfe's entry; the bullet points are from the editor):

"(The headmaster went so far as to inform Mr. and Mrs. Beiderbecke, about Bix, "that certain parents have objected strenuously to their sons' association with him.)"

• The parentheses are dishonest - either you take responsibility their content and for the space they take in section.
• If the item is retained, the headmaster's florid prose needs to be replaced with something consise.

No wonder he was angry. The editor's comments are riddled with grammatical errors ("does this implies", "alcoholic mistaken") and misspellings ("irrevelant", "consise"). While this is, admittedly, a behind-the-scenes discussion, the editor's critique is so sloppily written that it verges on the indecipherable.

It's an example of what Jaron Lanier called the "fallacy of the infallible collective" in his critique of Wikipedia. Someone clearly ill-suited for vetting articles for quality was in a position of doing just that. Eventually, however, the collective worked out its kinks: The first editor ceased reviewing the article, and another took over and awarded it the "good" rating.

And there it is, Bix Beiderbecke explained in great detail on Wikipedia. Perhaps, as Wolfe hopes, it will spark interest in the musician that might lead people to his forthcoming book. Equally likely, sadly, is that it will scare other scholars off from investing their time in creating well-rounded Wikipedia entries.

-- Carolyn Kellogg

Image credit: Columbia Jazz Masterpieces

Comments

[Ryan]

I don't get this. Someone made a rude comment to someone else on the internet, and this is supposed to be newsworthy? Is the LA Times now a tabloid that posts about irrelevant drama not even associated with celebrities?

[The_ed17]

Hello there. There is more to this story than you might think. Despite its reward of a higher-quality rating, the good article process normally functions as more of a one-person peer review. This means that occasionally--maybe even normally--articles will pass when reviewed by "someone clearly ill-suited for vetting articles for quality."
However, for Wikipedia's highest quality rating, "featured", an article will typically undergo a rigorous reviews from editors both familiar and unfamiliar with the topic in question, allowing for views on the article from both perspectives.

[lulu]

No, the article pointed out that knowledgeable scholars are being "reviewed" and approved by semi-literate children who don't generally have the depth of understanding about the subject they are editing.

Most of the original contributors, many with vast in-depth knowledge about a variety of subjects, left wikipedia several years ago and pseudo-intellectual teens/20-somethings have taken over. The shock is that people still believe in or rely on the accuracy of wikipedia

[Grouchy prof]

Face, it is a general fact of life that your work occasionally gets judged by people who simply are not up to the task.

Academics who submit articles to peer-reviewed journals and conferences sometimes have their work reviewed by some incompetent colleague, or some inexperienced graduate student to whom an overworked professor has sub-contracted his refereeing.

Engineers and academics alike report to managers that often simply don't know what they're doing, and apply formal criteria of evaluation.

Now, the real question is whether this happens so much of the time — the proportion. In the case of the Wikipedia "good article" process, another question is whether it has real impact on the site. Just citing one example of one person being rude to another on an online forum is not an analysis of the situation.

[lulu]

Touche "Grouchy prof". However, having seen and watched from the inside (knowing many of the people personally), I can honestly tell you that the preponderance of the "editors" are NOT up to the task and their personal/political agenda is more important than the original goal of the site. Any idea who the editors are today? I can tell you that there is an unusually high incidence of Asperger's or highly functioning autistic boys/men who are young with a limited education. And although they might have an interest in a particular subject, they are hardly authorities as would be found in peer reviews.

At this point, I wouldn't trust Wikipedia for anything, except maybe a link to something else ... like the original source.

[BT]

The "Good Article" (GA) review system is essentially one editor who supposedly has some experience on the wiki stating what has to be done to meet the criteria he or she is interpreting. As such it depends entirely on the reviewer. I've seen great reviewers who are both comprehensive and treat the review as a dialogue intended to improve the article, and also little dictators demanding that every one of their random whims, regardless of the possible negative affects on the article itself, be met before they deign to bestow a rating upgrade on the lowly supplicant.

The GA process is so dysfunctional that I no longer try to increase the ratings of the articles I work on, regardless of how good I think they are. It's far too capricious to be as important as the GA reviewers think it is.

If Wolfe has already passed GA, he will find the "Featured Article" (FA) process, the process to identify the best articles on the site for featuring at the top of the Main Page, to be relatively sane. This is largely result of the fact that the FA process involves multiple reviewers, with the final decision to pass or not taken by a separate person facilitating the process who can decide if an objection by an reviewer is unreasonable. I'd suggest putting the article on the "Peer Review" page first before suffering the slings and arrows of an FA review.

[The_ed17]

BT: yes, the GA process is completely dysfunctional, but I doubt that anything will change. I've started to take my articles straight to the Military history WikiProject's (Milhist) A-class review system and nominate it for good article as an afterthought. Milhist has some very knowledgeable who will go through the article for accuracy as well as prose quality. The problem is that this doesn't benefit anyone who works outside the realm of military history; in most cases, they are forced into the GA process.

I agree with you regarding FAC, but I just hope that he doesn't get angry if someone finds problems with, say, his references...

[ForNoReasonAtAllInC]

Bix Lives!

[Tim1965]

So, is the Los Angeles Times going to report on the tens of thousands of Wikipedia article reviews which had no problems, or the hundreds of cases where academics contributed to articles and there were absolutely no problems? I guess not.

The reporter of this story fails to note that the reviewer of the Bix Beiderbecke article failed to adhere to Wikipedia's guidelines for review. "Good article" criteria don't ask the reviwer to start making judgment calls (such as those cited above) about "taking credit" for cited statements or judging whether a quotation is "florid prose". Every contributor to Wikipedia has the right to seek a reviewer who understands the rules and applies them fairly. Wolfe sought that change, and Wikipedia worked.

[Carl Isaacson]

While the editor's comments are grammatically difficult, the point is not. If Dorsey and Murray are not part of the alcoholic marring of the 2nd number, then why mention them? On the second issue - the concision of the "florid prose," I'm not sure I understand. Yes, democracy sometimes allows the lesser to evaluate the greater - but overall it does produce a better product that can be edited and improved over time. No one's writing, not even my own, fails to benefit from editing, even when the editorial suggestions and/or demands are phrased in a way we don't like.

[SierraHotel058]

I empathize with Mr Wolfe....it must be very frustrating to be the world's foremost expert on a particular subject (which he may very well be in regards to Bix Beiderbecke) and be subject to scrutiny and criticism by some unknown person...... who is by definition less qualified, and perhaps unqualified to edit period.

This is the frustrating and sometimes ugly part of the Wikipedia process. I don't know of any easy way around it, because in any truly democratic exercise (as I think Wikipedia strives to be) the journey to the end result is rife with conflict and frustration (think chaotic town hall meeting).

I do know that Wikipedia is my "go to" resource for information. It is remarkably up-to-date and all-incompassing. If accuracy is paramount, it is still an excellent starting point for a list of reference material.

In the case of the article in Wikipedia on Bix Beiderbecke, where else on the web can we easily find this information? Possibly nowhere. That is the point. The consumers and beneficiaries of Mr Wolfe's work may not be aware of his frustration, but they will certainly be better informed as a result of his efforts.

[Einar E Kvaran]

I've been a wikipedia editor for a while and am familiar with some of the issues Brendan ran into. I guess that to me the point of the article was in the phrase, "the collective worked out its kinks" and the article received its "good" rating. Wikipedia is a bureaucracy and it sometimes takes a while to slog through it. But every time an editor does that wikipedia, and all of its users are better off for it. Einar

[gwern]

I'll admit, I don't understand this column's criticisms.

> No wonder he was angry. The editor's comments are riddled with grammatical errors ("does this implies", "alcoholic mistaken") and misspellings ("irrevelant", "consise"). While this is, admittedly, a behind-the-scenes discussion, the editor's critique is so sloppily written that it verges on the indecipherable.

The article author seemed to understand fine, as did the column author. You'll find many misspellings in Wikipedia - the edit summaries alone verge on universally having problems.

> "the second number was marred by alcohol consumed by the musicians, who included Tommy Dorsey on trombone and Beiderbecke's best friend, Don Murray, on clarinet". Does this implies that Tommy Dorsey and Don Murray contributed to the alcoholic mistaken in the 2d number? Tommy Dorsey and Don Murray contributed to Beiderbecke's life or work at last, except as members as the same band for a while? If these 2 musicians NOT contributed in at least of one of these aspects, they are irrevelant - just name dropping.

And isn't he right? This is worrisome as insinuation, or guilt by association. (Imagine rewriting it with 'heroin': "The band's second number was a disaster due to massive heroin consumption. 2 of the 10 band members were...") If they have some later connection, this is a good point to bring it out - X played in band Z with our topic, Y, and X would later collaborate with Y on award-winning album A. Random name-dropping is confusing and bad & bulky prose.

I would defend the headmaster criticism, except as I have already noted, pointing out grammar & spelling errors is the most trivial & vapid possible point to make and in no way refutes the criticism.

[Bill]

Seriously? The worst you can say is that the reviewer's comments were grammatically incorrect? Go through the review again. You'll notice many valid suggestions for improvement. If only the LA Times could get some peer reviewers...

[Brendan Wolfe]

Thanks for the all the feedback. I wanted to add a couple things: I am not, at all, by any stretch, one of the world's foremost experts on Bix Beiderbecke. Nor, I hope, do I strut around Wikipedia (or anywhere else!) as if I were. And my frustrations regarding the Good Article Review were not about receiving criticism -- about my sentence that name-dropped Don Murray and Tommy Dorsey or about anything else. Nor, finally, was it my intention to criticize mere bad spelling and grammar in the context of such feedback.

My frustration was centered on two things: first, that a self-described "ignoramus about jazz" felt qualified and empowered to parse decisions I made about about content; and second, that some of these comments were not merely full of spelling and grammatical errors, they were largely incoherent. I think that when the subject is writing and editing, the reviewer -- presumably the expert here, if not on Bix Beiderbecke then on writing and editing -- ought to be able to demonstrate minimum qualifications on the subject, which is to say s/he ought to be able to write not just clearly but coherently. The sentence of mine that mentioned Dorsey and Murray was infinitely more clear than the one that criticized it.

I realize this sounds like some angry-old-man rant. It's not intended to be. While I admitted to be "enraged," I meant it at least somewhat ironically, and anyway, it all worked out. Wikipedia admirably self-corrected and I was pleased that the article received its stamp of approval, even if I'm still not sure why that was so important to me in the first place.

[Loya]

Like it or not, Wikipedia articles that pass the review process and reach the Feature level are the best on the Internet and the best in all encyclopedias.

[George H. Smith]

Ms. Kellog begins by referring to "coronetist Bix Beiderbecke." What, pray tell, is a "coronetist"? Maybe Bix was a skilled artisan who made crowns to supplement his
income. Or maybe "coronetist" means "king of the cornet players." These are just wild guesses, of course.

[Colin Howell]

Repeating here a comment I posted to Brendan Wolfe's blog...

In fairness to Philcha, it should be pointed out that he was apparently suffering from a medical problem which was affecting his writing. During the review, he eventually passed the task to another editor named "Geometry guy", after having left him the following message (see my name for a link to the discussion page):

"I'm very sorry, I am ill - I can read words but cannot write (I often write gibberish), and most of this post is pasted words found in other places. The GA Reviews I started for others will need to be done by others ... "