Wednesday, October 24, 2007
It was bound to happen
Many years ago, I had a conversation with my high school debate teacher (yes, it was that long ago) about falsifying the research the debate team used. He pointed out that fabricated information has a tendency to be shared by all team members, and mentioned an instance not long before where it harmed someone -- not the person who fabricated it -- in a debate.
I was reminded of this discussion when I stumbled onto a discussion at Wikipedia: Administrators' Noticeboard/Incidents. Someone, to everyone's shock, had managed to insert over the space of two years, a surprising amount of fabricated or misrepresented information, into a number of articles. He evaded notice by being civil, quick to back down in a conflict, and by focussing his attention on a number of subjects that non-experts were not likely to challenge his edits -- especially when he provided what appeared to be, at a casual examination, reliable sources.
As other editors worked their way down his rabbit-holes of his sources and evidence (another phrase that comes to mind is "mares nest"), it became obvious that he had constructed an elaborate collection of unacceptable sources. He would cite works that did not support his assertions; they either contradicted his assertion or were entirely irrelevant. Some were relevant, but clearly outdated. Still others appeared to be peer-reviewed literature, but on closer examination were not; they were published by groups with deceptively similar names, or self-published. And many of these papers and monographs leaned on each other: publication A would cite publication B, which would cite publication C, which would cite publication A.
I'm not mentioning the user name because my I'm not writing about this specific user, but about the problem he revealed. Despite the fact Wikipedians are always reviewing each others' work and challenging each other's conclusions, there is an irreducable level of trust between all of us. If someone makes statements about a given source, we fidn ourselves assuming that they are telling us the truth about that source.
Wikipedia functions on a certain level of mutual trust, and in the most part, people do not violate this trust. Even the troublemakers, the self-promoters, and the tendentious editors almost always honestly report the contents of their sources. Those who don't -- until now -- are also unable to show enough self-control to be effective in the give-and-take that makes up much of the Wiki environment, stop being civil or clearly violate one of the customs of the Wikipedia culture, and are quickly banned from the site and their edits reverted.
However, this affair has pointed out that this level of trust may not work any more; if an editor claims that an uncommon resource -- say a rare book or an article published in an obscure technical journal -- says something, how do we know that this editor is not lying? While this is less of a concern for established editors than new ones, the fact remains that this specific user contributed 8000 edits over two years, before someone noticed.
This is not a theoretical problem; in my own case, I have been using more and more uncommon works on the history of Ethiopia. Not every article in Wikipedia is a reworking of what is posted on the Internet; some of us do use sources printed on paper! This leads to the problem that not all of these printed works are easily accessible. In my case, like many serious Wikipedians, I have gradually accumulated a collection of books on my topic of interest to overcome this problem. Since not everyone using Wikipedia can do this, should I continue to use them because they cannot verify what I report these sources contain?
Maybe this is a just problem that was bound to happen eventually. As a group attracts more people, the chances that at least one person will abuse that trust increases until it happens. We should be glad that Wikipedia went almost 6 years before this became a serious problem.
Geoff
Technocrati tags: online communities, reliability, research, trust, Wikipedia
I was reminded of this discussion when I stumbled onto a discussion at Wikipedia: Administrators' Noticeboard/Incidents. Someone, to everyone's shock, had managed to insert over the space of two years, a surprising amount of fabricated or misrepresented information, into a number of articles. He evaded notice by being civil, quick to back down in a conflict, and by focussing his attention on a number of subjects that non-experts were not likely to challenge his edits -- especially when he provided what appeared to be, at a casual examination, reliable sources.
As other editors worked their way down his rabbit-holes of his sources and evidence (another phrase that comes to mind is "mares nest"), it became obvious that he had constructed an elaborate collection of unacceptable sources. He would cite works that did not support his assertions; they either contradicted his assertion or were entirely irrelevant. Some were relevant, but clearly outdated. Still others appeared to be peer-reviewed literature, but on closer examination were not; they were published by groups with deceptively similar names, or self-published. And many of these papers and monographs leaned on each other: publication A would cite publication B, which would cite publication C, which would cite publication A.
I'm not mentioning the user name because my I'm not writing about this specific user, but about the problem he revealed. Despite the fact Wikipedians are always reviewing each others' work and challenging each other's conclusions, there is an irreducable level of trust between all of us. If someone makes statements about a given source, we fidn ourselves assuming that they are telling us the truth about that source.
Wikipedia functions on a certain level of mutual trust, and in the most part, people do not violate this trust. Even the troublemakers, the self-promoters, and the tendentious editors almost always honestly report the contents of their sources. Those who don't -- until now -- are also unable to show enough self-control to be effective in the give-and-take that makes up much of the Wiki environment, stop being civil or clearly violate one of the customs of the Wikipedia culture, and are quickly banned from the site and their edits reverted.
However, this affair has pointed out that this level of trust may not work any more; if an editor claims that an uncommon resource -- say a rare book or an article published in an obscure technical journal -- says something, how do we know that this editor is not lying? While this is less of a concern for established editors than new ones, the fact remains that this specific user contributed 8000 edits over two years, before someone noticed.
This is not a theoretical problem; in my own case, I have been using more and more uncommon works on the history of Ethiopia. Not every article in Wikipedia is a reworking of what is posted on the Internet; some of us do use sources printed on paper! This leads to the problem that not all of these printed works are easily accessible. In my case, like many serious Wikipedians, I have gradually accumulated a collection of books on my topic of interest to overcome this problem. Since not everyone using Wikipedia can do this, should I continue to use them because they cannot verify what I report these sources contain?
Maybe this is a just problem that was bound to happen eventually. As a group attracts more people, the chances that at least one person will abuse that trust increases until it happens. We should be glad that Wikipedia went almost 6 years before this became a serious problem.
Geoff
Technocrati tags: online communities, reliability, research, trust, Wikipedia
Labels: speculations, wikipedia, writing
Comments:
<< Home
We should be glad that Wikipedia went almost 6 years before this became a serious problem.
Indeed. But, IMHO, this will tend to appear more and more often.
The key here is the reputation inflection point. Beyond it, one could earn more reputation by vandalism than by constructive editions.
Wikipedia will tend to become an institution rather than a novelty. What will occurs if medias praise to the skies the one that "challenged Wikipedia comunity" ?
The question raised here is "do the intrinsicaly good nature of Wikipedia will save it ?".
Indeed. But, IMHO, this will tend to appear more and more often.
The key here is the reputation inflection point. Beyond it, one could earn more reputation by vandalism than by constructive editions.
Wikipedia will tend to become an institution rather than a novelty. What will occurs if medias praise to the skies the one that "challenged Wikipedia comunity" ?
The question raised here is "do the intrinsicaly good nature of Wikipedia will save it ?".
Oh yes... I have seen this in a few instances... I guess that gaming the system is easy in a community that is based in extending good faith to its contributors.
I would argue that it is a fair price to pay...
[[User:Jossi]]
Post a Comment
I would argue that it is a fair price to pay...
[[User:Jossi]]
# posted by 
: 3:43 PM, November 09, 2007
: 3:43 PM, November 09, 2007 << Home
