Tag Archives: bibliometry

Academic Search Engine Spam and Google Scholar’s Resilience Against it

[Journal of Electronic Publishing, Volume 13, Issue 3, December 2010]

Academic Search Engine Spam and Google Scholar’s Resilience Against it

Joeran Beel and Bela Gipp

In a previous paper we provided guidelines for scholars on optimizing research articles for academic search engines such as Google Scholar. Feedback in the academic community to these guidelines was diverse. Some were concerned researchers could use our guidelines to manipulate rankings of scientific articles and promote what we call ‘academic search engine spam’. To find out whether these concerns are justified, we conducted several tests on Google Scholar. The results show that academic search engine spam is indeed—and with little effort—possible: We increased rankings of academic articles on Google Scholar by manipulating their citation counts; Google Scholar indexed invisible text we added to some articles, making papers appear for keyword searches the articles were not relevant for; Google Scholar indexed some nonsensical articles we randomly created with the paper generator SciGen; and Google Scholar linked to manipulated versions of research papers that contained a Viagra advertisement. At the end of this paper, we discuss whether academic search engine spam could become a serious threat to Web-based academic search engines.

[continue at JEP]

Tagged , , , ,

Self-Selected or Mandated, Open Access Increases Citation Impact for Higher Quality Research

[Gargouri Y, Hajjem C, Larivière V, Gingras Y, Carr L, et al. (2010) Self-Selected or Mandated, Open Access Increases Citation Impact for Higher Quality Research. PLoS ONE 5(10): e13636. doi:10.1371/journal.pone.0013636]

Articles whose authors have supplemented subscription-based access to the publisher’s version by self-archiving their own final draft to make it accessible free for all on the web (“Open Access”, OA) are cited significantly more than articles in the same journal and year that have not been made OA. Some have suggested that this “OA Advantage” may not be causal but just a self-selection bias, because authors preferentially make higher-quality articles OA. To test this we compared self-selective self-archiving with mandatory self-archiving for a sample of 27,197 articles published 2002–2006 in 1,984 journals.

Methdology/Principal Findings
The OA Advantage proved just as high for both. Logistic regression analysis showed that the advantage is independent of other correlates of citations (article age; journal impact factor; number of co-authors, references or pages; field; article type; or country) and highest for the most highly cited articles. The OA Advantage is real, independent and causal, but skewed. Its size is indeed correlated with quality, just as citations themselves are (the top 20% of articles receive about 80% of all citations).

The OA advantage is greater for the more citable articles, not because of a quality bias from authors self-selecting what to make OA, but because of a quality advantage, from users self-selecting what to use and cite, freed by OA from the constraints of selective accessibility to subscribers only. It is hoped that these findings will help motivate the adoption of OA self-archiving mandates by universities, research institutions and research funders.

[continue at PLoS One]


What’s in a name?

[EMBO Rep. 2008 December; 9(12): 1171–1174.
doi:  10.1038/embor.2008.217]

What’s in a name?

Howard Wolinsky

An international author identification system could allow scientists to receive credit for all their scientific contributions and would solve the problem of identity in a world of limited surnames.

In Romeo and Juliet, William Shakespeare (1546–1616) asked: “What’s in a name? That which we call a rose / By any other name would smell as sweet” (2.2.1–2). But, unlike Shakespeare’s characters, whose names were a burden to them, names for scientists are extremely important and are attached to discoveries, publications, careers and even fame. In her poem Sacred Emily, American poet Getrude Stein (1874–1946) wrote, “Rose is a rose is a rose is a rose”, which is reminiscent of the trouble that scientists face. However, unlike two roses, two scientists with identical names are extremely different. Indeed, this growing problem of how to unambiguously identify members of an ever-growing international community has triggered a serious debate.

Matthew Falagas, Director of the Alfa Institute of Biomedical Sciences (Athens, Greece) and Adjunct Associate Professor in the Department of Medicine at Tufts University School of Medicine (Boston, MA, USA) described the problem. “Every country has its own common names: Smith and Jones in English-speaking countries; here in Greece we have Papapapolous; in China, Li is very common,” he said. “When you enter the last name, or even with the initial of the first name, you may produce thousands of papers; you cannot understand who produced what. The result is that you have difficulties finding the best collaborator or the best person to ask to do peer review.” Falagas, who serves on the editorial board of the scientific journal PLoS One (San Francisco, CA, USA), said that he has heard of cases in which mistaken identity has resulted in the wrong person being invited to work on a project, appear on a television programme or to undertake the peer review of an article.

Names are also important with regard to the funding of research. Kyle Brown, who founded ResearchCrossroads (San Mateo, CA, USA; www.researchcrossroads.org), explained that the company started as a database to help funding agencies and researchers look up who was getting money from government and private foundations, including the National Institutes of Health (Bethesda, MD, USA), the National Science Foundation (NSF; Arlington, VA, USA) and the Community Research and Development Information Service run by the European Union (CORDIS; Brussels, Belgium). Brown discovered the confusion inherent in similar names almost immediately as researchers contacted him to try to correct their information posted on the ResearchCrossroads website. “We struggle with this constantly,” he said. “I get e-mails every day or two saying: ‘My profile is combined with this other guy’s profile. Can you help us untangle them?’ And we’ll do that.” But he added that this is often a tough assignment as scientists can change jobs and even names, and do not use their middle initials consistently.

The same problem applies to publishing, in which it is not always clear whether the authors of different articles are the same person or not. “I’d love a user of Nature.com to be able to click on an author’s name and to be able to see a list of everything that we publish by them,” commented Timor Hannay, Publishing Director of Nature.com (London, UK). “And that kind of thing, which seems really trivial, should be very straightforward, but actually isn’t because we don’t have identifiers associated with them […] We’ve got a world in which scientists have assigned numbers to all kinds of things: to genes, to species, to stars, to molecules, to the articles they write. The one thing they left out was themselves,” he said, adding, “it does create real problems for us as publishers wanting to provide certain services.”

Yet, Hannay also has an idea of how to solve this problem: “A global author ID does bring you the same benefits that you already have from [a] unique article ID, and you can locate an article very quickly and easily online if you know what its DOI [digital object identifier] is.” However, despite the elegance and simplicity of the idea, introducing a unique author ID for scientists is anything but a simple measure, as Peter Binfield, Managing Editor of PLoS One commented: “It’s not a simple problem at all. I have worked at a few publishing companies and nobody is able to make unique author IDs happen within their own databases. [Authors] may submit with a different middle initial, or [they may have] moved institutions. Even our internal databases get incredibly messy and cluttered. It’s hard to keep anything consistent.”

[continue at EMBO Reports]

Tagged ,

Bibliometrie in der Forschungsbewertung

[Forschung & Lehre 11/2011]

Aussagekraft und Grenzen

Werner Marx

Von den einen als bloßes Zitatezählen kritisiert, wird die Bibliometrie in der Forschungsbewertung von anderen hochgelobt. Was bringt die Bibliometrie für die Bewertung von Forschungsleistung? Kann sie Gutachter gar ersetzen oder ist ein Zusammenspiel von Peer Review und Bibliometrie die sachgerechteste Lösung? Eine Analyse.

In der Wissenschaft werden die Publikationswürdigkeit von Arbeiten, die Förderungswürdigkeit von Forschungsvorhaben und die Qualifikation von Stellenbewerbern von erfahrenen Fachkollegen (Peers) beurteilt. Diese „Wächter der Wissenschaft“ sollen im Rahmen des sogenannten Peer Review Verfahrens die Qualität der Forschung gewährleisten. Dabei besteht jedoch die Gefahr, dass Gutachter die Bedeutung einer wissenschaftlichen Arbeit nicht erkennen oder sachfremde Einflüsse eine objektive Bewertung beeinträchtigen. Wenn es um die Einschätzung von Fachkollegen geht, sind auch Wissenschaftler nicht unbedingt objektiv – und manchmal auch überfordert. Gutachtergremien setzen sich außerdem meist nur aus wenigen Experten zusammen, was die Gefahr der Subjektivität noch erhöht.

[weiter auf Forschung & Lehre]

Tagged ,

The use and misuse of bibliometric indices in evaluating scholarly performance

Ethics in Science and Environmental Politics (ESEP), Vol. 8, No. 1, 2008.

Quantifying the relative performance of individual scholars, groups of scholars, departments, institutions, provinces/states/regions and countries has become an integral part of decision-making over research policy, funding allocations, awarding of grants, faculty hirings, and claims for promotion and tenure. Bibliometric indices (based mainly upon citation counts), such as the h-index and the journal impact factor, are heavily relied upon in such assessments. There is a growing consensus, and a deep concern, that these indices — more-and-more often used as a replacement for the informed judgement of peers — are misunderstood and are, therefore, often misinterpreted and misused. The articles in this ESEP Theme Section present a range of perspectives on these issues. Alternative approaches, tools and metrics that will hopefully lead to a more balanced role for these instruments are presented.

Tagged , , ,