Open access to peer-reviewed research: making it happen
Pritpal S Tamber, Fiona Godlee, Peter Newmark
Lancet 2003; 362: 1575-77
See Commentary
The difficulty with charging users
How can access to subscription journals be improved?
An alternative to subscriptions: article processing charges
The way forward
References
At the start of 2003, the Public Library of Science announced that it had secured funds to enable it to launch two high-level open-access journals.1 The first issue of PLoS Biology was published in October, 2003, and the launch of PLoS Medicine is scheduled for mid 2004. This iniative, welcomed by many, arose from the failure of the library's original mission to persuade major publishers to convert existing subscription journals to open access. Here, we summarise the arguments that all scientific research should be freely and immediately available online.
Publishers use a range of business models. Some do not charge users, preferring to cover costs through sponsorship or advertising; most do impose a charge, either to cover costs or to make a profit. The publishing model based on charging users (ie, subscriptions) has served us well for over 200 years, but it now hinders rather than helps science communication (figure). Research articles, which make up the bulk of scholarly publishing, remain a cash cow for the publishing industry. Journal prices have grown out of proportion with inflation and library budgets, making the research they contain increasingly inaccessible.
With the growing use of the internet, new publishing models are emerging, which are committed to providing free access to the full text of research articles. Many society-owned journals now offer their archives online free of charge, while retaining subscription-only access to newer material. The challenge now is to make access free from the moment of publication in a way that has long-term sustainability. Authors and funders of research should understand the problems with the current publishing model, know about the new models, and add their voices to the debate on the future of science communication.
The difficulty with charging users
A business model that charges users a fee to access research articles has several important adverse effects. First, to protect subscription revenues, publishers must ensure that authors do not distribute the material by any other route, which means that authors must assign copyright to the publisher. Although scientists may be unhappy about this enforced loss of ownership,2 they are under pressure to publish (for career enhancement and funding applications), and since almost all journals require transfer of copyright, they have little choice but to comply. The BMJ allows authors to retain copyright3 and Nature and its sister journals have recently announced that they will do the same.4 However, these journals still insist on an exclusive licence, the terms of which mean that authors are not free to distribute their articles or allow open access to them.
Second, the need to protect subscription revenues carries with it large costs. Beyond the essential functions of a journal to control quality and disseminate knowledge, publishers employ teams of people to attract and monitor subscribers. We estimate that these activities account for up to 30% of journal publishers' costs. These costs are passed on to the scientific community as part of subscription charges.
Third, publishers have what is effectively a monopoly over the distribution of scientific research,5 since authors hand over copyright and libraries have little choice but to subscribe to journals. This situation has enabled some publishers to test library budgets to their limit. By increasing the size or frequency of journals, publishers can justify linear increases in prices. Some libraries cancel subscriptions, but the loss to publishers is small compared with the gain from continuing subscribers--the net effect is increased revenues. For publishers supposedly interested in communicating science, there is a peverse incentive to limit access to such information to boost profits. Perhaps more importantly, the spiralling cost of journals affects science funding, since library budgets are part of the overall budget for science. Between 1999 and 2002, the global medical publishing sector grew by an estimated 20%, taking its revenue to US$2·69 billion.5 Growing profits for publishers mean less money for research.
A fourth adverse effect of the subscription-based model is that it increases the risk of excluding large parts of the world's scientific community from full participation in scientific debate.2,6 The Health InterNetwork Access to Research Initiative (HINARI), WHO's initiative in which publishers provide free or cheaper access to their content for people in the developing world7 is a contribution to greater access. But many global inequities and inefficiencies remain: small, subspecialty, or emerging areas of inquiry are unable to sustain journals because the number of subscribers is too low to cover running costs or to make a worthwhile profit for publishers, and some subscription journals fold after a few issues, thus wasting resources and making the research published in them irretrievable.
Finally, limited access to the full text of research articles is bad for science. Such restrictions make it difficult for researchers to build on the entirety of what has gone before and for readers to check whether they have done so.8 The practice might contribute to citation bias since authors will only reference journals they can access.9 Cross-disciplinary interaction could be restricted, thus delaying the transfer of knowledge of useful techniques. Furthermore, researchers and clinicians will sometimes make do with the abstract of an article they cannot obtain in full, which in many cases will give them inadequate, if not misleading, information.10
How can access to subscription journals be improved?
Attempts to improve access have largely focused on limiting the effect of user charges (subscriptions) rather than abolishing them altogether. Three such initiatives deserve special mention: PubMed Central, the Open Archives Initiative, and Scholarly Publishing and Academic Resource Coalition (SPARC).
PubMed Central (http://pubmedcentral.nih.gov), set up by the US National Institutes of Health, aims to become a centralised digital library that provides free access to the full text of all peer-reviewed life-science research articles--in essence, a full text version of PubMed. Although a few publishers have joined the scheme in the belief that it will increase their visibility and citation rates,11,12 most have so far refused to participate because of concerns that their subscription revenues will suffer. Two concessions from PubMed Central have made participation more likely. First, publishers can impose delays of up to a year after publication before making articles freely accessible; and second, the material can remain on the journal's website rather than having to be posted on the PubMed Central website. Journals can, therefore, benefit from being part of a centralised searching facility without losing visitors to their own websites.
The Open Archives Initiative (http://www. openarchives.org) aims to create a global online archive of all published research and is funded by the Joint Information Systems Committee, part of the UK government's Higher Education Funding Councils of England, Scotland, and Wales.13 Its chief proponent, Stephen Harnad of Southampton University, UK, calls for all research, after publication, to be posted on personal or institutional websites and tagged in a standardised form, making it searchable, navigable, and retrievable. If publishers do not allow authors to post their articles on personal or institutional websites, Harnad suggests they post the submitted draft together with a corrigendum file highlighting the differences between it and the published version. Although this approach is not an alternative to the current subscription-based publishing model, it could improve access within it.
The SPARC (http://www.arl.org/sparc/) aims to return "science to scientists" in three ways: by encouraging scientists to create journals that directly compete with those thought to be overpriced; by giving confidence to scientists to create journals in new areas of inquiry; and by backing scientists who create web-based resources other than journals for their communities. SPARC began in 1998 as part of the Association of Research Libraries (ARL) in the USA, but is now a worldwide network of libraries and research institutions, each offering funding to the overall cause. What SPARC offers, as well as know-how, is a potential customer base. Of the many SPARC projects, some have open access, whereas others remain subscription-based. Their important contribution is that they are cheaper than, but attempt to be as prestigious as, established journals.
An alternative to subscriptions: article processing charges
The success of PubMed Central, the Open Archives Initiative, and SPARC will help to improve access within the subscription-based model. Other initiatives challenge the subscription-based model outright and offer an alternative, article processing charge. Most prominent among these are BioMed Central (http://biomedcentral.com) and the Public Library of Science. BioMed Central is an independent commercial publisher, committed to providing free and immediate online access to the full text of peer-reviewed biomedical research. Authors retain copyright. BioMed Central has more than 90 peer-reviewed journals spanning the fields of biology and medicine, and provides free technical support and hosting for groups of researchers wanting to run online, open access, peer-reviewed journals under their own editorial control. The company receives no support from governments or from scientific societies. Instead of charging users, BioMed Central covers the costs of peer review and publication by charging authors for processing manuscripts.14 The charge, US$500 per published article in 2003, could be paid directly by authors, usually from their research funds, or via their institutes through BioMed Central's membership scheme. In 2003, BioMed Central has 291 institutional members from 29 countries. The charge is waived for authors from developing countries and others who are unable to pay.
The business model used by BioMed Central has several important benefits. First, savings are made through obviation of the need to attract and keep track of subscribers. The cost of processing charges is small in relation to the cost of research and could become a normal part of grant applications. Second, if all journals adopted the model, there would be substantial savings to the scientific community, which pays about US$5000 per published article (estimated from publishers' gross revenues from journal subscriptions).14,15 Widespread adoption of a US$500 charge per published article would represent a ten-fold saving for science and society. Third, authors and funders would be fulfilling their responsibility to disseminate the results of their work as widely as possible. Finally, article processing charges create potential for the proper operation of market forces, allowing authors to choose where their article is published based on the level of service, journal prestige, and cost. The adoption of article processing charges by the Public Library of Science is an important endorsement of this new business model, and their charge of US$1500 per article shows that market forces are already at work.
The way forward
Scientific research should be freely accessible to all. Free access is a public good--much research is publicly funded and involves members of the public as participants. Authors and peer reviewers provide their their work free of charge. The cost of peer review and dissemination can, and should be, covered in ways that do not limit access to information and so do not hinder scientific communication.
Funding agencies, academic institutions, promotion and tenure committees, and authors can all work to promote open access. Funding agencies and institutions can encourage their researchers to publish in open-access journals. They can also explore a range of ways of shifting budgets away from journal subscriptions, including allowing processing charges to be routinely payable from research grants. Promotion and tenure committees can encourage their members to judge each piece of work for what it is rather than where it is published, and can give credit for open-access publication. And authors? For the sake of equity and science, send your next paper to an open-access journal.
Conflict of interest statement
P S Tamber is an employee of BioMed Central. P Newmark is an employee and shareholder in BioMed Central.
References
1 Public Library of Science. About the PLOS journals. http://www.plos.org/journals/index.html (accessed Feb 28, 2003).
2 Brown P. What must scientists do to exploit the new environment? http://www.biomedcentral.com/info/brown-tr.asp (accessed July 23, 2001).
3 Smith J. Giving something back to authors. BMJ 2000; 320: 6.
4 Anon. Nature in 2003. Nature 2003; 421: 1.
5 Tamber P. Is scholarly publishing becoming a monopoly? BMC News and Views 2000 1: 1.
6 Horton R. North and South: bridging the information gap. Lancet 2000; 355: 2231-36. [Text]
7 Health InterNetwork. Second phase of HINARI underway: journals available at drastically reduced prices. http://www.healthinternetwork.org/ (accessed Feb 23, 2003).
8 Clarke M, Chalmers I. Discussion sections in reports of controlled trials published in general medical journals: islands in search of continents? JAMA 1998; 280: 280-82. [PubMed]
9 Gøtzche PC. Reference bias in reporting of drug trials. BMJ 1987; 295: 654-56.
10 Haynes RB. More informative abstracts: current status and evaluation. J Clin Epidemiol 1993; 46: 595-97. [PubMed]
11 Delamothe T, Smith R. PubMed Central: creating an Aladdin's cave of ideas. BMJ 2001; 322: 1-2. [PubMed]
12 Cozzarelli N. Power to the people: PubMed Central. Proc Natl Acad Sci 2000; 97: 1-1. [PubMed]
13 Open Archives Initiative. www.eprints.org Site accessed Feb 23, 2003.
14 Velterop J. Author charges are the future. BMC News and Views 2001; 2: 2.
15 Odlyzko A. Tragic loss or good riddance? The impending demise of traditional scholarly journals. http://www.dtc.umn.edu/~odlyzko/ doc/tragic.loss.txt (accessed Feb 23, 2003).
The Journal
Protocol Reviews
eResearch Archive
Info for Authors
Profile
Help
Feedback
Open access to peer-reviewed research: making it happen
4 posts
• Page 1 of 1
Open access to peer-reviewed research: making it happen
by malernee » Sat Nov 08, 2003 8:55 pm
- malernee
- Site Admin
- Posts: 462
- Joined: Wed Aug 13, 2003 5:56 pm
Evidence-Based Vet Medicine & Internet Resources
by guest » Wed Mar 17, 2004 11:41 am
Evidence-Based Medicine & Internet Resources
Western Veterinary Conference 2003
Ronald D. Smith, DVM, PhD
University of Illinois College of Veterinary Medicine
Urbana, Illinois, USA
OBJECTIVES
Evidence-based medical resources and relevant on-line information sources.
Key Points
Evidence-based medicine (EBM) provides a disciplined approach to identifying and meeting information needs during case workup or management.
The practice of EBM also promotes self directed lifelong learning.
Online EBM evidence sources include discussion groups, diagnostic support systems, and the peer-reviewed medical literature. Most of these resources are free.
Sophisticated search algorithms greatly facilitate retrieval of information from medical bibliographic databases such as PubMed.
The validity of online information varies greatly and requires that veterinarians learn to critically evaluate all medical claims.
OVERVIEW
What is EBM?
Evidence-based medicine (EBM) is the process of turning clinical problems into questions and then systematically locating, appraising, and using contemporaneous research findings as the basis for clinical decisions.1,2 Evidence-based medicine follows four steps:
1. Formulate a clear clinical question from a patient's problem
2. Search the literature for relevant clinical articles
3. Evaluate (critically appraise) the evidence for its validity and usefulness
4. Implement useful findings in clinical practice
The result is a disciplined approach to recognizing and meeting information needs during case workup or management. The practice of EBM also promotes lifelong learning. Identifying and meeting information needs in the context of a particular case ("just-in-time learning") increases the likelihood that it will be retained compared with passive information acquisition ("information foraging") typical of routine reading of journal articles and attendance at medical conferences.
EBM Evidence Sources
One of the most immediate evidence sources for case management is one's own personal experience and that of one's peers. This evidence source is often the most clinically-relevant, but is subject to many biases. The peer-reviewed literature represents a valuable alternative source of medical information. Online discussion groups, bibliographic databases and user-friendly software for searching these sources have made it easier for busy clinicians to retrieve clinically relevant information when it's needed.
Online EBM Evidence Sources
1. Electronic discussion groups such as VetProf/VetPlus <http://www.vetprof.com/> and the "Vet-to-Vet" message boards of the Veterinary Information Network (VIN) <http://www.vin.com/> provide online forums where practitioners can seek and exchange information to support case management. VetPlus is a traditional e-mail based electronic discussion group and is free. VetProf is a Web-based enhancement of VetPlus with added features such as archive searching. There is a minimal annual subscription fee for this service. VIN is a premium online information provider which includes discussion groups, rounds, continuing education, reference and bibliographic services for veterinarians. There is an annual subscription fee.
2. The Consultant diagnostic support system for veterinary medicine <http://www.vet.cornell.edu/consultant/consult.asp> provides immediate access to brief synopses on diseases of species of veterinary importance. The CONSULTANT database consists of approximately 500 signs/symptoms, about 7,000 diagnoses/causes, and about 18,000 literature references including over 3000 web references. Each disease synopsis includes 1) a general description, 2) species affected, 3) the signs/symptoms associated with it, and 4) a list of recent literature references. Disease information can be accessed directly by searching on a diagnosis of interest, or indirectly by entering one or more signs and generating a list of compatible diseases. CONSULTANT also includes a link from each disease synopsis to relevant Web references and the National Library of Medicine's PubMed database of biomedical literature.
3. PubMed <http://www.ncbi.nlm.nih.gov/PubMed/>, a service of the National Library of Medicine, provides access to over 12 million MEDLINE citations back to the mid-1960's and additional life science journals. PubMed includes links to many sites providing full text articles and other related resources. The PubMed Web interface provides a powerful set of tools for retrieving just what you're looking for:
4. Basic Search-the user can simply enter relevant text into the search box and perform an unstructured search of the MEDLINE database. Boolean operators, AND, OR, NOT, e.g., canine OR dog, may also be used but must be entered in upper case. This is the easiest way to see what's available on a particular topic, especially if repeated searches with different word combinations are performed.
5. Related Articles-each citation in PubMed has a link that will retrieve a pre-calculated set of PubMed citations that are closely related to the selected article. Click on Related Articles to the right of each citation to display the related set of articles. PubMed creates this set by comparing words from the title, abstract, and MeSH (Medical Subject Heading) terms using a powerful word-weighted algorithm. Citations are displayed in rank order from most to least relevant with the linked from citation displayed first. This is a great way to find additional articles similar to one that may be of particular interest.
6. Single Citation Matcher-this is a fill-in-the-blank form that allows the user to enter any combination of journal citation information (journal, volume, year, first page, author's name, title words) and retrieve the complete citation including abstract (if available). This is useful for retrieving abstracts of articles cited in other sources, such as the disease synopsis page of Consultant.
7. Clinical Queries-This specialized search is intended for clinicians and has built-in search "filters" based largely on Haynes et al.3 Four study categories are provided (therapy, diagnosis, etiology, prognosis), and the emphasis may be more sensitive (i.e., most relevant articles but probably some less relevant ones) or more specific (i.e., mostly relevant articles but probably omitting a few).
"Signal to Noise Hierarchy" in Peer-Reviewed Clinical Research
A number of factors can influence the validity and relevance of peer-reviewed clinical research articles. First and foremost are the study design and number of subjects used in the study. In general validity tends to be highest and bias lowest in well-controlled studies with large numbers of subjects. The following is an approximate ranking of study designs from lowest to highest validity:
1. Case report
2. Case series
3. Retrospective studies (cross-sectional, case-control)
4. Uncontrolled clinical trial
5. Non-randomized controlled clinical trial
6. Randomized controlled clinical trial
Additional factors influencing the validity of research reports are discussed in a companion paper (VET 431-"Skills in Evaluating Online Information").
Summary
Evidence-based medicine (EBM) provides a disciplined approach to identifying and meeting information needs during case workup or management. The practice of EBM also promotes self directed lifelong learning. Online EBM evidence sources include discussion groups, diagnostic support systems, and the peer-reviewed medical literature. Sophisticated search algorithms greatly facilitate retrieval of information from medical bibliographic databases such as PubMed. The validity of online information varies greatly and requires that veterinarians learn to critically evaluate all medical claims.
References
1. Rosenberg W, Donald A. Evidence based medicine: an approach to clinical problem-solving. BMJ. 1995;310:1122-1126.
2. Sacket DL, Richardson WS, Rosenberg W, Haynes RB. Evidence-Based Medicine. How to Practice and Teach EBM. NY: Churchill Livingstone, 1997.
3. Haynes RB, Wilczynski N, McKibbon KA, Walker CJ, Sinclair JC. Developing optimal search strategies for detecting clinically sound studies in MEDLINE. J Am Med Inform Assoc. 1994;1:447-58
Speaker Information
(click the speaker's name to view other papers and abstracts submitted by this speaker)
Ronald D. Smith, DVM, PhD
Professor, Pathobiology
University of Illinois
Urbana, IL
Western Veterinary Conference 2003
Ronald D. Smith, DVM, PhD
University of Illinois College of Veterinary Medicine
Urbana, Illinois, USA
OBJECTIVES
Evidence-based medical resources and relevant on-line information sources.
Key Points
Evidence-based medicine (EBM) provides a disciplined approach to identifying and meeting information needs during case workup or management.
The practice of EBM also promotes self directed lifelong learning.
Online EBM evidence sources include discussion groups, diagnostic support systems, and the peer-reviewed medical literature. Most of these resources are free.
Sophisticated search algorithms greatly facilitate retrieval of information from medical bibliographic databases such as PubMed.
The validity of online information varies greatly and requires that veterinarians learn to critically evaluate all medical claims.
OVERVIEW
What is EBM?
Evidence-based medicine (EBM) is the process of turning clinical problems into questions and then systematically locating, appraising, and using contemporaneous research findings as the basis for clinical decisions.1,2 Evidence-based medicine follows four steps:
1. Formulate a clear clinical question from a patient's problem
2. Search the literature for relevant clinical articles
3. Evaluate (critically appraise) the evidence for its validity and usefulness
4. Implement useful findings in clinical practice
The result is a disciplined approach to recognizing and meeting information needs during case workup or management. The practice of EBM also promotes lifelong learning. Identifying and meeting information needs in the context of a particular case ("just-in-time learning") increases the likelihood that it will be retained compared with passive information acquisition ("information foraging") typical of routine reading of journal articles and attendance at medical conferences.
EBM Evidence Sources
One of the most immediate evidence sources for case management is one's own personal experience and that of one's peers. This evidence source is often the most clinically-relevant, but is subject to many biases. The peer-reviewed literature represents a valuable alternative source of medical information. Online discussion groups, bibliographic databases and user-friendly software for searching these sources have made it easier for busy clinicians to retrieve clinically relevant information when it's needed.
Online EBM Evidence Sources
1. Electronic discussion groups such as VetProf/VetPlus <http://www.vetprof.com/> and the "Vet-to-Vet" message boards of the Veterinary Information Network (VIN) <http://www.vin.com/> provide online forums where practitioners can seek and exchange information to support case management. VetPlus is a traditional e-mail based electronic discussion group and is free. VetProf is a Web-based enhancement of VetPlus with added features such as archive searching. There is a minimal annual subscription fee for this service. VIN is a premium online information provider which includes discussion groups, rounds, continuing education, reference and bibliographic services for veterinarians. There is an annual subscription fee.
2. The Consultant diagnostic support system for veterinary medicine <http://www.vet.cornell.edu/consultant/consult.asp> provides immediate access to brief synopses on diseases of species of veterinary importance. The CONSULTANT database consists of approximately 500 signs/symptoms, about 7,000 diagnoses/causes, and about 18,000 literature references including over 3000 web references. Each disease synopsis includes 1) a general description, 2) species affected, 3) the signs/symptoms associated with it, and 4) a list of recent literature references. Disease information can be accessed directly by searching on a diagnosis of interest, or indirectly by entering one or more signs and generating a list of compatible diseases. CONSULTANT also includes a link from each disease synopsis to relevant Web references and the National Library of Medicine's PubMed database of biomedical literature.
3. PubMed <http://www.ncbi.nlm.nih.gov/PubMed/>, a service of the National Library of Medicine, provides access to over 12 million MEDLINE citations back to the mid-1960's and additional life science journals. PubMed includes links to many sites providing full text articles and other related resources. The PubMed Web interface provides a powerful set of tools for retrieving just what you're looking for:
4. Basic Search-the user can simply enter relevant text into the search box and perform an unstructured search of the MEDLINE database. Boolean operators, AND, OR, NOT, e.g., canine OR dog, may also be used but must be entered in upper case. This is the easiest way to see what's available on a particular topic, especially if repeated searches with different word combinations are performed.
5. Related Articles-each citation in PubMed has a link that will retrieve a pre-calculated set of PubMed citations that are closely related to the selected article. Click on Related Articles to the right of each citation to display the related set of articles. PubMed creates this set by comparing words from the title, abstract, and MeSH (Medical Subject Heading) terms using a powerful word-weighted algorithm. Citations are displayed in rank order from most to least relevant with the linked from citation displayed first. This is a great way to find additional articles similar to one that may be of particular interest.
6. Single Citation Matcher-this is a fill-in-the-blank form that allows the user to enter any combination of journal citation information (journal, volume, year, first page, author's name, title words) and retrieve the complete citation including abstract (if available). This is useful for retrieving abstracts of articles cited in other sources, such as the disease synopsis page of Consultant.
7. Clinical Queries-This specialized search is intended for clinicians and has built-in search "filters" based largely on Haynes et al.3 Four study categories are provided (therapy, diagnosis, etiology, prognosis), and the emphasis may be more sensitive (i.e., most relevant articles but probably some less relevant ones) or more specific (i.e., mostly relevant articles but probably omitting a few).
"Signal to Noise Hierarchy" in Peer-Reviewed Clinical Research
A number of factors can influence the validity and relevance of peer-reviewed clinical research articles. First and foremost are the study design and number of subjects used in the study. In general validity tends to be highest and bias lowest in well-controlled studies with large numbers of subjects. The following is an approximate ranking of study designs from lowest to highest validity:
1. Case report
2. Case series
3. Retrospective studies (cross-sectional, case-control)
4. Uncontrolled clinical trial
5. Non-randomized controlled clinical trial
6. Randomized controlled clinical trial
Additional factors influencing the validity of research reports are discussed in a companion paper (VET 431-"Skills in Evaluating Online Information").
Summary
Evidence-based medicine (EBM) provides a disciplined approach to identifying and meeting information needs during case workup or management. The practice of EBM also promotes self directed lifelong learning. Online EBM evidence sources include discussion groups, diagnostic support systems, and the peer-reviewed medical literature. Sophisticated search algorithms greatly facilitate retrieval of information from medical bibliographic databases such as PubMed. The validity of online information varies greatly and requires that veterinarians learn to critically evaluate all medical claims.
References
1. Rosenberg W, Donald A. Evidence based medicine: an approach to clinical problem-solving. BMJ. 1995;310:1122-1126.
2. Sacket DL, Richardson WS, Rosenberg W, Haynes RB. Evidence-Based Medicine. How to Practice and Teach EBM. NY: Churchill Livingstone, 1997.
3. Haynes RB, Wilczynski N, McKibbon KA, Walker CJ, Sinclair JC. Developing optimal search strategies for detecting clinically sound studies in MEDLINE. J Am Med Inform Assoc. 1994;1:447-58
Speaker Information
(click the speaker's name to view other papers and abstracts submitted by this speaker)
Ronald D. Smith, DVM, PhD
Professor, Pathobiology
University of Illinois
Urbana, IL
- guest
Improving peer review: who's responsible?
by guest » Mon Mar 22, 2004 9:57 pm
Improving peer review: who's responsible?
Peer review needs recognition at every stage of scientific life
Peer review matters. Why? Firstly, scientific assertions can't be proved; they can only be disproved. The doubts raised by peer reviewers are therefore a crucial element in scientific reasoning. More specifically, as Francis Bacon put it in 1605, the "registering and posting of doubts has a double use:" it not only guards us "against errors," but also furthers the process of inquiry, causing issues that would otherwise be "passed by lightly without intervention" to be "attentively and carefully observed." Moreover, since scientific findings effectively don't exist until they're in written form, the doubts raised during editorial peer review come at a particularly crucial step in the overall scientific process.1 Secondly, the exchange of information for professional recognition is the principal instrument of social control within the scientific community.2 Approval by peer review is perhaps the single most powerful expression of that recognition. Thirdly, and more pragmatically, journal editors depend heavily on peer review to accomplish their two main tasks—selecting papers and improving their quality1 3—even though editors themselves are apparently the source of substantive improvements to manuscripts more often than either peer reviewers or statisticians.4
The quality of many manuscript reviews is excellent, but in many others it is, unfortunately, still far from optimal. Therefore editors of journals might understandably look for ways to improve reviewers' performance. The paper in this issue by Schroter et al reports on a carefully controlled randomised trial of educational interventions designed to do just that.5
Sadly the result is essentially negative: the small improvements found in some measures of review quality were judged to be "not of editorial significance," and faded over time. Sadly, this result is also highly predictable. The report provides few details, but the educational programmes seem to have been short, cognitively focused, and largely didactic (passive)—all features that decades of educational research tell us are relatively ineffective in producing meaningful changes in practice.6-8
Peer reviewing is after all an applied skill—like architecture, flying aeroplanes, or clinical medicine—a practice, rather than a purely cognitive attribute, that is, knowing a lot. All practitioners need to become good at "reflection in action," the mix of analytical, judging, and performing skills that is essential for handling the complex, unfamiliar, and ill formed problems they're called on to handle.9 And, as John Dewey and others have told us for the past 65 years, acquiring these applied skills requires "learning by doing," a process, sometimes referred to as "experiential learning,"10 that differs fundamentally from traditional cognitive (or "rationalist") classroom learning. Although the experiential learning "cycle"10 includes important cognitive elements (reflection, integration of experience with established explanatory models), the true curriculum in experiential learning is the experience itself11; it requires hands-on involvement by learners—fully, openly, and without bias—in the relevant tasks: a "practicum"9; and experiential learners need to receive their guidance from coaches, rather than absorbing information from lecturers.9
Whether by intention or not the present report provides strong evidence that experiential learning is important in peer review. It cites extended training in epidemiology and statistics—a hands-on, problem solving experience under expert guidance—as a known characteristic of high quality peer reviewers. And it makes us realise that most editors are themselves highly skilled reviewers. How do they get that way? They critically assess the quality of manuscripts, day after day, thousands of them over the years. And they receive repeated, timely, on-the-job feedback from a variety of coaches, including their editorial and statistician colleagues, not to mention peer reviewers. The difficulty that many peer reviewers have in producing high quality reviews shouldn't, therefore, surprise us, any more than the difficulty most lawyers would have in flying aeroplanes: most reviewers, chosen for their expertise in clinical or research areas, simply haven't had the requisite opportunities for experiential learning in critical assessment.12
So does this largely negative study help? Yes. It can discourage further use of precious time, energy, and funds for the kind of educational intervention that's unlikely to be effective. Further, it can focus attention on more productive research questions—for example: What's the actual learning curve for high quality peer review (in particular, how many reviews are required)? How does the length of time a reviewer spends on a review affect its quality? And what's the optimal mix of input from peer reviewers, editors, and statisticians in the overall editorial process?
Finally, the study suggests that editors, for all their strengths, can hardly be expected to fix the problems of peer review on their own, any more than schools can be expected to solve the problems of education singlehandedly. True, in the short term, editors might be more effective if they were to "train the trainers," rather than trying to train reviewers directly. But there is probably no quick fix here. Peer review is such a fundamental element of critical scientific thinking that the entire scientific and scholarly community should arguably take on the responsibility for improving and maintaining its quality—a major, long term commitment. Why isn't substantial, formal training in peer review an important and integral part of all graduate level training in basic science in clinical medicine? It should be. Moreover, serious consideration should be given to developing certification in peer review, and certified reviewers might be required to do a defined minimum number of reviews every year to maintain their credentials. And, finally, why isn't peer review considered worthy of serious academic recognition? It should be. Promotion, tenure, and funding decisions should take the quantity and quality of candidates' peer reviewing into account.
Much good can come out of this study if it serves as a wake up call. The message: the broader scientific and scholarly communities need to get serious about making high quality peer review an integral part of all aspects of professional life, including training, practice, and reward systems.
Frank Davidoff, editor emeritus, Annals of Internal Medicine
143 Garden Street, Wethersfield, CT 06109, USA (fdavidoff@earthlink.net)
Papers p 673
Competing interests: None declared.
References
Davidoff F. Editor's world. Ann Intern Med 2001;134: 1144-6.[Free Full Text]
Hagstrom WO. The scientific community. Carbondale and Edwardsville, IL: Southern Illinois University Press, 1965.
Jefferson T, Wager E, Davidoff F. Measuring the quality of editorial peer review. JAMA 2002;287: 2786-90.[Abstract/Free Full Text]
Purcell G, Donovan SL, Davidoff F. Changes in manuscripts and quality: The contribution of peer review [abstract]. Proceedings of the fourth international congress on peer review in biomedical publication. Barcelona, 2001: 31.
Schroter S, Black N, Evans S, Carpenter J, Godlee F, Smith R. Effects of training on quality of peer review: randomized controlled trial. BMJ 2004;328: 673-5.
Davis DA, Thomson MA, Oxman, AD, Haynes RB. Changing physician performance: a systematic review of the effect of continuing medical education strategies. JAMA 1995;274: 700-5.[Abstract]
Oxman AD, Thomson MA, Davis DA, Haynes RB. No magic bullets: a systematic review of 102 trials of intervention to improve professional practice. CMAJ 1995;153: 1423-31.[Abstract]
Mazmanian PE, Davis DA. Continuing medical education and the physician as a learner. Guide to the evidence. JAMA 2002;288: 1057-60.[Free Full Text]
Schön DA. Educating the reflective practitioner. Toward a new design for teaching and learning in the professions. San Francisco: Jossey-Bass, 1987.
Kolb DA. Experiential learning: experience as the source of learning and development. Englewood Cliffs, NJ: PTR Prentice Hall, 1984.
Ende J, Davidoff F. What is a curriculum? Ann Intern Med 1992;116: 1055-7.[ISI][Medline]
Davidoff F. Masking, blinding, and peer review: The blind leading the blinded. Ann Intern Med 1998;128: 66-8.[Free Full Text]
Peer review needs recognition at every stage of scientific life
Peer review matters. Why? Firstly, scientific assertions can't be proved; they can only be disproved. The doubts raised by peer reviewers are therefore a crucial element in scientific reasoning. More specifically, as Francis Bacon put it in 1605, the "registering and posting of doubts has a double use:" it not only guards us "against errors," but also furthers the process of inquiry, causing issues that would otherwise be "passed by lightly without intervention" to be "attentively and carefully observed." Moreover, since scientific findings effectively don't exist until they're in written form, the doubts raised during editorial peer review come at a particularly crucial step in the overall scientific process.1 Secondly, the exchange of information for professional recognition is the principal instrument of social control within the scientific community.2 Approval by peer review is perhaps the single most powerful expression of that recognition. Thirdly, and more pragmatically, journal editors depend heavily on peer review to accomplish their two main tasks—selecting papers and improving their quality1 3—even though editors themselves are apparently the source of substantive improvements to manuscripts more often than either peer reviewers or statisticians.4
The quality of many manuscript reviews is excellent, but in many others it is, unfortunately, still far from optimal. Therefore editors of journals might understandably look for ways to improve reviewers' performance. The paper in this issue by Schroter et al reports on a carefully controlled randomised trial of educational interventions designed to do just that.5
Sadly the result is essentially negative: the small improvements found in some measures of review quality were judged to be "not of editorial significance," and faded over time. Sadly, this result is also highly predictable. The report provides few details, but the educational programmes seem to have been short, cognitively focused, and largely didactic (passive)—all features that decades of educational research tell us are relatively ineffective in producing meaningful changes in practice.6-8
Peer reviewing is after all an applied skill—like architecture, flying aeroplanes, or clinical medicine—a practice, rather than a purely cognitive attribute, that is, knowing a lot. All practitioners need to become good at "reflection in action," the mix of analytical, judging, and performing skills that is essential for handling the complex, unfamiliar, and ill formed problems they're called on to handle.9 And, as John Dewey and others have told us for the past 65 years, acquiring these applied skills requires "learning by doing," a process, sometimes referred to as "experiential learning,"10 that differs fundamentally from traditional cognitive (or "rationalist") classroom learning. Although the experiential learning "cycle"10 includes important cognitive elements (reflection, integration of experience with established explanatory models), the true curriculum in experiential learning is the experience itself11; it requires hands-on involvement by learners—fully, openly, and without bias—in the relevant tasks: a "practicum"9; and experiential learners need to receive their guidance from coaches, rather than absorbing information from lecturers.9
Whether by intention or not the present report provides strong evidence that experiential learning is important in peer review. It cites extended training in epidemiology and statistics—a hands-on, problem solving experience under expert guidance—as a known characteristic of high quality peer reviewers. And it makes us realise that most editors are themselves highly skilled reviewers. How do they get that way? They critically assess the quality of manuscripts, day after day, thousands of them over the years. And they receive repeated, timely, on-the-job feedback from a variety of coaches, including their editorial and statistician colleagues, not to mention peer reviewers. The difficulty that many peer reviewers have in producing high quality reviews shouldn't, therefore, surprise us, any more than the difficulty most lawyers would have in flying aeroplanes: most reviewers, chosen for their expertise in clinical or research areas, simply haven't had the requisite opportunities for experiential learning in critical assessment.12
So does this largely negative study help? Yes. It can discourage further use of precious time, energy, and funds for the kind of educational intervention that's unlikely to be effective. Further, it can focus attention on more productive research questions—for example: What's the actual learning curve for high quality peer review (in particular, how many reviews are required)? How does the length of time a reviewer spends on a review affect its quality? And what's the optimal mix of input from peer reviewers, editors, and statisticians in the overall editorial process?
Finally, the study suggests that editors, for all their strengths, can hardly be expected to fix the problems of peer review on their own, any more than schools can be expected to solve the problems of education singlehandedly. True, in the short term, editors might be more effective if they were to "train the trainers," rather than trying to train reviewers directly. But there is probably no quick fix here. Peer review is such a fundamental element of critical scientific thinking that the entire scientific and scholarly community should arguably take on the responsibility for improving and maintaining its quality—a major, long term commitment. Why isn't substantial, formal training in peer review an important and integral part of all graduate level training in basic science in clinical medicine? It should be. Moreover, serious consideration should be given to developing certification in peer review, and certified reviewers might be required to do a defined minimum number of reviews every year to maintain their credentials. And, finally, why isn't peer review considered worthy of serious academic recognition? It should be. Promotion, tenure, and funding decisions should take the quantity and quality of candidates' peer reviewing into account.
Much good can come out of this study if it serves as a wake up call. The message: the broader scientific and scholarly communities need to get serious about making high quality peer review an integral part of all aspects of professional life, including training, practice, and reward systems.
Frank Davidoff, editor emeritus, Annals of Internal Medicine
143 Garden Street, Wethersfield, CT 06109, USA (fdavidoff@earthlink.net)
Papers p 673
Competing interests: None declared.
References
Davidoff F. Editor's world. Ann Intern Med 2001;134: 1144-6.[Free Full Text]
Hagstrom WO. The scientific community. Carbondale and Edwardsville, IL: Southern Illinois University Press, 1965.
Jefferson T, Wager E, Davidoff F. Measuring the quality of editorial peer review. JAMA 2002;287: 2786-90.[Abstract/Free Full Text]
Purcell G, Donovan SL, Davidoff F. Changes in manuscripts and quality: The contribution of peer review [abstract]. Proceedings of the fourth international congress on peer review in biomedical publication. Barcelona, 2001: 31.
Schroter S, Black N, Evans S, Carpenter J, Godlee F, Smith R. Effects of training on quality of peer review: randomized controlled trial. BMJ 2004;328: 673-5.
Davis DA, Thomson MA, Oxman, AD, Haynes RB. Changing physician performance: a systematic review of the effect of continuing medical education strategies. JAMA 1995;274: 700-5.[Abstract]
Oxman AD, Thomson MA, Davis DA, Haynes RB. No magic bullets: a systematic review of 102 trials of intervention to improve professional practice. CMAJ 1995;153: 1423-31.[Abstract]
Mazmanian PE, Davis DA. Continuing medical education and the physician as a learner. Guide to the evidence. JAMA 2002;288: 1057-60.[Free Full Text]
Schön DA. Educating the reflective practitioner. Toward a new design for teaching and learning in the professions. San Francisco: Jossey-Bass, 1987.
Kolb DA. Experiential learning: experience as the source of learning and development. Englewood Cliffs, NJ: PTR Prentice Hall, 1984.
Ende J, Davidoff F. What is a curriculum? Ann Intern Med 1992;116: 1055-7.[ISI][Medline]
Davidoff F. Masking, blinding, and peer review: The blind leading the blinded. Ann Intern Med 1998;128: 66-8.[Free Full Text]
- guest
Open Access could reduce cost of publishing by 30%
by guest » Thu May 06, 2004 8:45 am
Wellcome Trust say Open Access could reduce cost of scientific publishing by 30%
The Wellcome Trust, a major UK based research charity, published Costs and Business Models in Scientific Research Publishing last week, and the report gives strong backing to Open Access.
A press release announcing the report stated, "The Wellcome Trust report shows for the first time that the open access model of scientific publishing - where the author of a research paper pays for peer reviewed research to be made available on the web free to all who wish to use it - is economically viable, guarantees high quality research and is a sustainable option which could revolutionise the world of traditional scientific publishing."
Dr Mark Walport, Director of the Wellcome Trust said: "Up to now there have been unanswered questions about the economic and practical viability of this system. Our report now shows this is a win-win situation: high quality peer reviewed research available to everyone free of charge within a sustainable on-line market - plus savings of as much as 30 per cent."
The report is available online and will be passed to the Science and Technology Committee for use in their inquiry into scientific publications. For more information, read the press release.
--------------------------------------------------------------------------------
Why a growing number of authors choose BioMed Central
At BioMed Central, we believe in offering our authors the best possible publishing service. Every author who submits a manuscript to a BioMed Central journal is asked for the three most important reasons why they chose our journal over the hundreds of others that are available.
In 2003, the four reasons given most often for submitting manuscripts to BioMed Central journals were:
speed of peer review and publication
aspects of the journal including scope and reputation
our Open Access policies, and
the fact that our journals and submission process are online.
The Wellcome Trust, a major UK based research charity, published Costs and Business Models in Scientific Research Publishing last week, and the report gives strong backing to Open Access.
A press release announcing the report stated, "The Wellcome Trust report shows for the first time that the open access model of scientific publishing - where the author of a research paper pays for peer reviewed research to be made available on the web free to all who wish to use it - is economically viable, guarantees high quality research and is a sustainable option which could revolutionise the world of traditional scientific publishing."
Dr Mark Walport, Director of the Wellcome Trust said: "Up to now there have been unanswered questions about the economic and practical viability of this system. Our report now shows this is a win-win situation: high quality peer reviewed research available to everyone free of charge within a sustainable on-line market - plus savings of as much as 30 per cent."
The report is available online and will be passed to the Science and Technology Committee for use in their inquiry into scientific publications. For more information, read the press release.
--------------------------------------------------------------------------------
Why a growing number of authors choose BioMed Central
At BioMed Central, we believe in offering our authors the best possible publishing service. Every author who submits a manuscript to a BioMed Central journal is asked for the three most important reasons why they chose our journal over the hundreds of others that are available.
In 2003, the four reasons given most often for submitting manuscripts to BioMed Central journals were:
speed of peer review and publication
aspects of the journal including scope and reputation
our Open Access policies, and
the fact that our journals and submission process are online.
- guest
4 posts
• Page 1 of 1
Return to evidence based vet guidelines
Who is online
Users browsing this forum: No registered users and 1 guest