Browse through our Journals...

Publication bias by general psychiatry journals against addictive disorders

Abstract

Background: Publication bias and discrimination are increasingly recognised in medicine and has overwhelming ethical significance. Funding for medical research often depends on the capacity of researchers to publish in “high-impact” journals. Despite the tremendous impact of addictive disorders on mental health, anecdotal reports suggest that some general psychiatry journals selectively discriminate against original research reports from the addictions field. We tested this hypothesis based on the assumptions that each general psychiatry journal would receive the same proportion of articles on addictive disorders, deliberate self harm and psychiatry of learning disability.

Methods: A retrospective review of all the original data-based research reports was conducted for six high-impact “international” general psychiatry journals from 2000-2005.

Results: Addictive disorders was the principle theme of 322 out of 5110 research papers (6.3%). There was a statistically significant difference between the journals in the proportion of published research reports on addictive disorders (chi-square = 50.66; p < 0.0001) with the worst performing journal publishing only 19 papers (2.6%) (expected 47; odds ratio 0.35; confidence interval 0.21 to 0.55; two sided p < 0.0001). There was no significant difference between the journals in the proportion of published research reports on learning disability (Chi² = 4.75; 1% of all reports; range 0.76%-1.42%; p = 0.4469) or deliberate self-harm/suicide (Chi² = 10.06; 4.6% of all reports; range 3.77%-5.93%; p = 0.0734). Two other journals from the country whose journals published the lowest proportion of articles on addiction were examined but found no evidence to explain the results due to a regional lack of productivity in addictions research.

Conclusions: Some general psychiatry journals are selectively discriminating against research reports on addictive disorders. Alternatively, researchers in addictions are selectively boycotting general psychiatric journals.

Key words: Publication bias, substance use disorders, discrimination

Declaration: The work was funded entirely by the authors. There are no conflicts of interest.

Introduction

Despite being widely condemned by all professional bodies, discrimination is increasingly recognised in medicine, particularly in regard to racial groups and people with mental health problems (Sashidharan, 2001; Crisp et al, 2000). Some reports have also shown potential discrimination by psychiatric journals against research performed from developing countries (Patel & Kim, 2007). For example only 3.7% of published research emerges from these less affluent countries, European journals were twice as likely to publish these articles than American journals while articles from less developed countries were more likely to be rejected than those from high-income countries.

Stigma is a social construction that devalues people as a result of a distinguishing characteristic or mark (Biernat & Dovido, 2000). The World Health Organisation and World Psychiatric Association both recognize that the stigma and discrimination attached to mental disorders are strongly associated with suffering, disability and poverty (Corrigan PW & Watson AC, 2002). Crisp et al’s large survey (2005) for the Changing Minds Campaign showed that people with alcoholism and drug addiction are the most stigmatised group of all those with mental illness. Although it has been explicitly stated that discrimination “seems to exist in every area of life” particularly for those suffering with drug dependence there are no reports of discrimination by professional journals against people with these disorders or researcher who work with them (Byrne, 2000).

Publication bias is a widely recognised phenomenon whereby positive results have an increased likelihood of being published and are published in more influential journals (Chalmers 1990; Easterbrook et al, 1991). Publication bias distorts conclusions of meta-analysis and leads to a misleading impression of the effectiveness of some treatments (Egger & Smith, 1995). Even more seriously, funding for medical research depends on either industrial sponsorship, which has been the subject of increasing concern, or national funding bodies such as the UK Medical Research Council and Wellcome Trust (Bekelman et al 2003). Many of these organisation base funding decisions on the publication record of the individual researchers and those of their host institutions (Hobbs & Stewart, 2006). Research funding is more likely for authors of research articles published in “high-impact” journals. The ethical implications of discrimination by medical journals is tremendous. For example, if high-impact medical journals are discriminating against research from particularly disciplines, this will reduce the likelihood of further research funding and impede improvements in treatment. It will also discourage the more able researchers from entering these disciplines.

There is overwhelming evidence of the impact of addictive disorders on mental health and their importance to practitioners in general psychiatry (Regier et al, 1990; Kessler et al, 1997; Weaver et al, 1999; UK Department of Health, 2002). However anecdotal reports from other researchers have suggested that some high-impact “international” journals, that claim to represent a broad interest in psychiatry, are selectively discriminating against original research reports from the addictions field. This has been reported in respect to eating disorders in the past (Frost et al, 2003). We therefore examined the proportion of original data-based research reports related to addictive disorder from various journals and compared this to the proportion of reports on deliberate self harm and psychiatry of learning disability.

Method

A retrospective review of all the original data-based research reports (including brief reports) published by six of the most high impact international academic journals of general psychiatry was conducted using citation impact factors based on the reports of Altman et al (2003) and Frost et al (2003) (Table 1). This included one journal to represent researchers from the Southern Hemisphere. Papers related to addictive disorders were defined as those original scientific reports whose principle aim was to report the non-therapeutic effects of addictive drugs and those papers who principle aim was to report results relevant to the aetiology, diagnosis and management of people with substance use disorders including co-morbidity with other mental disorders and tobacco smoking. Papers on gambling and other “behavioural addictions” were also included. Published reports over the six year period were also examined to determine the proportion whose principle subject was relevant to the sub-speciality of learning disability and those reports who principle subject concerned suicide or deliberate self-harm.

Results

Overall substance misuse was the principle theme of 322 out of 5110 research papers (6.3%) for the six general psychiatry journals over the 6 year period. The results showed a statistically significant difference over the 6 year period in the proportion of substance misuse papers published between these journals (chi-square = 50.66 with 5 degrees of freedom; p < 0.0001). Overall the worst performing journal published only 19 of 740 (2.5%) research papers on substance misuse over the 6 year period (expected 47; odds ratio 0.35; confidence interval 0.21 to 0.55; two sided p < 0.0001). It published only 1 out of 123 original research papers in 2004. This journal consistently published a lower proportion of original research papers on substance misuse than any of the other journals for five of the six years included in the study.

Table 1. Proportion of original research papers related to substance use disorders published in selected journals 2000-2005

Journal	2005	2004	2003	2002	2001	2000
BJPsych	4/143 2.8%	1/123 0.8%	5/101 5%	3/101 3%	4/117 3.4%	2/136 1.5%
Am J Psych	14/272 5.1%	24/260 9.2%	27/265 10.2%	15/258 5.8%	25/256 9.7%	17/268 6.3%
Psychol Med	6/148 4.0%	12/132 9.1%	9/126 7.1%	5/116 4.3%	2/130 1.5%	3/126 2.3%
Acta Psych Sc	6/81 7.4%	3/77 4.0%	5/96 5.2%	5/107 4.7%	5/114 4.4%	8/118 6.7%
Archives	20/122 16.1%	15/106 14.2%	14/114 12.3%	11/93 11.8%	12/104 11.5%	5/92 5.4%
Aust NZ J Psych	8/97 8.2%	6/94 6.4%	5/67 7.4%	6/73 8.2%	4/72 5.5%	6/83 7.2%
BMJ	14/250 5%	12/260 4.6%	15/234 6.4%	10/242 4.1%	9/234 3.8%	11/294 3.7%
Psychiatric Bulletin	4/48 8.3%	3/47 6.4%	5/56 8.9%	5/51 9.8%	6/62 4.8%	1/55 1.8%

Journals cited were:

BJPsych - British Journal of Psychiatry
Am J Psych - American Journal of Psychiatry
Psychol Med - Psychological Medicine
Acta Psych Sc - Acta Psychiatrica Scandinavica
Archives - Archives of General Psychiatry
Aust NZ J Psych - Australia and New Zealand Journal of Psychiatry
BMJ - British Medical Journal

There was no significant difference between the journals in the proportion of published research reports on learning disability (Chi² = 4.75; 1% of all reports; range 0.76%-1.42%; p = 0.4469) or deliberate self-harm/suicide (Chi² = 10.06; 4.6% of all reports; range 3.77%-5.93%; p = 0.0734).

The editorial boards of each of the journal under review claimed to be “international” presumable implying that articles originating from different countries would be assessed according to the same criteria. However it is widely believed that editorial boards will favour articles that are of interest to their local readership (Patel & Sumathipala, 2001). The lowest publication rates for substance use disorders were observed in general psychiatry journals based in Britain (The British Journal of Psychiatry). In order to compensate for local bias we also examined the publication patterns of the British Medical Journal (BMJ; a general medical journal with a much wider readership than mental health professionals). We were surprised to find that substance use disorders were represented at a higher rate in absolute and relative terms (4.3%) in the BMJ than the worst performing general psychiatry journal even though the BMJ’s impact factor was higher. We also looked at a Psychiatric Bulletin, a general psychiatry journal originating from the same organisation as the British Journal of Psychiatry. Once again Psychiatric Bulletin published a greater proportion of articles on substance misuse (7.2%) than the British Journal of Psychiatry.

Discussion

One explanation for the great variation in publication of addiction research between the journals is that clinical researchers in Britain are less interested in studying addictive disorders than those overseas, especially in the US. To address this we examined the proportion of papers published by two British Journals, the British Medical Journal and Psychiatric Bulletin. It was apparent that there were a significant higher proportion of papers published in both these journals than the worst performing general psychiatry journal in the study. This indicates that there is an active addictions research community in Britain who produce research of a sufficiently high standard to merit publication.

We assumed that each general psychiatry journal would receive the same proportion of articles on each sub-speciality. We were unable to confirm this as we do not have access to the details of papers rejected by the various journals under study. Unfortunately the suggestion that previously rejected submissions be analysed is quite impractical. Firstly, the additional work load would be too onerous for the staff of many medical journals. Furthermore the records of submissions are often erased after rejection of the articles. Finally medical journals often regard the submission process as confidential and do not permit outsiders access to this information. However we examined the proportion of published reports for two other psychiatric sub-specialities. There was no significant difference between the journals in the proportion of research reports that were relevant to the psychiatry of learning disability (approximately 1%) or suicide-deliberate self-harm (approximately 5%). This supports the supposition that submission of sub-speciality research are comparable between the general psychiatry journals. (However there was a significant difference in the total proportion of reports published with reports on learning disability consistently under-represented in all journals. Presumably this represents the severe shortage of research funding for this sub-speciality and also the extremely problems in obtaining ethical approval and informed consent for research on this patient group.)

There remain two equally unpalatable explanations. Firstly, some general psychiatry journals are selectively discriminating against research papers on addictive disorders. Secondly, researchers in addictions are selectively boycotting general psychiatric journals in Britain. We are not aware of any support for the second explanation either anecdotally or in published comments. Selective boycotting of some general psychiatry journals by addiction researchers in unlikely to explain why one British journal apparently received only one substance misuse paper suitable for publication in 2004 while Psychological Medicine and the British Medical Journal published 12 each. Furthermore each of the general psychiatric journals in the study had a higher impact factor than any specialist journal in the addictions field and would therefore be a more attractive option for researchers in substance misuse. Unfortunately the current peer review process and editorial conduct has recently come under severe criticism from a former editor of the British Medical Journal (Smith 2006). While some commentators have dismissed these comments an “outrage”, these results again cast doubt on the integrity of the editorial process (Finch 2006).

Altman ( 2005) states “Selective underreporting of research might be more widespread and more likely to have adverse consequences for patients than publication of deliberately falsified data”. If some high impact psychiatry journals are actively discriminating against researchers in addictive disorders, whether this is intentional or not, this unfairly inhibits the careers of researchers in this field and also reduces the opportunity to obtain further research funding. Ultimately this stifles research in these sub-specialities and will damage patient care. We fully support Chalmer’s claim (1990) that, “Underreporting research is scientific misconduct”. The ethical dimensions of this problem are obvious.

AUTHORS

Dr Jason Luty PhD MRCPsych
Consultant in Addictions Psychiatry. South Essex Partnership NHS Trust
Honorary Consultant in Addictions Psychiatry, Cambridge & Peterborough Mental Health Partnership NHS Trust
The Taylor Centre
Queensway House
Essex Street
Southend on Sea
Essex SS4 1RB

Dr Harish Rao MBBS; MD; MRCPsych.
Specialist Registrar in Psychiatry
Peter Bruff Unit
Clacton and District Hospital
Tower Road
Clacton on Sea CO15 1LH

Dr Prabhavathy Sethi MB BS MRCPsych
Registrar in Psychiatry
South Essex Partnership NHS Trust
Runwell Hospital
Runwell Chase
Wickford
Essex

Dr Rofique Ali BSc, BM
Registrar in Psychiatry
Basildon & Thurrock University Hospitals NHS Trust
Basildon Hospital
Nethermayne
Basildon
SS16 5NL

REFERENCES

Altman D.G. (2005). Endorsement of the CONSORT statement by high impact medical journals: survey of instructions for authors. British Medical Journal 330:1056-1057

Bekelman JE, Li Y, Gross CP. (2003) Scope and impact of financial conflicts of interest in biomedical research: a systematic review. Journal of the American Medical Association 289: 454-65

Biernat, M. & Dovidio, J.F. (2000) Stigma and stereotypes. In The Social Psychology of Stigma (eds T.F. Heatherton, R.E. Kleck, & M.R. Hebl), pp 88-125. New York: Guilford Press.

Chalmers I. (1990) Underreporting research is scientific misconduct. Journal of the American Medical Association 263: 1405-8.

Corrigan, P.W. & Watson, A.C. (2002). The impact of stigma on people with mental illness. World Psychiatry 1: 16-19.

Crisp, A. H., Gelder, G., & Rix, G. (2000) Stigmatisation of people with mental illness. British Journal of Psychiatry, 177, 4-7.

Department of Health (2002). Mental Health Policy Implementation Guide: Dual Diagnosis Good Practice Guides. London: Department of Health.

Docherty, J.P. (1997) Barriers to the diagnosis of depression in primary care. Journal of Clinical Psychiatry, 58, 5-10.

Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR. (1991) Publication bias in clinical research. Lancet 337: 867-72

Egger M, Smith GD. (1995) Misleading meta-analysis. British Medical Journal 310: 752-4

Finch R (2006). Medical Journals accused of bias. Hospital Doctor 28 October 2006.

Frost S, Murphy R, Webster P & Schmidt U (2003). Are top journals biased against eating disorders topics? American Journal of Psychiatry 160: 363-365.

Hobbs FD & Stewart PM (2006). How should we rate research? British Medical Journal 332: 983-984.

Kessler R, Crum RM, Warner LA (1997). Lifetime co-occurrence of DSM-III R alcohol abuse and dependence with other psychiatric disorders in the National Comorbidity Study. Archives of General Psychiatry 54: 313-321.

Patel V & Sumathipala A (2001). International representation in psychiatric literature
Survey of six leading journals. British Journal of Psychiatry 178: 406-409.

Patel V & Youl-Rim K (2007) Contribution of low- and middle-income countries to research published in leading general psychiatry journals. 2002-2004. British Journal of Psychiatry 190: 77-78.

Sashidharan SP (2001) Institutional racism in British psychiatry. Psychiatric Bulletin 25: 244-247

Smith R (2006) The trouble with medical journals. London: Pavillion Press.

Regier DA, Farmer ME, Rae DS, Locke BZ, Keith SJ & Judd LL (1990). Co-morbidity of mental disorder with alcohol and other drug abuse: results from an epidemiological catchment area (ECA) study. Journal of the American Medical Association 264: 2511-2518.

Weaver T, Renton A, Stimson G & Tyrer P (1999) Severe mental illness and substance misuse. British Medical Journal 318: 137-138.

First Published June 2007

Click on these links to visit our Journals:
Psychiatry On-Line
Dentistry On-Line | Vet On-Line | Chest Medicine On-Line
GP On-Line | Pharmacy On-Line | Anaesthesia On-Line | Medicine On-Line
Family Medical Practice On-Line

Home • Journals • Search • Rules for Authors • Submit a Paper • Sponsor us