|Year : 2017 | Volume
| Issue : 2 | Page : 73-78
Comparative Analysis of Study Design and Statistical Test Utilization in Indian Journal of Community Medicine, Indian Journal of Public Health and Bulletin of the World Health Organization
Sandeep Sachdeva, Nidhi Dwivedi
Department of Community Medicine, North DMC Medical College and Hindu Rao Hospital, New Delhi, India
|Date of Web Publication||28-Jun-2017|
Department of Community Medicine, North DMC Medical College and Hindu Rao Hospital, New Delhi-110007
Source of Support: None, Conflict of Interest: None
Objective: To analyze study topic, study design, usage of statistical test, and other selected document parameters in published original research articles. Materials and Methods: Three journals (1) Indian Journal of Community Medicine (IJCM), (2) Indian Journal of Public Health (IJPH), and (3) Bulletin of World Health Organization (WHO) were reviewed and all original research article published during three years, that is, January 2014 to December 2016, were considered, and selected document parameters were recorded using checklist. During this period, IJCM released volumes 39 to 41, IJPH released 58 to 60, and WHO released 92 to 94. Results: A total of 318 original articles were reviewed with the contribution of 28.9% in IJCM, 27.6% in IJPH, and 43.3% in WHO bulletin. Out of all the publications, highest (46.5%) original article belonged to research domain category-I [communicable, maternal and child health (MCH)] followed by 104 (32.7%) category-II (noncommunicable diseases) and 66 (20.8%) by category-III (health system, medical education, and environment). Overall 79.2% original articles were cross-sectional in study design followed by cohort (15.4%), intervention/experimental study (3.4%), and case-control (1.8%). Average number of authors per article in WHO bulletin was 8.1 [95% confidence interval (CI): 7.1–9.0] followed by IJCM 4.2 (95% CI: 3.8–4.6), and IJPH (3.84, 95% CI: 3.49–4.19). There were 95 (29.9%) articles where-in all the authors were from a single institution, whereas in 223 (70.1%) articles, co-authors were from different institutions. It was also found that most (80.1%) of the study papers did not contain any background statement regarding statistical sample size estimation; 239 (75.2%) articles used statistical software; majority had utilized SPSS (61.08%), followed by STATA (19.24%), SAS (8.76%), R-software (5.45%), EPI-info (3.76%), and EXCEL (1.25%); nearly 210 (66%) original research articles utilized at least one of the inferential statistics (basic, advance, and survival analytical methods). Higher proportion of advance (60.2%) inferential statistical test and survival analysis (92.3%) were used in Bulletin of WHO; 82% of references used were within 10 years of publications of articles in WHO while this figure was 66% for ICJM and 63% for IJPH. Conclusion: This study provides an objective snapshot of national and international public health journal that may aid researcher for better comprehension, situational, and trend analysis as well as assist them in their future research endeavor.
Keywords: Author mapping, bibliometric evaluation, citation, document properties, developing countries, global burden of disease, research publications, statistical test, study-design, study topic, statistical software
|How to cite this article:|
Sachdeva S, Dwivedi N. Comparative Analysis of Study Design and Statistical Test Utilization in Indian Journal of Community Medicine, Indian Journal of Public Health and Bulletin of the World Health Organization. MAMC J Med Sci 2017;3:73-8
|How to cite this URL:|
Sachdeva S, Dwivedi N. Comparative Analysis of Study Design and Statistical Test Utilization in Indian Journal of Community Medicine, Indian Journal of Public Health and Bulletin of the World Health Organization. MAMC J Med Sci [serial online] 2017 [cited 2017 Jul 21];3:73-8. Available from: http://www.mamcjms.in/text.asp?2017/3/2/73/209016
| Introduction|| |
The conveyance of information by research publication is responsible for growth and development of the subject area, journals as well as authors and readers. This further provides new insights and inspiration for new researchers to work on the same or related fields. Role of scientific journal ranges from building a collective knowledge base, communicating information, validating the quality of research, distributing rewards, and building scientific communities across the globe.,, During last decade there has been increased thrust being given to research in the developing countries, especially India due to various internal and external driving forces. Interalia there has been rapid expansion of research publications and appearance of new journals with a considerable fear of diluting the quality of research standards. An attempt has been made in the present study to undertake selective bibliometric evaluation of original research articles published in reputed national and international public health journals.
Bibliometrics are quantitative technique of measuring output, mapping pattern, emerging trend and obsolesce of publications, authorship, citation impact, and use of literature. It encompasses measurement of properties of documents and document related processes., However, surveying peer-reviewed medical journals for their contents in terms of study design and statistical methods is not an uncommon practice.,, This review is expected to serve as an objective approach for improving the quantitative and qualitative content of journal and learning amongst readers as there are very limited studies available to throw light on this dimension.
Indian Journal of Community Medicine (IJCM) and Indian Journal of Public Health (IJPH) are peer-reviewed, historically trusted, widely acknowledged quarterly English language publications in the domain of community medicine and public health managed by two independent professional bodies in the country. Both these free to access journals are indexed in large number of database and earliest records of IJPH and IJCM available in PubMed dates back to year 1961 and 1974 respectively. IJCM is available online (www.ijcm.org.in) since September 15, 2007 and IJPH is available online (www.ijph.in) since September 25, 2010. For comparison purpose, we also evaluated international journal of public health, Bulletin of the World Health Organization (WHO) (available in PubMed since 1948) on the same parameters. Both IJCM and IJPH publish peer reviewed quarterly (four issues per year) while bulletin of the WHO is published monthly (12 issues per year). SCImago Journal Rank of IJCM was 0.62 with 1.14 cites/document; IJPH was 0.44 with 0.83 cites and Bulletin of the WHO was 2.82 with 4.97 cites/document.
| Materials and Methods|| |
Three journals (1) IJCM, (2) IJPH, and (3) Bulletin of World Health Organization (WHO) journal were reviewed. All original research published either as full-length paper or brief/short communication during last three calendar years that is, January 2014 to December 2016 were considered, downloaded from the respective website, and reviewed by the authors. Thus editorial, personal view-point, commentary, oration, review article, student research section, field report, essay, news or letters to editors were excluded from the present analysis. During this period, IJCM released volumes 39 to 41; IJPH released 58 to 60 and WHO released 92 to 94.
For every original research article, following document parameters were recorded using check-list: thematic topic of research domain, study design, statistical methods, software, inputs for proper calculation of sample size, number of authors, department, and institution of first/corresponding authors, number of pages and references used. The research domain was classified and adapted according to global burden of disease categories: category-I = communicable, maternal, perinatal, congenital, food-borne, vaccine coverage, and nutritional disorders; category-II = noncommunicable diseases, mental health, associated risk factors, geriatrics, drug abuse, injuries, burn, occupational diseases, domestic violence, and oral health; and category-III = miscellaneous (health systems research, medical education, environment, and information technology).
Papers containing statistical analysis beyond descriptive statistics (percentage, mean, median, and standard deviation) were classified into three categories of inferential statistics—basic, advance, and survival analysis. We utilized SPSS ver. 16 (IBM, Armonk, New York, USA) to summarize our study findings by descriptive statistics and calculating chi-square value.
| Results|| |
A total of 318 original articles were reviewed with the contribution of 28.9% in IJCM, 27.6% in IJPH and 43.3% in WHO bulletin with 256 (80.5%) original research articles being full-length papers while rest (62, 19.5) were brief/shorter format of original research articles.
[Table 1] and [Figure 1] depict thematic study domain of published original research articles. Out of all the publications, highest (46.5%) original article, belonged to research domain category-I (communicable, MCH etc.) followed by 104 (32.7%) category-II (noncommunicable diseases etc.) and 66 (20.8%) by category-III (health system, medical education, environment, etc.). Highest research articles with category-I (communicable diseases, MCH etc.) domain was published in bulletin of WHO (70, 47.3%) followed by IJCM (41, 27.7%) and IJPH (37, 25.0%). Amongst noncommunicable category, both the Indian journals had similar but slightly higher publications than WHO while category-III research domain was dominated in WHO bulletin.
|Table 1: Broad thematic domain of original research publications (2014–2016)|
Click here to view
|Figure 1: Research domain of published original articles according to journal (2014–16), category-I (communicable, MCH etc), category-II (noncommunicable diseases etc), and category-III (health system, medical education, and environment).|
Click here to view
The study design of 79.2% original research articles was cross-sectional in nature followed by cohort (15.4%), intervention/experimental study (3.4%) and case-control (1.8%). Details are shown in [Table 2] and [Figure 2].
|Table 2: Distribution of study design in original research articles (2014–2016)|
Click here to view
|Figure 2: Distribution of published original research articles by study design|
Click here to view
Average (mean) number of authors per article in WHO bulletin was higher with 8.1 (95% CI: 7.1–9.0) followed similarly by IJCM 4.2 (95% CI: 3.8–4.6), and IJPH (3.84, 95% CI: 3.49–4.19). There were 95 (29.9%) articles which reported all the authors from a single institution and 223 (70.1%) articles where-in authors were from more than two institutions. Is it a true indication of collaborative effort or reflection of gifted authorship, difficult to infer. Highest articles with multiinstitutional associated authors were reported in bulletin of WHO (93.4%) with similar representation in both Indian journals (50.0% each). Overall, 168 (52.8%) authors were associated with medical colleges, 18 (5.7%) were associated with hospitals other than medical colleges, 29 (9.1%) were involved with research institute, and 103 (32.4%) had professional linkage with other miscellaneous organization. Amongst all publications in Indian journals, there were 10 (5.8%) research articles where-in there was international coauthorship/collaboration. Selected details are shown in [Table 3]. Highest number of research publications with a source of funding was published in bulletin of WHO (98, 71.01%).
|Table 3: Selected characteristic of published original research manuscript (2014–2016)|
Click here to view
There were 239 (75.2%) research articles that had utilized statistical software while 79 (24.8%) articles used none. Out of all the research articles using statistical software, majority had utilized SPSS (61.08%), followed by STATA (19.24%), SAS (8.76%), R-software (5.45%), EPI-info (3.76%), and EXCEL (1.25%).
Average (mean) number of references used were 35.4 (95% CI: 32.91–38.01) in WHO bulletin while it was nearly similar in both Indian journals (IJCM = 20.2 and IJPH = 17.8). However, utilization of more recent (within 10 years of publications) references was reported in WHO bulletin than the Indian journals ([Table 3]). This might be suggestive of stringent quality parameters, such as focus of journal on advancement of knowledge base, build upon the current evidence and transmission of latest information to the readers, and/or updated extensive literature review by researchers.
[Table 4] depicts range of statistical tests utilized. It was observed that all articles used descriptive statistics (percentage, mean, median, and standard deviation) as per need. On further scrutiny, 210 (66%) original research articles utilized at least one of the inferential statistics (basic, advance, and survival analytical) methods, whereas 108 (34%) articles used none, including policy-level research papers. The number of statistical method exceeds the total number of articles since some of them reported more than one statistical method; 92 (28.9%) articles used single statistical method while 118 (37.1%) articles reported two or more methods. Most commonly utilized statistical method was chi-square test (52.8%) followed by multivariate logistic regression (30%), and z test (23.3%) and so on.
|Table 4: Type and frequency of inferential statistical methods used in all published original articles (2014–2016)|
Click here to view
[Table 5] depicts the usage pattern of inferential statistics. Out of all the inferential statistical methods employed, highest (352, 67.56%) were basic test followed by advance tests (143, 27.44%) and only 26 (4.99%) had utilized survival analysis. Amongst basic statistical methods, 42.7% were applied in WHO bulletin followed by IJCM (30.8%) and IJPH (26.5%), respectively. Higher proportion of advance tests (60.2%) and survival analysis (92.3%) were used in Bulletin of the WHO.
| Discussion|| |
This study, probably first of its kind, carried out comprehensively and simultaneously on Indian and international public health journals revealed interesting findings that may aid a scholar to uptake high-quality research work with more advance statistical methods in future albeit that would require capacity building. Currently, many of the document properties such as mapping and others are being undertaken by publication houses with increasing usage of newer information technology tools and process in the form of bibliometrics, almetrics, and other measurement indicators. However, this study is unique considering that learning emerging from peer-review of content evaluation would still hold its superiority and importance over artificial intelligence.
The quality of publications in both Indian journals has evolved for better since last couple of years. Both the Indian community medicine (I = 44.5%, II = 40.2%, and III = 15.2%) and public health (I = 42.4%, II = 40.9%, and III = 17.04%) journal have published similar proportion of original articles under broad three-research categories. However, bulletin of the WHO published category-I = 50.7%, category-II = 22.4%, and category-III = 26.8%, reflecting the higher burden of communicable disease/maternal and child health issues amongst substantial population of developing and under-developed countries of the world.
Another study (2004–2009) carried out on four Indian journals also depicted that 43% of all published articles were on noncommunicable diseases (category-II). It was also reported that IJCM published about 30% of articles on non-communicable diseases (NCDs). In our study (2014–2016), higher proportion (40.2%) of articles on NCDs were published reflecting the time evolution and priority being received by NCDs in the recent times. Since developing country like India is undergoing epidemiological transition and experiencing double burden of disease, both the Indian journals seems to be adequately reflecting the ground-realities.
Considering the feasibility and applicability perspective, cross-sectional studies are easy to conduct, and hence the predominance (79.2%) of such designs in public health research articles as noted in the present analysis. This finding seems to corroborate with other studies performed amongst international primary care, general, community medicine, or public health journal too. There are limited studies available on public health journals for comparison purpose, especially in Indian setting. However, this is in contrast to another study evaluating predominantly 14 Indian clinical journals wherein 60.8% research articles were prospective study and least common was cross-sectional study (6.2%) design. McDermott, way back in 1991, found that 35.0% papers published in JAMA, The Lancet, and New England Journal of Medicine were clinical trials. Another recent study carried on these three clinical journals for the period (2005–2009) reflected randomized clinical trial (30.9%), clinical trial (3.7%), cohort and case-control studies (19.2%), cross-sectional studies (2.4%), and case reports (43.8%).
In our analysis, there was no article with ambiguous research design in any of the three journals that is suggestive of good-quality research/write-up and/or editorial screening decision. This is in contrast to Pakistan Journal of Medical Sciences (2005–2015), where nonclarity of research design ranged from 5 to 16%.
An interesting study assessing the quality of reporting of cross-sectional studies (2010–2011) in IJCM described that 58% of the articles reported less than 15 items of the desirable 22 in the Strengthening the Reporting of Observational Studies in Epidemiology statement checklist. This information is needed to give readers a clear idea as to what was planned and what was done. Similarly, in our study, it was observed that most (80.1%) of the study papers did not contain any background statement regarding statistical sample size estimation, which is very crucial to any efficient study design. Another important aspect noted was that 134 (42.1%) articles did not report clear inclusion/exclusion criteria for the sample recruitment. One of the probable reasons for this could be the word-count restriction imposed by journals in manuscript.
Rigby et al. found that articles reporting no statistics constituted 29.1, 32.4, and 40.7% studies in the British Medical Journal, British Journal of General Practice, and Family Practice, respectively. Nonutilization of statistics in our analysis was 34%. On a corollary, 66% of articles had utilized some statistical test that is in contrast as reported in study published in Journal of Family and Community Medicine (71%) and Chinese Medical Journal (78.1%).,
Majority of health journals prescribe Vancouver style of reference formatting with increasing focus on recent journal publications as references. Number of references/article “allowed” is directly related to type of manuscript, strength/detail of write-up, word-limit restriction, and/or number of pages per articles and should be visualized accordingly. The Indian journals prescribe limits of 6 authors with 30 references in full-length original articles and up to 6 authors with 10 references in short/brief research article. In our analysis, mean number of references used were 20.2 (95% CI: 18.3–22.1) in IJCM and 17.8 (95% CI: 16.1–19.5).
Both the Indian journals have evolved for better in terms of quality of research and publications, yet the difference observed on review of literature could perhaps be due to specialty (clinical vs. public health journal). The scope of publication in Indian journals could be enhanced with young public health researchers exploring clinical epidemiology in future. Both Indian journals have demonstrated certain strength and similarities in present study and discussed by authors elsewhere also., Introspection and constructive criticism when taken in positive perspective facilitates growth and development leading to learning, expansion, and retention of journal users in long run.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Momen H. The role of journals in enhancing health research in developing countries. Bull World Health Organ 2004;82:163.
Ding Y, Foo S, Chowdhury G. A Bibliometric analysis of collaboration in the field of information retrieval. Int Inf Lib Review 1999;30:367-76.
Doloon DJ. The role of peer review for scholarly journals in the information age. J Electron Publ 2007;10:1-10.
Borgman CL, Furner J. Scholarly communication and bibliometrics. Annul Rev Inform Scie Tech 2002;36:3-72
Emerson JD, Colditz GA. Use of statistical analysis in the New England Journal of Medicine. N Engl J Med 1983;309:709-13.
Thomas T, Fahey T, Somerset M. The content and methodology of research papers published in three United Kingdom Primary Care Journals. Br J Gen Pract 1998;48:1229-32.
Wang Q, Zhang BH. Research design and statistical methods in Chinese Medical Journals. JAMA 1998;280:283-85.
Mathers CD, Lopez AD, Murray CJL, editors. The burden of disease and mortality by condition: data, methods, and results for2001. Global Burden of Disease and Risk Factors. New York: Oxford University Press; 2006.
Mony PK, Srinivasan K. A bibliometric analysis of published non-communicable disease research in India. Ind J Med Res 2011;134:232-4.
Aljoudi AS. Study designs and statistical methods in the Journal of Family and Community Medicine 1994–2010. J Fam Community Med 2013;20:8-11.
] [Full text]
Jaykaran X, Solanki P, Saxena D, Kantharia ND. Study design and statistical methods in Indian Medical Journals. J Pharm Negative Results 2011;2:35-8.
McDermott MM, Lefevre F, Feinglass J, Reifler D, Dolan N. Changes in study design, gender issues, and other characteristics of clinical research published in three major medical journals from1971 to 1991. J Gen Intern Med 1995;10:13-8.
von Ballmoos MCW, Ware JH, Haring B. Clinical research Quo Vadis? Trends in reporting of clinical trials and observational study designs over two decades. J Clin Med Res 2015;7:428-34.
Akhtar S, Shah SWA, Rafiq M, Khan A. Research design and statistical methods in Pakistan Journal of Medical Sciences. Pak J Med Sci 2016;32:151-4.
Jeelani A, Malik WR, Haq I, Aleem S, Mujtaba M, Syed N. Cross-sectional studies published in Indian Journal of Community Medicine: evaluation of adherence to strengthening the reporting of observational studies in epidemiology statement. Ann Med Health Sci Res 2014;4:875-8. [Full text]
Rigby AS, Armstrong GK, Campbell MJ, Summerton N. A survey of statistics in three UK general practice journal. BMC Med Res Methodol 2004;4:28.
Jin Z, Yu D, Zhang L, Meng H, Lu J, Hong M et al.
A retrospective survey of research design and statistical analyses in selected Chinese Medical Journals in 1998 and 2008. PLoS One 2010;5:e10822. doi:10.1371/journal.pone.0010822
Dwivedi N, Sachdeva S. Forecasting visitor accession trend of two prominent Indian health journal websites for the period 2015–2020 using time-series analysis. Digit Med 2016;2:57-63. [Full text]
Sachdeva S, Sachdev TR, Sachdeva R. Bibliometric evaluation of published Indian research on “knowledge, attitude, and practices”. Indian J Health Sci Biomed Res (KLEU) 2017;10:208.
[Figure 1], [Figure 2]
[Table 1], [Table 2], [Table 3], [Table 4], [Table 5]