Adopting and Promoting the New TOP Study Design Guidelines as a University

Adopting and Promoting the New TOP Study Design Guidelines as a University

By Dr. Michael Solomon and Dr. James Gillespie

Are you tuned in to the debate surrounding the increase in studies retracted due to poor design? What about the growing chorus in the scholarly community for a forceful and focused response to scientific misconduct? For example, these recommendations for advancing solutions to improve research  from PsycNet.apa.org or this article reviewing the literature in promoting research integrity as it applies to nursing.

This issue rattles scholarly research at its core and reinforces skeptics’ views; intensifying the gap of misunderstanding and fear. It’s also an opportunity for the University’s Research Centers to lead the way in supporting innovative, rigorous, and transparent research.

The Transparency and Openness Promotion (TOP) guidelines recently published in the journal Science provide a systematic framework for achieving this goal. Two of the eight “transparency standards” are already baked into guidelines of most peer-reviewed journals with high impact factors – specifically the citation standards and design and analysis transparency – all that’s needed is thorough and consistent compliance. However, the remaining guidelines require the attention of our scholar-practitioner community.

In this piece,  we take a brief look at each of the six standards and why adopting and promoting the standards raises the quality of research conducted by SAS faculty and students and in turn the reputation of the Research Centers.

Data, Analytic Methods, and Research Materials Transparency

These three standards all have the same basic requirement: to make accessible the documentation, data, and coding needed for another researcher to replicate the study (level 2) with verification by the journal prior to publication (level 3). These standards codify a best practice that all experienced scholar-practitioners strive to follow. Adopting these standards as criteria for original research papers and dissertations provides a powerful mechanism for increasing the quality of the Centers’ research.

Preregistration of Studies and Analysis Plans

Although many journals require preregistration of controlled trials. These two interlocking standards raise the bar by calling on researchers to preregister with an independent registry all confirmatory and exploratory studies, describing the study design and including the data analysis plan prior to conducting the research.  Journal boards that adopt level 3 will only accept manuscripts for preregistered studies. These standards should raise the visibility of studies that are not ultimately published; providing greater exposure to researchers and also offering a valuable resource to doctoral candidates who are searching for ideas and examples of study designs.

Study Replicability

The last transparency standard is intended to encourage researchers to replicate studies and submit the results for publication. Adoption of level 3 of this standard by journals provides researchers with a two-step approach to conducting replication studies: first proposing a plan for approval, and then conducting the research and submitting the manuscript. Promoting the study replication and providing a mechanism for publication opens a new and much-needed window of opportunity for the Centers’ scholar-practitioners.

Implications

Setting aside the serious consequences of being an author of a retracted research article, why embark on what is arguably additional and significant burdens on researchers?  

First, the Research Centers can springboard from the TOP Guidelines to promote our work as scholar-practitioners conducting rigorous, replicable research to advance knowledge in our broad portfolio of domains. Second, adoption of the TOP guidelines serves as an infusion to the scholar-practitioner criteria for research, including dissertations. Third, in the increasingly competitive space of academic journals, we should anticipate editorial boards adopting the level 2 and level 3 standards as a means of standing out from the pack. Executing a strategy to ultimately achieve level 3 for all these standards enables our research community to be proactive and will lead to an increase in the rate of acceptance of papers to the most prestigious journals in each of the Centers’ domains.

Lastly, but arguably the most important point, it is incumbent on scholars to adopt these guidelines because it’s the right thing to do. All research intended for publication by scholar-practitioners and doctoral candidates should adhere to these basic principles of high-quality research.

In fact, because of the momentum building for the TOP guidelines, all research institutions striving to be leaders in their respective fields are now on notice to:  

(a) Begin to incrementally adopt the TOP guidelines for all applicable scholar-practitioner research intended for publication; and

(b) Provide incentive for faculty to incorporate these guidelines into the criteria used for evaluating dissertations and developing research proposals.

By taking these steps, the School of Advanced Studies has the opportunity to leapfrog many of its peer organizations that haven’t recognized the strategic significance of the TOP guidelines to the research community.

What do you believe are the implications of the TOP guidelines? What should be the SAS Research Centers’ response to these guidelines, in the short-term? Longer-term? 

Visit Our Blog

Visit the Research Process Blog for insights and guidance from University researchers Go >>