Researchers investigating the outcomes of academic research have estimated that the results of many medical studies are inflated or not as strong as reported. If another expert tried to reproduce the results, the previously achieved research results could not be matched. This causes a significant problem in determining the usefulness of novel therapies in patient care.
Further, the problem of lack of reproducibility, not limited to medicine, is also observed in other disciplines such as economics, animal research and humanities. Many times the research outcomes are true for the limited conditions they were tested in. Brian Nosek & other experts at the ‘Reproducibility Project’ found that the results of many research studies are surely credible (not fraudulent), but the lack of reproducibility poses problems in their real world applications.
The lack of reproducibility can stem from a resource crunch. The project may not have sufficient funds or budget to include full-time consultants in key areas, for instance – statistics. Some studies do not test an optimum number of participants (lack of statistical power) as the tests may be expensive. Additionally, the project personnel such as research assistants, graduate student may be pulled in different directions due to their educational requirements and responsibilities.
During this decade, the emphasis on reproducible research is growing. The academia has been exploring multiple ways to effectively address this problem and find answers. Hiring a freelance expert who has advanced degree such as Ph.D. or a postdoc might provide a feasible solution. Here are the top three reasons for collaborations:
1) Experts provide a different point of view: While developing a research design, academic scholars focus on generalizability of the results. However, if their knowledge of statistics or research methodology is not current, then they may overlook certain conditions of testing. This will eventually result in the lack of reproducibility.
Further, there are multiple ways of testing a hypothesis and a statistics expert can identify a method that will not inflate the results. Hiring a statistics and research methodology expert will ensure that the current standards of research methods are met and statistical analysis are conducted appropriately. Listening to an alternate point-of-view that an expert has to offer might prove to be a useful exercise in the above mentioned scenarios.
2) Experts save dollars: In many projects, some services are needed only for a short while. For instance, including an expert programmer on a medical study may be necessary but not feasible. The budget of the project may not permit a long term full time consultant. Freelance experts can provide cost-effective services to meet such short-term needs.
3) Experts help in project replication: Some projects may need a quick double check to ascertain that the replication produces the same results. These replication efforts exhibit diligence and vigilance necessary for research projects. However, if the budget is limited then engaging primary personnel in double check efforts can prove time-consuming and costly. An external team of experts can be hired to replicate these projects.
Non-traditional models to integrate advice from the experts for various projects needs to be further explored; so that the society can benefit from the reproducible science.