Behavioural change, that has been at the centre of the Community Led Total Sanitation (CLTS) process has been truly remarkable in transforming communities and leading them to an Open Defecation Free status. The focus that it lays on a participatory community based approach rather than one reliant on subsidies makes it more sustainable in the long run. Since there have been occurrences of “slippage” where despite the construction of toilets, communities have gone back to open defecation; a constant review of the effectiveness of CLTS needs to be measured in more concrete terms rather than the arbitrariness of behavioural change. The number of toilets constructed and achievement of the ODF status are definitely the initial indicators of a positive outcome which are usually followed by the change in health outcomes. These changes in the health outcomes need to be monitored and evaluated to understand the long term impacts of the CLTS process on health.

What is a Cluster Randomised Controlled Trial?

A fundamental issue with the process of reviewing any sanitation process, be it CLTS or a subsidy led scheme is that it is usually in the short run. Long run causal relationships with respect to outcomes in health or infrastructure or even change in behaviour have yet to be established. The process of reviewing CLTS is ongoing and so far, a few methods of evaluation have been highlighted and even these are not looking at any significant long run evaluations. One of the methods is a Cluster Randomised Controlled Trial (henceforth RCT). This method is primarily used to review the effectiveness of CLTS to eliminate open defecation, reduce diarrhoea and other gastrointestinal diseases, and decrease stunting in children. Usually, the trial monitors changes in the above indicators for a period of a year after CLTS has been implemented. These trials are also used to compare the effectiveness of other sanitation programmes such as India’s Total Sanitation Campaign (TSC) when assessing the effectiveness of the approach. It has been used in rural Mali, where CLTS was implemented via a cluster RCT between 2011 and 2013. The study concluded that CLTS with no monetary subsidies did not affect diarrhoea incidences, but substantially increased child growth (thereby reducing stunting), particularly in children under two years of age.

Randomised Controlled Trials (henceforth RCTs) are in essence, an experimental assessment tool. It has been used in medical science for more than 50 years now and is considered in many ways the standard rational method for measuring the efficacy of drugs. In medical trials, the new treatment is randomly assigned to people participating in the experimental trial. Two groups are selected randomly and are compared, one receiving the new treatment, and the other the ‘control’ group receiving (usually) a placebo drug. This experimental process seeks to establish a better causal link between the treatment drug and the patient’s response, seemingly eliminating to a large extent problem of selection bias and confounding variables. Cluster RCT simply means it’s not individuals but clusters or groups which are being treated separately to decide effectiveness. Cluster RCTs are more frequently used in Social Sciences. Since the early 2000s, Randomised Controlled Trials have begun to have been used in the social sciences, most famously (and originally) by a group of economists from MIT’s Abdul Latif Jameel Poverty Action Lab (commonly known as JPAL). The ‘lab’ itself was founded in 2003 with the express purpose of carrying out randomised trials in the ‘fight against poverty’.

Drawbacks of Cluster Randomised Controlled Trials

Certain qualities of cluster RCTs make them unsuitable as a tool for evaluating the effectiveness of the CLTS process. They have being primarily used to judge the effectiveness of the process at the stage of implementation and not so much at the follow up stage, but the paucity of reviewing tools at the moment have increased the reliance on cluster RCTs. To begin with they are not equipped to judge long term impacts.  In the case of Mali, a year long impact was observed. Cleaner latrines and improved hygiene were observed in CLTS villages but no conclusive remarks could be made about impacts on child diarrhea and respiratory diseases, in another case study of assessing the effectiveness of a rural sanitation intervention, within the context of the Government of India’s Total Sanitation Campaign that took place in rural Odisha, India the impact was studied over a period of two and a half years, 2010 May to 2013 December.

The trial in Odisha also made similar inferences about the intervention. Increased latrine coverage was generally believed to be effective for reducing exposure to faecal pathogens and preventing disease; but the trial showed that this outcome could not be assumed.

Randomised controlled trials began in the field of medicine to judge the effectiveness of drugs and experimental treatments. However, their adoption by the field of social sciences has raised questions in the ethical front. This is a second drawback of the cluster RCTs, where they are essentially being used as a tool to judge effectiveness of developmental programmes. With the introduction of this method it has become a preferred tool of intervention in development economics and public health. The Poverty Action Lab has worked or is working in more than 11-12 countries and they have evaluated projects that have affected more than 300 million people. RCT method of evaluation is called ‘the gold standard’ in policy evaluation. However, a crucial point to be noted here is that unlike clinical trials, in social sciences the treatment variable is a developmental intervention, like building roads, erecting bridges or building toilets in the case of sanitation. Which raises the question should a mode of impact assessment involve social ‘experimentation’ on human beings at all. RCT practitioners argue this based on a sort-of utilitarian line, by claiming that their projects are only carried out if there is sufficient evidence to show that the potential positives of conducting a trial outweigh the possible negative effects.

Another more fundamental and more widespread criticism is with regards to the methodology itself. RCTs in medicine, as successful and widespread as they are, have recently come under increasing criticism for their usage in clinical trials itself. However, in social sciences, it is being argued that beyond the moral and ethical dimensions are they as effective in establishing causation as they claim to? RCTs have been claimed to be the most efficient way of isolating true causal variables whereas it has been argued that their solution faces most of the same problems that regular experimental economics does, and some argue that this method does not represent any great advance on other impact assessment methods.

Cluster RCTs have been successful to decide feasibility of CLTS in certain areas, however, newer methods need to be developed to monitor effectiveness of CLTS in health and other developmental outcomes. At the moment self reporting of diarrhoea is considered as one of the tools to assess health impacts as diarrheal and respiratory diseases have been directly linked to sanitation and hygiene, efforts have to be taken to ensure the quality of such reporting as well.

By

Debadrita Gupta

M.A. Development Studies, TISS, Mumbai

Debadrita graduated with B.Sc. Economics(Hons) from Symbiosis School of Economics, Pune. She is currently pursuing her M.A. in Development Studies from TISS, Mumbai. Her areas of interest are Public Health and Development Economics.