Trend Analysis and Forecast Methodology

Can we trace change in corruption using perception indicators?

Measuring corruption across time is one of the most difficult tasks in governance studies. Not only we do not have specific enough measurements, but due to their aggregate nature from various sources the most well-known corruption indicators (like Corruption Perception Index) capture little change. The World Bank researchers Daniel Kaufmann and Art Kraay calculated transparently that about half the variance over time in the World Bank Institute Governance Indicators (computed by a similar technique with CPI) results from changes in the sources used and the weights assigned to different sources. They produced a confidence error interval which highlights only the changes above this threshold as significant. But as the statistical noise is so high, very little change is captured. So, not only do we have indicators which rely on non-specific perceptions (we do not know what, ultimately, is perceived and what actually changed versus what did not), but we must also disregard any change lower than the statistical noise. Please find a full discussion on lagging indicators in our working paper Beyond the lag. How to predict and understand evolutions towards good governance?

The world of lagging control of corruption and lagging corruption indicators looks like the two charts below, which average change across income groups and continents. The world seems entirely flat, but for the two higher income groups where the trendline is declining. However, this is not a significant decline, indicating that the world is really flat when governance is concerned.

During the period represented by this relatively static trend line, the world has invested more than ever in good governance following the adoption of United Nations Convention against Corruption in 2004. While we do not have country success stories in abundance, it is also implausible that no evolution at all has taken place. We need, however, better tools to trace this evolution.

Why not use the Index for Public Integrity, created by ERCAS in 2016? Well, for the very simple reason that the IPI is still relatively recent – some of its components do not go far back enough to allow us to compile it retrospectively. Also, despite its superior specificity (we know exactly what components it includes, what do each measure and how they interact), the IPI is also an aggregate, and the components are also standardized and normalized to allow a ranking in any given year, thus generating statistical noise itself, even if less than the perception indicators.

However, it remains very important to trace the evolution of control of corruption in time. Various government agencies use governance indicators to program and condition foreign aid. Governments and civil societies also need a tool to gauge the effectiveness of their policies. How can we reliably have an instrument that both captures change more sensitively and tells us what drives the change?

How can a more specific and sensitive forecast be developed?

To solve this riddle, we proceed as follows:

1. TREND ANALYSIS AND THRESHOLD TEST

   We use the disaggregated components of IPI, and we observe their changes since 2008. To eliminate changes which may just be random we compare our sample of 120 countries with a similar theoretical group of countries where average change is zero (like in the graphs above). We rate as significant change any change above or below the global standard deviation of average change against a control group with zero change. This is a more positive scenario than using a null hypothesis with average global change as baseline, but still eliminates small changes. It is also less arbitrary than just setting a confidence interval above/below which we would consider changes too small. As this exercise takes place in 2021 amidst a context of global democratic decline we opt for this variant because our indicators have mixed signs: some have negative average changes (political indicators), others positive (more technical indicators, like e-citizens). We do not want to miss reforms, but to encourage countries to engage in them by making them visible. As a complement to the Country page trends table where this approach is used, you can compare each country against the regional (continental) mean by using this Compare Trends button on the forecast map page.

   We then rate change as consistent if a country has progressed (or regressed) in at least two indicators and had not regressed (or progressed) in any. Some data is missing retrospectively: the Facebook users’ data, which is a component of the e-citizens indicator changes quality and coverage across years, so we use only Internet household connections to measure e-citizens; also, the World Bank has repeatedly changed methodology for Trade Openness, so we do cannot use this IPI indicator for the trend analysis.

2. POLITICAL CHANGE CHECK

   In case of inconsistency, as a mandatory step we check all radical political events (regime change or political violence) which occurred in the last 4 years in a country and if the sign of these events is in line with or contradictory to the long-term trends. A plus on the long-term trends can be canceled by a minus on political change, for instance a coup, a violent repression of civil society. Likewise, a negative on the long-term trends can be counter-balanced by an uprising that is followed by the election of an anticorruption government.

3. SOCIETAL DEMAND (NORMATIVE CONSTRAINTS) CHECK

   Finally, evidence shows that the demand for change from the society matters a lot. In case of inconsistencies, we thus use as final check the most recent value for the IPI e-citizens (digital citizens) component as proxy for demand for good governance. Additionally, for particularly puzzling cases or where there is change, but it is under the statistical threshold, we use the latest edition of Transparency International’s Global Corruption Barometer, which includes a question where citizens are asked if they perceive a change, and if so in what direction. They are also asked if they approve of the anticorruption stance of the government – all useful questions to estimate demand for change in a society.

What are the results?

The trend analysis and weighting described above produces three categories of countries: stationary cases, improvers and backsliders. These can be further refined by plotting the change sign against the IPI map, so to see if stagnation, for instance, occurs where such lack of change is problematic, such as where there is already systemic corruption. It is not a problem if a country with good public integrity stagnates. The forecasted trends are indicated as either red (declining) or green (improving). You can access the IPI map on this webpage under Forecast.

The final categories are established by our senior experts, professors Alina Mungiu-Pippidi and Michael Johnston, and reviewed by a group of experts knowledgeable in all continents and governance indicators.

This exercise returns the results that we hoped for. We identify change in over 30 countries, and encouragingly in over 20 of them this change is positive. For some, like Japan or Estonia, corruption is already rather the exception than the rule, while others have far more to travel to good governance, but they are on a positive trend. For a detailed legend of why a country changed or did not, please consult the country pages under Forecast.

Improvers and decliners in the 2022 forecast

Improving	Declining
Bulgaria, Costa Rica, Estonia, Latvia, Lithuania, Indonesia, Japan, Kenya, R. Korea, Kyrgyz Republic, Liberia, Moldova, Mongolia, Morocco, North Macedonia, Slovakia, Spain, Uruguay, Taiwan, Timor Leste, Vietnam, Zimbabwe	Bolivia, BiH, Cambodia, Egypt, Ethiopia, Myanmar, Russian Federation, Venezuela, Zambia

What to use this forecast for?

The forecast can serve as an evaluation tool for the anticorruption strategists in a country, as well as for a longer-term diagnosis completing the Index for Public Integrity, which offers only a snapshot in one moment in time. It is important to understand the trend a country is on to confirm or adjust your theory of change and your strategy accordingly. For some very basic choices for donors and civil societies, see the table below.

	STAGNANT AT HIGH CORRUPTION	ACHIEVER	BACKSLIDER
DONOR	This country has not changed and there is no signal it will in the near-term. Change your strategies; take stock of why existing theories of change have not worked, and the balance is both sub-optimal and stuck. The Index for Public Integrity will show what is wrong.	Understand why this country is on the upswing and support positive trends and domestic actors who promote change	There can be more harm than benefit to pushing classic anticorruption (ACA). Instead, go for targeted sanctions and support the endangered integrity warriors and free press in the country or diaspora
CIVIL SOCIETY	The power balance is not in your favor. Achieving far broader interest representation is a worthy goal – in some cases alliances with business, unions, or cultural/community entities – including outside of capitals. Create political vehicles, think-tanks, vibrant digital commons as in our De facto Transparency index. Use naming and shaming systematic campaigns to challenge corrupt status groups.	Check existing public accountability tools (for instance, on our www.againstcorruption.eu) and use them where the enabling contexts exist. Create political vehicles, think-tanks, a vibrant digital commons as in our De facto Transparency index,. Remove legal rents from legislation. Introduce public services evaluations based on social accountability.	Create coalitions, get external support, invest in legal representation, move critical media to servers outside the country.