Trend Analysis and Forecast Methodology


Can we trace change in corruption using perception indicators?

Measuring corruption across time is one of the most difficult tasks in governance studies. Not only we do not have specific enough measurements, but due to their aggregate nature from various sources the most well-known corruption indicators (like Corruption Perception Index) capture little change. The World Bank researchers Daniel Kaufmann and Art Kraay calculated transparently that about half the variance over time in the World Bank Institute Governance Indicators (computed by a similar technique with CPI) results from changes in the sources used and the weights assigned to different sources. They produced a confidence error interval which highlights only the changes above this threshold as significant. But as the statistical noise is so high, very little change is captured. So, not only do we have indicators which rely on non-specific perceptions (we do not know what, ultimately, is perceived and what actually changed versus what did not), but we must also disregard any change lower than the statistical noise. Please find a full discussion on lagging indicators in our working paper Beyond the lag. How to predict and understand evolutions towards good governance?

The world of lagging control of corruption and lagging corruption indicators looks like the two charts below, which average change ain the World Governance Indicator Control of Corruption (produced by World Bank Institute) cross income groups and continents. The world seems entirely flat, but for the two higher income groups where the trendline is declining. However, this is not a significant decline, indicating that the world is really flat where governance is concerned.

1. Control of Corruption World Governance Indicator
Ten Year Trends by Income Group

 

2. Control of Corruption World Governance Indicator
Ten Year Trend by Continent

During the period represented by this relatively static trend line, the world has invested more than ever in good governance following the adoption of United Nations Convention against Corruption in 2004. While we do not have country success stories in abundance, it is also implausible that no evolution at all has taken place. We need, however, better tools to trace this evolution.

Why not use the Index for Public Integrity, created by ERCAS in 2016? Well, for the very simple reason that the IPI is still relatively recent – some of its components do not go far back enough to allow us to compile it retrospectively. Also, despite its good specificity (we know exactly what components it includes, what do each measure and how they interact), the IPI is also a statistical aggregate, and the components are also normalized to allow a ranking in any given year, thus generating statistical noise itself, even if less than the perception indicators.

However, it remains very important to trace the evolution of control of corruption in time. Various government agencies use governance indicators to program and condition foreign aid. Governments and civil societies also need a tool to gauge the effectiveness of their policies. How can we reliably have an instrument that both captures change more sensitively and tells us what drives the change?

How can a more specific and sensitive forecast be developed?

To solve this riddle, we proceed as follows:

We use the disaggregated components of IPI, and we observe their changes for the past ten years. To eliminate changes which may just be random we compare our sample of 120 countries with a similar theoretical group of countries where average change is zero (like in the graphs above). We rate as significant change any change above or below the global standard deviation of average change against a control group with zero change. This is a more positive scenario than using a null hypothesis with average global change as baseline, but still eliminates small changes. It is also less arbitrary than just setting a confidence interval above/below which we would consider changes too small. We do not want to miss reforms, but to encourage countries to engage in them by making them visible. As a complement to the Country page trends table where this approach is used, you can compare each country against the regional (continental) mean by using this Compare Trends button on the forecast map page.

The six indicators used for the IPI 2023 are:

Administrative transparency De facto transparency of public contracts, business register, land cadaster and auditor general reports, as reported step by step and link by link in the T-index.
Online services The extent to which governments offer online services, as featured in the UN Survey. (Replaced Administrative burden based on the World Bank Doing Business)
Budget transparency The extent to which budget proposal and previous-year expenditures are and have been made public, using a fraction of the Open Budget Index survey.
Judicial independence The extent to which the judiciary is autonomous from private interest (including by government officials) as in the Global Competitiveness Report survey by the World Economic Forum
(Digital citizenship) E-citizenship Household broadband subscriptions and Facebook users per country measure the capacity of civil society
Freedom of the press Yearly indicator including economical and physical pressure on media

Some data is missing retrospectively: the Facebook users’ data, which is a component of the e-citizens indicator changes quality and coverage across years, so we use only Internet household connections to measure e-citizens. As administrative transparency is a new indicator, with direct observations of every country’s online transparency just 2 years old, it is not included in the forecast trends monitoring, but as an additional weight step.

The steps are as follows:

  1. Change significance test. Monitoring for the last decade of the factors of corruption (5) and rating change above one standard deviation of global change as significant.

  2. Consistency test. We then rate change as consistent if a country has progressed (or regressed) in at least two indicators and had not regressed (or progressed) in any. If a country has 2+ indicators that have changed positively and none negatively, the country enters the second round of evaluation with a positive sign and it is diagnosed as ‘improving’, unless some recent political fact is radically different. The same applies to negative trends if trends are consistent. If, however, the country has 3 pluses and one minus, or the other way around, the next step is where the decisions is taken, so it becomes more than just a check for consistency. Most inconsistent countries end with a diagnosis of ‘stationary’, as trends cancel one another.

  3. Political contingencies test. Political violence or radical change have the most important impact on corruption and state capture. The world has become very unstable in the past two years, so we consider the last year’s political impact of major events- coups, civil wars, assassinations, elections, use or misuse of anticorruption organizations – in order to complete the analysis of a country’s trend consistency.

  4. E-transparency and e-participation test. Finally, we add two more weights to complete consistency analysis: administrative transparency and the level of e-citizens. Regardless of the trend, digital citizenship needs to reach a critical mass to articulate demand for good governance. Administrative transparency, on the other side, shows to what extent the government has committed to allow citizens the main tools to fight against favoritism and discrimination. One measures the government’s will, the other the capacity of civil society. The interaction between the two explains the change in corruption across time. Therefore, these last two elements are used for the last step of the forecast, as they tell us to what extent active civil society exists and has the instruments to make the country change.


Our IPI and forecast methodology then provide three pictures:

  1. A snapshot- How the world is in the 2022-2023 IPI and why. Users can read the IPI by country and compare it against its region and income group on every component.

  2. A motion picture based on a time series- how countries changed over the past ten years and where they would likely be next year.

  3. A diagnosis - Open the forecast country page to see the individual trends, diagnosis and explicit legend so to understand where the country is on corruption risk, what it could do to improve, and where it will be next year.


This exercise returns the results that we hoped for. We identify change in a large number of countries, and we know exactly what changed and what did not.
 

Improvers and decliners in the 2023 forecast

Improving (22)

Declining (12)

Costa Rica, Côte d'Ivoire, the Dominican Republic, Ecuador, Estonia, Latvia, Lithuania, Indonesia, Ireland, Kazakhstan, Latvia, Liberia, Moldova, North Macedonia, Portugal, Rwanda, the Slovak Republic, Sri Lanka, Thailand, Ukraine, Uruguay, Vietnam.

Bolivia, El Salvador, Guatemala, India, Lebanon, Qatar, Russian Federation, Tajikistan, Tanzania, Tunisia, United States, Venezuela

 

What to use this forecast for?

The forecast can serve as an evaluation tool for the anticorruption strategists in a country, as well as for a longer-term diagnosis completing the Index for Public Integrity, which offers only a snapshot in one moment in time. It is important to understand the trend a country is on to confirm or adjust your theory of change and your strategy accordingly. For some very basic choices for donors and civil societies, see the table below.

STAGNANT AT HIGH CORRUPTION ACHIEVER BACKSLIDER
DONOR This country has not changed and there is no signal it will in the near-term.
Change your strategies; take stock of why existing theories of change have not worked, and the balance is both sub-optimal and stuck. The Index for Public Integrity will show what is wrong.
Understand why this country is on the upswing and support positive trends and domestic actors who promote change There can be more harm than benefit to pushing classic anticorruption (ACA). Instead, go for targeted sanctions and support the endangered integrity warriors and free press in the country or diaspora
CIVIL SOCIETY The power balance is not in your favor. Achieving far broader interest representation is a worthy goal – in some cases alliances with business, unions, or cultural/community entities – including outside of capitals. Create political vehicles, think-tanks, vibrant digital commons as in our De facto Transparency index. Use naming and shaming systematic campaigns to challenge corrupt status groups. Check existing public accountability tools (for instance, on our againstcorruption.eu) and use them where the enabling contexts exist. Create political vehicles, think-tanks, a vibrant digital commons as in our De facto Transparency index. Remove legal rents from legislation. Introduce public services evaluations based on social accountability. Create coalitions, get external support, invest in legal representation, move critical media to servers outside the country.