Would quality-of-life considerations improve financial assistance evaluations?
Evaluators are increasingly considering new approaches to assess the impact of policies and financial assistance programmes on people’s lives. Though such innovation might not always be easy to implement, it’s worthy of consideration. In this blog post, we examine how data on quality-of-life could benefit financial assistance.
Post-programme evaluations of financial assistance are an important element in improving crisis responses. So far, EFSF and ESM evaluations have, in line with the ESM’s mandate, focused on how financial assistance affects economic growth and resilience. However, the consequences for citizens’ lives and well-being remained outside the main interest and evaluators’ toolkits.
We found that assessing the impact of crisis resolution on people’s everyday lives makes it more likely to detect potentially unintended consequences and informs about national policy preferences for defining programme objectives.
A quality-of-life focus offers relevant indicators that help demonstrate the impact of financial assistance on shared prosperity, stability, and well-being. Considering the wider impact of reform programmes could increase trust in institutions, improve governance, strengthen reform programme ownership, and render international financial assistance more effective, less costly, and sustainable.
Why examine quality of life when evaluating financial assistance?
International financial institutions place great importance on evaluations of their financial assistance in beneficiary countries. These assessments boost institutional learning and spur efforts to improve policies that ensure organisations’ credibility and effectiveness.
Today’s pressing challenges, such as climate change, security risks and rising income inequalities, add new dimensions to financial assistance evaluations. Evaluators are therefore thinking about how best to enhance their efforts to collect evidence and draw lessons from the assistance provided in an increasingly complex and interconnected environment.
A quality-of-life lens provides a wider angle for the evaluator that looks beyond the limited ability of traditional economic indicators to capture living conditions in society.
We tested how the OECD well-being framework, as one way of measuring quality of life, could contribute to this assessment.
The OECD framework includes 11 dimensions that we clustered based on their relevance in evaluating financial assistance (see Figure 1).
Figure 1: Quality-of-life indicators
Our approach focused on six categories:
- income, wealth, and housing,
- employment and education,
- life satisfaction, work-life balance, and social connections, and
Financial stability, the ESM’s primary goal, underpins several of the analysed categories and is thus an important factor in quality of life within the euro area. Still, we recognise the difficulties associated with drawing conclusions on causal pathways due to the multiple, simultaneous policy initiatives at play during a crisis.
What did we find applying a quality-of-life approach?
First, the most vulnerable bear more hardship
During financial assistance programmes women, young people, and the less educated were typically disproportionately affected. Income differences between social groups and pressures on household budgets tended to widen initially in the crisis, although they began to reverse as reforms progressed. Education results deteriorated during the crisis and personal health assessments of the poorer and the elderly recovered at a slower pace.
Second, the environment is impacted
We also looked at the countries’ ability to mitigate and adapt to climate change. While there is no evidence to suggest that the ESM programmes negatively impacted environmental quality, the implications for the capacity to act on climate issues at a national level are important. In some countries, environmental policies have succeeded in improving living conditions.
For example, because of crisis-triggered economic slowdowns, the material footprint per capita decreased in all assisted countries and, in particular, in Greece. While the programmes’ conditionality measures targeting environmental factors were relatively few, a good life quality for all can only be sustained over time when the necessary resources are preserved and risks to economic, natural, and societal systems are recognised and managed appropriately. Therefore, we found that it could be more useful to consider how crisis resolution strategies interact with environmental policies rather than just the environmental quality.
Third, quality-of-life indicators help uncover the drivers for reform ownership
National acceptance of reforms agreed with international institutions determines success of a reform programme. Drivers of national preferences and determination to see reforms through are multifaceted, especially in countries facing economic instability.
Focusing on quality of life offers pertinent additional metrics to more clearly demonstrate that the ultimate objective of financial assistance is to ensure more equally shared prosperity, stability, and well-being. It also emphasises increasing trust in institutions, improving governance, and strengthening reform programme ownership, thereby rendering the reform programmes more effective, efficient, and sustainable. There are indications that countries scoring low in governance indicators tend to struggle more in other aspects.
Better alignment between national preferences and programme objectives could uphold programmes’ long-term sustainability and limit the risk of rising public discontent and political instability. Country ownership is conditioned by leadership in the strategic processes needed for success, including ensuring alignment with national policies and institutional capacity for implementation and engaging stakeholders in the reforms.
Need for reliable data and further research
The successful implementation of the quality-of-life indicators and the overall inclusive growth perspective in evaluations will rest mainly on data availability and quality.
First, more granular, and timely data beyond the country aggregates could help to better understand the impact of reform programmes on the most vulnerable groups. Our analysis suggests that the living conditions of young people and women tend to be hardest hit by economic and financial distress. Beyond these, institutional indicators for trust and governance can help to understand a country’s capacity to deliver on reform efforts.
The European statistical office and OECD have already started taking steps to respond to international initiatives and the European Green Deal. A G20 data governance initiative that will cover key areas of climate change, household distributional information, fintech, and financial inclusion could help to further extend data coverage.
Second, we need more frequent and broader opinion surveys generating perception data. Despite their limitations in country comparisons, such data are key to understanding countries’ social fabric and could inform international institutions’ assessments of the context in which they intervene.
Finally, there is a need for policy discussions on including quality-of-life dimensions among the programme performance indicators. Although the causality and links between programme action and social and environmental outcomes are difficult to establish, further research should investigate whether these indicators have improved or worsened in the run-up to and/or at different points in time during financial assistance programmes, and how persistent such changes might be.
A common approach to statistics and data reporting could help evaluate policy consequences, unintentional or otherwise, in increasingly important areas.
The better we evaluate programmes, the more we can improve future financial assistance and ensure that reforms benefit all citizens – the ultimate goal of institutions providing assistance.
Climate Coalition of Finance Ministers for Climate Action, October 2021 Joint Ministerial Statement. Joint Ministerial Statement - October 2021.pdf (financeministersforclimate.org)
ESM, Evaluation of EFSF/ESM programmes: Independent evaluation on Greek financial assistance | European Stability Mechanism (europa.eu)
Khan, M., & Sharma, S. (2001). IMF Conditionality and Country Ownership of Programmes. IMF Working Paper, WP/01/142, 2001, Available at: https://www.imf.org/external/pubs/ft/wp/2001/wp01142.pdf
Almunia, J. (2020) Lessons from financial assistance to Greece: Independent evaluation report. European Stability Mechanism, June 2020. Available at: https://www.esm.europa.eu/sites/default/files/document/2021-07/lessons-financial-assistancegreece.pdf.
About the ESM blog: The blog is a forum for the views of the European Stability Mechanism (ESM) staff and officials on economic, financial and policy issues of the day. The views expressed are those of the author(s) and do not necessarily represent the views of the ESM and its Board of Governors, Board of Directors or the Management Board.