The What, Why, and How of Impact Assessment

What is an impact assessment?

An impact assessment is a research study that measures how the services of a microfinance institution (MFI) impact the lives of its clients in such areas as employment, income, nutrition, education, health, and gender equity. It is the primary tool used to determine the effectiveness of microfinance as a development intervention.

How are impact assessments used?

Depending on an MFIs objectives, goals for an assessment, the type of information required and cost considerations, impact assessments can be used in a number of ways including:

• As an internal learning tool for MFI staff and help them determine whether their services are having a positive impact on the poor clients they seek to serve. At the very least, they can use an assessment to establish whether their services are producing any negative impacts (e.g., growing client indebtedness) and take action to correct such effects.
• As a management tool to improve operational efficiency, product design, and social effectiveness. Impact assessments encourage an MFI to include impact data in its information systems and decision making. Regular collection and analysis of impact data enables an MFI to better identify and serve the needs of its clients, and thus function more effectively.
• As a fundraising tool to prove the positive impact of financial services to external donors

Why are impact assessments important?

Microfinance programmes are one of the most important interventions to reduce poverty in developing countries. Recent years have seen huge growth in the number and size of microfinance organisations, the volume of microfinance clients, and the provision of subsidised donor funding. Poverty reduction is part of the mission of a large proportion of microfinance providers, and donor funding is allocated to microfinance on this basis. There is, therefore, a basic need to not only understand through measurement but also improve through management the impact of microfinance institutions (MFIs) as a key premise of successful poverty reduction and as a marketing tool to attract more donor funding.

It is critical for MFIs, particularly those working with very poor and vulnerable clients, to be sensitive to more than just financial performance and market indicators. They need to be sensitive to the social impacts of their work - particularly to the potential negative impacts that their services may produce. An MFI should know that it is having a positive impact or, at the very least, that it is not having a negative impact on some people. See the Practitioner Pages to read how and why MFIs have integrated impact assessment into their business.

Impact Assessment is for Practitioners and Clients, not just Donors

All MFIs have knowledge about how their work affects their clients but it is often not formalised. Many organisations, for example, use market research to understand client needs and reactions, and develop management information systems that can detect problems in operations or methodology.

A great deal of impact assessment work conducted in the past, however, has been donor driven and mostly serves donor needs. The emphasis of these assessments has been on justifying funding, rather than helping MFI management to learn and improve their work. Increasingly, however, a more "practitioner-oriented" approach to impact assessment is emerging. This approach seeks to:

• Fit into existing MFI work patterns and provide frequent and timely information
• Learn from and build on existing knowledge and experience of an MFI
• Produce results that can be easily used by MFI management
• Feed into management and product design processes, allowing MFIs to improve their services and, thus, their impact on clients

The Impact Assessment Centre provides guidance and information about how impact assessments can better answer the needs of MFIs (i.e., how they can be designed to provide credible information to improve MFI operational practices). It also considers how to design studies and systems that balance the needs and objectives of different stakeholders. For more detailed information, see Developing an Impact Assessment.

Credible Impact Assessment

Impact assessment is hard to do well. The key is to provide information that is sufficiently credible for the specific purpose of the assessment. Practitioners do not need information that is 99.9 percent accurate. They can accept a margin of error, provided the information is credible and can assist management in operational decisions and product design. An emphasis on timely and useful results leads to impact assessments that are more cost-effective and simpler than donor studies, which have attempted to demonstrate causality.

A considerable amount of debate has centered on the question of causality and attribution in microfinance impact assessment, that is, whether it is possible to demonstrate that the particular intervention of an MFI has led to a specific change in a client's life. Client livelihoods and the communities in which they live are complex. Clients, for example, may have multiple income sources. The credit provided by an MFI is, howeover, fungible and not necessarily used for the purpose for which it was requested. These conditions make it difficult to attribute specific impacts to an MFI. The difficulty of impact attribution is exacerbated by the challenge of establishing effective control-group mechanisms that can create a counter-factual (i.e., what would have happened to a potential client without the MFI's intervention). The methods that can be used to establish such a counter-factual are, moreover, time-consuming, costly, and complex.

Even a low-cost impact assessment needs to be credible. It is important that the assessment process is carefully thought through. Indicators must be developed based on a sound conceptual framework for the type of impacts being examined. Systems used to collect information need to be effective and produce reliable results. Analysis needs to be carefully planned to ensure that results will provide the right type of information. Using a mix of methods and triangulating information from one method against another is a crucial technique that can both improve the credibility of results and lower the costs of an impact assessment.

The Impact Assessment Centre aims to provide a practical guide to enable people to develop impact assessment processes and systems that are appropriate to the context, objectives, and stakeholder needs of their organisation. An appropriate system is one that provides information that is credible and sufficiently reliable for the specific objectives of an individual MFI.