SUMMARY - A Toolkit for Planning Conducting and Monitoring Pilot-Tests – Savings Products

By Michael J. McCord, Graham Wright and David Cracknell (MicroSave)

Abstract

MicroSave has developed a toolkit to design and implement pilot-tests in a controlled and scientific manner. It is designed to help organizations prepare a pilot-test for a savings product from the very initial stages of planning, along every step of the pilot process and up to the final evaluation of the product.

Key issues presented in the toolkit:

• A pilot-test demands great institutional commitment, including resources, staff and management time
• The pilot-test team should always keep their loyalty to the MFI and not to the product, being always opened to terminate the test if necessary
• Objectives should be easy to quantify, realistic and should commence with baseline data
• Continuous monitoring allows the team to implement prompt changes where needed by comparing projections to actual results. This analysis will also lead the final evaluation of the project

Summary

This toolkit based on Microsave's experience outlines the process of planning, implementing and monitoring a pilot-test in ten steps:

1. Composing the pilot-test.
   The team should incorporate a 'product champion' as leader and probably from 5 to 10 other people that would be representational of all major departments of the MFI. The size and people involved depend on the size and specific characteristics of the institution.
   The team should be created formally, presenting TOR that describe the roles, responsibilities, resources, duration and expected results of the test. It should also assign accountabilities and reflect the consequences of non-compliance.
   The minimum skills that should be available within the team are: Team leader; Finance/Accounting; IT/MIS; Marketing; Training; Operations Management; Operations/Frontline; Research; Audit/Controls.
   
   
2. Developing the Testing Protocol or 'Roadmap'
   The roadmap defines what will be done by whom and when. Among the key issues that need to be defined at this stage are:
   Number of clients that you want to reach - should be a sample that's large enough to be representative, but not too large that puts the institution at risk.
   Location - The place or branch where it will be implemented should have enough infrastructure and space for the new product. Also, should have enough potential clients.
   Duration - Depending on the product. If product-cycle is short (6 months or less), the pilot should cover one full cycle; for longer or indeterminate cycles plan a pilot for 9 to 15 months. For hi-risk products, double the length.
   Reporting Dates - The team should meet regularly and be formally informed. During the fist month of the pilot, meetings should take place weekly. Later, meetings can be hold on a monthly basis.
   Data to be analyzed - The data should allow the team to review the objectives sheet and to compare the financial projections with the actual results.
   Options available as data is evaluated - Specify the parameters under which the project would be i. terminated or suspended, ii. changed and continued, iii. continued or iv. roll out. A pilot test should be terminated when protocol reporting is not on schedule, profitability targets are away by more than 20% or if break-even point is beyond 2 years.
   
   
3. Defining the objectives
   Objectives provide the criteria to evaluate the pilot-test. They should be very specific, quantifiable and should cover the full period of the pilot-test. There should be objectives that refer to issues of profitability, growth (number of accounts and balances), efficiency, customer service and marketing. Having baseline data is important to set realistic targets.
   Setting objectives is usually done in two stages: In the first one, general objectives for profitability, efficiency and growth and their respective targets are set (usually around 5-7 objectives are sufficient). The second one takes place when the product has been fully modeled and in this stage end-point and month-to-month targets are identified (step 5).
   Target numbers must be calculated not guessed for. When possible, use previous the performance of similar products to estimate your targets. When the product is dramatically different to all products of the MFI, use market research tools to estimate the 'qualified available market'.
   
   
4. Preparing all systems
   Systems should be in place and tested about 2 to 4 weeks before commencement of the pilot-test. Not all MFIs can afford a computerized system and it might not be necessary for all organizations.
   MFIs sometimes take too long waiting for the perfect product or designing their own. There are several available in the market and it is likely that at least one of them will be sufficient for your institution's needs.
   Keep an effective manual back-up system in case the electricity goes down or you have any other problems with the system.
   Have the system reviewed by internal and external auditors.
   
   
5. Modeling Financial Projections
   Profitability is one of the most important objectives of the new product. If the product being tested exists in another market or at a smaller scale, it is possible to extrapolate that information. However, for a completely new product, projections are based on assumptions.
   Assumptions must be reasonable and justifiable, though conservative. Some key assumptions for savings products are: number of accounts, transaction time, investment yields, product price and volume of transactions.
   The main purpose of doing financial projections is to find the right balance between cost and price. This will define the financial impact of the product on the institution. There are several models that can help an institution to find the right revenue/expense mix, such as the one developed by MicroSave. It is that the chosen model is flexible and that it is detailed by transaction and cost factors.
   Projections should be done for 3-5 years and they can take between 5-10 person/days. Having projections ready, allows the institution to finalize end-point and monthly targets.
   
   
6. Documenting the Product
   This is an important step for those implementing the product, who need to understand the policies as well as the procedures of its operation.
   Documentation should cover front and back office as well as key accounting procedures.
   
   
7. Training
   Staff must be trained or re-trained on issues that go from proper application of product policies and customer service to optimal use of systems and marketing.
   
   
8. Develop customer marketing materials and methods
   The marketing plan should be based on a clear statement of the benefits or value clients can expect from the product. This will come from the MFI's market research and from understanding the target market.
   The sales strategy should be a combination of push and pull strategies: personal selling, public relations, direct marketing, advertising and sales promotions.
   Keep the large promotions and big campaigns for the big roll-out because this could bias the result of the test. Concentrate on the area where the pilot is taking place and be clear on which branches are offering the product to avoid customer's being confused.
   In order to track the results of the marketing, there are several strategies you could use. The cheapest and easier to implement is including a questionnaire at the end of the account opening documentation.
   
   
9. Commencing the pilot-test
   Before launching the pilot-test make sure you inform the branch manager with all the details of the new product and how it will operate. Also, it is important to send a letter to all other branches with general information about the pilot-test and an overview of the product, in case there is some interference with their own operations, such as customers asking about the product.
   
   
10. Evaluating the test
    During the implementation the team collects and analyzes information with one question in mind: 'Is launching this product worth the investment to our MFI?
    The team will be reviewing monthly reports, which should include financial performance, but also, client's perceptions, staff perceptions, marketing, training, infrastructure.
    Data analysis should involve a comparison of projections with objectives and protocol, plus, it should lead to decision-making based on the facts learned and the limits set up in the protocol.
    Finally, once the pilot is over, a recommendation must be issued, suggesting whether to roll out the product, continue with the test (i.e. significant changes were introduced and data is inconclusive) or terminate the product (i.e. results show lack of profitability).
    Next step would be to form a roll out team and TOR, which should build on the pilot-test team and lessons.