Evaluation of Growing Forward 2 Cost-shared Programming Strategic Initiatives – Summary Report

About the Evaluation

• This evaluation reported on relevance issues, and issues of design and delivery. The case studies provide illustrative examples of the kinds of outputs and outcomes that resulted from program activities
• The evaluation is national in scope and examined programming over the first three years of the five-year program cycle, from 2013-14 through 2015-16. The focus was on federal government activities, with no intent to evaluate provincial or territorial program delivery

Cost-shared Programming Description

• Cost-shared programming under Growing Forward 2 funds initiatives designed and delivered by provincial and territorial governments on a 60:40 cost-shared basis
• The total investment in Cost-shared programming is $2 billion over five years. Provinces and territories determine the suite of programs that best suit their regional needs from three priority areas
• Competitiveness and Market Development Innovation; and, Adaptability and Industry Capacity. The ultimate objective across the three priority areas is the enhancement of agricultural Gross Domestic Product growth

What the evaluation found

Program Relevance

• While both the domestic and international markets continue to evolve, the need to support Canada’s agriculture and agri-food industry through broad-based programming essentially remains the same as it was during the time of Growing Forward
• The Growing Forward 2 framework outlines a vision of Canada’s agricultural and agri-food sector as a world leader in food safety, innovation, and environmentally responsible production, as well as an engine of Canadian economic growth
• The cost-shared programming model is aligned with federal responsibilities as outlined in the 2015-16 Departmental Performance Report (DPR) and Agriculture and Agri-Food Act

Program Performance

• In the first three years of a five-year program, program spending is on track
• Indicator targets were found to vary widely and, in some cases, bore little relation to actual performance and did not paint a clear picture of achievements
• The survey, key informant interviews, and case studies all provided evidence of progress toward the achievement of expected immediate and intermediate outcomes

Program Design and Delivery

• Agriculture Agri-Food Canada (AAFC) is effective in administering the financial aspects of the program at the national level and from the work of the regional officials is efficient in supporting provinces and territories
• With the Growing Forward 2 Cost-shared Programs, provinces and territories have greater flexibility in programming choices. Though most capital projects are approved, provinces and territories perceive that a number of administrative processes are constraints to the timely awarding of funding capital
• Provincial and territorial representatives indicated during interviews their willingness to share their data with AAFC, beyond existing AAFC requirements to supply data on the output indicators in their respective bilateral agreements. As the performance measurement strategy was in place since 2014-15 and as the Performance Measurement Working Group meets regularly to discuss performance data, discussions on the steps needed to collect data against all the indicators in the performance measurement strategy could have resulted in the existence of robust data for the purpose of this evaluation

What We Learned

Cost-shared Programming Remains Relevant and is Aligned to Priorities

• There is a continued need for programming to support the Canadian agriculture and agri-food sector toward increased market share, improved productivity and environmental sustainability, improved resilience, and overall enhancement of agricultural Gross Domestic Product growth
• Cost-shared programming is aligned with federal government priorities and departmental strategic outcomes

Examples illustrate progress toward achievement of outputs and outcomes

• The analysis of the output data suggests the likelihood that the program will meet its targets should progress continue as seen to-date
• Progress is being made toward the achievement of the expected outcomes of cost-shared programming

Limitations Regarding Data Hampered the Ability to Assess the Program Performance

• While the program collected data from the provinces and territories, the lack of consistency in language when entering the indicators into the bilateral agreements made it difficult to roll the data up at national level hampering the ability of the Department to fully determine the performance of the program
• As communicated by interviewees during the evaluation, most provinces and territories collect data and are willing to share with AAFC which would lead to a much broader set of output and outcome indicators

Program Financial Administration is Effective, but There is Room for Improvement in Program Design and Delivery

• Cost-shared programming operates economically. A consolidated analysis of efficiency was beyond the scope of the evaluation; however, instances of efficiency were observed
• Growing Forward 2 Cost-shared Programming gives provinces and territories greater flexibility in programming choices; however, departmental approval restrictions delays provinces and territories efforts to award funding capital
• Bilateral communications are generally effective. However, coordination and communication between provinces and territories could be strengthened

Recommendations

Recommendation 1: Programs Branch and Strategic Policy Branch should include in each bilateral agreement a requirement for standardized data against common indicators that will enable AAFC to assess the effectiveness of the Cost-shared Program.

Management Response Agreed. The Multilateral Framework Agreement for the Canadian Agricultural Partnership includes a provision that states that all parties agree to develop consistent and comparable performance indicators and that the process for reporting will be set in each provincial/territorial Bilateral Agreement.

Programs Branch is working with Strategic Policy Branch and with its provincial/territorial counterparts to develop and implement a Performance Measurement Strategy for the Cost-Shared Strategic Initiatives under the Canadian Agricultural Partnership. The Strategy will include a logic model that links indicators to outcomes and a comprehensive data dictionary for outputs, outcomes and indicators. The outcomes and indicators for which the PTs are responsible to reporting on annually will be included in the Bilateral Agreements.

In addition, a provision will be added to PT Bilateral Agreements indicating that data to support progress against the outcomes in the Performance Measurement Strategy will be collected annually through performance measurement reports. These reports will be reviewed annually by AAFC to ensure the quality and integrity of the data.

Recommendation 2: Programs Branch, in collaboration with other Branches, should leverage its existing communication tools and mechanisms to enable sharing and to coordinate communication among provinces and territories supporting innovations, operational best practices, and other areas of common interest.

Management Response: Agreed. The Multilateral Framework Agreement principles underscore that collaboration among stakeholders is a key factor to success and approaches for information sharing are developed in a transparent way.

Programs Branch will encourage federal co-chairs of federal, provincial, territorial working groups (for example, federal, provincial, territorial Innovation Working Group) to incorporate, in their mandate, a role for strengthening the sharing and coordination of information on areas of common interest among provincial/territorial governments.