Language selection

Search

Evaluation of AgriMarketing Stream D: Assurance Systems

March 30, 2017

1 List of Abbreviations

AAFC
Agriculture and Agri-Food Canada
AMS
Agricultural Marketing Services
BRM
Business Risk Management
CCIA
Canadian Cattle Identification Agency
CFIA
Canadian Food Inspection Agency
CIFSI
Canadian Integrated Food Safety Initiative
EU
European Union
FDA
Food and Drug Administration (United States)
FPT
Federal Provincial Territorial
FSA
Food Standards Agency
FSANZ
Food Standards Australia New Zealand
FSRP
Food Safety Recognition Program
GF2
Growing Forward 2
G&C
Grants and Contributions
GFSI
Global Food Safety Initiative
HACCP
Hazard Analysis and Critical Control Point
IGAC
Industry-Government Advisory Committee
ISO
International Organization for Standardization
LSTS
Lab Sample Tracking System
MISB
Market and Industry Services Branch of AAFC
MOU
Memorandum of Understanding
NAFTS
National Agriculture and Food Traceability System
NASD
National Assurance Systems Development Program
NLTO
National Livestock Traceability Operations
OAE
Office of Audit and Evaluation
PAA
Program Alignment Architecture
PB
Programs Branch of AAFC
PID
Premise Identification
PPMRMS
Program Performance Measurement and Risk Management Strategy
RPP
Report on Plans and Priorities
SSBS
Service Sector Biosecurity Standards
TBS
Treasury Board Secretariat
TNIP
Traceability National Information Portal
TTT
Federal Provincial Territorial Traceability Task Team
UK
United Kingdom
US
United States
USDA
United States Department of Agriculture

2 Executive Summary

Background

This report presents the findings of the evaluation of the program, AgriMarketing Stream D: Assurance Systems (hereafter referred to as the Assurance Systems program). The evaluation was conducted by the Office of Audit and Evaluation (OAE), AAFC, in fiscal years 2015-2016/2016-2017 and is identified in AAFC's Five-Year Departmental Evaluation Plan (2015-2016 to 2019-2020). The evaluation responds to the requirements of the Financial Administration Act and the Treasury Board (TBS) Policy on Evaluation (2009) which has since been replaced by the TBS Policy on Results (2016).

The evaluation assessed the relevance and performance for all three components of the Assurance Systems program, encompassing all program activities from 1 April 2013 to 31 March 2016, covering the first three years of the five-year program.

About the Program

Growing Forward 2 is a five-year Federal-Provincial-Territorial (FPT) policy framework (April 2013 to March 2018) for Canada's agricultural and agri-food sector. AgriMarketing is one of the three federal-only funded programs. The objective of AgriMarketing is to advance the Growing Forward 2 priority of improving the agricultural sector's competitiveness in domestic and international markets. AgriMarketing consists of four streams (A, B, C and D), of which the Assurance Systems program is Stream D.

Under AAFC's Program Alignment Architecture the Assurance Systems program is sub-program 1.2.4 of Program 1.2: Market Access, Negotiations, Sector Competitiveness, and Assurance Systems. This sub-program contributes to AAFC's Strategic Outcome 1: A competitive and market-oriented agriculture, agri-food and agri-based products sector that proactively manages risk.

The Assurance Systems program also contributes to the Canadian Food Inspection Agency's (CFIA) Strategic Outcome: A safe and accessible food supply and plant and animal resource base.

Program Objectives, Key Activities and Targeted Results

The overall objective of the Assurance Systems program is to "support the Canadian agriculture, agri-food and agri-based products industry to meet buyer and market demands for assurance and to enhance competitiveness through support for the development of Canadian assurance systems or standards". (See Appendix A for program Logic Model).

To contribute to these overall objectives, the Assurance Systems program consists of three distinct components:

  1. National Assurance Systems Development, Grants & Contributions Component

    This component provides Grants and Contributions (G&C) funding to eligible recipients to help the sector increase its capacity to develop and maintain assurance systems in five areas:

    1. food safety,
    2. animal and plant health surveillance,
    3. market attributes,
    4. traceability, and
    5. integration of systems.
  2. National Livestock Traceability Operations Component (Vote 1)

    This component provides operational funding, known as Vote 1 funding, to the Market and Industry Services Branch (MISB) to facilitate engagement between FPT governments, the Canadian Food Inspection Agency (CFIA) and industry to support the establishment of a National Agriculture and Food Traceability System beginning with livestock and poultry in Canada.

  3. CFIA-delivered Technical Expertise Initiatives (Vote 1)

    Pursuant to the Memorandum of Understanding (MOU) between AAFC and the CFIA, titled Delivery of the Growing Forward 2 Assurance Systems Stream Technical Expertise Initiative, this component provides operational funding to the CFIA in support of three initiatives that are complementary to the other two components of the program:

    1. the delivery of the Food Safety Recognition Program (FSRP),
    2. to develop, maintain and update national biosecurity standards, and
    3. to support the development of traceability regulatory infrastructure in Canada.

Conclusions and Findings

Overall, the program has demonstrated alignment to AAFC and Government of Canada priorities, shown progress towards the production of outputs and the achievement of outcomes and has generally been delivered efficiently.  The program has effectively met its outputs (or is on track to meet them) and outcomes related to systems and standards development. At the same time, insufficient information was available three years into the program, to determine the extent to which expected intermediate and end outcomes will be achieved.  

Program Relevance

The Assurance Systems program enhances the industry's ability to maintain consumer confidence and entry into domestic and international markets. It also enables the industry to realise cost savings by reducing the risks associated with plant and animal health and food safety incidences. In addition, the program offers an avenue by which the industry can benefit from CFIA's international reputation as a competent authority for attesting food safety schemes.

At the midway point through its Growing Forward 2 funding, the Assurance Systems program remains in alignment with departmental and federal government priorities related to innovation, competitiveness and market development. The combined activities of the program demonstrate that government plays an appropriate role in supporting the industry in creating national systems and standards and ensuring plant, animal and human health. The program continues to address the need to enhance the agriculture and agri-food sector's capacity to access domestic and international markets and enhance its market competitiveness, although Canada seems unique in providing government funding to industry to develop voluntary assurance systems. The exception to this is support for the development of traceability systems. The governments of Australia, Uruguay and the United Kingdom have provided financial support for developing traceability systems.

Some examples of how the Assurance Systems program contributes to the sector's capacity to access markets and enhance market competitiveness are:

Finally, social licence (also used interchangeably in AAFC with public trust) will continue to be a driver for the development of assurance systems. The government's role in the social licence between producers and consumers may require further exploration in the development of future programs.

Performance (Effectiveness)

The evaluation determined that the Assurance Systems program is on track to meet its planned outputs. In some instances, such as the number of projects funded under the National Assurance Systems Development G&C component, the targeted outputs were close to being met as of March 2016. Furthermore, systems are being both "advanced" and/or "completed" as planned, as per the stated immediate and intermediate outcomes of the program. The exception to the achievement of outputs is the publishing of proposed regulatory amendments, related to traceability, to the Health of Animal Regulations in Canada Gazette for bison, cattle, sheep, goat and cervids, which is behind schedule.

The evaluation determined that the program's influence on market access could go beyond simply supporting the development of standards and assurance systems, because success, to a large extent, is dependent on the industry implementing the assurance systems and standards that have been developed. Implementation is deferred to provincial and territorial governments through the Cost-Shared program or left to industry to voluntarily implement systems or standards. In this regard, the evaluation identified enabling factors that could facilitate the implementation of assurance systems and standards. The enabling factors include industry having compatible infrastructure in place, and keeping stakeholders informed and supported. Also, barriers that could impede the implementation of assurance systems and standards include a general resistance to change and/or cost prohibitions, particularly for smaller, less mature sectors.

Finally, while the evaluation found some examples of how G&C projects have incrementally built upon previously funded projects to attain or expand market access, the data was not sufficient to conclude that funding helped sustain any such expanded market access under Growing Forward 2. Given the evaluation was conducted at the midway point of Growing Forward 2, more time will likely need to pass, as systems are completed and implemented, to better assess this outcome.

Performance (Efficiency)

Communications is an important activity in the delivery of the Assurance Systems program. The evaluation found that there are efficient knowledge exchange mechanisms in place, particularly through the Traceability Task Team and the Industry-Government Advisory Committee forums and that the program provided value to stakeholders by way of knowledge exchange mechanisms and the sharing of best practices. While effective communication channels existed between AAFC and the CFIA in delivering all aspects of the program, there were challenges related to roles and responsibilities delineated in the MOU where there are shared responsibilities.

At the time this evaluation was conducted, one unresolved gap remained with industry on CFIA's proposed traceability regulations. This was related to the requirement to make all livestock movement reporting mandatory. Finding common ground with industry on livestock movement reporting will be important in moving forward with regulatory amendments as it could impact the industry's ability to implement traceability systems that meet regulatory requirements and ultimately access markets and respond to market opportunities. Since this evaluation was completed, progress has been made towards amendments to the Health of Animal Regulations. In addition, the Traceability National Information Portal, as the single window into traceability information, has yet to be fully populated with partner information.

Regarding the National Assurance Systems Development G&C component, the evaluation determined that the application process was clear and transparent and that technical reviewers considered that their feedback was incorporated appropriately. At the same time, the AAFC-led internal audit noted that the Assurance Systems program could improve on documenting the eligibility assessment process.

The evaluation also determined that although actual G&C expenditures were 53% lower than planned for the three-year period from April 2013 to March 2016, the number of projects that were funded was on pace to meet the target. A lack of awareness about the Assurance Systems program amongst stakeholders may have been a contributing factor for the G&C underspending. Also, although industry organizations considered the 75:25 (government: funding recipient) ratio to be appropriate, smaller, resource-limiting organizations suggested that the recipient share could be lower than 25%.

Finally, the evaluation determined that the Assurance Systems program has been successful in supporting the sector in developing assurance systems and standards.

The program theory puts the role of supporting the implementation of systems and standards to provincial and territorial governments through the Growing Forward 2 Cost-Shared program. At the same time, the program theory does not clearly demonstrate how it influences provincial and territorial governments to support implementation and ultimately contribute to improving the industry's ability to access markets, respond to market opportunities and compete domestically and internationally. However, given that the evaluation was conducted at the midway point of Growing Forward 2, more time will likely need to pass, as systems are completed and implemented, to better assess this outcome. The exception to this is livestock traceability systems which, once regulated, industry participation will be required.

Recommendations

The following recommendations are based on the findings and conclusions of the evaluation and are intended to facilitate programming for the Next Policy Framework. While the programming for the Next Policy Framework is currently at the development stage, this evaluation takes into consideration the commitments outlined in the Calgary StatementFootnote 1 and presents the following recommendations in order to facilitate program design under the Next Policy Framework and also to inform AAFC's transition to the Results and Delivery agenda.

3 Introduction

3.1 Background

This report presents the findings of the Evaluation of the Assurance Systems program. The evaluation was conducted by the Office of Audit and Evaluation (OAE), AAFC, in fiscal years 2015–2016/2016-2017 and is identified in AAFC's Five-Year Departmental Evaluation Plan (2015–2016 to 2019–2020). The evaluation responds to the requirements of the Financial Administration Act and the TBS Policy on Evaluation (2009) which has since been replaced by the TBS Policy on Results (2016).

The evaluation assessed the relevance and performance for all three components of the Assurance Systems program, encompassing all program activities from 1 April 2013 to 31 March 2016 covering the first three years of the five year program.

Under AAFC's Program Alignment Architecture the Assurance Systems program is sub-program 1.2.4 of Program 1.2: Market Access, Negotiations, Sector Competitiveness, and Assurance Systems. This sub-program contributes to AAFC's Strategic Outcome 1: A competitive and market-oriented agriculture, agri-food and agri-based products sector that proactively manages risk.

The Assurance Systems program also contributes to CFIA's Strategic Outcome: A safe and accessible food supply and plant and animal resource base.

An evaluation strategy for the program was jointly developed in 2014 by AAFC and CFIA. Based on the evaluation strategy and additional input from program managers, an evaluation framework and a Terms of Reference were developed. The Terms of Reference was approved in May 2016.

3.2 Program Profile

3.2.1 Context

Growing Forward 2 is a five-year policy framework (April 2013 to March 2018) for Canada's agricultural and agri-food sector consisting of Business Risk Management (BRM) and non-BRM programming. The non-BRM programming consists of three Federal-only programs. AgriMarketing is one of the three federal-only programs, and is administered by AAFC.

The objective of AgriMarketing is to advance the Growing Forward 2 priority of improving the agricultural sector's competitiveness in domestic and international markets. AgriMarketing consists of four Streams (A, B, C and D). This evaluation focuses on AgriMarketing Stream D: Assurance Systems (hereafter referred to as the Assurance Systems program)

3.2.2 Overview of Program

The program theory, outlined in the Program Performance Measurement and Risk Management Strategy (PPMRMS), reasons that Canada is experiencing various challenges as a result of the evolving nature of the global agriculture and agri-food sector. Being trade-oriented, the Canadian sector's sustainability and growth rests on its capacity to remain competitive in both domestic and global markets; this, in turn, depends on its capacity to compete on costs and also satisfy new consumer demands. The emergence and growth of agriculture and agri-food sectors in developing countries, and the rapid evolution of global food retailing, as a result of new technologies and new consumer demands, are shaping the evolution of both domestic and international markets.

Satisfying consumer demands or preferences often means being able to demonstrate to the consumer that the food they are buying is safe to eat and that it has been produced in a sustainable and ethical manner. Consumer preferences, also known as market attributes, could also include characteristics or quality standards that are desirable, such as religious requirements (for example, halal or kosher foods) or animal welfare standards. To show that a food has certain market attributes or quality standards often means being able to demonstrate that the food satisfies established assurance systems or standards.

The overall objective of the Assurance Systems program is to "support the Canadian agriculture, agri-food and agri-based products industry to meet buyer and market demands for assurance and to enhance competitiveness by supporting the development of Canadian assurance systems or standards."

Program Components

The Assurance Systems program consists of three distinct componentsFootnote 2: (1) National Assurance Systems Development, (2) National Livestock Traceability Operations, and (3) CFIA delivered Technical Expertise Initiative.

  1. National Assurance Systems Development

    The National Assurance Systems Development component provides G&C funding to eligible recipients to help the sector increase its capacity to develop and maintain assurance systems in five areas. Eligible recipients include not-for-profit organizations (including cooperatives) and academic institutions. The term "systems" includes everything from certification processes, training materials and communications products, to databases, software or hardware. The five eligible areas of funding are:

    • Food Safety Systems: The Assurance Systems program provides funding to industry organizations to develop food safety systems and submit them for review to CFIA's voluntary Food Safety Recognition Program, which is also funded though the Assurance Systems program. Project activities could include the preparation of producer/enterprise manuals, training of auditors/trainers and the management system.Footnote 3
    • Animal and Plant Health Surveillance Systems: The Assurance Systems program provides funding to industry-led projects focused on developing a national disease surveillance system for all farmed animals and/or cultivated plants.Footnote 4
    • Market Attributes and Quality Standards: The Assurance Systems program provides funding to industry-led projects designed to develop market-relevant attributes, product quality standards and tools that demonstrate that industry is adhering to established standards.Footnote 5
    • Traceability Systems: The Assurance Systems program provides funding to industry-led projects that develop and implement a system that can track and trace the movement of a plant, animal or product along the supply chain.Footnote 6
    • Integration of Assurance Systems: The program supports industry-led efforts to integrate different assurance systems for a specific commodity, as well as benchmarking existing domestic standards to international standards.

    The maximum G&C funding that AAFC contributes to a project normally will not exceed $1,000,000 per project.Footnote 7 Projects are cost-shared with recipients. The ratio of Federal Government to recipient funding is 75:25 respectively. If funding is received from other government institutions as part of the applicant's share (commonly called "government stacking"), the total contribution from all levels of federal, provincial and territorial governments cannot exceed 85%.Footnote 8

  2. National Livestock Traceability Operations

    A key objective of the Assurance Systems program is to facilitate the establishment of a national livestock traceability system for Canada for six species. This objective honours the 2006 federal, provincial and territorial (FPT) Ministers' commitment for a National Agriculture and Food Traceability System beginning with livestock and poultry. Such a system requires the collaboration of FPT governments, CFIA and the industry. The Assurance Systems program provides operational funding (Vote 1) to the National Livestock Traceability Operations, MISB, to facilitate FPT engagement and information sharing among these three players. Specifically, National Livestock Traceability Operations activities include the following:

    • Supports to various forums to promote collaboration and communications between FPT governments and industry regarding livestock traceability implementation. Two such forums are the Traceability Task Team and the Industry-Government Advisory Committee. The Traceability Task Team is an FPT committee that provides leadership and coordination on livestock traceability among jurisdictions. The Industry-Government Advisory Committee is an advisory body, comprised of FPT governments, CFIA and industry, that leads on the development and implementation of the livestock component of the National Agriculture and Food Traceability System.
    • Developing and operating the Traceability National Information Portal, which is an information sharing database that allows authorized users to simultaneously search for traceability information to effectively address animal disease situations and sanitary issues.Footnote 9 The tool can accept multiple species' information, as well as provincial-territorial identification information. The Portal is a key component of a national livestock traceability system because its intent is to house key information on livestock traceability by industry and governments. The CFIA, through its Information Management/Information Technology function, is funded by AAFC to provide support, maintenance and future enhancements to the Traceability National Information Portal.
    • AAFC is also responsible for creating and disseminating traceability-related communications material within AAFC and to the Industry-Government Advisory Committee.
  3. CFIA-Delivered Technical Expertise Initiatives

    The Memorandum of Understanding (MOU) between AAFC and CFIA, Delivery of the Growing Forward 2 Assurance Systems Stream Technical Expertise Initiative, provides operational funding (i.e., Vote 1 funding) to the CFIA to carry out three initiatives:

    • The Food Safety Recognition Program Initiative: Funding is provided for the delivery of the Food Safety Recognition Program that provides government recognition to on-farm and post-farm food safety systems developed by national industry organizations. The Food Safety Recognition Program is based on the internationally recognised science-based Hazard Analysis Critical Control Points (HACCP) standards. (Note: the National Assurance Systems Development component provides G&C funding to industry organizations to develop the food safety systems and apply to the Food Safety Recognition Program).
    • National Biosecurity Standards Development Initiative: Farm-level biosecurity is a set of practices used to minimize the transmission of pathogens and pests in animal and plant populations including their introduction (bio-exclusion), spread within the populations (bio-management), and release (bio-containment. Funding is provided to CFIA to develop, maintain and update national biosecurity standards.
    • Traceability Management Office Regulatory Infrastructure Initiative: Funding is provided to CFIA to facilitate the development of the policy and regulatory infrastructure to put traceability authorities, agreements, protocols and other elements in place for livestock and poultry and amend traceability regulations pertaining to the Health of Animal Regulations.

    These three initiatives are complementary to the other two components of the program. CFIA provides AAFC with annual progress reports on its performance based on outputs outlined in the MOU.

Governance

The three components of the Assurance Systems program are managed by AAFC's Programs Branch and Market Industry Services Branch as follows:

Key Stakeholders

In general, the key stakeholders of the Assurance Systems program are those organizations promoting and representing the Canadian agriculture, agri-food, fish and seafood sector, including national industry associations and alliances. Canadians are the ultimate beneficiaries of the Assurance Systems program activities; however, their stake is dependent on the implementation of activities by others in the system.

Specifically, the three components of the Assurance Systems program each target different stakeholders more directly. For the G&C component, eligible applicants are mainly national (or regional organizations representing national interests) not-for-profit industry organizations (such as, industry associations) in the agriculture or agri-food sector, academic institutions or other non-governmental entities.

The National Livestock Traceability Operations component services agriculture and agri-food national industry stakeholders along the value chain and FPT governments. The information generated by the National Livestock Traceability Operations component also informs decisions, by veterinarians and scientists, related to mitigating risks and responding to animal disease emergency or sanitary issues. The use of this information to support risk mitigation and the emergency management function of traceability is one that is more closely aligned to CFIA's roles and responsibilities and is not directly linked to the outcomes of the Assurance Systems program.

Finally, the CFIA delivered component targets on-farm and post-farm organizations eligible for the Food Safety Recognition Program, the National Biosecurity Standard Development Initiative targets national commodity organizations and farm service sectors, and the Traceability Management Office Regulatory Infrastructure Initiative targets national organizations, producers and FPT governments.

3.2.3 Program Resources

The Assurance Systems program is a Growing Forward 2 federal-only program that has a notional allocation of $105,477,500 over five years from April 2013 to March 2018. Of this amount, $71,350,000 is provided in the form of G&C and $34,127,500 is provided as operational funding to support the management and delivery of the program, the National Livestock Traceability Operations and the CFIA MOU for Technical Expertise (such as, Vote 1, which includes non-pay-operating, salaries and employee benefit plans).

Table 1 - Assurance Systems Program Expenditures (Budgeted and Actuals)

Assurance Systems (1.2.4) - Salary
2013-2014 2014-2015 2015-2016
Vote % Actual Budget Actual % Actual Budget Actual % Actual Budget Actual
Salary 1 100% $2,669,842 $2,669,842 100% $1,656,068 $1,656,068 100% $1,881,213 $1,881,213
Program Branch 1 69% $1,832,733 $1,832,733 64% $1,065,277 $1,065,277 74% $1,389,661 $1,389,661
Market and Industry Services Branch 1 31% $837,110 $837,110 31% $515,033 $515,033 26% $488,295 $488,295
Information Systems Branch 1 0% 0 0 0% 0 0 0% 0 0
Corporate Management Branch 1 0% 0 0 5% $75,758 $75,758 0% $3,257 $3,257
Science and Technology Branch 1 0% 0 0 0% 0 0 0% 0 0
Assurance Systems (1.2.4) - Non-Pay Operational
2013-2014 2014-2015 2015-2016
Vote % Actual Budget Actual % Actual Budget Actual % Actual Budget Actual
Non-Pay Operational 1 100% $342,729 $342,729 100% $648,955 $648,955 100% $652,562 $652,562
Program Branch 1 20% $69,032 $69,032 5% $32,271 $32,271 15% $94,981 $94,981
Market and Industry Services Branch 1 78% $268,546 $268,546 43% $281,684 $281,684 12% $76,741 $76,741
Information Systems Branch 1 0% 0 0 52% $335,000 $335,000 74% $480,840 $480,840
Corporate Management Branch 1 0% 0 0 0% 0 0 0% 0 0
Science and Technology Branch 1 2% $5,151 $5,151 0% 0 0 0% 0 0
Assurance Systems (1.2.4) - Total
2013-2014 2014-2015 2015-2016
Vote % Actual Budget Actual % Actual Budget Actual % Actual Budget Actual
Salary 1 100% $2,669,842 $2,669,842 100% $1,656,068 $1,656,068 100% $1,881,213 $1,881,213
Non-Pay Operational 1 100% $342,729 $342,729 100% $648,955 $648,955 100% $652,562 $652,562
G&C 10 N/A $14,270,000 $2,121,640 Note 1 $6,270,000 $5,414,761 Note 2 $10,183,188 $6,903,917
Employee Benefit Plan N/A N/A $434,576 $434,576 N/A $281,646 $281,646 N/A $298,186 $298,186
Subtotal N/A N/A $17,717,147 $5,568,787 N/A $8,856,669 $8,001,430 N/A $13,015,149 $9,735,878
Canadian Food Inspection Agency - Memorandum of Understanding 1 Note 3  $2,500,000  $2,500,000 Note 4  $2,441,163  $2,441,163 Note 5  $2,500,000  $2,500,000
Total N/A N/A $20,217,147 $8,068,787 N/A $11,297,832 $10,442,593 N/A $15,515,149 $12,235,878
Circovirus 10 Note 6 $1,000,000 $511,244 N/A 0 0 N/A 0 0
Notes: 

Note 1

2014-15 Vote 10 Authority $14.27M less in-year transfers ($2M to WED and $6M to Enabling Commercialization).

Note 2

2015-16 Vote 10 Authority $14.27M less in-year transfers ($1M WED and $3M to Market Development).

Note 3

2013-14 $2.5M was transferred in Supps B.

Note 4

2014-15 CFIA MOU amount was reduced by unspent funds in FY 2013-14.; transferred in Supps B.

Note 5

2015-16 $2.5M was transferred in Supps A.

Note 6

2013-14 Circovirus was included under this PAA as no box was available on the PAA for this program.

Note 7

The above figures are as reported in the DPR. 

N/A = Not Applicable

3.3 Evaluation Scope and Methodology

The Evaluation of the Assurance Systems program was conducted in 2015-2016 as part of AAFC’s Five-Year Departmental Evaluation Plan (2015-2016 to 2019-2020). The evaluation fulfils the requirements of the Financial Administration Act and the TBS Policy on Evaluation (2009), which has since been replaced by the TBS Policy on Results.

The evaluation used multiple lines of evidence to assess the program activities undertaken and delivered from April 2013 to March 2016. This includes a literature review, document review, financial review, and key informant interviews (a full description of the evaluation scope and methodology can be found in Appendix J).

4 Evaluation Findings

4.1 Relevance

4.1.1 Continued Need for the Program

Finding 1: The Assurance Systems program is relevant because it supports the industry's need to maintain consumer confidence and to maintain access to existing markets.

The Assurance Systems program enhances the industry's ability to maintain consumer confidence and entry into domestic and international markets. It also enables the industry to realise cost savings by reducing the risks associated with plant and animal health and food safety incidences. In addition, the program offers an avenue by which the industry can benefit from CFIA's international reputation as a competent authority for attesting food safety schemes.

Access to International Markets

It was not possible to identify instances where a Growing Forward 2 funded assurance system project contributed to accessing international markets. This may be because the evaluation looked at the first three years of the program. However, examining projects funded under the previous framework, Growing Forward, helped to illustrate the importance of assurance systems in maintaining consumer confidence and maintaining access in existing markets. Some interviewees indicated that Canada's products are seen as high quality and safe by the international market, and thus consumers are willing to pay more for them. It is not clear whether recipients funded through the G&C component of the program, however, obtained a premium in the marketplace.

The potential for maintaining existing markets and setting the groundwork for improving access to new international markets was also highlighted through the case studies:

Cost Savings for Industry

The Assurance Systems program facilitates cost savings for producers by reducing the risks associated with plant and animal health and food safety incidences. According to a few interviewees, a well implemented assurance system can not only reduce the need for testing and sampling requirements, but also, a strong assurance system can, for example, helps reduce the risk of infectious diseases such as bovine tuberculosis, thus helping the industry avoid costs associated with such outbreaks. 

The Manitoba Cattle Producers Association received G&C program funding to proactively address such risks. The goal of the funded project was to enhance the scientific tools available for surveillance of bovine tuberculosis in a region of Manitoba. The tools are expected to lead to a surveillance system modelled after a system developed by CFIA, and is expected to have the potential to be adapted and applied to other areas across Canada.

Food Safety Recognition Program

Industry capacity to export agriculture and agri-food products is enhanced by CFIA standing among Canada's trading partners as Canada's competent authority in attesting the safety of Canadian food products. Funding provided by AAFC to CFIA through the Assurance Systems program for CFIA's Food Safety Recognition Program provides government recognition for on-farm and post-farm food safety systems developed by national industry organizations (via a three-step technical review process). Such recognition contributes to making the sector more competitive by meeting internationally recognized HACCP standards and allowing producers to "brand" products as being recognized as safe. The program funding specifically enabled two organizations (Dairy Farmers of Canada, 2016 and Chicken Farmers of Canada, 2013) to apply for and achieve the final stage of the Food Safety Recognition Program.Footnote 10 In addition, there were 20 organizations in the various stages of review, as of March 2016.

Confidence by Canada's trading partners that Canada's food system is safe is an important factor in ensuring market access. Internationally, governments recognize that market access is facilitated by formal recognition of a trading partner's food safety system (one that meets the national standard and is at least equivalent to its own standards). This directly influences a company's or sector's ability to move food and animals across provincial, territorial and international borders. For example, the US Food and Drug Administration's (FDA) Food Safety Modernization Act (2011) implemented new regulations for trading partners, like Canada, who must now verify that their foreign suppliers have adequate preventive controls in place to ensure safetyFootnote 11 The CFIA is viewed by Canada's international trading partners as Canada's competent authority in attesting the safety of Canadian food products. As such, countries, such as, the United States and the European Union, allow imports from Canada in cases where the their food safety authority has recognized that a food safety system is at least equivalent to their country's food safety system, for those commodities that are regulated in the respective countries.Footnote 12Footnote 13 The critical element is that CFIA, being recognised as a competent authority, is well positioned to attest that Canadian food products are safe.

4.1.2 Alignment with Government Priorities

Finding 2: The overall objective of the Assurance Systems program was aligned to AAFC's and the Government of Canada's priority to support innovation, competitiveness and market development in agriculture.

The objective of the Assurance System program is to "support the Canadian agriculture, agri-food and agri-based products industry to meet buyer and market demands for assurance and to enhance competitiveness through support for the development of Canadian assurance systems or standards." Evidence demonstrates the program's alignment to the priorities of AAFC and the Government of Canada.

The Assurance Systems program supports AAFC's Strategic Outcome 1: A competitive and market-oriented agriculture, agri-food and agri-based products sector that proactively manages riskFootnote 14 by contributing to market competitiveness and food safety assurance, as demonstrated in the Department's Reports on Plans and Priorities (RPP). AAFC's Program Alignment Architecture has the Assurance Systems sub-program under Program 1.2: Market Access, Negotiations, Sector Competitiveness, and Assurance Systems. This was consistent with AAFC's 2014–15 RPP.Footnote 15

The program objective was also consistent with the Government of Canada priority to support the agriculture sector. For example, Budget 2013–14 committed support to innovation, competitiveness and market development in agriculture. Seven federal documents produced since 2013 mention the importance of the agricultural sector in Canada (Figure 1). Budget 2016 also draws the link between food safety and export opportunities by emphasizing the importance of Canada's reputation in instilling trust in foreign buyers, thereby maintaining or attaining market access. Finally, in 2006, FPT agriculture ministers committed to phasing in a National Agriculture and Food Traceability System (NAFTS), beginning with livestock and poultry.Footnote 16

Figure 1 - Key federal documents that make reference to the importance of the agricultural sector and competitiveness
Description of this image follows.
Description of above image

Figure 1 shows the key federal documents that made reference to the importance of the agricultural sector and competitiveness from 2013 to 2016, as follows:

  • The 2013 Throne Speech and the Federal Budget
  • The 2014 Federal Budget
  • The 2015 Throne Speech, AAFC's Ministerial Mandate Letter and the Federal Budget
  • The 2016 Federal Budget

Source: References Footnote 17 Footnote 18 Footnote 19 Footnote 20 Footnote 21 Footnote 22 Footnote 23

The contribution of agriculture to the Canadian economy is an additional factor in the program's importance to Government of Canada objectives. The agricultural sector has historically contributed billions to the Canadian economy annually.

Economic Value of the Agriculture and Agri-Food Sector

In 2014, Canadian export sales were valued at $51.5 billion. The agriculture and agri-food sector:

The sector's annual GDP growth averaged 1.1% from 2010 to 2014.

Source: Agriculture and Agri-Food Canada, "An Overview of the Canadian Agriculture and Agri-Food System, 2016

Looking forward, the world population is expected to rise to more than 7.6 billion by 2020. Income growth, especially in emerging economiesFootnote 24, will lead to an increased demand for food. Furthermore, the food industry is expanding to accommodate a range of consumer demands (e.g., organics, low-carbon footprint, products with health attributes, increased convenience, local production). The increase in global demand will lead to increased competition from low-cost commodity-producing competitors. In order to remain competitive, the Canadian industry will be required to compete on attributes. The program remains relevant and will continue to be so, especially if it can continue to embrace emerging consumer demands for new market attributes.

4.1.3 Alignment with Federal Roles and Responsibilities

Finding 3: AAFC and CFIA play an appropriate role in delivering the Assurance Systems program, with the provincial governments and industry playing a complementary role related to the implementation of systems.

Government of Canada

The Canadian food safety system comprises several government players. Health Canada develops food safety and nutrition standards and policies and assesses food safety risks. The CFIA, guided by the Safe Food for Canadians Act, verifies that industry is meeting federal food safety and regulatory requirements, and sets standards to detect and prevent risks to Canada's food supply. The Public Health Agency of Canada conducts food-related illness surveillance and outbreak investigations. The role of AAFC is to facilitate growth, competitive, innovation and sustainability of the Canadian agriculture and agri-food sector.Footnote 25Footnote 26 Each of these federal departments and agencies contribute to the agriculture and agri-food industry's capacity to maintain and access markets.

The literature shows that these roles closely follow international trends as food safety regulation is traditionally seen as a responsibility of governments, generally in the form of compulsory standards as a basis for guaranteeing that food is safe when delivered to consumers.Footnote 27 Research also suggests that government generally plays an important role regarding public confidence in the food safety system, because it enforces rules or regulations, and oversees industry and recalls.Footnote 28

Agriculture and Agri-Food Canada

The Department's market access-oriented mandate is relevant in delivering all components of the program. This includes its funding to the CFIA. Those interviewed from both government and industry confirmed that there was a clear role for the Federal Government in ensuring public good as it relates to food safety, traceability and biosecurity. For example, the Federal Government plays a role in ensuring that food safety systems are developed in line with internationally recognised standards.  Moreover, in a 2014 survey of 3,020 Canadian adults conducted by Ipsos Reid for AAFC, 56% of respondents were "very or completely confident in the safety of Canadian food products" and about half were confident in the Canadian food system's management of concerns regarding animal diseases.Footnote 29

Those interviewed also pointed to the appropriateness of the federal role in cases where projects are of national scope. Examples include support for national traceability systems through the activities of the National Livestock Traceability Operations component, as well as support for scientific research on animal disease and development of national biosecurity standards. Specifically, under the MOU between AAFC and CFIA, funding is allocated to develop "nationally consistent plant and animal farm-level biosecurity standards as well as national service sector biosecurity standards". The CFIA completed eight national farm-level commodity specific biosecurity standards (bees, beef, dairy, sheep, goat, fur-bearing, potato, and grains and oilseeds) and one service sector standard (poultry) under Growing Forward. Growing Forward 2 funding focuses on completing national commodity specific standards for the equine, cervid, fruit/tree nut, and greenhouse sectors and one service sector standard for livestock, poultry and deadstock transport.

Canadian Food Inspection Agency, Provincial/Territorial Governments and Industry

Those interviewed recognise the CFIA as an enforcement agency, while AAFC is viewed as a system funder and facilitator in the context of assurance systems. There is also a perception that the federal role ends at the system development stage, leaving flexibility for the provincial governments and industry to implement assurance systems and to track uptake and impact. The evaluation identified some provincial programs that fund similar assurance system initiatives on a regional level (See Appendix B). Provincial government programs were perceived by interviewees as complementary to federal funding with the federal programs focused on projects with a national scope while provincial government programs support local initiatives.

Finding 4: The federal role in supporting the development of voluntary assurance systems appears to be unique internationally (with the exception of Traceability systems).

"Making a specialty claim about an attribute is key [to market access abroad], along with the need to substantiate the claim." (Government stakeholder)

"If a demand is from inside the industry (e.g., where a processor wants to be able to say, 'All our ingredients are sustainable') and if there is not a huge demand from consumers for more information than that, and if consumers are satisfied, then there is no real need for government to step in." (Government stakeholder)

"Virtually everything we do is in response to marketplace demand, which manifests itself either in trade barriers or buying choices." (Industry stakeholder)

Based on the review of literature, Canada appears to be unique in its approach to providing funding for the development of voluntary assurance systems through the G&C component of the program. In comparable international jurisdictions, such as Australia, the United Kingdom, and the United States, the evaluation did not find similar federal- or national-level funding for such activities. Rather, assurance systems in these countries have developed from private or market-based incentives, and the government's role is focused on enforcement of mandatory compliance and providing guidance or technical support on standards implementation. A 2013 evaluation of the Canadian Integrated Food Safety Initiative drew the same conclusion, stating that "Canada differs significantly from many other industrialized countries in the financial support it offers for the development of these systems."Footnote 30 Appendix C provides a short profile of these international jurisdictions. The exception to this is support for traceability systems from the governments of Australia, Uruguay, and the U.K., which have mandated and financially supported the development of traceability systems.

While there is evidence that government policies help maintain access to international agricultural markets, consumer demand is also a strong driver for development of assurance systems. This was reflected in the interview data, where some interviewees from both government and industry placed the responsibility on industry to respond to a consumer demand.

Finally, the emerging concept of social licence (also used interchangeably in AAFC with public trust) could shape the federal role in programs focused on supporting the development of voluntary assurance systems. Social licence is defined by the Canadian Federation of Agriculture as the "ongoing level of acceptance, approval and trust of consumers regarding how food is produced."Footnote 31 It encompasses issues such as environmental impact, biotechnology and animal welfare, among others. Increasingly, the agricultural sector is facing scrutiny across many of these areas, influencing its social licence with consumers. Some interviewees suggested that the public trust concept is already rooted in some sectors. Traditionally, the Canadian agricultural industry has responded to consumer concerns through communication, the development of assurance systems and by changing farming practices. The literature suggests that the social licence is a two-way conversation between the industry and consumers. The role of government is to support industry in cultivating good relationships with consumers, but not play the role of intermediary. The focus of social licence is for industry to proactively establish a strong level of trust with consumers.Footnote 32 A paper from the Canadian Federation of Agriculture suggests that social licence should be considered during development of the Next Policy Framework. It encourages government and industry to develop programs together that allow producers to illustrate continuous improvement in the themes that impact agriculture's social licence.Footnote 33

4.2 Performance

4.2.1 Achievement of Expected Outcomes (Effectiveness)

Finding 5: Assurance systems and standards are on track for advancement or completion as intended, although the development of traceability regulations is behind schedule.

Program documentation shows outputs are on track to meet intended targets for all three components of the Assurance Systems program (see Appendix E). Additionally, with regard to the advancement and completion of assurance systems, most funded under the program are on track to meet their stated immediate and intermediate outcome targets by March 2018 (definitions for "advanced" and "completed" are provided in Appendix F). Table 2 (page 25) provides a summary of outputs and outcomes achieved under each component. Additional supporting information on systems advancement and completion is provided in Appendix G.

The exception to the achievement of outputs is the publishing of proposed regulatory amendments to the Health of Animal Regulations for cattle, bison, sheep, goat and cervids. These amendments were scheduled to be published in Canada Gazette by December 2015. At the time of the evaluation, this output was behind schedule, although the regulatory amendments for pig traceability were published in 2014. While traceability systems could be developed in the absence of regulations, this may have impacted the demand for funding under the Assurance Systems program.

Table 2 - Targeted achievements of program activities
Funded Component Activity Target
(on track/not on track)
Achieved as of
March 2016
National Assurance Systems Development Systems advanced or completed
  • By 2018, 22 advanced and 18 completed - On track
  • 17 advancing
  • 19 completing
National Livestock Traceability Operations Traceability system implementation
  • System fully in place for 7 species by 2018 - On track
  • 2 system complete (poultry, hogs) and 5 in progress (cattle, bison, goats, sheep and cervids)
CFIA Technical Expertise Food Safety Recognition Program
  • Service standards met (90%) - On track
  • Standards (for example, issuing letters of completion, conducting reviews and providing reports to AAFC) met for 2015-16
CFIA Technical Expertise Biosecurity standards completed
  • 2-4 commodity sector by 2017-18 – On track
  • 1-6 service sector by 2017-18 - On track
  • Up to 3 national standards reviewed and updated - On track
  • 2 completed and posted (greenhouse, fruit/nut trees), 2 in progress
  • 0 complete but preliminary work plan and timelines identified for development of 1 transport sector standard (livestock, poultry, deadstock)
  • 1 standard (Avian) identified for review and revision
CFIA Technical Expertise Traceability Management Office
  • 10 provinces ratify data sharing agreements by March 2018 - On track
  • 5 administrator agreements signed by 2018 - On track
  • Proposed regulatory amendments to the Health of Animal Regulations for pigs in Canada Gazette by March 2014 - On track
  • Proposed regulatory amendments to the Health of Animals Regulations for cattle, bison, sheep, goat, cervid and equid in Canada Gazette by December 2015 – Behind schedule
  • 8 provincial agreements ratified (Manitoba, Alberta, Saskatchewan, Prince Edward Island, Ontario, New Brunswick, Nova Scotia, and British Columbia), 1 drafted but not signed (Quebec), 1 no data (Newfoundland)
  • All regulated species have a responsible administrator. 2 administrator agreements signed (Canadian Cattle Identification Agency and Canadian Pork Council) and 2 in draft (Canadian Sheep Federation and Agri-Traçabilité Québec
  • Regulatory amendments for pig traceability published in Canada Gazette in 2014
  • Regulatory amendments behind schedule

Finding 6: While the Assurance Systems program supports industry-led projects in which assurance systems and standards are developed, there are enabling factors that could facilitate industry in implementing systems and standards, such as having compatible infrastructure in place and keeping stakeholders informed. Similarly, there are barriers to industry implementing systems and standards, such as cost prohibition and a general resistance to change.

The performance of the Assurance Systems program is dependent on industry-led projects that develop systems and standards the successful implementation of the systems and standards that have been developed. Despite the Assurance Systems program being dependant on provincial and territorial governments supporting implementation and/or industry voluntarily implementing systems and standards, understanding the factors that enable or limit implementation could help the Department identify approaches to encourage implementation. The evaluation identified enabling factors and barriers that influence the industry's capacity to implement systems or standards:

  1. Industry requires modern infrastructure to be in place in order to implement modern assurance systems. The absence of modern infrastructure may impede the adoption of systems and standards. The literature showed that, globally, modern electronic infrastructure has facilitated the implementation of assurance systems and standards. For example, many countries are using electronic databases to facilitate storing and transferring of information when animals or products are moved from one location to another. An example is the Canadian Cattle Identification Agency's Canadian Livestock Tracking System that identifies live cattle and bison with radio-frequency identification tags. The use of such technology is intended to increase efficiency and accurate data collection when there is a need to track diseased animals.Footnote 34
  2. Encouraging the integration of systems may facilitate implementation and respond to international market demands. Increasingly, industry groups are shifting from developing individual systems to integrating systems to create one comprehensive system or platform for sector management. A commonly cited example in both documents and interviews was the proAction initiative, from the Dairy Farmers of Canada, which encompasses six independent systems (milk quality, food safety, animal care, traceability, biosecurity and environmental sustainability) in one integrated package.  
  3. The literature suggested that the implementation of assurance system is facilitated by the availability of adequate information for planning, execution and monitoring functions and the support of company managers or decision-makers.Footnote 35 Having engaged and informed leaders at both the project and program level was also cited as an enabling factor by some of those interviewed for this evaluation: "It boils down to the people that were leading [the initiative]. They worked to build support and engagement early on in the project; their leadership is critical."
  4. Interview data indicates that small industries, with fewer resources, will wait until larger industries have implemented standards, before doing so themselves, which could be a barrier. For example, although the Turkey Farmers of Canada have passed the CFIA Food Safety Recognition Program technical review part II (management manual) for commercial meat production, a module was only recently added for the turkey breeders (a much smaller group). Moreover, a study in Ontario found that the main barriers faced by firms in implementing a HACCP system included financial constraints and perceptions. There was a perception that  implementation was not appropriate/necessary and that the scale of change required to achieve implementation was overwhelming.Footnote 36
  5. Finally, a barrier noted only by recipients of G&C funding is specifically applicable only at the project level. The key challenge expressed by program recipients was in not having enough flexibility in what constituted eligible expenses. For example, some program recipients that were interviewed said that in various cases project funding was not approved or was lower than requested for communications activities, long-term strategic planning, international travel, consultant salaries and administrative overhead. The interviewees explained that these types of activities are critical to project success overall, and that having to source external funding for such activities often delayed implementation.

Finding 7: Assurance Systems programming has effectively built upon advancements made during Growing Forward, thus maximizing the value of Growing Forward 2 investments. However, it is too soon to observe sustained, expanded market access resulting from the Growing Forward 2 funding.

All lines of evidence indicate that Growing Forward 2 maximized value by incrementally building on advancements made during Growing Forward.

Expansion of Assurance Systems

In two of the case studies conducted, projects were building directly on previous AAFC investments. The Pulse Canada project, under Growing Forward 2, refined and improved upon sustainability indicators that were developed under AAFC's Agricultural Flexibility Fund (2009–2014), so that they could be used nationally. In addition, the program funding allowed Pulse Canada to develop new indicators on biodiversity and water quality.Footnote 37

Similarly, AAFC has provided funding for several consecutive food safety projects conducted by the Jewish Community Council of Montreal since Growing Forward, beginning with the development of food safety standards for kosher oil, confectionery and bakery products in 2008. A completed Growing Forward 2 project built on a Growing Forward project for oil, confectionery and bakery products, to create a standard known as Canada Kosher 22000™. This standard, which includes a set of methods, software and audit guidelines, integrates kosher elements with food safety elements set out by the International Organization for Standardization (ISO). An ongoing Growing Forward 2 project now aims to adapt the Canada Kosher 22000™ scheme for other specialty foods such as halal, allergen-free, and lactose-free.

Biosecurity Standards

Biosecurity standards continue to be developed for new sectors. Under Growing Forward, the CFIA completed eight national farm-level biosecurity standards (bees, beef, dairy, sheep, goat, fur-bearing, potato, and grains and oilseed) and one service sector biosecurity standard (poultry); Growing Forward 2 funding is intended to focus on completing standards for the cervid, equine, fruit/tree nut, and greenhouse sectors, and for the transportation of livestock, poultry and deadstock. At the time this evaluation was conducted, standards had been developed for greenhouse/floriculture and fruit/tree nut and progress was being made toward the other Growing Forward 2 goals through program funding (refer to Table 9).

Traceability

The development of a national traceability system began prior to Growing Forward. Funding has continued to help develop policy and traceability regulatory infrastructure (via the CFIA–AAFC MOU), governance and stakeholder collaboration (via the Traceability Task Team and the Industry-Government Advisory Committee) and the supporting electronic infrastructure (via Traceability National Information Portal). Recent work on the Traceability National Information Portal brought together multiple data repositories from both industry and government (data providers) with the intent of reducing the time to respond to animal disease crises or sanitary issues and thus, regain market access quickly if lost.

This progress notwithstanding, it is too soon to observe instances of sustained, expanded market access as a consequence of the Assurance Systems program. On this point the evaluation relies on the conclusion of the Canadian Integrated Food Safety Initiative evaluation, which determined that the development of Hazard Analysis and Critical Control Point systems facilitated market access in some sectors (grains, pork, and honey).Footnote 38

4.2.2 Demonstration of Efficiency

Communication between AAFC and CFIA

Finding 8: AAFC and CFIA have established effective formal and informal communication channels, facilitating the delivery of the program; however, they have been challenged by a lack of clarity on the roles and responsibilities delineated in the MOU where there is a shared responsibility.

Communication Channels

Good formal and informal communication channels exist. Overall, the evaluation identified several avenues that facilitated regular communication between AAFC and the CFIA regarding program activities.

One example is the formal mechanism in place for performance reporting of CFIA to AAFC (on Food Safety Recognition Program, biosecurity and traceability activities) as laid out in the terms of the AAFC–CFIA MOU on Technical Expertise. Managers and staff members involved in the program indicated that each organization has a secretariat whose role is to oversee MOU amendments and performance reporting. Another example is related to the Traceability National Information Portal, where AAFC and the CFIA reached a decision on the continued operation and funding of the Traceability National Information Portal. Lastly, government officials that were interviewed also indicate that regular communications between AAFC and the CFIA take place informally as necessary on all aspects of the program. Also, documents show that the CFIA has engaged AAFC when seeking input into the forthcoming traceability regulations.

Greater Clarity of Individual Roles and Responsibilities of Agriculture and Agri-Food Canada and Canadian Food Inspection Agency

Government officials that were interviewed suggested that communications between AAFC and CFIA could have been more effective in cases where it was necessary to come to a common ground, such as with work plans. In addition, an audit report on the MOU was completed by CFIA in September 2016. While the audit report found that administrative oversight roles and responsibilities were clearly defined in the MOU, the report also determined that there was a "lack of clear delineation in the MOU of the individual roles and tasks of each organization in carrying out the financial and performance related activities where there is shared responsibility. The clear delineation of individual roles and tasks would better support relations between AAFC and CFIA and reduce any confusion about individual roles and tasks". The subsequent finding of the audit was:

Specific individual roles and tasks for AAFC and CFIA related to MOU administration and management were not delineated in the MOU, resulting in shared responsibilities not being consistently carried out between initiatives.

Finally, the MOU underwent seven amendments on items ranging from budget transfers to revision of activities and performance measures from June 2014 to August 2015 (Summarised in Appendix H).

Knowledge Exchange Mechanisms to Facilitate Assurance and Traceability Systems Development

Finding 9: AAFC and the CFIA have developed efficient and effective approaches to fostering information sharing between provincial and territorial governments and industry, on the development of traceability regulations.

Forums: The Traceability Task Team and the Industry-Government Advisory Committee

The Traceability Task Team and the Industry-Government Advisory Committee are important forums for facilitating knowledge exchange among industry and provincial partners. Documented evidence related to the Traceability Task Team, Industry-Government Advisory Committee and FPT Assurance Systems Program Coordination Working Group meetings show that knowledge is shared between and among provincial-territorial governments and industry. Some examples are:

Drafting Amendments to the Health of Animal Regulations

CFIA has also proactively responded to feedback from partners through its consultation process.

Finding 10: Over the course of Growing Forward 2, core strategy elements of the National Livestock Traceability System have been at various stages of implementation. These core strategy elements have included (1) CFIA finalising amendments to the Health of Animals Regulations for their publication in Canada Gazette, and (2) MISB and CFIA ensuring that the Traceability National Information Portal has access to all data sources, in anticipation of its use by authorized industry and government users following the implementation of new regulations.

Traceability is the ability to follow the movement of a plant or animal through specific stages of production, processing and distribution. The purpose of a traceability system is twofold. First, it acts as an emergency management tool, enabling the flow of information across the agri-food supply chain, improving the effectiveness and timeliness of isolating animal health emergencies. Responsibility for emergency management lies with CFIA. Second, a traceability system can support product claims of market attributes to improve national and international market access, industry competitiveness, and consumer confidence.Footnote 39 Market access lies within the mandate of AAFC and CFIA.Footnote 40Footnote 41

MISB's National Livestock Traceability Operations work in implementing the National Livestock Traceability system has been ongoing prior to GF2. Its implementation has required working collaboratively with CFIA, provincial and territorial governments and industry to implement core strategy elements. While many elements of the strategy have progressed, at the time this evaluation was conducted, two key elements were still in progress:

  1. Some gaps on regulatory amendments for livestock traceability

    AAFC is supporting CFIA in developing the policy and regulatory infrastructure for traceability. In this regard, CFIA is in the process of drafting amendments to the Health of Animals Regulations in relation to traceability requirements for livestock. These regulations will require three fundamental elements to be in place:

    1. animal/product identification,
    2. premises identification, and
    3. animal/product movement reporting for cattle, bison, sheep, goats, cervids and pigs.Footnote 42

    The regulatory amendments for pig traceability were published in 2014. However, at the time this evaluation was conducted, CFIA publishing the amendments to other livestock traceability systems in Canada Gazette was behind schedule. There is evidence to suggest that this delay is due to some gaps between proposed regulatory amendments and challenges in implementing them.Footnote 43 These gaps were identified through the consultation process between government and industry stakeholders (led by CFIA with AAFC) and confirmed through key informant interviews and AAFC documentation.

    According to those interviewed, four gapsFootnote 44 were identified in Canada's livestock traceability systems, of which stakeholders had reached a general consensus on three. At the time this evaluation was conducted, the one gap that remained unresolved related to the proposed requirement to make all livestock movement reporting mandatory. This is an area not presently regulated by the CFIA. While the CFIA considers such a requirement necessary from a science-based perspective, the possibility of it being enforced is a cause for some concern to some industry stakeholders, which views such reporting as costly, given the high volume of animal movement and the lack of adequate infrastructure. The CFIA's view is that, its role as the regulating organization responsible for enhancing the health and wellbeing of Canada's people by safeguarding food, animals and plants, under the Minister of Health, must be balanced with CFIA's work on assurance systems in support of improving industry's ability to access markets and compete domestically and internationally. Finding common ground with industry on livestock movement reporting will be important in moving forward on amendments to the Health of Animal Regulations. Since this evaluation was completed, CFIA has concluded its consultations with industry and feedback has been incorporated in the proposed amendments to the Health of Animal Regulations. Publication in Canada Gazette 1 is scheduled for fall 2017.

  2. The Traceability National Information Portal was not fully utilised

    The concept for a national portal, such as the Traceability National Information Portal, began under Growing Forward and continued under Growing Forward 2. The idea was to build a tool that would allow authorized users to run reports for authorized purposes, such as to effectively address animal disease situations and sanitary issues.Footnote 45 A full explanation of the Traceability National Information Portal development is provided in Appendix D. Despite visible and ongoing progress on the Traceability National Information Portal, AAFC has recognized that the portal is still not fully populated with traceability data.

    Finally, through the National Assurance Systems Development G&C component, funding was provided to an industry organization to develop and implement a national industry database that would collect and manage regulated industry data for multiple species.

    The eventual goal was for the national industry database to be integrated into the Traceability National Information Portal to realise the Traceability National Information Portal vision of a single window access to information on multiple species. The national industry database was targeted to be operational by 2015. However, the funding agreement was mutually terminated in 2016 due to uncertainty regarding the projects future. This decision has not impacted the sectors ability to trace disease outbreaks as industry is still using existing databases.

Performance Measurement Systems

Finding 11: The success of the program in implementing systems and standards is dependent on provincial-territorial governments supporting implementation and/or industry voluntarily implementing systems and standards.

Assurance Systems is a complex program performing a broad range of activities spanning AAFC and CFIA. The program theory offers the rationale for the Canadian agriculture and agri-food sector to develop assurance systems, so that the sector can respond to consumer and market demand, and thus improve its ability to compete domestically and internationally. At the same time, the program theory acknowledges the challenge in attributing the achievement of the program's outcomes solely to the program activities, as noted in the PPMRMS below:

One key performance measurement challenge the program has is how to effectively attribute the development of national assurance systems to an industry's ability to obtain access and/or compete in specific export markets, particularly as the assistance these assurance systems convey (if implemented) are often one piece of a broader technical/regulatory puzzle. […] additional work is needed to better identify the end results of assurance systems. To this end, efforts are underway to identify potential case studies which could be used to increase our understanding and track the benefits derived from developing and implementing assurance systems.(page 11, PPMRMS)

This evaluation has determined that the Assurance Systems program has been successful in supporting the sector in developing assurance systems and standards (See Section 6.2). Success, to a large extent, is dependent on the industry implementing the systems and standards. The program theory puts the role of supporting the implementation of systems and standards to provincial and territorial governments through the Growing Forward 2 Cost-Shared program. At the same time, the program theory does not clearly demonstrate how it influences provincial and territorial governments to support implementation and ultimately contribute to improving the industry's ability to access markets, respond to market opportunities and compete domestically and internationally. However, given that the evaluation was conducted at the midway point of Growing Forward 2, more time will likely need to pass, as systems are completed and implemented, to better assess this outcomeFootnote 46. The exception to this is livestock traceability systems which, once regulated, industry participation will be required.

4.2.3 Program Design and Delivery

National Assurance Systems Development Project Review Process

This program component's main activities are to process applications, contribution agreements, and recipient financial claims. AAFC's Market Industry Services Branch and Strategic Policy Branch, as well as the provincial governments, all play a role in the funding application approval process. The CFIA also provides support to this activity by reviewing funding applications for developing food safety (HACCP) systems.

A DG Review Committee (with representatives from the Market and Industry Services Branch, Strategic Policy Branch, Programs Branch and CFIA) is in place to discuss the recommendations for projects and recommend projects for Ministerial approval. Occasionally, the committee may be asked to exercise program oversight.

Finding 12: The application process for the National Assurance Systems Development G&C component was well understood, but it could improve on documenting the eligibility assessment process.

A review of the online Applicant Guide, the internal Program Officer Manual and the Technical Assessment Form determined that the application process (the guidance on how to apply to the G&C component and guidance on how applications are assessed) was well described, easy to understand and transparent. Project reviewers and program administrators were knowledgeable about the process and program applicants who were interviewed generally had positive feedback on the process and the expertise and helpfulness of AAFC program officers. In addition, many applicants had previous experience with the program under Growing Forward. It should be noted, however, that the program does not collect feedback from unsuccessful applicants to understand and assist with program design improvements.

However, AAFC internal audit of the National Assurance Systems Development G&C component, which was conducted at the same time as this evaluation, concluded that the assessment process could improve on documenting the eligibility assessment process of applications. The Audit of the AgriMarketing Program – Assurance Systems recommended that the Assistant Deputy Minister of Programs Branch review documentation requirements for the eligibility assessment of applications to strengthen the decision-making process.

Efficient Program Delivery (Economical Use of Resources)

Finding 13: The program outputs appear to have been delivered on target with fewer resources than planned over the three-year time frame of the evaluation.

While it was sometimes difficult to link actual reported outputs to the intended targets presented in the PPMRMS, generally, it appears that targets were on track to be met, or were exceeded. Some of the output tables presented in Appendix E identify targets drawn from the PPMRMS; however, they could not be identified in all instances due to a lack of consistency between the descriptions of expected versus actual outputs.

Data on full-time equivalents (FTEs) obtained through the 2013–14, 2014–15 and 2015–16 departmental performance reports show that 62 FTEs were utilized for the delivery of the program. This was seven fewer FTEs than planned for the period. A total of 644 outputs were produced during this period, accounting for 34 funded projects (8 finished at 31 March 2016) under the National Assurance Systems Development G&C component, 323 under the National Livestock Traceability Operations component and 287 under the CFIA Technical Expertise component. Additional details are shown in Table 3.

Table 3 - Summary of full-time equivalents, expenditure and outputs in the delivery of program activities (1 April 2013 to 31 March 2016)
Programs Branch (NASD Program) Market and Industry Service Branch (NLTO)* CFIA Technical Expertise Total
Full-Time Equivalents 44 18 N/A 62
Salary + Non-salary expenditure $4,483,955 $2,467,410** $7,441,163 $14,392,528
Outputs 34*** 323 287 644

Note:

*Assumes all expenditure linked to the Market and Industry Services Branch is for delivery of the National Livestock Traceability Operations.

**MISB and ISB were responsible for 88% of all non-pay operating costs.

***Reflects all projects with a signed contribution agreement as of 31 March 2016. It does not reflect other process-oriented tasks such as reviews, letters issued, etc., which are discussed elsewhere in the report.

Source: Table 1 and AAFC 2013-14, 2014-15 and 2015-16 Departmental Performance Reports

In addition, a number of service standards are defined for the AgriMarketing Program (all streams) on AAFC's website.Footnote 47

The program's PPMRMS target is to meet these service standards for 95% of the projects. The evaluation found that from 2013-14 to 2015-16 the first standard (100 days) was met for 73% of the projects. The second standard (30 days) met its overall target. Public reporting for the year 2015-16 showed that standards improved in that year, with 88% of projects receiving notification letters within the 100-day range.Footnote 48

Finding 14: The number of G&C projects expected to be funded is on pace to achieve its target number. Actual G&C expenditure is 53% lower than planned for the three year period from April 2013 to March 2016.

Data shows that the planned number of G&C projects to be funded was on pace to achieve its target of 40 by April 2018 (n=34). At the same time the National Assurance Systems Development component underspent G&C funding by 53% over the three year time frame. Figure 2 shows actual spending against budgeted spending over the three year time period that the evaluation covered.

Figure 2 - Budgeted versus Actual G&C spending, 2013-2014 to 2015-2016
Description of this image follows.
Description of above image

Figure 2 shows budgeted versus actual G&C spending from 2013-2014 to 2015-2016 as follows:

  • In 2013-2014 budgeted expenditure was $14,270,000 and actual expenditure was $2,121,640
  • In 2014-2015 budgeted expenditure was $6,270,000 and actual expenditure was $5,414,761
  • In 2015-2016 budgeted expenditure was $10,183,188 and actual expenditure was $6,903,917
  • For the three years, total budgeted expenditure was $30,723,188 and actual expenditure was $14,440,318

One explanation provided for the difference in budgeted and actual spending in 2013-2014 is that there were delays in the project approval process and that some G&C applicants withdrew their application when the government: recipient funding ratio changed from 90:10, in Growing Forward, to 75:25, in Growing Forward 2.

"Government should proactively reach out to us [and make it easier for us to find funding. They need to be more visible. They need to tell us what they want to support so that when we are doing our planning we can take account of their funding and policy priorities." (Industry stakeholder)

"Our industry is not well versed in knowing what federal funding is out there. Would love for AAFC to do more visibility, marketing of these programs; we as a province are happy to play supporting role there." (Provincial government official)

Finding 15: The Assurance Systems program was not sufficiently visible to organizations that did not have prior awareness of it.

The program was not visible enough to target groups who did not have prior awareness of it, according to government officials responsible for managing or administering program activities, and some other interviewees (for example, G&C applicants or provincial technical reviewers). Comments from interviewees illustrate a low level of awareness. This low awareness may have contributed to the lower than expected uptake.

Finding 16: The funding ratio of 75:25 (government: recipient funding) was generally considered appropriate, although smaller, resource-limiting organizations suggested that the recipient share could be lower than 25%.

In general, funding recipients agreed that the funding ratio of 75% to 25% government to applicant was an appropriate balance and demonstrated a commitment from both parties. However, recipients from resource-limited organizations (e.g., industry associations with fewer than 50 members) found it more difficult to leverage external funding. These groups suggested the required applicant share be lower than 25%.

4.2.4 Qualitative Value of Assurance Systems

Finding 17: Stakeholders reported that the program facilitated the exchange of knowledge and sharing of best practices.

"Everyone involved is learning about sustainability, about what the market is asking for, about how agronomics, economics and environmental outcomes are linked. We are learning that sustainability doesn't have to cost money, it can actually make money." (Industry stakeholder)

Those interviewed as part of three case studies shows that the program provides value by way of knowledge exchange mechanisms and the sharing of best-practices. This in turn can contribute to cost savings for their businesses and facilitate access to international markets, as indicated by a comment from an industry stakeholder.

 
Pulse Canada – Aligning Canadian Sustainable Agriculture Metrics to the Sustainability Needs of the Global Food Industry:

The purpose of this project was to develop sustainability indicators and a calculator tool that was closely based on a similar US resource called the Field to Market Fieldprint CalculatorFootnote 49.  Since US companies source pulses from Canadian producers, the intent is to align data reporting and usage between operations in both countries, thereby helping Canadian producers maintain that market access. An immediate outcome of this project is knowledge creation and awareness-building. This not only relates to filling acute information gaps (such as a much-needed baseline survey on farm fertilizer management practices), but extends to a new way of thinking about the link between environmental sustainability and profitability. A large and diverse number of stakeholders were involved in the Pulse Canada project, including major industry players such as General Mills and Syngenta, industry associations such as Canadian Canola Growers Association and the Barley Council of Canada, and non-profits such as Ducks Unlimited. Each stakeholder is learning and sharing information along the way.

Beekeepers Commission of Alberta – Honey Bee Health Surveillance in Canada:

The Canadian laboratory at the helm of this project was able to forge new international partnerships by mirroring survey protocols already implemented in the US, Europe and New Zealand. This is enabling the laboratory to grow its presence in the international community of diagnostic bee laboratories and to benefit from an exchange of best practices and international expertise. In the immediate term, knowledge translation has already occurred via presentations of early survey results to beekeepers associations/honey producers associations in New Brunswick, Manitoba, British Columbia, Alberta, Quebec, Ontario and Saskatchewan. Those interviewed noted that a budget was allocated for publishing in scientific journals and technical reporting so final results can be diffused to the scientific and academic community. In the intermediate term, reliable data on a national bee health baseline is envisioned to lead macro-management of the industry. This means that the traditional model, where responsibility for disease management rests with the individual beekeeper, will evolve. Those interviewed see a model where data on health trends at the regional or provincial level will allow for better decision-making and widespread dissemination of best practices. Another example is management of bees in the agri-food chain as crop pollinators. Bees can currently be shipped across provinces to pollinate, but having the latest health surveillance data could help provinces make informed decisions about hive movement and mitigate the spread of disease.

Jewish Community Council of Montreal – Canada Kosher 22000™:

This case study demonstrated how pursuing specialty certification could make Canadian companies more competitive and/or could enhance their market share. Those interviewed said that El Al, the major Jewish international airline, and some Air Canada routes have now required their baked goods suppliers to have the Canada Kosher 22000™ certification. Typical knowledge translation activities have occurred because the AAFC investment has helped fund JCCM attendance at food safety conferences in England, Chicago, Ontario and Montreal, where ongoing project progress has been shared with a variety of international food industry and specialty-food stakeholders.

5 Conclusions

At the midway point through its Growing Forward 2 funding, the Assurance Systems program remains in alignment with departmental and federal government priorities related to innovation, competitiveness and market development. The combined activities of the program demonstrate that government plays an appropriate role in creating national systems and standards and ensuring plant, animal and human health. The program continues to addresses the need to enhance the agriculture and agri-food sector’s capacity to access domestic and international markets and enhance its market competitiveness.

The Assurance Systems program is on track to meet its planned outputs with the exception of publishing proposed regulatory amendments, related to traceability, to the Health of Animal Regulations in Canada Gazette, for bison, cattle, sheep goat and cervids. While the evaluation found some examples of how activities have incrementally built upon previously funded projects to attain or expand market access, the data were not sufficient to conclude that funding helped sustain any such expanded market access under Growing Forward 2. Given the evaluation was conducted at the midway point of Growing Forward 2, more time will likely need to pass, as systems are completed and implemented, to better assess this outcome.

The program could go beyond simply supporting the development of systems and standards to facilitating the implementation of assurance systems and standards, although this function has been deferred to provincial-territorial governments through the Cost-Shared program or left to industry to voluntarily implement the systems or standards.  

The Assurance Systems program has efficient knowledge exchange mechanisms in place, particularly through the Traceability Task Team and the Industry-Government Advisory Committee forums. The program also provides value to stakeholders by way of knowledge exchange mechanisms and the sharing of best practices. At the same time, there remain challenges related to roles and responsibilities of AAFC and CFIA delineated in the MOU where there is a shared responsibility. Also, at the time this evaluation was conducted, one unresolved gap remained with industry on CFIA’s proposed traceability regulations. This was related to the requirement to make all livestock movement mandatory. Finding common ground with industry on livestock movement reporting will be important in moving forward with regulatory amendments as it could impact the industry’s ability to implement traceability systems that meet regulatory requirements and ultimately access markets and respond to market opportunities. Since this evaluation was completed, progress has been made towards amendments to the Health of Animal Regulations. In addition, the Traceability National Information Portal, as the single window into traceability information, has yet to be fully populated with partner information.

Regarding the National Assurance Systems Development G&C component, the application process was clear and transparent although the program could improve on documenting the eligibility assessment process. Actual G&C expenditures was 53% lower than planned for the three-year period from April 2013 to March 2016, although the number of projects that were funded was on pace to meet its target. A contributing factor to underspent funding could be the lack of awareness about the Assurance Systems program amongst stakeholders.

Finally, while the Assurance Systems program has been successful in supporting the sector in developing assurance systems and standards, its program theory could more clearly demonstrate how the program influences the sector to implement the systems and standards that have been developed and ultimately contribute to improving the industry’s ability to access markets, respond to market opportunities and compete domestically and internationally.

6 Recommendations and Management Response and Action Plans

The following recommendations are intended to facilitate programming for the Next Policy Framework. While the programming for the NPF is currently at the development stage, this evaluation takes into considerations the commitments outlined in the Calgary StatementFootnote 50 and presents the following recommendations in order to facilitate program design in the NPF and also to inform AAFC’s transition to the Results and Delivery agenda.

Recommendation 1

Moving forward in designing the Assurance System program under the Next Policy Framework, Programs Branch, in working with the Market and Industry Services Branch and the Strategic Policy Branch, should foster closer collaboration and coordination with provincial and territorial governments and industry to:

Management Response and Action Plan Recommendation 1

Agreed.

Programs Branch agrees that closer collaboration and coordination with provincial and territorial governments will facilitate the implementation of assurance systems and standards and will encourage greater awareness of the program. Programs Branch will revisit the mandate and operation of the existing Federal, Provincial and Territorial Assurance Coordination Working Group to further position it to be a forum for greater information sharing and consultation regarding program design, delivery, performance and project implementation.

Target Date: December 2018

Responsible Position: Director, Competitiveness Division, Programs Branch.

Recommendation 2

In support of the commitment made by the FPT agriculture Ministers in 2006 to phase in the livestock component of the National Agriculture and Food Traceability System, the Market and Industry Services Branch, in partnership with CFIA, should ensure that the Traceability National Information Portal is available to authorized industry and governments following the implementation of new regulations.

Management Response and Action Plan Recommendation 2

Agreed.

MISB will continue to work with the CFIA through the Traceability National Information Portal (TNIP) Management Committee to ensure integration of industry and provincial data systems. A User Acceptance Test (UAT) is planned to take place near the end of March 2018 to ensure all system integrations have been completed successfully.

Target Date: June 2018

Responsible Position: Director, Horticulture and Cross Sectoral Division, Market and Industry Services Branch

Finding 8: AAFC and CFIA have established effective formal and informal communication channels, facilitating the delivery of the program; however, they have been challenged by a lack of clarity on the roles and responsibilities delineated in the MOU where there is a shared responsibility.

Recommendation 3

In the event that an MOU is established between AAFC and CFIA for the Next Policy Framework, Programs Branch should build upon the effective communication channels that already exist and ensure that shared responsibilities are clearly defined.

Management Response and Action Plan Recommendation 3

Agreed.

Programs Branch agrees that leveraging existing communication channels will benefit the development of the MOU between AAFC and CFIA for the Next Policy Framework.

Programs Branch will review the existing communication channels and will adjust or reorient these channels if or as required to ensure roles and responsibilities between program, service and policy areas are clearly established and documented in the MOU with CFIA.

Target Date: December 2018

Responsible Position: Director, Competitiveness Division, Programs Branch

Appendix A: Logic Model of AgriMarketing Stream D, as presented in the Program Performance Measurement and Risk Management Strategy

Logic Model of AgriMarketing Stream D
Description of this image follows.
Description of above image

Appendix A presents the logic model for AgriMarketing Stream D: Assurance Systems, outlining the objective, activities and outputs, immediate outcomes, intermediate outcomes and end outcomes, as follows:

  • Objective - The Assurance Systems Stream D seeks to support the Canadian agriculture, agri-food and agri-based products industry to meet buyer and market demands for assurance and to enhance competitiveness through support for the development of Canadian assurance systems or standards, such as food safety, biosecurity, animal and plant health surveillance, market attribute (such as animal care) and traceability and related tools; facilitating the integration of systems and benchmarking existing standards to international standards.
  • Activities and outcomes of the three components: National Assurance Systems Development, CFIA Technical Expertise and National Livestock Traceability Operations.
  • Immediate Outcome - The Canadian agriculture and agri-food sector is developing assurance systems that respond to consumer, buyer, and market demands and protect against threats to plant, animal and human health
  • Intermediate Outcomes:
    • The Canadian agriculture and agri-food industry has the tools available to implement assurance systems that respond to consumer, buyer and market demands, gain/maintain market access and protect against threats to plant, animal & human health
    • Implementation of Assurance systems and standards by producers and industry funded by provinces under 1.2.5 Cost-Shared Assurance Programming
  • End Outcome - Improved industry ability to access markets, respond to market opportunities and demands, and compete domestically and internationally

Appendix B: Provincial Programs that Fund Assurance Systems Development

Table 4 – Provincial Programs that Fund Assurance Systems Development
Province Program Objective Eligible recipients Funding type
Quebec[1] Salubrité, biosécurité, traçabilité et santé et bien-être des animaux
Program in effect until 31 March 2018
Encourage the establishment of food safety systems, biosecurity and traceability under three key themes:
  1. outreach activities,
  2. advisory services, equipment procurement and facility upgrades,
  3. development of support tools for the implementation of a system
Funds food businesses, industry organizations, farms looking to meet demands of domestic and international markets Government contribution of 70% of eligible costs (up to $2,000 for theme 1, $5,000 for theme 2, $20,000 for theme 3)
Alberta[2] Food Safety Systems Producer Program
Applications accepted until January 2018
Help producers invest in equipment and tracking systems to improve On-Farm Food Safety practices, enhancing producers' business competitiveness and food safety performance

Producers in Alberta who have an established primary business in production of crops or livestock

Horticulture packers directly involved in grading and packing horticultural produce for human consumption

Government contribution of 70% of eligible activities up to $5,000
Alberta[3]

Livestock Welfare Producer Program
Applications accepted until April 2016

Provide financial support for Alberta livestock producers to adopt best management practices and technologies that improve livestock welfare, thereby enhancing competitiveness and sustainability Producers in Alberta raising livestock with a Premise ID and who are participating in existing industry animal welfare programs Reimbursement for 50% of approved capital and non-capital costs, to a maximum amount of $50,000
Ontario[4] Growing Forward 2 Cost-share Funding Assistance Program
Assurance systems is 1 of 6 possible focus areas
Applications accepted until October 2016
Supports projects where the industry uses recognized standards and puts systems in place to effectively manage risk, access new markets, and retain and expand existing markets. Assurance systems include traceability, animal welfare and food safety. Not-for-profit, Ontario-based involved in the agriculture, agri-food and agri-based bioproducts sector including but not limited to associations, marketing boards, municipalities, economic development offices, and Aboriginal groups

Up to 50% cost-share funding of total eligible costs
Funding requests above 50% must demonstrate how the project is innovative and may receive up to 75% of total eligible costs
Maximum amount not listed on website

British Columbia[5] Biosecurity and Surveillance Program Risk management approach focusing on awareness and implementation of on-farm biosecurity plans, as well as plant and animal disease surveillance projects Industry and commodity groups Not listed on website

[1] Ministère de l'Agriculture, des Pêcheries et de l'Alimentation du Québec (MAPAQ), "Salubrité, Biosécurité, Traçabilité et Santé et Bien-Être Des Animaux," Transformation et Distribution Alimentaires, 2015, http://www.mapaq.gouv.qc.ca/fr/Transformation/md/programmesliste/ameliorationsalubrite/Pages/Appuiimplantationdesystemessalubrite.aspx

[2] Alberta Agriculture and Forestry, "Food Safety Systems Producer," Government of Alberta: Growing Forward 2 - Programs - Programs, accessed May 4, 2016, http://www.growingforward.alberta.ca/Programs/index.htm?contentId=FOOD_SAFTY_PRD_PRG&useSecondary=true.

[3] "Government of Alberta: Growing Forward 2 - Livestock Welfare Producer," accessed May 4, 2016, http://www.growingforward.alberta.ca/Programs/index.htm?contentId=LIVESTK_PRD_PRG&useSecondary=true&active=no.

[4] Ontario Ministry of Agriculture, Food and Rural Affairs, "Growing Forward 2 - Collaborations and Organizations," Growing Forward 2 for Organizations and Collaborations, 2016, http://www.omafra.gov.on.ca/english/about/growingforward/gf2-org.htm#T3.

[5] Ministry of Agriculture, "Biosecurity & Surveillance - Province of British Columbia," accessed May 4, 2016, http://www2.gov.bc.ca/gov/content/industry/agriculture-seafood/programs/growing-forward-2/biosecurity-surveillance.

Appendix C: Government Support for Assurance Systems in International Jurisdictions

The following text examines the federal approach to developing assurance standards in three international jurisdictions: the US, the United Kingdom (UK) and Australia. These were chosen as they are considered fairly comparable to Canada in terms of geography, demographics, and cultural/legislative context.

United States: In the US, the focus of regulation is shifting from a predominately prescriptive process to more flexible performance-based standards that allow food businesses greater choice in the mode of implementationFootnote 51. For example, the Food Safety Modernization Act (2011) gives the US FDA more power to establish new safety standards for products, greater oversight capabilities to ensure compliance, and greater authority to respond to violations by suspending facility registrations. However, this overhaul of the regulations did not include developing or implementing a mandatory national food safety system for industry to abide byFootnote 52. Rather, the US tends to rely on market-based incentives to develop safety systems. As the US Department of Agriculture (USDA) reports, government policies that reward producers (such as safety certification for products), or policies that punish and increase the costs of violating safety standards (e.g., recalls, fines and plant closure) are preferable to imposing mandatory food safety system processesFootnote 53.

Similarly, the US Federal Government does not provide financial support or fund industry to develop national traceability systems because the private sector has already developed a substantial capacity to trace, including third-party safety/quality audits and industry-maintained standards. Thus, from the US perspective, it is proposed that the best role for government is to ensure that unsafe foods are quickly removed from the system (via recall standards, increased penalties, increased surveillance), while allowing firms the flexibility to determine how food is to be kept safeFootnote 54.

One department that administers a wide variety of market programs is the Agricultural Marketing Service (AMS) of USDA. Regarding assurance systems specifically, the AMS provides an audit and accreditation service based on ISO standards and HACCP principles. It "provides producers and suppliers of agricultural products the opportunity to assure customers of their ability to provide consistent quality products or services. AMS verifies their documented programs through independent, third-party audits."

These audits are voluntary and are paid for through hourly fees. Audits can be done for food safety systems (e.g., fruit and vegetable good handling practices) or for market attributes (e.g., non-hormone-treated cattle)Footnote 55. While the AMS does provide some grant funding, none of the funded opportunities are specifically related to assurance systems, so in this respect it is not comparable to AAFCFootnote 56. Similarly, while there are many federal and state funding opportunities in the agricultural sectorFootnote 57 (including several that appear to fund the development of market attributes such as phasing out of certain pesticides, sustainable agriculture, building soil healthFootnote 58), the literature search found little evidence of funds specifically dedicated to assurance systems or traceability development.

United Kingdom: In the UK, appropriate HACCP-based food safety systems must be implemented in accordance with the wider European Union (EU) Regulation 852/2004, Article 5Footnote 59. Food assurance schemes (such as the Red Tractor Assured Food Standards, the Lion Eggs logo, Quality Meat Scotland, Farm Assured Welsh Livestock and more) are mainly voluntary certifications accredited by the UK Accreditation Service, although this certification is often a requirement for producers supplying the UK or the EU. These schemes use regular independent inspections to check that members are meeting standards. In addition to food safety, there are a number of small niche schemes that aim to meet particular consumer demands, such as higher animal welfare and environmental or organic standardsFootnote 60. Similarly, traceability in the UK food chain follows the legal framework of the European Regulation 178/2002, Article 18Footnote 61.

The UK government's Food Standards Agency (FSA) and the Department for Environment, Food and Rural Affairs are not responsible for implementing or funding any of these specific food assurance schemes, but do monitor whether claims made by assurance schemes are accurate. FSA also undertakes periodic reviews of these schemes to make best-practice recommendations.Footnote 62.

FSA offers guidance and resources to support the capacity of the food industry to develop strong HACCP-based food safety systems. Some of the resources available online include a meat plant HACCP manual and CD-ROM, a short guide to creating a HACCP plan, a food safety management diary to record food safety activities year-round, manuals for caterers and retailers handling high-risk foods, and manuals to help create food safety management plans and business record-keepingFootnote 63.

Australia: Here the government agency known as Food Standards Australia New Zealand (FSANZ) develops and administers the Food Standards Code, which requires food businesses to implement a food safety program based upon the HACCP concepts. The food safety program is implemented and reviewed by the food business, and is subject to periodic audit by a qualified food safety auditorFootnote 64. Importantly, FSANZ does not decide any overarching food policy, nor does it enforce the Code. Rather, each state and territory government agency is responsible for enforcing and interpreting the CodeFootnote 65. Thus, in Australia, the onus is on the producers/suppliers to implement a safety system that is required by wholesalers/processors, as enforced by individual statesFootnote 66Footnote 67Footnote 68.

Although the Australian federal government does not appear to fund the development of assurance systems, it has developed a partnership with state governments and with industry to create the SAFEMEAT initiative. Funded by industry, this initiative has developed voluntary and mandatory standards, which have been introd9uced from farm to fork.Footnote 69.

Appendix D: Development of the Traceability National Information Portal

The timeline below summarizes the major milestones of the Traceability National Information Portal (TNIP) development from 2009 to 2016. The information has been collated from multiple documents (referenced in the Table 9) and validated by interviewees.

Table 5 - Timeline of Traceability National Information Portal Development
Policy Framework Date Item
Growing Forward 1 2009
  • – AAFC authorized $1.1M to the CFIA via MOU (Growing Forward) Program Initiatives Development Memorandum of Understanding) for the initial planning of TNIP (then known as TIS - the Traceability Information Sharing Solution)[1]
Growing Forward 1 2011
  • – Second MOU (official title not found) signed between AAFC and CFIA, specifically for the development and launch of TNIP by CFIA
  • – AAFC budget for TNIP was $12,745,069 in Vote 1 from 2009–10 to 2012–13. (This included the $1.1M from 2009–10)[2]
Growing Forward 1 2013
  • – TNIP was officially launched, collecting data from three providers; Premises ID from the provincial governments of Alberta and Manitoba and Animal ID from Canadian Livestock Tracking System (CLTS) maintained by the Canadian Cattle Identification Agency (CCIA)
  • – Responsibility for management and operation of the portal was transferred from CFIA to AAFC
Growing Forward 2 2013–2018
  • – Under the National Livestock Traceability Operations component of Assurance Systems Stream D, AAFC is responsible for the operation (aggregation of data and ensuring access for authorized users) of TNIP.
  • – CFIA is funded by AAFC via MOU (Memorandum of Understanding between AAFC and CFIA on the Delivery of Growing Forward 2 Assurance Systems Stream Technical Expertise Initiatives) to negotiate data sharing agreements with provincial governments, ensuring that provincial PID data will be linked to TNIP.
  • – As data sharing agreements are finalized with individual provincial governments, provincial data is linked into TNIP
  • – A TNIP management committee was formed with representation from the AAFC Market and Industry Services Branch (Assurance Systems), CFIA Program Policy Integration (Traceability Section) and CFIA Information Management and Information Technology (IM/IT) group. The committee's (ongoing) responsibility is to develop a TNIP enhancement plan, to monitor uptake and to troubleshoot ad hoc issues that arise
Growing Forward 2 2015
  • – MOU (Memorandum of Understanding between CFIA's IM/IT Organization and AAFC's Information Systems Branch for the Maintenance of the TNIP Solution) to provide information technology (IT) support, maintenance and future enhancements to the TNIP.
Growing Forward 2 2016
  • – Current TNIP data providers are the CCIA CLTS, as well as the PID repositories from the government of AB, MB, PEI, SK, ON
  • – TNIP was recently connected to the CFIA's lab sample tracking system (LSTS), a repository for identified animal samples (tissue, blood etc.) and lab analysis results. This animal health information is expected to bring added value to TNIP in order to better manage outbreaks.

[1] Canadian Food Inspection Agency Government of Canada, "Audit of the Growing Forward Program Initiatives Development Memorandum of Understanding," June 28, 2012, sec. 2.1, http://www.inspection.gc.ca/about-the-cfia/accountability/other-activities/audits-reviews-and-evaluations/growing-forward-program/growing-forward-program-report-/eng/1340860998452/1340861066766.

[2] Agriculture and Agri-Food Canada and Government of Canada, "Evaluation of the Canadian Integrated Food Safety Initiative." http://www.agr.gc.ca/eng/?id=1421671560787

Appendix E: Progress on Outputs

Targets identified in the series of tables below reflect the expected target as of March 31, 2018. The number of outputs shown reflects the current state at March 31, 2016.

Table 6 - National Assurance Systems Development Program Outputs
National Assurance Systems Development Program Target Number Produced
Number of producer / enterprise / management systems, standards and tools that advance Assurance Systems 22 19
Number of producer / enterprise / management systems, standards and tools that complete Assurance Systems 18 17
Total Outputs (2013-14 to 2015-16) 40 36*

Note: *Two of the 34 funded projects were to both advance and complete assurance systems
Source: Program Data

Table 7 - National Livestock and Traceability Operations Outputs
Output Categories Target Number Outputs
Traceability Plans Developed N/A 7
Communications Products 25 84
Industry-Government Advisory Committee Action Items Completed* N/A 39
Industry-Government Advisory Committee Management Group Action Items Completed* N/A 99
Traceability Task Team Action Items Completed* N/A 94
Total Outputs N/A 323

Note: *Only items reported as "Completed and/or ongoing" were accounted for in the tallies.

Source: Program Data

N/A: Not Applicable

Table 8 - Food Safety Recognition Program Output Data
Food Safety Recognition Program: Activities and Outputs Target Number Produced
Activity 1: Ongoing technical review and assessment of on farm and post farm food safety programs for recognition and maintenance of the FPT policy and protocol documents as required N/A 34
1.1 Service standards met N/A N/A
1.2 Number reviews per program stage N/A N/A
1.2a&b Technical Review Part 1&2 8 6
1.2c Letter of recognition 4 2
1.2d 18-month review N/A 25
1.2e Post recognition monitoring N/A 1
Activity 2: Provide scientific and technical support as needed with respect to food safety to AAFC and AAFC stakeholders such as reviewing and assessing funding applications submitted to AAFC and where applicable, review, assess and advise on applications dealing with integration and/or benchmarking of systems N/A 30
2.1 Service standards of 7 business days for review of all applications and completion of assessments N/A N/A
2.2 Applications reviewed (G&C - National Assurance Systems Development) N/A 30
Total Output (2013–14 to 2015–16) N/A 64

Source: Program Data

N/A: Not Applicable

Table 9 - National Biosecurity Standards Development Output Data
Biosecurity: Activities and Outputs Number Produced
Activity 1: Develop and approve farm level biosecurity standards for 2–4 priority commodity sectors 93
1.1 Number consultative meetings with commodity stakeholders and / or number of external consultant sector readiness assessment reports 82
1.1a Sector readiness reports 3
1.2 Meetings of advisory council 2
1.3 Milestones identified in project plan 4
1.4 Number commodity specific farm-level national biosecurity standards completed on time and within budget 0
1.4a Number standards posted 2
Activity 2: Develop and approve a minimum of 1–6 service sector biosecurity standards 6
2.1 Delivery of work plan 0
2.2 Recommendations applied 0
2.3 Service sector biosecurity standards, 2.3a Consultative meetings 6
Activity 3: Maintain and Update existing national farm-level biosecurity standards as needed and third party submission review and evaluation as required 5
3.1 Standards reviewed and updated 1
3.1a Meetings/webinars 4
Total Output (2013–14 to 2015–16) 104

Note: In the case of this element of CFIA Technical Expertise, the targets appear to be defined within the activity description, but they don't appear to reflect the information reported, nor do they seem to reflect targets listed in the PPMRMS.
Source: Program Data

Table 10 - Traceability Management Office Regulatory Infrastructure Output Data
Trace: Activities and Outputs Target Number Produced
Activity 1: Regulatory framework development (Part 1) 1 1
1.1 Publish regulatory amendments for pigs in Canada Gazette 1 1
Activity 2: Regulatory framework development (Part 2) N/A 16
2.1 Regulatory consultation documents posted to CFIA website N/A 6
2.2 Regulatory drafting and supporting documentation for amendments for cattle, bison, sheep, goats, cervids and possibly equid (6) completed N/A 10
(not published in CG1)
Activity 3: Information Management and Sharing N/A 14
3.1 Traceability information sharing agreements in place with provinces, where possible 8 8
3.2 Privacy Impact Assessments and Threat Risk Assessments are initiated where necessary N/A 6
Activity 4: Administrator agreements (part 1) N/A 1
4.1 Finalize administrator agreements with the Canadian Pork Council N/A 1
Activity 5: Administrator agreements (part 2) 5 3
5.1 Continue progress toward, and finalize, administrator agreements for other regulated sectors N/A 3
Activity 6: Policy development and analysis N/A 33
6.1 Documents are prepared as necessary N/A 26
6.2 Research and analysis to support policy and regulatory development N/A 7
Activity 7: Stakeholder engagement and outreach N/A 48
7.1 Continue stakeholder outreach, engagement, consultation, awareness and/or stakeholder relationships management activities N/A 48
Activity 8: Evaluation Framework for Traceability N/A 1
8.1 Developing a method for evaluating the efficacy of the traceability system N/A 1
Activity 9: System enhancement and collaboration N/A 2
9.1 Support the enhancement of the Traceability National Information Portal N/A 2
Total Output (2013–14 to 2015–16) N/A 119

Source: Program Data

N/A: Not Applicable

Appendix F: Definitions of Advanced and Completed as per the Stream D PPMRMS

Working Definitions for "Advancing a System"

Immediate Outcome
"Advancing a system" in this instance means an organization moving through the stages of the recognition process and receiving letters of completion for those stages. It may also include activities involving integration of a system. Chicken Farmers completed Recognition under Growing Forward so are not included in this baseline.
Traceability
"Advancing a system" in this instance means an organization working on a traceability system to enable animal identification, movement tracking and reporting.
Biosecurity
"Advancing a system" means CFIA or industry organizations initiate development or update/revise an existing system.

Note: in Growing Forward, all biosecurity standards initiated were also completed in the Growing Forward period. Thus, 11 standards were completed by March 2013. So 'completed standards' will not count as part of the Immediate baseline but only as part of the Intermediate baseline.

Post-Farm Food Safety (PFFS)
"Advancing a system" in this instance means an organization moving through the stages of the recognition process and receiving letters of completion for those stages. It may also include activities involving integration of a system.
Market Attributes and Quality Standards
"Advancing a system" in this instance means working on the development of a system, set of standards or a tool in relation to an attribute. A single organization may develop multiple standards in this type of assurance. In this case, each standard/tool will be counted individually (by sector). For example, the National Farm Animal Care Council (NFACC) coordinates and leads the development of all animal care codes of practices across all sectors. Each sector code will be counted individually even though NFACC will have developed them all.

Working definitions for "a Completed System/Standard"

Intermediate Outcome

On-Farm Food Safety (OFFS)
A "completed system" is one that has received a letter of recognition from CFIA. CFIA recognizes a system based on how it is managed to ensure the integrity of the food safety system as a whole. Anything under a single management system is considered a single system, even if it has multiple modules or component. Over time, an organization may add to the system, but as long as the additions operate under the same management system or "umbrella," it would still receive only a single letter of recognition. Further, if an organization sought recognition for only certain modules one year and recognition of additional modules a few years later, CFIA would consider it the same single, recognized system because the modules all fall under the same umbrella. Should technical systems be integrated (for example, CHC's CanadaGAP with CMPA's food safety system) and then operate under one management system, it would be considered one recognized system as well.
Traceability
A "completed system" is one that can collect animal ID, track movements and provide reports as required.
Biosecurity
A "completed standard" is one that has been approved by CFIA. The nature of the MOU between AAFC and CFIA will result in all standards initiated in Growing Forward 2 also being completed by 31 March 2018.
Post-Farm Food Safety (PFFS)
A "completed system" is one that has received a letter of recognition from CFIA.
Market Attributes and Quality Standards
A "completed system" is one that the industry identifies as complete (by stating in plans when a system will be implementable) or when a tool is operational and can be used/implemented.
Benchmarking
Benchmarking could be considered a form of recognition, but by a third party or private entity/scheme (for example, GFSI). A "completed system" would be one that receives documentation stating benchmarked or equivalency status.

Appendix G: Supporting Performance Data for National Livestock Traceability Operations and G&C Programs

Table 11 - Status of Species Traceability Implementation as of June 2016
Species Elements already in place Anticipated requirements of future federal traceability regulations
Hogs Animal ID and movement reporting as per CFIA Health of Animals Regulations as of 2014 Premises ID
Cattle Animal ID as per CFIA Health of Animals Regulations Movement reporting and premises ID
Bison Animal ID as per CFIA Health of Animals Regulations Movement reporting and premises ID
Goats Nothing currently regulated Animal ID, premises ID, movement reporting
Sheep Animal ID as per CFIA Health of Animals Regulations Movement reporting and premises ID
Cervids Animal ID and movement reporting as per CFIA Health of Animals Regulations Premises ID
Poultry Animal identification, movement reporting and premises ID (but not federally regulated) Not anticipated
Equine Nothing regulated at this time Not anticipated

Source: Program Data

National Assurance Systems Development (G&C): Among the initiatives funded under this component for advancement or completion, several types of systems are eligible, namely food safety, traceability, biosecurity, market attributes and benchmarking systems. As such, several projects funded under the G&C program are developing multiple types of systems (Table 12). In other words, one project can develop more than one system type.

Table 12 - Types of Systems Being Advanced or Completed
System type Number moving toward advancement Number moving toward completion Examples of types of funded projects
Food Safety (FS) 6 4*
  • Working group to establish public/private sector consensus on Canadian approach to qualifications of FS certification personnel
  • Enhancement of existing on-farm FS assurance systems for turkey farmers by adding a breeder-specific module
Biosecurity 0 3
  • All three projects aim to integrate biosecurity standards into existing assurance programs
Traceability 6 2
  • Building a framework to prepare the Canadian goat industry for mandatory national identification and traceability
  • Adapting traceability data for use in zoning and disease management
Surveillance 2 2
  • Research projects to monitor animal health and/or develop testing procedures for specific diseases
Animal Welfare 8 7
  • Develop a national feedlot animal care assessment program
Market Attributes 0 9
  • Develop tools to measure the sustainability performance of Canadian agriculture

Note: *One project is marked as N/A but appears to be food safety related.

Source: Program Data

Appendix H: Amendments to the Agriculture and Agri-Food Canada-Canadian Food Inspection Agency Technical Expertise Memorandum of Understanding

Table 13 – Amendments to the Agriculture and Agri-Food Canada-Canadian Food Inspection Agency Technical Expertise Memorandum of Understanding
Amendment Date signed Content
Amendment 1 to Annex D: Budget – Traceability Management Office Regulatory Infrastructure Initiative June 2014 Amendment to transfer funding from non-salary operating budget to salary ($10,000) and EBP ($2,000) in year two of the MOU to support the completion of the regulatory amendment for pig traceability.
Amendment 2 to Annex B: Performance Reporting – National Biosecurity Standards Development and Annex C Work Plan – National Biosecurity Standards Development June 2014 Amendment to allow a shift in the completion of service sector biosecurity standards from three to six standards over the lifetime of the MOU to one to six so that priority is given to commodity-specific biosecurity standards.
Amendment 3 to the MOU text July 2014 Amendment to reflect CFIA's change in reporting from the Minister of AAFC to the Minister of Health Canada, to update a financial clause (clause 5.3.3.2), to change two directorates names in the governance section (clause 6.2.1 and 6.2.2) and to add a clause (clause 11.1) to provide further flexibility to facilitate future directorate name changes and other changes that are minor or administrative in nature at the DG and ED level.
Amendment 4 to Annex D of the MOU February 2015 Amendment to provide flexibility for the National Biosecurity Standards Development initiative to transfer funds between cost categories that differ from the planned Budget, but remain within the total annual fiscal allotment.
Amendment 5 to the MOU text March 2015 Amendment to authorize financial transfers to the CFIA through the earliest possible Supplementary Estimates, when there are extenuating circumstances that may impact availability of Supplementary Estimates B.
Amendment 6 to Annex B, Annex C March 2015 Amendment to add activities under the Traceability Management Office Regulatory Infrastructure work plan as well as performance measures in relation to these new activities. New activities included "Developing a method for evaluating the efficacy of the traceability system" and "Support the enhancement of the Traceability National Information Portal".
Amendment 7 to Annex B, Annex C, Annex D of the MOU August 2015 Amendment to update the Performance Reporting document (Annex B of MOU); to update the Work Plan (Annex C of MOU); to update Budget (Annex D of MOU), including adjustments to total cost category column due to an error in Amendment 1 and the addition of the following wording. "Fiscal Years 2015–2016, 2016–2017 and 2017–2018: When funds allocated to this traceability initiative are intended for a specific cost category cannot be used or effectively spent by the CFIA, the Participants acknowledge that CFIA may, within a given Fiscal Year, transfer these funds to another cost category of this traceability initiative."

Source: MOU amendments as provided for this evaluation

Appendix I: Key Definitions and Concepts

Assurance Systems
Systems that allow the agriculture and agri-food sector to  respond to consumer demand, at home and abroad. Processes and procedures that provide consumers and buyers with the confidence that associated risks along the food supply chain are adequately addressed – that food is safe, it is sourced from a healthy resource base, and that it incorporates the attributes they seek and pay for. They are not information technology (IT) systems.
Advanced and Completed Assurance Systems
Please refer to Appendix F for detailed definitions.
Biosecurity Systems
Biosecurity may be defined as: "A set of practices used to minimize the transmission of pathogens and pests in animal and plant populations including their introduction (bio-exclusion), spread within the populations (bio-management), and release (bio-containment)."

Preventive on-farm measures address animal, human, and   vehicle access, animal health and operations. The Canadian Food Inspection Agency (CFIA) develops national biosecurity standards, protocols and strategies in collaboration with producer organizations,   provincial/territorial (PT) governments, and academia. A full list of national biosecurity standards is available on the CFIA website: National Biosecurity Standards and Biosecurity Principles

Effectiveness
The impact of a program, policy or other entity, or the extent  to which it is achieving its expected outcomes.
Efficiency
The extent to which resources are used such that a greater level of output/outcome is produced with the same level of input, or a lower level of input is used to produce the same level of output/outcome.
Growing Forward 2
Growing Forward 2 (GF2) is a five-year (2013-2018) policy framework for Canada's agricultural and agri-food sector. This $3 billion investment by federal, provincial and territorial (FPT) governments is the foundation for government agricultural programs and services.
National Assurance Systems Development Program
Refers to the Grants and Contribution (G&C) funding component allocated under AgriMarketing Stream D:  Assurance Systems, as part of Growing Forward 2.
Relevance
The extent to which a program, policy or other entity addresses  and is responsive to a demonstrable need.
Service Sector Biosecurity Standards (SSBS)
A set of guidelines aimed at limiting the opportunity for introducing and spreading disease.

Biosecurity standards that apply specifically to three categories of groups supporting the agrifood sector:

  1. Service providers that specialize in agricultural services and may come directly into contact with crops and animals;
  2. Service providers that specialize in providing agricultural  services that are on farm premises but without direct contact with crops or animals; and
  3. Service providers whose customer base contains a certain percentage of crop and livestock producers.
Social Licence
The ongoing level of acceptance, approval and trust of consumers regarding how food is produced.  It encompasses issues such as environmental impact, biotechnology and animal welfare.
Traceability
Traceability is the ability to follow the movement of a product or  animal through specified stages of production, processing and  distribution. It can be limited to critical points along the supply  chain, such as, livestock from birth to slaughter, or can be full-  chain from farm to fork, such as, meat and fresh produce.  Food traceability requirements are proposed under the Safe Food for Canadian Regulations. Canadian regulatory requirements for traceability are available on the CFIA website: Description of regulatory requirements
Traceability National Information Portal
A technology platform to provide authorized users with a single point of access to traceability information (animal identification,    premises identification and movement information) collected in    distributed Federal/Provincial/Territorial and industry databases.
Traceability System
A traceability system is one that can track and trace a plant, animal or product to support product claims of market attributes or for the purposes of emergency management in the plant or animal sector. Traceability systems in Canada are based on three basic elements: animal identification, animal movement and premises identification. The ability to rapidly trace an animal throughout its life cycle is essential to isolating animal health emergencies and can help limit the economic, trade, environmental and social impacts of such emergencies.

Appendix J: Evaluation Scope and Methodology

The objective of this evaluation was to assess the relevance and performance for all three components of the Assurance Systems program, encompassing all program activities from 1 April 2013 to 31 March 2016. The evaluation addressed the core issues of relevance, effectiveness and efficiency. Table 2 presents the themes that were examined.

Table 14 - Evaluation issues and associated questions
Evaluation Issue Evaluation Questions
Relevance R1. To what extent do the objectives of AgriMarketing Stream D align with the priorities of AAFC and the Government of Canada?
Relevance R2. To what extent do the major activities of AgriMarketing Stream D facilitate the ability of industry stakeholders to meet their objectives (needs) regarding market competitiveness?
Relevance R3. To what extent is the implementation of AgriMarketing Stream D an appropriate role of the Federal Government?
Performance – Effectiveness P1. To what extent have standards and assurance systems been advanced, completed and shown progress toward implementation?
Performance – Effectiveness P2. What are the facilitators and barriers to the implementation of standards and assurance systems by producers and industry?
Performance – Effectiveness P3. To what extent have AgriMarketing Stream D activities contributed to industry’s ability to access/expand markets?
Performance – Efficiency P4. To what extent has communication and collaboration between the CFIA and AAFC facilitated efficient delivery of AgriMarketing Stream D?
Performance – Efficiency P5. Are knowledge exchange mechanisms working to ensure stakeholders have access to information to facilitate development of assurance and traceability systems?
Performance – Efficiency P6. To what extent is the G&C project review process clearly understood and transparent?
Performance – Efficiency P7. To what extent are performance data associated with AgriMarketing Stream D systematically collected and used for decision-making purposes?
Performance – Efficiency P8. To what extent have AgriMarketing Stream D activities been delivered in an efficient manner?
Performance – Efficiency P9. What is the qualitative value brought to stakeholders through AgriMarketing Stream D?

Also, the Assurance Systems program was the subject of an internal audit in 2016, the results of which have been taken into consideration in this evaluation.

Evaluation Methodology

Technical reports were produced for each line of evidence. These reports, as well as data triangulation, served as the basis for this evaluation.

Literature review:

A literature review focused on identifying relevant peer-reviewed or grey literatureFootnote 70, as well as publicly available secondary data sources, external to the program’s documentation. Information was gathered to contextualize the operating environment of the Assurance Systems program and to identify any points of comparison with similar programs or systems in other jurisdictions. The literature review served mainly to address evaluation questions on relevance.

Document review (including administrative and performance data):

A review of program documents, databases and activities was conducted as part of the document and data review to support an assessment of the program’s relevance and performance. This information provided a program profile and was important for understanding the historical performance of activities through to 31 March 2016. The information reviewed included an analysis of performance data, service standards and financial data.

Financial review/Cost-benefit analysis:

The evaluation engaged external stakeholders to seek their perspective on the value of the Assurance Systems program. While the review assessed quantitative data, the qualitative data obtained provided more meaningful information on the value of the program.

Key informant interviews:

Key informant interviews were an important source of information for understanding some of the operational challenges and successes in the implementation of program activities to date, as well as the progress toward the production of outputs and early outcomes.

Thirty-three interviews spanning 40 individuals were conducted as part of this evaluation. For tallying purposes, these interviews were separate and distinct from the interviews conducted as part of the case studies methodology, although some of the same individuals were interviewed for both purposes. The breakdown by category is shown in Table 15.

Table 15 - Breakdown of Interviews
Stakeholder Group Number of Interviews Conducted
Program management and administration (AAFC and CFIA) 10
Traceability Task Team and Industry-Government Advisory Committee members 4
Trade commissioners 2
G&C project technical reviewers and DG Review Committee members 5
G&C funding recipients 12
Total 33

Case studies:

Four case studies were conducted, primarily to verify progress towards the achievement of outputs and outcomes, but also to highlight lessons learned as a result of the challenges and successes associated with activities related to each case. Three of the four case studies looked at up to two types of assurance systems.

Projects for each case study were selected from the full list of G&C projects funded up to March 2016. Table 16 shows the G&C projects that were selected against the types of assurance systems being studied for each case study.

Table 16 - Assurance systems addressed by each Selected Case
Animal and Plant Health Surveillance Biosecurity Food Safety Integration of Assurance Systems Other Market Attributes Traceability
Pulse Canada: Aligning Canadian Sustainable Agriculture Metrics to the Sustainability Needs of the Global Food Industry N/A N/A N/A N/A Addressed N/A
Beekeepers Commission of Alberta (BCA): Honey Bee Health Surveillance Project in Canada Addressed Addressed N/A N/A N/A N/A
Jewish Community Council of Montreal (JCCM): Canada Kosher 22000 N/A N/A Addressed Addressed N/A N/A
Traceability N/A N/A N/A N/A N/A Addressed

Source: AAFC internal document

N/A: Not Applicable

The intent with three of the case studies was to look for early, illustrative examples of outcomes achieved as of March 2016:

The scope for these three cases was limited to Growing Forward 2 funded activities. They included a review of project progress reports and any relevant, publicly available documentation. For each case, two interviews were conducted, one with the funding recipient and one with the responsible AAFC project officer. The case studies provided a snapshot of the progress made toward the achievement of outputs and outcomes, and an overview of key challenges and lessons learned.  

Table 5 shows the planned G&C funding for the projects selected for the three case studies. Generally, program administrators that were interviewed noted that project expenditure had matched planned spending through to March 2016, and any deviations were considered immaterial.

Table 17 - Contribution agreements budgeted, funding by project and year
Project recipient and number 2013–14
(Actual)
2014–15
(Actual)
2015–16
(Actual)
2016–17
(Estimated)
2017–18
(Estimated)
Total
Actual and
estimated
Pulse Canada, GF2-AS-022 $63,296 $396,858 $348,185 $332,863 $236,408 $1,377,610
Beekeepers Commission of Alberta (BCA), GF2-AS-027 $0 $109,940 $209,480 $296,850 $383,730 $1,000,000
Jewish Community Council of Montreal (JCCM), GF2-AS-003 $341,800 $257,500 $60,000 $0 $0 $659,300
Jewish Community Council of Montreal (JCCM), GF2-AS-042 $0 $0 $303,000 $410,000 $0 $713,000

Note: Expenditures to date were not available, although interviews indicated that spending has generally followed approved budgets each year.

Source: AAFC internal document

The fourth case study looked at traceability and was broader in scope. It examined key activities, from Growing Forward to Growing Forward 2, that contributed to developing traceability systems in Canada. This included a review of the development of:

  1. traceability regulations,
  2. the Traceability National Information Portal, and
  3. a national industry database. The primary purpose was to identify lessons learned.

This case study included five interviews with representatives of AAFC, CFIA and one province. It also included a detailed review of relevant project files. The time period looked at for this particular case study (from Growing Forward to Growing Forward 2, or April 2008 to May 2016) is broader in scope than the other case studies because it was considered important to outline the evolution of livestock traceability in Canada. 

Evaluation Limitations

The evaluation identified some challenges and limitations to data collection:

  1. Interviews/Case Studies. Thirty-three interviews were completed, spanning 40 individuals. Coverage was adequate across the various interviewee categories with the exception of livestock traceability. Based on advice from AAFC program officials, non-government stakeholders with expertise in traceability were not interviewed, due to the evolving nature of the livestock traceability file. As a result, the information gathered in connection with traceability was derived from federal and provincial government representatives only. This also led to an inability to conclude on certain aspects of relevance and performance with respect to the traceability activities, even though conclusions may have been made for other program components in these areas.
  2. Literature Review. The primary limitations for the literature review were the resource and scope constraints linked to a more complete exploration of assurance systems in comparable jurisdictions. The assessment was based largely on publicly available information on the internet. Publicly available information, however, did not always contain details of other countries’ approaches to funding assurance systems.  In this case, best efforts were made to identify trends or information that may be relevant for this evaluation.
  3. Financial Review/Cost-Benefit Analysis. The review attempted to establish a relationship between resource inputs to outputs using financial data. However, this approach was limited by a lack of quantifiable output data. As a result, the value of the program was determined through qualitative information derived from interviews with stakeholders.
Report a problem on this page
Please select all that apply:
Date modified: