Customs Action Plan (CAP)

Interim Evaluation Study

Evaluation Division
Corporate Planning, Research & Evaluation Directorate
Strategy and Coordination Branch
Canada Border Services Agency

March 2005

Table of Contents

• Executive Summary
• Evaluation Study
  • Evaluation Purpose
  • Background
  • Issues Covered by the Evaluation
  • Methodologies
  • Findings and Recommendations
  • Approval of the Study
• Annex A - Customs Action Plan (CAP) Interim Evaluation Study Management Response
• Issue 1: CAP Design Management Response
• Issue 2: Implementation Progress Management Response
• Issue 3: ICS Effectiveness Management Response
• Issue 4: Enforcement/Security Management Response

---

Executive Summary

Purpose of Evaluation Study

The purpose of this evaluation was to provide Agency senior management with an evaluation of the progress to date regarding the implementation of, and results achieved by, the CAP. The study was undertaken as an "interim study", as outlined in the Customs Action Plan Evaluation Framework Study, approved by the Internal Audit and Program Evaluation Committee of the former Canada Customs and Revenue Agency (CCRA) in October 2002.

Early in the evaluation, the focus and scope of the study were modified to include initiatives introduced under the umbrellas of Public Security and Anti-Terrorism (PSAT) and the Smart Border Declaration (SBD). The scope was also expanded to include an interim evaluation of the Integrated Customs System (ICS), which was originally intended to be the focus of a separate evaluation.

Background

The CAP was launched in April 2000. It introduced a new, comprehensive risk management model of program delivery based on the principles of self-assessment, advance information and pre-approval. The Plan consisted of 17 initiatives, with proposed implementation over five years. The initiatives were intended to address four commitments, which are the overall objectives of CAP:

1. continue to improve service;
2. ensure that business and travellers play by the rules;
3. intensify efforts to stop illegal activity and threats to health and safety; and
4. promote certainty and consistency for exporters and for Canadians travelling abroad.

The majority of the new processes to be developed under the change program involve a significant information technology component, most of which is being developed under the Integrated Customs System (ICS) initiative. ICS is both a systems development framework and a systems architecture, within which all-new customs systems and modifications to existing systems are being delivered.

In response to increased North American security concerns, following the terrorist attacks in September 2001, the Government of Canada introduced a considerable number of security-related initiatives, under the umbrellas of Public Security and Anti-terrorism (PSAT) and the Shared Border Declaration SBD. For Customs, PSAT and SBD resulted in expanded and expedited CAP initiative development and several new initiatives designed to bolster enforcement and security capacity. The Agency's budget for developing and implementing all the initiatives is about $650 million.

Key Evaluation Findings

• The CAP model of risk-based program delivery has proven resilient and effective.
• Low participation rates limit effectiveness of pre-approval programs for low-risk participants.
• Risk assessment processing for pre-approval programs is reliable.
• Significant progress has been made in implementing the high priority CAP, PSAT and SBD initiatives, but overall implementation is currently about two years behind schedule.
• Budgeted funds are insufficient to complete implementation of the initiatives and for on-going delivery of the new programs.
• Project management shortcomings have contributed to difficulties in developing and implementing the initiatives.
• Fewer ICS applications have been implemented to date than originally planned. Those available are generally meeting business requirements and delivering expected systems performance and functionality.
• ICS is providing a sound framework for automated systems development and is positioned to meet the Agency's future business requirements.
• The Agency's enforcement capacity has been strengthened considerably since the terrorist attacks in 2001.
• Targeting and enforcement effectiveness have not improved significantly since 2000. Some key components designed to improve capabilities, however, are not yet fully implemented.
• Initiatives designed to improve intelligence, analysis and targeting have yet to realize full benefits.
• Many Customs officers believe that the new provisions included in Section 107 of the Customs Act make it difficult for them to work closely with law enforcement and security partners (for example, on integrated teams).
• New detection equipment is increasing examination and detection capacity.
• Several factors have limited the effectiveness of the new Vehicle and Cargo Inspection System (VACIS) machines.

Key Recommendations

• Consider development of a second Customs Self Assessment / Free and Secure Trade (CSA/FAST) option which provides participants with border streaming privileges, but has a less onerous accounting component.
• Conduct a full examination of the pre-approval programs with a focus on program rationale, contributions to program model effectiveness, and cost effectiveness.
• Develop a long-term funding strategy to enable completion and effective on-going delivery of the initiatives.
• Implement the key project management lessons learned during the development of these initiatives.
• Develop a strategy to improve effectiveness of VACIS machine use, including establishment of dedicated VACIS teams for all deployed machines
• Conduct an evaluation of the Partners-in-Protection (PIP) program to develop recommendations to improve program performance.

Evaluation Context / Approval of the Study

An evaluation framework for the CAP evaluation study was completed by the CCRA in early 2001. This framework was amended in August 2003 to take into account PSAT and Smart Border Action Plan Initiatives that had been introduced after the September 11, 2001 terrorist attacks. Though some administrative data used in the report were collected in the fall of 2004, most primary data collection occurred between June 2003 and May 2004. As a result, the key evaluation findings and recommendations pertain largely to program operations and issues that were evident, in many cases, more than two years ago. With the transition to CBSA, reporting of results did not occur until March 2005. By that time, numerous program changes and enhancements had occurred. For further details on these program changes, please refer to the March 2005 Management Response in Annex A.

The Management Response was prepared by the three key branches implicated in the evaluation in response to the recommendations outlined in the study. The Audit and Evaluation Committee of CBSA approved the evaluation, including its key findings and recommendations, on March 17, 2005.

This report concludes the last of the Customs specific evaluations carried over from the CCRA. There will be no further "CAP" evaluations. Individual CAP, PSAT and SBD initiatives will be the subject of evaluations during the coming years.

Evaluation Study

A. Evaluation Purpose

The purpose of this study is to provide Agency senior management with an evaluation of the progress to date regarding the implementation of, and results achieved by, the Customs Action Plan (CAP). This study was undertaken as an "interim study", as outlined in the CAP Evaluation Strategy, approved by the Internal Audit and Program Evaluation Committee (IA/PE) of the former Canada Customs and Revenue Agency (CCRA) in June 2001. The CAP Evaluation Framework Study, approved by the IA/PE Committee in October 2002, provided a plan for conducting this interim study.

Early in the evaluation, the focus and scope of the study were modified to reflect the revised change program that emerged after the terrorist attacks in 2001. This program comprises initiatives introduced under the umbrellas of the CAP, Public Security and Anti-Terrorism (PSAT) and the Smart Border Declaration (SBD). The evaluation also assesses the progress made in developing and implementing the Integrated Customs System (ICS). Originally, the ICS was to be the focus of a separate evaluation. At an early stage, however, this was felt to be impractical due to the interdependencies between ICS development and CAP development. The Customs Branch of the former CCRA approved Terms of Reference for the Study, including its revised scope, in December 2003.

The scope of the study includes initiatives designed to transform the Customs program only. It does not include those affecting immigration or food and plant inspection program delivery.

The CAP Evaluation Strategy called for a "final" CAP evaluation, to be undertaken in Fiscal Year 2005-06. As a result of the changed environment after the terrorist attacks in 2001, and the formation of the Canada Border Services Agency, it is clearly no longer appropriate to continue with the original plan.

B. Background

The Customs Action Plan was introduced in April 2000. The launching of the CAP culminated an extensive consultation process with stakeholders and clients that had begun in October 1998 with the publishing of the Customs and Trade Administration Blueprint - A Discussion Paper (Blueprint).

The CAP introduced a new, comprehensive risk management model of program delivery, based on the principles of self-assessment, advance information and pre-approval. It had two main features:

• Risk-based processes for travellers and traders
  • Streamlined processing for low risk.
  • Intensified processing for higher and unknown risk.
• A fair and effective sanctions regime.

There were 17 initiatives covered in the Plan. These initiatives were intended to address four commitments (the so-called "Blueprint commitments"), which are the overall objectives of CAP:

1. Continue to improve service;
2. Ensure that business and travellers play by the rules;
3. Intensify efforts to stop illegal activity and threats to health and safety; and
4. Promote certainty and consistency for exporters and for Canadians travelling abroad.

The new model of program delivery represented a shift from Customs' previous transactional approach, in which every cross-border movement was subjected to similar processing and scrutiny. The new model is designed to allow Customs to achieve an adequate balance between facilitation and enforcement, in an environment marked by increasing volumes of travellers and goods entering Canada. The new approach is intended to help Customs deliver on the first Blueprint commitment of improving service, while at the same time intensifying its efforts to stop illegal activity and threats to health and safety (the third commitment).

Customs intends to obtain as much information as possible, in electronic format, prior to the arrival of goods and people at the border, to enable informed decisions at arrival and speed up movement of legitimate travellers and goods. The Agency is also striving to enrol significant numbers of low-risk travellers and importers on pre-approval programs in which participants self-assess their Customs obligations, subject to periodic inspection or verification.

The majority of the new processes to be developed under the change program involve a significant information technology component, most of which is being developed under the Integrated Customs System (ICS) initiative. ICS is both a systems development framework and a systems architecture, within which all-new customs systems and modifications to existing systems are being delivered. The ICS approach involves building systems within a single, component-based architecture, premised on the reuse of common components and a single, consolidated front end for users, allowing applications to have a common look and feel.

All new applications are being built on ICS; eventually legacy applications will be replaced with ICS ones, installed on ICS, or interfaced with ICS. Within the current change agenda, the intention has been to build the ICS base within which all future automated applications will be built, and to build solutions for all the current change initiatives. Integration of all Customs automated systems is a long- term project, which can only be completed long beyond the time frame of the current change program.

The terrorist attacks in the United States in September 2001 had a significant impact on the Customs environment. In response to increased North American security concerns, the Government of Canada passed the Public Security and Anti-terrorism (PSAT) legislation and signed the Shared Border Declaration (SBD) in late 2001. Under these umbrellas, the Government introduced a great number of security-related initiatives, involving agencies with law enforcement and security responsibilities. For the Customs change program, PSAT and SBD has resulted in expanded and expedited CAP initiative development and several new initiatives designed to bolster enforcement and security capacity.

Broadly speaking, Customs has been increasing its enforcement capacity in three main areas. The first is increasing interoperability and cooperation with Customs' key partners, both in Canada and the US. This includes involvement in integrated enforcement teams. It also involves working more closely with the US Department of Homeland Security to improve container targeting, exchange lookouts and targets, test joint targeting of airline passengers, and tackle other border vulnerabilities. Customs is also being funded to increase and enhance its inventory of detection equipment, as well as increasing the staff to operate the equipment. The third key area is improved automated systems to support enforcement and intelligence and to provide better tools for risk assessment and risk management.

The total budget for initiative development and implementation is $655.2 million, over the period 2000-2008. This includes $99.4 million over five years that the Treasury Board agreed to provide CCRA in 2000 for CAP initiative development and $39.2 million from internal funding that the CCRA committed at the same time. It also includes Treasury Board's funding of $278.4 million for PSAT and $238.2 million for SBD.

Key Initiatives of the CAP, PSAT and SBD

Streamlined reporting processes for pre-approved low-risk clients

• For Travellers: NEXUS-Highway, NEXUS-Air and CANPASS Air
• For Traders: Customs Self-Assessment / Free and Secure Trade (CSA/FAST)

Risk assessment programs and automated systems:

• Advance Passenger Information / Passenger Name Record (API/PNR)
• Advance Commercial information (ACI)
• Titan, Paxis, and Integrated Primary Inspection Line (IPIL)

Architecture and framework for delivery of automated solutions

• Integrated Customs System (ICS)

Sanction Regime to promote compliance

• Administrative Monetary Penalty System (AMPS).

Intelligence, analysis and targeting enhancements:

• National Risk Assessment Centre (NRAC)
• Integrated Border Enforcement Teams (IBETS)
• Integrated National Security Enforcement Teams (INSETS)
• In-Transit Container Targeting Units
• Partners in Protection (PIP).

Detection Equipment

• New detection equipment, such as Vehicle and Cargo Inspection System (VACIS) machines, X-Ray Scan Trailers and radiation detection equipment
• Increased quantities of previously deployed equipment.

C. ISSUES COVERED BY THE EVALUATION

D. METHODOLOGIES

The evaluation used several methodologies to address the evaluation issues and to measure the progress to date of the implementation of the Customs change agenda and results achieved to date:

• 71 interviews of: Agency officials in Headquarters and in the Regions; officials from the RCMP and Citizenship and Immigration Canada; and representatives from the private sector;
• On-line survey of randomly selected Customs officers across Canada in October 2003, to which 353 officers provided responses;
• Review of a sample of 309 applications for the Customs Driver Registration Program (CDRP) to verify whether acceptance and rejection criteria were applied properly;
• Review of agency, program and project documentation and data.

E. FINDINGS AND RECOMMENDATIONS

Issue 1: CAP Design

1.1 The CAP model of program delivery addresses the changing dynamics of the Customs environment, and provides for balance between service and facilitation

As noted in the "Background" section above, the CAP introduced a new model of program delivery, founded on the principles of risk-based processing. The evaluation found that this model has proven to be resilient in light of changing imperatives following the terrorist attacks on the United States in September 2001. It has allowed the Agency to keep low-risk border traffic moving, while providing for increased efforts to address health and safety threats. New initiatives introduced after September 2001 have focused largely on enhancing enforcement capacity, but have been achieved within the model. It should also be noted that this model has now become the cornerstone of US Customs program delivery. Since the terrorist attacks, the US administration has embraced the concept of pre-approval processing in both the travellers and commercial streams. This has opened the doors for joint Canada-US programs in both streams.

As noted in various parts of this Study, the evaluation found problems with the operation and functioning of some of the new programs. From a design perspective, however, the CAP model provides for an appropriate balance between facilitation and enforcement, in light of conflicting pressures of limited Agency resources, increased border volumes, the need for more streamlined and expedited processing, and the imperative to protect Canadian society.

Achieving an appropriate balance between facilitation and enforcement was a key consideration for Customs in designing its new initiatives. Instead of dealing separately with facilitation and enforcement, the design of the new program delivery effectively links facilitation of low risk cross-border movements of people and goods to intensified scrutiny of higher risk movements. In essence, the Agency can deliver improved service responsibly, when it is able to ensure that the borders are protected effectively. The model's focus on pre-approval processing and advance information in both the travellers and commercial streams facilitates a more comprehensive and effective approach to handling risk at the border. In the past, the Customs inspector at the Primary Inspection Line (PIL) had to differentiate between high and low risk travellers and goods on a transactional basis for every cross-border movement, largely without the assistance of pre-arrival analysis or information. The CAP model reduces the amount of transactional processing of low risk, and provides officers with better information and tools for targeting and examination of higher and unknown risk people and goods.

1.2 The importing community feels that CSA/FAST and ACI are too constraining, but Agency imperatives necessitated reengineering of the Customs commercial process

As a result of the CAP, all previously available import service options are being replaced with two new ones: Advance Commercial Information (ACI) and Customs Self-Assessment/Free and Secure Trade (CSA/FAST). This reflects Customs' need to reduce the burden of managing and maintaining multiple service options, as well as balancing service and enforcement objectives. More specifically, the CSA/FAST design balances the extension and strengthening of streamlined processing with the necessity to ensure the process remains secure, and to leverage the program's overall benefits. ACI provides Customs with a more effective process (involving increased information about shipments in advance of their arrival at the border) for managing the risk of non-CSA participants, without negatively impacting the speed of border processing.

While the importing community recognizes the Agency's need for increased information and enhanced border security, many in the community feel that the move to these two commercial import service options does not meet their needs very well. In general, they find ACI to be onerous on a transactional basis and do not believe that CSA/FAST participation proves a beneficial alternative due to the significant investments required to comply with its self-assessment and accounting requirements. Many importers were initially interested in CSA because it offers streamlined border clearance, which provides greater certainty to their business operations. They also liked the concepts of extended accounting time frames and self-assessment of Customs obligations. For many potential participants, the systems changes required for self-assessment and accounting, however, are burdensome and cost more to put into place than they are prepared to pay.

1.3 Despite the Agency's significant communication efforts, many members of the trading community still lack knowledge about new program requirements

The evaluation found that the Agency has made significant efforts to communicate with the trading community about the new initiatives and changed requirements. It has made particularly significant efforts in relation to informing traders about Administrative Monetary Penalty (AMPS) and ACI requirements. Despite these efforts, many smaller companies are not very knowledgeable about the requirements of these programs. It is particularly important that all traders, however, are well aware of their ACI obligations prior to initiative implementation in the highway mode in 2006, since most small and infrequent traders transport goods into Canada by truck. Most will have to make changes to their systems and processes to be in compliance. There is a significant risk that lack of trader preparation could cause border-processing slow-downs.

1.4 Low Participation Rates Limit Effectiveness of CSA/FAST

CSA/FAST is designed to play a key role in Customs program delivery. Streamlined processing of a significant proportion of commercial imports is intended to allow the Agency to shift resources to processing higher and unknown risk traffic. Another CSA objective is to improve the level of participants' compliance with accounting and trade data reporting requirements. The relative importance of maximizing streaming benefits has increased, however, since the terrorist attacks in 2001.

The Agency's target for CSA/FAST was 50% of overall commercial import releases by March 2005. CSA/FAST releases currently account for about 20% of releases at key Southern Ontario ports, but only about 7% nationally. There has been a recent increase in applications, but the program does not appear positioned to achieve the target in the short or medium term. It should be pointed out that CSA/FAST importers account for about 20% of the Value for Duty of imports into Canada.

Some factors external to the CSA/FAST program have delayed client take-up. The delayed implementation of ACI, which increases pre-arrival reporting requirements for traders, removed a compelling incentive to participate in CSA/FAST. ACI is now scheduled to be implemented in all modes in 2006. In addition, CSA marketing efforts were overtaken by the decision in 2001 to implement FAST by late 2002, which resulted in a diversion of program resources to FAST design and development.

Four main factors continue to inhibit importer participation:

1. Lack of dedicated lanes at most commercial crossings;
2. High participant costs;
3. Program exclusions re: commodities, countries of origin and modes of transport; and
4. Lack of compelling incentive to participate.

The Agency is addressing several of the factors inhibiting participation. It will be eliminating other expedited processes (e.g. the Frequent Importer Release System (FIRST), and the Aerospace and Automotive programs) and has been working with Other Government Departments (OGDs) to ease their border release requirements. Processes are also being designed to relax current CSA/FAST country of origin and mode restrictions. Though some progress has been made, the ability to offer dedicated lanes at many major border crossing is restricted by physical limitations at these locations.

The high participant costs are largely due to the accounting requirements of CSA/FAST. While each importer situation is unique, most importers have to make significant investments to establish the system linkages required to ensure complete and accurate reporting of imports and provide adequate audit trails. Many prospective participants do not believe that the benefits to them of CSA/FAST are worth the investment required, at least in the current environment.

Offering traders a CSA/FAST option that provides them with streamlined border release, but has less onerous accounting requirements, may make the program more attractive to traders and increase participation. It is noteworthy that the US FAST program does not have a specific accounting component. Like the Canadian FAST program, applicants for the US program must be assessed as low risk for border criminality and border security, and are required to participate in the Customs and Trade Partnership Against Terrorism (C-TPAT), which is the US equivalent of Partners in Protection (PIP):

It should be noted that under the current program design the level of compliance of CSA/FAST importers with Customs' accounting and trade data reporting requirements has been increasing.

1.5 Limited participation in CANPASS and NEXUS programs minimizes their contribution to realization of program model objectives

There are several traveller pre-approval programs in operation or under development. These programs offer streamlined processing for low-risk travellers. The NEXUS programs are operated jointly with the US, while the CANPASS programs are Canada-only programs. NEXUS-Highway is operational at all higher-volume ports. CANPASS-Air is available to participants at the international airports in Vancouver, Toronto, Halifax, Montreal, Calgary and Winnipeg, and is scheduled to be operational at the other two major Canadian airports (Edmonton and Ottawa) by the end of 2005. The Agency is currently running a NEXUS-Air pilot project at Vancouver International Airport. The key objective of the traveller pre-approval programs is similar to CSA/FAST: to facilitate resource shifts to processing of higher risk movements.

Though participation rates are increasing, they are lower than projected. The Agency set a goal of processing 30% of travellers through CANPASS Highway (now NEXUS-Highway) at participating ports by the end of Fiscal-Year 2003-04. NEXUS-Highway passages, however, still currently constitute less than 10% at these ports and only approach the 30% target at some Pacific Region ports.

CANPASS-Air passages currently constitute less than 1% of total traffic at airports where it is available. Studies suggest that participation in air traveller pre-approval programs should increase significantly over time.

The evaluation identified the following main reasons for low participation rates:

1. There is a low proportion of commuters and frequent travellers compared to overall traveller traffic;
2. The lack of designated lanes at most busy highway ports limits the participants' ability to realize fully streamlined border clearance;
3. The application fees (Cdn $80.00 for NEXUS; $50.00 for CANPASS-Air) are a disincentive to many travellers, particularly non-commuters. The fees are also non-refundable, even to unsuccessful applicants;
4. Any past infraction makes a traveller ineligible for participation in NEXUS programs; and
5. The limited number and inconvenient location of NEXUS and CANPASS enrolment centres.

The evaluation found that there is limited potential for expansion of traveller pre-approval programs to smaller ports due to insignificant wait times and limited interest, in addition to the above-listed disincentives.

Without adequate traveller participation in the program, Customs' ability to achieve savings and reallocate staff to handling higher risk movements is limited. As a result, as with CSA/FAST, limited participation restricts the effectiveness of the new program delivery model.

1.6 Pre-Approval risk assessment processing is reliable

The evaluation study reviewed decisions made by officers conducting risk assessment of applicants for the Commercial Driver Registration Program (CDRP) and for NEXUS-Highway programs. We found a high level of compliance of officers in applying existing risk assessment criteria for these programs (over 99% for both programs).

For the most part, the risk assessment process for the pre-approval programs is enabling the Agency to ensure participants are low risk, within current risk definitions. Though compliance with criteria is high, however, operational issues affect accuracy and validity of risk assessment decisions, including: keying errors, quality and completeness of photocopies of drivers' licenses and citizenship documentation, and coordination of Customs and immigration officers involved in risk assessment.

Expansion of systems checks could enhance the risk assessment process. At present, some potentially relevant systems are not consulted for all applicants.

In addition, applicants for pre-approval programs may only be rejected for Customs or immigration offences or criminal convictions. The Agency has recognized the limitations of these rejection criteria, so an approach is being developed to enable exclusion of high-risk applicants also on the basis of information other than criminal convictions or offences.

Issue 2: Implementation Progress

2.1 The Agency has made significant progress in implementing its high-priority initiatives, but there have been some delays

Despite the difficult environment in which CAP, PSAT and SBD initiative development has taken place, the evaluation found that the high priority initiatives are being implemented. Development has been complicated by changing priorities after the 2001 terrorist attacks, the need to speed up implementation of some initiatives, and the dynamics of joint Canada-US project work.

Changing priorities have resulted in decisions not to implement parts of some initiatives, at least not in the short term. AMPS for the Travellers stream, and streamlined clearance of passengers from visa-exempt countries and expedited baggage handling for CANPASS-Air have been delayed indefinitely, as have passage processing functionality for CSA/FAST and ACI.

A number of the key change initiatives were implemented more or less on time, such as NEXUS-Highway, FAST, In-Transit Container Targeting and the first phases of IPIL and API./PNR. Overall, however, implementation of the initiatives has been behind schedule from an early stage, before the 2001 terrorist attacks. For example, implementation of the first two major initiatives, CSA and AMPS, was about six months late (December 2001 instead of June 2001).

Delays involve both initiatives developed on automated systems and those not requiring development of such support. However, the delay in automated systems construction has been the most significant cause of later than planned completion of initiatives.

Most initiatives are being developed with automated support, mostly within ICS, so the pace of automated systems construction has largely become the critical path for these initiatives. There are significant interdependencies amongst these initiatives due to the dynamics of the ICS environment, based as it is on the use of common components and a consolidated front-end. The scope of this work was larger than expected, which made many initiative implementation schedules unrealistic. In addition, initial plans did not take into account initial ICS infrastructure development, which was a significant feature of the first year of ICS development. Overall ICS development is currently about two years behind schedule, with completion of ICS implementation now scheduled for the second quarter of 2006.

Beyond automation, however, various initiative-specific factors have contributed to delayed implementation of the initiatives. For instance, the introduction of PSAT and SBD initiatives further increased the scope and complexity of automated business requirements. The length of time required for client and stakeholder consultation was underestimated, which resulted in some delay in AMPS and API/PNR implementation. Full implementation of the latter initiative has also been delayed by privacy concerns raised by the European Union. The decision to harmonize ACI processes with directions taken by the US also had an impact on the timing of final implementation of this initiative. The duration of the procurement process was a key factor in the delayed deployment of detection equipment while the delay in the development of supporting policies has postponed Customs Controlled Areas implementation.

Timelines for most initiatives have generally been very tight, with new automated systems (or significant changes to existing systems) completed just in time to support the launching of the new initiatives. This has not allowed sufficient opportunity for employee training and client testing on the functioning systems or for systems adjustments prior to initiative implementation.

2.2 While many key initiatives have been implemented, some key risk management and enforcement functionality is still under development, so full effectiveness and benefits from the new program delivery model have not yet been realized

The new program delivery model is intended to result in expedited processing of low-risk transactions and more effective scrutiny of unknown and high-risk movements. Achievement of full benefits requires the implementation of the key components of the model. These include the pre-approval initiatives intended to streamline border processing, as well as functionality designed to provide risk management underpinnings and improve enforcement capabilities.

Significant progress has been made in putting the new program delivery model into operation. Though there is still further work to be done in expansion of the CANPASS and NEXUS programs, introduction of CSA/FAST, NEXUS-Highway and CANPASS-Air has expanded the principle and application of expedited processing for low risk transactions. Several initiatives designed to improve enforcement, risk identification and assessment are now operational. These include parts of IPIL, API/PNR and the new detection equipment.

Some risk management elements of the model, however, have not yet been implemented. In particular, some risk identification and risk assessment functionality is not yet in operation.

Delayed implementation of other enforcement initiatives has also postponed expected improvements in enforcement capabilities. For example, delays in the implementation of Customs Controlled Areas and the deployment of certain radiation detection equipment have resulted in protection benefits associated with these initiatives being delayed.

In addition, delays in implementation of ACI removed an incentive for traders to participate in CSA/FAST initially, contributing to lower than expected current participation rates. ACI was implemented in the marine mode in April 2004, and is scheduled to be implemented in other modes in 2005. The delay in putting ACI into place not only postpones the realization of risk assessment benefits expected from this initiative, but also the full effectiveness of the new program delivery model. From the onset of the change program, the Agency has recognized that achieving the expected level of client participation in CSA/FAST would be dependent on the implementation of ACI.

2.3 There are insufficient funds available to complete initiative implementation, provide for automated systems maintenance or pay for program delivery of implemented initiatives

The change program has been under-funded since its inception in 2000. For CBSA, initiative development shortfalls are compounding the shortages in A-Base funding for on-going operations that have been identified in other reviews. In 2000, the Treasury Board obliged the Agency to reduce its funding request for the CAP initiatives to just under $100 million, to put the request under the threshold for a "Crown Project". CCRA adjusted by allocating about $39 million from internal Agency funds. It also cut all funding for the program administration of CAP initiatives once they have been developed and implemented, and for most on-going automated systems maintenance and capital replacement required for these new initiatives.

To cope with this situation, Customs has pursued a funding strategy that has paid development costs to date and enabled development of high priority initiatives. The Agency decided not to develop several PSAT initiatives and to reduce the scope of FAST. Funds made available through these decisions were reallocated to higher priority initiatives. The PSAT initiatives for which development plans were suspended include: Automated Trade Chain Partners (Automated TCPs), Customs On-Line for Small Business, ACROSS-OGD Interfaces with Environment Canada and Health Canada, and enhancements to ACROSS to support full multi-modal warehouse capabilities. The scope of FAST was reduced by reducing implementation to 12 ports instead of 21, decreasing the number of PIL lines at which FAST processing is available at these ports, and decreasing the number of client enrolment offices for the program. The most significant reallocation provided additional funding to the high-priority API/PNR initiative.

Generally, the changes resulted in funding transfers from lower priority to higher priority initiatives, with limited impacts on the functionality of major initiatives. The Automated TCPs initiative is the most significant exception, however, as this initiative is intended to enhance the risk assessment of entities involved in the transportation of goods for CSA/FAST participants. The impact of FAST descoping has been limited to date, however, since lower than expected participation rates have minimized the impact of fewer ports and dedicated PIL lines.

Spending in excess of budgets has compounded the impact of shortfalls. As examples, it cost about $7 million more for CSA than budgeted, about $10 million more for AMPS and about $6 million more for API/PNR.

There are several reasons why development costs have exceeded budgets. Unrealistic deadlines obliged project teams to spend more money for overtime than budgeted, and delayed implementation also increased costs. In addition, the scope of many initiatives was underestimated. Developing accurate cost estimates was made more difficult by funding processes that required estimates based on early analysis of requirements. In order to obtain Treasury Board funding, the Agency had to make commitments concerning the initiatives before it could conduct detailed analysis. This was a particular problem with PSAT and SBD initiatives, for which the Agency had very restricted timeframes to develop estimates.

It should be noted, however, that the Agency's estimates could have been better. The Agency needs to improve its ability to estimate project costs, especially when significant information technology components are involved.

2.4 Project management shortcomings have contributed to difficulties in initiative development and implementation

The evaluation found that project teams responsible for developing and implementing the initiatives have not been able to achieve the level of coordinated and integrated project development required to meet the demands of interrelated automated systems development within the ICS infrastructure. This has contributed to delayed initiative development and difficulties in initiative implementation. Key factors inhibiting project management have been shortage of project management knowledge and experience, intermittent existence of central project control offices, and lack of authority of these offices over the project teams.

The evaluation found that there are no comprehensive and accurate schedules and budgets maintained to track initiatives overall. This has made it difficult to manage initiative development and implementation on an on-going basis.

The Agency has been able to mobilize a large number of knowledgeable staff, who have been able to develop and implement CAP, PSAT and SBD initiatives successfully. There have been so many initiatives, however, that the demand for qualified staff has outstripped supply. The Agency has been unable to mitigate fully the shortage of sufficient staff with skills, knowledge and experience in automated systems development, project management, and Customs program and operations. This shortfall has contributed to delayed initiative development.

Issue 3: Effectiveness of ICS

3.1 Available ICS applications are generally meeting business requirements and delivering expected systems performance and functionality.

Since automated development is behind schedule, fewer automated applications have been implemented on ICS than originally planned. In addition, several do not yet have all planned functionality in place.

The evaluation found that those that have been implemented, however, are providing the user functionality that was expected of them. Agency staff is by and large satisfied with the performance and functionality of the new applications, as indicated by responses to the Customs officer survey and in interviews of Customs officials.

Some difficulties have been encountered with a few applications. IPIL, Paxis and Titan have data matching problems with certain names, which limit their ability to match names in lookouts with those appearing in travel documents. There have also been some initial problems with Titan functionality, which upcoming systems releases are designed to fix.

Some initial problems with functionality of the ICS Enforcement system identified by officers during the Customs officers survey (such as difficulties in issuing and printing Notices of AMPS Penalty Assessments) have been fixed. Remaining issues generally reflect officers' difficulties with the complexities of the AMPS program, rather than with ICS Enforcement itself.

ICS applications have largely maintained common look and feel. They have been largely stable and generally available, although data concerning systems outages suggest that there have been some isolated problems.

Response times of ICS applications have generally been adequate to facilitate Customs program delivery. Some initial problems have been encountered with Titan (launched in December 2004), but upcoming systems releases include fixes designed to increase its speed. Further tests for ICS will come with completion of implementation of ACI in all modes in 2006, since this will substantially increase the amount of transactions processed through ICS.

Within the current initiative project development time frames, ICS integration objectives are limited. Delays in ICS application development have resulted in a slower pace of integration than planned. Officers still need to log on separately to many key systems. Regional staff interviewed for the evaluation have expressed disappointment with the limited impact of ICS enforcement on systems interoperability to date.

ACROSS Stability and Availability Initiative

The Accelerated Commercial Release Operations Support System (ACROSS) was implemented in 1996. One of its key features is a release process, which allows participating traders to send release information electronically to the Agency. In the late 1990s, there were significant problems with the reliability of ACROSS, with a high level of system outages and instability. In an effort to improve reliability, in 2000 the Agency introduced the "ACROSS Stability and Availability Initiative" (A CAP initiative) to reengineer the Customs Electronic Platform supporting ACROSS.

The evaluation found that this initiative has succeeded in increasing ACROSS stability and availability. Traders interviewed for the evaluation indicate that there have been far fewer system outages and that ACROSS performance has improved markedly since 2000 to the point that this no longer poses any significant concern. A large majority of officers who responded to the Customs Officer survey agreed that the number and duration of ACROSS outages and slowdowns have decreased significantly.

3.2 ICS is providing a sound framework for automated systems development

In selecting the ICS concept as a framework for CAP initiative development, the Agency is in line with recent thinking and trends in the information technology industry. Over the past ten years, there has been a growing preference to develop automated solutions based on component-based architecture. Results to date have shown that systems built in such architectures have generally been providing significant benefits, including increased systems interoperability, lower maintenance costs, increased flexibility and ease of adding additional components and applications, and more user-friendly systems.

ISTB staff (on both the business development and systems development sides) believe that ICS is a good choice as systems architecture and will produce significant benefits to the Agency, though some objectives will only be fully realized in the long term. They noted that ICS is a viable alternative to the stove-piped systems development process previously used. The previous approach resulted in systems that cannot be maintained and supported very efficiently. In addition, different features and user interfaces made it more difficult for officers to become proficient with the various applications required for their work.

Common components are a key feature of the ICS architecture. The evaluation found that those that have been built to date have been successfully implemented and have been able to support multiple applications, as necessary.

The evaluation also concluded that the ICS development framework has reduced development risks compared to previous approaches because it involves systems building in an iterative manner, which facilitates detection of problems at an earlier stage. Building applications on a proven platform also reduces risks.

The integrated nature of ICS does not allow for the same level of flexibility in satisfying business requirements as in distributed systems development. Flexibility has to be restricted because the specifications of automated applications built on ICS have to be compatible with the common components and comply with ICS standards. In practice, though, it has been possible to make alterations to existing ICS code tables and common components to accommodate some specific initiative requirements.

Integrated systems development necessitates greater planning and coordination amongst project teams and between business and systems development teams. The requirements of each initiative have to be accommodated on a single architecture, requiring more consultation among project development teams and individuals to deal with initiative issues. Agency project management shortcomings have made it difficult to manage this challenge.

3.3 ICS appears to be positioned to meet the Agency's future business requirements

The evaluation found that ICS appears to be positioned to meet Customs future business requirements. The groundwork has been laid by achieving a common look and feel, reusable common components and systems stability, as noted in Section 3.1 above. It should become easier for ICS to satisfy Customs business requirements after the ICS base has been substantially completed, which should occur in 2006 when ACI is fully implemented. At that point all major common components will be available and only minimal programming thereafter should be required for new applications built on ICS. This should result in several benefits, in comparison to previous automated systems development, including reduced time for developing new applications and decreases in incremental costs of constructing these applications.

The groundwork that has been laid to date also provides the basis for future integration of Customs automated systems. Substantial integration and systems interoperability, however, will only be achieved over the long term. The Agency should ensure that future systems development plans ensure steady progress towards interoperability and integration goals.

Automated systems maintenance has been easier for ICS than for stovepipe systems, since it is not necessary to conduct maintenance to similar components existing in separate systems. Ease of ICS maintenance should continue as long as deviations from ICS standards, and to common look and feel, are kept to a minimum. The Agency's ability to realize overall maintenance benefits, however, is limited by on-going use of several legacy systems, which are not connected to ICS.

Issue 4: Effectiveness of Implemented Security and Enforcement Initiatives

4.1 The Agency's enforcement capacity has been strengthened since the terrorist attacks in 2001

Since the terrorist attacks in September 2001, the Agency has made significant progress in shifting its focus to include effective management of security risks. This has involved changing employee mindsets, and developing new expertise, methodologies and processes. These changes cannot be completed in the short-term so are still on going.

Customs examination capacity has also increased since 2001. The traveller examination rate increased from 3.4% in 2000-2001 to 3.6% in 2003-04. The commercial examination rate rose from 1.5% to 2.3% during the same period.

Since 2001, the value of detection equipment deployed across Canada has increased from $12 million to over $60 million. The Agency has installed new equipment (such as VACIS machines and x-ray scan trailers) in addition to additional quantities of previously deployed equipment. After completion of radiation equipment installation in 2005, the Agency will have the capacity to conduct radiation checks on almost all marine containers arriving at the ports of St. John, Halifax, Montreal and Vancouver (which together receive nearly all container traffic arriving at Canada's marine ports of entry).

New risk assessment systems and programs have been, or are being, developed to enhance Customs' risk assessment capacity. These include Advance Passenger Information/Passenger Name Record (API/PNR) (supported by the automated system PAXIS), Advance Commercial Information (ACI) (supported by the Titan automated system), Integrated Passenger Inspection Line (IPIL) and Customs Query. These systems facilitate more effective referral and targeting activities by officers.

Intelligence, analysis and targeting capacity enhancements have increased the level of Agency interoperability with key partners, and provided better tools for analysis of intelligence and cross-border movements. Of particular note, the National Risk Assessment Centre (NRAC), established in January 2004, has increased risk analysis capacity and is facilitating sharing of risk data with the Agency's law enforcement partners.

4.2 There has been little improvement in targeting and enforcement effectiveness since the launching of CAP in 2000

While available information indicates that the incidence and magnitude of cross-border criminal activity has not decreased, there have not been sustained increases in the hit rate for targeted examinations or in the volume and value of contraband seizures. The hit rate for targeted examinations has remained between 15% and 18% since 2000, while the value of seized contraband has varied between about $300 million and $700 million during the same period (the largest volume was intercepted in 2000).

Since September 2001, Agency officers have managed adequately the national security threats they have encountered. At this point, however, it is difficult to assess how effective the Agency is in managing national security threats overall, due to the nature of these threats and the Agency's shared role with other security and enforcement organizations.

The lack of availability of timely and useful information, and inadequate tools have been key factors limiting targeting and enforcement effectiveness. There are several CAP, PSAT and SBD initiatives designed to address these shortfalls, but key components are still outstanding. Significant targeting improvements can be expected following successful and full implementation of ACI, API/PNR, ICS secondary and ICS passage history.

During the evaluation, concerns were raised that targeting effectiveness has also been limited by Customs' approach to targeting and interdiction. This approach places too much reliance on indicators that have resulted in interdictions in the past, and is not responsive enough to changes in smuggling patterns.

4.3 The new risk assessment systems and programs are positioned to facilitate improved targeting and enforcement capacity in future

The Integrated Primary Inspection Line (IPIL) allows primary inspection officers to conduct queries against Customs and immigration enforcement databases in real time by swiping traveller documents in a document reader or manually keying tombstone data on travellers. It can identify whether the traveller is the subject of a Customs or immigration lookout or has previous Customs infractions, helping officers to determine whether they should be referred to secondary for further questioning and/or inspection.

IPIL is improving the quality of information available to officers to support primary processing. Planned enhancements should increase benefits, especially ICS Secondary Processing which will provide secondary officers with access to the reasons for referrals and primary officers with feedback about the results of referrals.

IPIL usage rates at the major airports are now approaching 100%. The Agency, however, needs to continue efforts to attain a usage rate as close to 100% as possible at other locations.

Customs is considering the deployment of IPIL to other modes, such as highway commercial and highway. The Customs Branch of the former CCRA prepared an internal "Evaluation Report" to assess the results of an IPIL Commercial pilot project conducted at the port of Lansdowne from September 2002 to March 2003. The report concluded that IPIL is workable in the commercial stream, except for processing CSA/FAST shipments.

Titan and PAXIS increase the Agency's targeting capacity. Titan is the automated system that supports the ACI initiative, while PAXIS supports API/PNR. They conduct automatic risk scoring of commercial and passenger data, respectively, which assists officers in identifying high-risk cross-border movements requiring further scrutiny. To ensure that these systems are responsive to the changing dynamics of border criminality, it is key that the risk scoring formulae be assessed, and adjusted as necessary, on an on-going basis.

As noted above, in Issue 3 above, IPIL, PAXIS and Titan have data-matching problems, which limit their ability to match names in lookouts with those appearing in travel documents.

Through Advance Passenger Information / Passenger Name Record (API/PNR), the Agency receives information about air passengers in advance of their arrival in Canada. This information includes the passengers' "tombstone data" (such as name and date of birth) as well as other information including itinerary, routing and ticket payment method. Prior to the implementation of this program, access to such data was possible only after the plane carrying the travellers arrived in the country. Advance access permits officers to analyze the information, with the assistance of the PAXIS automated system, so that high risk travellers requiring further scrutiny can be identified on arrival.

Data from the initial period of API/PNR's existence suggest that this program has been improving targeting effectiveness. Between October 2002 and December 2003, the proportion of referrals resulting from data collected and analyzed through API/PNR that led to enforcement actions was 24%, compared to the overall travellers' hit rate of less than 2%. No similar data is currently available for the period beginning January 2004.

There are some significant challenges to full API/PNR effectiveness. The Agency currently receives inconsistent and poor quality API/PNR data from airlines due to lack of data input standards and keying errors by airline staff. In addition, airlines frequently submit PNR data containing many empty data fields. In addition, though the Agency was able to justify access to 38 PNR data elements collected by airlines, in response to objections raised by the Office of the Privacy Commissioner it decided not to obtain access to certain other ones. Many of these data elements would have been useful for targeting. Finally, current legal obstacles bar airlines from European Union countries from providing PNR data. It should be pointed out, however, that Canada is currently negotiating a treaty with the EU to provide a legal basis for European airlines to submit PNR data to the Agency.

Advance Commercial Information (ACI) is intended to provide Customs with more effective risk management processes and tools to manage commercial clients and their commercial shipments more effectively (particularly those posing a high and unknown risk). This initiative provides Agency officers with pre-arrival data regarding commercial shipments entering Canada. Advance access permits officers to analyze the information, with the assistance of the Titan automated system, to identify high-risk commercial traffic requiring further scrutiny on arrival.

ACI is being implemented in phases. It has been operational in marine mode in April 2004, and will be implemented in the air mode in December 2005. Final implementation in the remaining modes will occur in 2006. Since implementation is still on going, it is premature to analyze the initiative's effectiveness.

4.4 Initiatives designed to improve intelligence, analysis and targeting capabilities have yet to realize full benefits

The National Risk Assessment Centre (NRAC) was established at Agency headquarters in January 2004. Officers at the Centre conduct analysis of risk-scored API/PNR and API data for "Tier 1" targets (terrorism and security threats), and conduct strategic trend analysis. They also exchange lookouts and risk scored information with the US Bureau of Customs and Border Protection (CBP) concerning travellers arriving by air from both countries. Finally, NRAC serves as a 7/24 enforcement point of contact for CBSA staff, domestic and international agencies, and an information receipt line for enforcement and security information from the public and other stakeholders.

The NRAC has the potential to improve the Agency's intelligence, analysis and targeting effectiveness. It has already assisted officers in carrying out some enforcement actions and in dealing with a few national security threats. It should be positioned to achieve its objectives over time, as it refines its methodologies and processes, its officers become more experienced, and the remaining risk assessment and enforcement initiatives are implemented.

The Integrated Border Enforcement Teams (IBETS) involve law enforcement organizations (including CBSA) from Canada and the United States. The RCMP is the lead organization for this initiative. The Agency has a Regional Intelligence Officer attached to each of the 14 IBETS across Canada, as well as a liaison officer assigned to the National IBET Co-ordination team. The purpose of the IBETS is to coordinate law enforcement activity in the area between the ports at the land border. The first priority of the units is to deal with national security threats, followed by organized crime and other criminality. They are intended to be intelligence-led, identifying threats and coordinating cross-border enforcement activities that deal with these threats.

The IBETS have produced few enforcement results to date. Until 2003, there was little effective management direction provided to the individual IBETS, inadequate application of the "intelligence-led" concept, and insufficient focus on key priorities of national security and organized crime. Lack of shared accommodations and limited US participation has reduced interoperability of the units. All these problems have been acknowledged, and steps taken to rectify them. The measures taken show promise for improvement in IBET performance over time.

The concept for the Integrated National Enforcement Teams (INSETS) is similar to the IBETS. With the RCMP as the lead agency, four INSETS have been established across Canada. Each comprises representatives from selected Canadian federal, provincial and municipal law enforcement and security agencies, including CBSA. The purpose of these teams is to detect threats to national security and to facilitate efforts to prevent terrorist attacks and activities inimical to national security from occurring.

Preliminary observations indicate that the individual INSETS have had success in disrupting terrorist activities, but the evaluation could not confirm the Teams' effectiveness independently. Agency and INSET management are satisfied with Customs' participation in the INSETS during periods of imminent security threats. There is a misunderstanding, however, concerning Customs' role in on-going investigations. INSET management want Agency officers to be active team members on investigations, while the Agency prefers that assigned officers function largely as liaison officers to the INSETS.

The In-Transit Container Targeting initiative is a joint Canada-US initiative intended to improve effectiveness in identifying high-risk containers at their first point of arrival in North America. CBSA has stationed targeters at the marine ports of Seattle-Tacoma and Newark, while US targeters are deployed in Vancouver, Montreal and Halifax. These units have produced interoperability benefits, by facilitating the exchange of information between Canadian and US Customs officials and access to each others' automated systems. Few contraband seizures can be attributed to their activities, but their priority is interdiction of major security threats, including weapons of mass destruction. These units' effectiveness should improve following the implementation of Titan in December 2004.

Partners-in-Protection (PIP) is a program designed to enlist the cooperation of the trading community in enhancing border security, combating organized crime and terrorism, increasing awareness of Customs compliance issues, and helping to detect and prevent contraband smuggling. Participating companies sign a Memorandum of Understanding with CBSA, which focuses on the security of the trading company's cross-border operations (particularly the security of its trade chain), information exchanges, and joint training activities. Any company may become a PIP participant, but importer and carrier applicants for the Canadian FAST program are obliged to take part. Over 1,400 companies are now participating in PIP. The US has established a similar program, Customs-Trade Partnership Against Terrorism (C-TPAT).

To date PIP has had some success in assisting participating companies in improving their trade chain security. A few enforcement actions have also resulted from participants' tips. The Agency has conducted little training of company participants on security or contraband matters to date, however, and limited efforts have been made to develop effective contacts amongst participants to assist in detection of suspicious border-related criminality. Recognizing that there might be other obstacles to the realization of the objectives of the program, the lack of sufficient Regional Liaison Officer (RLO) was identified as a barrier.

In November 2001, significant amendments to Section 107 of the Customs Act came into force. Amongst the changes was a re-written Section 107, replacing sections 107 and 108 of the old Act. The changes, as well as legal interpretations of the revised section, have placed restrictions on officers' ability to provide Customs information to individuals and to other organizations. Many Agency officers believe that these restrictions have made it very difficult for the Agency to work cooperatively with other law enforcement and security organizations, including on integrated teams like the IBETS and INSETS.

4.5 Several factors have impaired the effectiveness of utilization of the new detection equipment

PSAT and SBD funding allowed the Agency to purchase a number of pieces of new detection equipment. One of the key pieces of new equipment is the Vehicle and Cargo Inspection System (VACIS). The VACIS is a gamma ray scanning system that captures an image, similar to an x-ray, of the contents of a container or vehicle to enable officers to detect anomalies that might suggest the presence of inadmissible goods or people. Between January and August 2003, Customs deployed 11 mobile VACIS machines across Canada at the ports and areas with the highest volumes. Another key piece of new detection purchased was X-Ray Scan Trailers, for use at major airports.

To date, few seizures (and no significant seizures) have been made with the assistance of the new detection equipment. The evaluation identified several factors limiting the effective use of the new equipment.

Firstly, many officers lack sufficient proficiency with the VACIS machines, particularly in interpreting x-ray images. At many ports, officers are not assigned to VACIS operation for extended periods, preventing them from gaining necessary practice and experience. VACIS training has also been insufficient, though Customs has been developing its own training program.

In addition, limited targeting effectiveness also impedes officers' ability to identify shipments that should be examined with detection equipment. Targeting effectiveness improvements resulting from CAP, PSAT and SBD initiatives should contribute to more effective detection equipment use.

There have been challenges to effective VACIS use at highway ports. One machine is shared amongst all ports in districts with the highest traffic volumes. VACIS machines are also readily visible, so their presence can be reported to individuals approaching the border. Transport between ports appears to increase the incidence of mechanical problems. Configuration of highway border installations makes it impossible to conduct scans in series; instead vehicles have to be referred one by one.

There have been difficulties in deploying X-Ray scan trailers at several airports, due to their size and difficulties in manoeuvring them. The Agency is developing alternate equipment that would function better in the airport environment.

F. Approval of the Study

The three key branches implicated in the evaluation prepared a "Management Response" to the evaluation study (see Annex A). These branches include the Innovation Science and Technology Branch (ISTB), the Enforcement Branch, and the Strategy and Coordination Branch. This document includes the actions that these branches will take (or are already undertaking) in response to the recommendations.

The Audit and Evaluation Committee of CBSA approved the evaluation, including its key findings and recommendations, at its meeting on March 17, 2005.

ANNEX A

CUSTOMS ACTION PLAN (CAP) INTERIM EVALUATION STUDY MANAGEMENT RESPONSE

The Customs Action Plan, launched in 2000, represented a major, ambitious reengineering of the Customs program. It consisted of a wide range of trader, traveller, joint initiatives, and infrastructure initiatives. The scope of the work was larger than expected, and was behind schedule from the beginning.

After September 2001, the CAP was overtaken by the response to the terrorist attacks. In response to increased North American security concerns, the Government of Canada passed the Public Security and Anti-terrorism (PSAT) legislation and signed the Shared Border Declaration (SBD) in late 2001. For the CCRA (and now for CBSA), this resulted in expanded and expedited CAP initiative development, and several new initiatives designed to bolster enforcement and security capacity. Joint program development with the US on many key initiatives added another dimension of complexity to the development process. In essence, what was already an ambitious plan to reengineer the Customs program became yet more ambitious.

Another challenge to initiative development has been funding. The Customs program pursued a funding strategy enabling the development and implementation of the high priority initiatives. This required reducing the scope of some initiatives, and not implementing others.

Despite the challenging environment in which CAP, PSAT and SBD initiative development has taken place, the high priority initiatives are being implemented, and significant benefits are being realized, both in terms of more effective security at the border and greater facilitation of trade.

ISSUE 1: CAP DESIGN MANAGEMENT RESPONSE

A. KEY RECOMMENDATIONS

1. Consider development of a second CSA/FAST option, which provides participants with border streaming privileges, but has a less onerous accounting component

Management Response: The CSA/FAST team is currently conducting consultations internally and externally to assess the need for, and interest in, modifications to the existing program, including: the identification of potential concerns among, and impacts on, existing and applicant CSA importers; determining the feasibility of modifying the program relative to the emerging commercial program; and raising awareness within the trade community of the benefits of the existing program. A detailed consultation report will be submitted to CBSA senior management in May 2005 for decision. Key to this decision will be whether a funding source has been identified, the level of participation in the program at the time and the ability to respond to existing CSA/FAST clients' concerns.

2. Conduct a full evaluation of the pre-approval programs with a focus on program rationale, contributions to program model effectiveness, and cost-effectiveness.

Management Response: Conducting a full evaluation would be premature at this point. Instead, evaluation should follow full implementation of the Global enrolment component (FAST/CDRP is to be integrated into Global Enrolment in Spring 2006) and the remaining CANPASS programs (Corporate Air, Private Air, and Boats will be implemented in Spring 2007). The Agency is currently developing its long-term evaluation plan and the appropriate timing for the full evaluation of these programs will be considered in that context.

B. OTHER RECOMMENDATION

3. Expand communication efforts in relation to ACI, to supplement current on-going activities, in order to facilitate smooth initiative implementation in the highway mode in 2005. As part of this communication effort, the Agency should consider an implementation transition period, to allow traders to adjust to the new requirements, similar to the approach taken for AMPS implementation in 2001-2002.

Management Response: The implementation in the highway mode will now take place in 2006, and not in 2005. The Highway community will be consulted in a similar fashion as in the other modes (1 year before implementation). During future consultations, the impact of non-compliance with ACI requirements (hold, penalty), and the implementation of a transition period will be taken into consideration.

ISSUE 2: IMPLEMENTATION PROGRESS MANAGEMENT RESPONSE

A. KEY RECOMMENDATIONS

1. Develop a long-term funding strategy to enable completion and effective management of the initiatives.

Management Response: A long-term funding strategy is being prepared to seek some maintenance funding for CAP initiatives such as AMPS, ICS Infrastructure and NEXUS Highway. In order to obtain the full maintenance funding for CAP initiatives, the CBSA is undertaking a capacity review and funding strategy for the maintenance of all CBSA applications. The target to complete this review is Fall 2005.

2. During future project development, implement the key project management lessons learned during CAP development.

Management Response: Lessons learned from every project are discussed and incorporated into the planning/management processes for future development. This post-implementation review is an integral part of the project management process at CBSA.

B. OTHER RECOMMENDATIONS

3. Consider development of a long-term staffing and training strategy to provide an on-going adequate supply of qualified staff for future project development

Management Response: The Innovation Science and Technology Branch is currently developing a Human Resources plan and a long-term staffing / succession plan will be prepared in 2005/2006.

4. Conduct a "Systems Under Development Audit" concerning CAP, PSAT and SBD, with a particular focus on project management and ICS development

Management Response: Internal Audit will consider conducting a "Systems Under Development Audit" of future major projects in the ongoing annual audit planning process.

ISSUE 3: ICS EFFECTIVENESS MANAGEMENT RESPONSE

A. KEY RECOMMENDATIONS

There are no key recommendations for this issue.

B. OTHER RECOMMENDATIONS

1. Give priority to on-going efforts to find a solution to the data-matching problems faced by IPIL, Paxis and Titan in order to optimize benefits from these automated systems and improve officers' ability to detect high-risk movements.

Management Response: The systems themselves do not have data matching problems; rather it is the search tool they utilize (SSA Name3) that is creating the data matching issues. The Research, Engineering and Development Supports (REDS) section within the IST Branch has formed an SSAName3 subject matter expert group to interoperate with development teams through the Lead Designers Forum, where SSAName 3 (data-matching) issues will be tabled and resolved as required.

ISSUE 4: ENFORCEMENT/SECURITY MANAGEMENT RESPONSE

A. KEY RECOMMENDATIONS

1. Develop a strategy to improve effectiveness of VACIS machine use, including establishment of dedicated VACIS teams for all deployed machines.

Management Response: A review of the VACIS program was undertaken when the program was still in its early stages. Since then, implementation has advanced significantly and most of the recommendations have been actioned. Resources have been obtained and distributed (in some cases staff are dedicated to VACIS; it is recommended that other areas adopt the dedicated staffing model). Overall, regions are becoming increasingly confident in using the equipment.

2. Conduct an evaluation of the PIP program to identify key problems and to develop recommendations to improve achievement of program objectives

Management Response: This recommendation is strongly supported. Scheduling of this evaluation will be determined once the CBSA Evaluation Plan is approved later in 2005-06.

B. OTHER RECOMMENDATIONS

3. Conduct a full evaluation of current risk assessment and targeting approaches and methodologies to identify ways to improve overall targeting effectiveness.

Management Response: Scheduling of this evaluation will be determined once the CBSA Evaluation Plan is approved later in 2005-06.

4. Reach agreement with INSET management to establish a shared and accepted understanding of the Agency's role and level of participation in the INSETS

Management Response: Funding has not yet been provided for CBSA's participation in the INSET program, but is under consideration. In the interim, there has been agreement with the Royal Canadian Mounted Police (RCMP) on part-time/interim participation in the INSETs. Once a determination on funding has been made, a more formal agreement with INSET management will be developed on the role and participation of the CBSA in this activity.