CIP Project Audit Report
Internal Audit Report Review of the Port Capital Improvement Program Current Practices Issue Date: April 05, 2011 Report No. 2011-06 Internal Audit Capital Improvement Program (CIP) Current Practices Table of Contents Transmittal Letter .................................................................................................................................... 3 Executive Summary ................................................................................................................................ 4 Audit Report ............................................................................................................................................. 5 Background .............................................................................................................................................. 5 Department Highlights and Accomplishments .................................................................................... 7 Audit Scope and Methodology ............................................................................................................... 8 Conclusion ............................................................................................................................................. 11 Table of Figures Figure 1 Project Costs ................................................................................................................................ 6 Figure 2 Central Procurement Office Training Classes ............................................................................. 7 Figure 3 Professional/Personal Services Agreements by Category .......................................................... 8 Figure 4 - Total Number of Small Works Contracts ...................................................................................... 8 2 Internal Audit Capital Improvement Program (CIP) Current Practices Transmittal Letter We have completed an audit of the Capital Development Division (CDD). The Division oversees and manages Port-wide procurement and the Capital Improvement Program (CIP). Our audit objective was to review and assess the effectiveness of management controls over the Capital Improvement Program. Our audit focused on the current CIP practices. Most of the policies and procedures governing the current CIP practices were established and implemented in 2009 or the early part of 2010. We conducted the audit using due professional care. The audit was planned and performed to obtain reasonable assurance that the controls implemented by the CDD are operating as intended to achieve the Division objective of effectively delivering Port-wide centralized procurement and CIP. Based on our review, the CDD has established adequate and effective controls over the CIP, and the Port has adequately addressed the 2007 State Auditor's Office (SAO) audit recommendations. We noted some weaknesses, not significant enough to include in the audit report, which we communicated to management in a separate letter. We extend our appreciation to CDD management and staff for their assistance and cooperation during the audit. Joyce Kirangi, CPA Internal Audit, Director 3 Internal Audit Capital Improvement Program (CIP) Current Practices Executive Summary Audit Scope and Objective The Port Chief Executive Officer (CEO) and the Managing Director, Capital Development Division (CDD), requested this audit. Our audit objective was to review and assess the effectiveness of management controls over the Capital Improvement Program. At the Port, the CDD oversees and manages Port-wide procurement and the Capital Improvement Program (CIP). We reviewed and assessed risks to the overall strategies, processes, policies, and other procedures that the CDD has established in order to effectively manage the Port-wide CIP. The established processes and strategies cover all phases of the CIP from planning, design, procurement and construction, and close-out of the capital projects. As part of the audit, we visited many business units across the Port and evaluated whether the established controls were carried out as intended. The CDD, with its 220 FTEs, manages the Port's CIP, which has annual project costs of approximately $300 million, as well as procurement of non-capital goods/services. Our audit focused on the current CIP practices. Most of the policies and procedures governing the current CIP practices were established and implemented in 2009 or the early part of 2010. Background The Port's CDD was created largely in response to the 2007 State Auditor's Office performance audit. As a result of the audit, the CEO implemented many policy changes including department re-alignment and re-organization. The CDD is comprised of the following groups that help in delivering Port-wide centralized procurement and CIP services: Aviation Project Management Seaport Project Management Engineering Services Port Construction Services (PCS) Central Procurement Office (CPO) Audit Result Summary The CDD has adequate and effective controls over the CIP. The controls provide reasonable assurance of management effectiveness and compliance. Additionally, the Port has adequately addressed the 2007 SAO recommendations related to the reviewed areas. We, however, noted some weaknesses, not significant enough to include in the audit report, which we have communicated to management in a separate letter. 4 Internal Audit Capital Improvement Program (CIP) Current Practices Audit Report Background The Port Chief Executive Officer (CEO) and the Managing Director, Capital Development Division (CDD), requested this audit. Our audit objective was to review and assess the effectiveness of management controls over the Capital Improvement Program (CIP). At the Port, the CDD oversees and manages Port-wide centralized procurement and delivery of CIP services. We reviewed and assessed risk to the overall strategies, processes, policies, and other procedures that the CDD has established in order to effectively manage the Port-wide CIP. The established processes and strategies cover all phases of the CIP from planning, design, procurement and construction, and close-out of the capital projects. As part of the audit, we visited many business units across the Port and evaluated whether the established controls were carried out as intended. The Port CDD was created largely in response to the 2007 State Auditor's Office performance audit. As a result of the audit, the CEO implemented many policy changes including department re-alignment and re-organization. The CDD is now one of five divisions of the Port and has been organized to deliver centralized procurement, projects, construction, and contract administration support. The following five major organizational units comprise the CDD and support its mission: Aviation Project Management The group provides planning, design and project management of projects at the Airport Division Seaport Project Management This group provides planning, design and project management of projects at Seaport and Real Estate divisions Engineering Services This group provides engineering and related design, contracting, administration, surveying, mapping, quality management, construction safety and construction management services Port Construction Services This group provides construction, building, renovation, remodeling, alteration, repair or improvement of real property for the Port that is estimated to be $200,000 or less (300K as of July 26, 2009) Central Procurement Office (CPO) This group provides construction contract services, goods & services (purchasing), and service agreements. Because the group is independent from the operating departments, its reviews of contract services and payment requests, among other things, is more objective and reliable for audit purposes. 5 Internal Audit Capital Improvement Program (CIP) Current Practices Procurement group oversees Purchase Requisitions and Purchase Orders, among other things. Construction Contract Services administers small works and major construction contract procurements and payments. Service Agreementprocures and/or provides technical support service agreements The following table summarizes project costs by division for the past three years. (in millions) 2008 2009 2010 Airport 229 66% 217 63% 205 85% Seaport 88 25% 46 13% 24 10% Corporate 8 2% 8 2% 6 3% Real Estate 21 6% 74 21% 5 2% Total 347 100% 345 100% 240 100% Figure 1 Project Costs Source: PeopleSoft Audit Objective Our audit objective was to review and assess the effectiveness of management controls over the Capital Improvement Program. Based on the results of our CIP risk assessment, our audit objective was further defined to focus on the following areas and to determine whether: 1. The CDD has established effective controls over its CIP. Additional audit evaluation and tests of compliance was focused in the following specific areas that we deemed high risk or significant to the CIP program: a. Procurement of Professional and Personal Service Agreements for the Port's defined Categories I & II, including compliance with the CPO requirements. b. Procurement of Small Works Contracts. c. Change Orders estimates evidence of due diligence over cost estimates, negotiation, monitoring, and approval. d. Contract Payments and effective cost management for: Personal & Professional Service Agreements--adequate support and review of contract payments. Small Works--adequate support and review of contract payments. Major Construction adequate support and review of contract payments. e. Performance measuresassess established performance measures for reasonableness, practicality, and effectiveness in measuring intended outcomes 2. Management has addressed the 2007 SAO recommendations related to the specific areas noted in objective number one above. 6 Internal Audit Capital Improvement Program (CIP) Current Practices Our audit focused on the current Capital Improvement Program practices. Most of the policies and procedures governing the current CIP practices mostly were established and implemented in 2009 or the early part of 2010. The Port expends millions annually in the CIP. The CDD employs approximately 220 FTEs (as of March 2011) to manage such project costs, as well as procurement of non-capital goods/services. Department Highlights and Accomplishments During the review, we observed considerable management efforts and significant improvements in the following areas: New Central Procurement Office (CPO) Policies and Procedures Staff Training on New CPO Policies and Procedures Reduction in Risk Exposure Related to Category 1 Professional/Personal Agreements Increasing Competition in Small Works Implementation and Continuous Refinement of Performance Measures The CDD has established and implemented a number of new policies and procedures. Beginning with CPO-1 on personal and professional services, the CDD has implemented procedures on every aspect of procurement from goods/services to competition waivers. To effectively manage newly implemented policies/procedures, the CDD has been training and assisting Port staff to promote the correct application of the new policies/procedures. The Port's Learning Management System indicates that approximately 1,000 Port staff has been attending CPO classes on a variety of subject areas. The CDD continues to offer classes to educate new staff and to maintain awareness/competency with existing staff. attendance Some CPO Course Description 2009 2010 Total Central Procurement Office Policy CPO-1 -- What is it and how do I use it? 381 90 471 Service Agreement Contract Administration and Review of EX-2, EX-2a 131 131 Evaluative Criteria for Contracts and Documenting Selection Decisions 82 82 Drafting Evaluative Criteria for Contracts and Documenting Selection Decisions 80 80 Insurance Requirements for Service Agreements 71 71 Standardization Of Project Manuals 55 55 Procurement Basics 18 18 Overview of Contracting Policies: Small Works 10 10 Figure 2 Central Procurement Office Training Classes Source: HR Learning Management System Prior to the formation of the CDD, a significant number of Category 1 Professional Services Agreements (<$50,000) were used to procure personal/professional services. Category 1 7 Internal Audit Capital Improvement Program (CIP) Current Practices PSAs were relatively free of compliance requirements as the dollar threshold was low. Due to the easy of compliance, there was a tendency to misuse Category 1 to avoid higher scrutiny of Category 2 and 3 PSA. Indeed the SAO report pointed a number of instances where Category 1 classification seemed to have been misused. The CDD implemented a number of things to control potential misuse, including new policies/procedures and extensive training to Port staff. As can be seen from the graph below, CPO has made significant progress to decrease risk exposure related to Category 1 PSA. Category 1 (<$50,000) Category 2 (>$50,000 and <$200,000) Category 3 (>$200,000) 500 Number of Contract 400 300 200 100 0 2005 2006 2007 2008 2009 2010 Calendar Year Figure 3 Professional/Personal Services Agreements by Category Source: PeopleSoft The SAO report indicated certain weaknesses in Small Works contracts specifically as it relates to fair competition and invoice splitting. As figure 4 below shows, competition has been promoted steadily, and statistics show that a fewer number of contractors are now getting multiple Small Works contracts. This is a clear indication that there is more competition with Small Works contracts. 2005 2006 2007 2008 2009 2010 Total Number of Small Works Contracts 91 97 99 60 52 25 Contractors with more than one contract 21 21 16 10 8 2 contractors with more than one contract as 23% 22% 16% 17% 15% 8% a percentage of the total Figure 4 - Total Number of Small Works Contracts Source: PeopleSoft The CDD is among the leading groups at the Port to proactively establish meaningful performance measures, and measure expected outcome. While the process as a whole is still maturing, the CDD has already implemented a number of measures to include, but is not limited to: 1) soft costs, 2) construction timeline, and 3) change orders. The measures are reported quarterly and are made available on the Division's intranet. Audit Scope and Methodology We conducted the audit to determine whether management controls in the Capital Improvement Program (CIP) were adequate to provide reasonable assurance of effective operations and compliance with Port policy/procedure. Our audit examined current practices but focused primarily on operations in 2009 for test of details purposes. Our work was conducted at various locations throughout the Port and involved CIP projects of all Port divisions. 8 Internal Audit Capital Improvement Program (CIP) Current Practices Our approach to the audit was risk-based from planning to test sampling. We reviewed and assessed risk of the strategies, processes, policies, and other procedures that the Capital Development Division has established in order to effectively manage the Port-wide CIP. The established processes and strategies cover all phases of the CIP from planning, design, procurement and construction, and close-out of the capital projects. As part of the audit, we visited many business units across the Port and evaluated whether the established controls were carried out as intended. We applied additional detailed audit procedures to areas with the highest likelihood of significant negative impact. We considered the nature of the activity and evaluated it within the context of our audit objectives. Our consideration included control (both manual and system driven) assessment and control testing, as necessary. Our additional detailed audit procedures can be grouped and summarized into procurement, disbursement, and performance measures. We approached each audit area with the following methodology: 1. Procurement For professional and personal service agreements, we evaluated whether management controls were adequate to ensure proper and compliant procurement practices for PSAs. We reviewed Category 1 (<$50,000) and 2 (>$50,000 and <$200,000) Professional Services Agreements (PSA) for risk of splitting. Splitting of PSA is a concern as it can be used to circumvent management's intentof competitive procurement. There were 101 consultants with Category 1 agreements in 2009. Of these, there were 19 consultants with multiple PSAs. We focused on the consultants who may have gotten multiple consulting agreements through splitting. We further eliminated consultants with multiple agreements whose contracts, as a whole, during 2009 did not exceed $50,000. Consulting agreements less than $50,000, as a whole, would not be at risk of contract splitting. We utilized the same approach for Category 2 PSAs with the dollar threshold of agreements between $50,000 and $200,000. There were 36 agreements under this category and included five consultants with multiple agreements. Our procedures resulted in four Category 1 and two Category 2 agreements for testing. We reviewed procurement documents for the evidence of splitting of contracts, significant negotiation, and compliance with current Port policies and procedures. For the Small Works, there were 53 contracts executed in 2009 with 41 contractors. To determine the adequacy of Small Works procurement practices, we selected contractors with two or more contracts equal to $200,000, and any contract over $200,000. We additionally excluded contracts with less than $50,000. Our sampling resulted in a group of 17 (or 32%) contracts. We reviewed the online Small Works Roster system and associated documentation to determine proper invitation to bid and procurement practices. For change orders, we reviewed to determine compliance with Port policies and procedures. There were 34 Major Construction contracts with a total of 554 change orders in 2009. We excluded change orders less than $10,000 as Port policy requires no cost estimates for such change orders. We focused on change orders with round figures (in the thousands) which could be indicative of an arbitrary estimate without an adequate engineering or scope based rational 9 Internal Audit Capital Improvement Program (CIP) Current Practices for the change order. Our sampling strategy resulted in 28 (or 5%) change orders from eight contracts (or 23%). We reviewed an assortment of documentation, including but not limited to the public bid materials, evidence of proper and adequate cost estimate analysis, negotiation, scope of work changes, and approval. 2. Payments For professional service agreements, there were 311 disbursements for a total amount of $4,886,821 under 47 agreements to 33 consultants in 2009. We considered disbursements to consultants with multiple agreements or a single agreement with disbursements to multiple projects in 2009. These transactions are more likely to have errors due to complexities involved with having to bill for similar service to a number of projects and under multiple agreements. We also considered an average payment amount as well as the timing of the payment. As a result, 19 disbursements from nine or 19% of 2009 agreements were selected for examination. The sample included 4 consultants or 12% of the total population of 33 consultants in 2009. We reviewed invoices and accompanying documentation for adequate support, proper disbursement, and approval. To determine whether Small Works disbursements were for actual hours worked and services rendered, we evaluated 80 Small Works contracts with disbursements in 2009 totaling $2.7 million dollars. We considered disbursements with six or more months apart among them. Small Works contracts are generally for short-term projects, and thus disbursements are expected every month or two. A resulting population was further excluded, based on an average number of projects assigned to the construction manager. The total number of contracts in 2009 was divided by the total number of construction managers to calculate an average number per construction manager. We then selected disbursements from projects assigned to Construction managers who had a higher number of projects than average. Consequently the likelihood of error and improper charges for labor and/or materials would be higher in these projects. Our sampling resulted in nine disbursements for a total of $447,983 (or 17%) from six Small Works contracts (or 7.5%). We reviewed contractor submitted time sheets and invoices for evidence of proper support for disbursements. For Major Construction contracts, we reviewed 2009 progress payments to determine whether they were properly based on verifiable progress. There were 45 Major Construction contracts with disbursements totaling $162 million dollars in 2009. We included contracts whose 2009 disbursements were in excess of 50% of 2008 or 2010 disbursements. The 50% threshold is considered a significant increase and thus could be an indicator that progress was overstated in order to receive payment for work not yet performed. We further considered contracts with multiple monthly vouchers. Progress payments are typically invoiced monthly, and as such multiple invoices in a month could be another indicator of improper progress payments. Our sampling resulted in seven (or 16%) contracts with $67,279,625 (or 41%) of the 2009 disbursements. We reviewed contractor submitted invoices and accompanying documentation for evidence of adequate support, and approval of progress payment requests. 10 Internal Audit Capital Improvement Program (CIP) Current Practices 3. Performance Measures We reviewed implemented performance measures for completeness with regards to types of measures, reasonableness, practicality, and effectiveness in measuring the intended result. We conducted the review from a risk-based perspective to ensure that the implemented measures are related to the Division's significant risks. Specifically, we reviewed the following measures for evidence of complete rational and accurate compilation: Construction Soft Costs measures quarterly the percentage of indirect construction costs. Cost Growth during Construction measures quarterly for the change in the original contract price made through change orders. Project Status quarterly reports the status of projects in terms of budget and schedule. Project Schedule Growth measures from the initial Commission authorization to start of design work to Planned/Actual Substantial Completion. Procurement Timeliness of Service Agreements measures milestone of Category 3 Services Agreements. Conclusion The Capital Development Division (CDD) has adequate and effective controls over the Capital Improvement Programs (CIP) to provide reasonable assurance of management effectiveness and compliance. Additionally, the Port has adequately addressed the 2007 SAO recommendations related to the reviewed areas. We, however, noted some weaknesses, not significant enough to include in the audit report, which we have communicated to management in a separate letter. 11
Limitations of Translatable Documents
PDF files are created with text and images are placed at an exact position on a page of a fixed size.
Web pages are fluid in nature, and the exact positioning of PDF text creates presentation problems.
PDFs that are full page graphics, or scanned pages are generally unable to be made accessible, In these cases, viewing whatever plain text could be extracted is the only alternative.