Attachment 2021_04_08_SCM_Audit_Noise_Monitor_Data_Accuracy_Audit_Report for Audit Meeting on Apr 08, 2021 2:30pm at Remote

2021_04_08_SCM_Audit_Noise_Monitor_Data_Accuracy_Audit_Report

INTERNAL AUDIT REPORT
OPERATIONAL AUDIT
NOISE MONITOR DATA ACCURACY

JANUARY 2017 – FEBRUARY 2021

ISSUE DATE: March 25, 2021
REPORT NO. 2021-05

INTERNAL AUDIT

Noise Monitor Data Accuracy

TABLE OF CONTENTS

Executive Summary .......................................................................................................................................... 3
Background ....................................................................................................................................................... 4
Audit Scope and Methodology .......................................................................................................................... 6
Schedule of Findings and Recommendations .................................................................................................. 7
Appendix A: Risk Ratings ................................................................................................................................ 10
Appendix B: Comprehensive Management Response .................................................................................... 11

Noise Monitor Data Accuracy
Executive Summary
Internal Audit (IA) completed an audit of Noise Monitor Data Accuracy, for the period January 2017
through February 2021. The audit was performed to assess compliance with policies, procedures, and
contractual obligations, and to evaluate internal controls governing noise program operations
specifically with regards to accuracy of noise data.
The Port owns 26 data monitoring terminals that capture airplane noise. The Port provides the data
from these terminals to a third party, L3Harris, who among other functions, correlates the information
to SEA flight data. Once the data is correlated, it is provided to the Port’s Business Intelligence team,
who using Tableau, generates reports for public viewing.
In general, Port Management complied with policies and procedures governing the Noise Program
Operations. However, our audit identified an opportunity where internal controls needed to be
enhanced.
This issue is listed below and discussed in more detail beginning on page seven of this report.
Medium – Our review of the noise monitoring program identified an opportunity to enhance the
monitoring process so that the information provided to the public can be relied on. We identified
instances where data was missing or appeared to be inconsistent, which could indicate that the
monitoring terminals were not working as intended or that internal processes didn’t process the data
completely or accurately. This information is provided as a public service and no regulatory
requirement exists for inaccurate or incomplete data. However, as a public agency, information that
can be relied on should be available regardless of how it is used.
We extend our appreciation to Port management and staff for their assistance and cooperation during
this audit.

Glenn Fernandes, CPA
Director, Internal Audit

Responsible Management Team
Elizabeth Leavitt, Senior Director Environment, Sustainability, and Engineering
Arlyn Purcell, Director, Aviation Environmental Services
Krista Sadler, Director, ICT Technology Delivery
Stan Shepherd, Senior Manager, Aviation Noise Programs

Noise Monitor Data Accuracy
Background
The Port of Seattle has established a Noise Monitoring Program. The airport’s flight tracking system,
by use of 24 permanent noise monitoring sites, helps the Port monitor aircraft operations and flight
paths, aircraft noise levels, and enables it to investigate citizen inquiries and identify trends.
The Port of Seattle offers the PublicVue flight tracking tool, which allows citizens to view flight activity
at SEA. This tool provides additional information about flights, including aircraft type, airline, and
altitudes. PublicVue can be accessed through the Port’s Noise Programs website.
The following data sets are available to the public:
• Equivalent Continuous Sound Level (Aircraft Noise) – a cumulative metric that averages aircraft
noise levels over a 24-hour period.
• Equivalent Continuous Sound Level (Community Noise) - a cumulative metric that averages
community noise (all recorded noise not correlated with an aircraft overflight) over a 24-hour
period.
• Equivalent Continuous Sound Level (Total Noise) - a cumulative metric that averages aircraft and
community noise over a 24-hour period.
• Sound Exposure Level – a single event measurement of the total sound energy for one aircraft
overflight.
The Port has contracted L3Harris, who provides the following services:
• Software – Provides the Port with user licenses for hardware and software use.
• Hosting Services – Application access and data storage of up to 10 years of data i.e. flight, noise,
complaint, and weather data.
• Support and Maintenance – Includes annual calibration of noise monitors and maintenance
response to meet service goals within contract.
• Complaint Management – Provides a web based application to capture public complaints.
• Public Portal – Provides a public to view noise data reports.

Noise Monitor Data Accuracy
Figure 1

Noise Monitor Data Accuracy
Audit Scope and Methodology
We conducted this engagement in accordance with Generally Accepted Government Auditing
Standards and the International Standards for the Professional Practice of Internal Auditing. Those
standards require that we plan and conduct an engagement to obtain sufficient, appropriate evidence
to provide a reasonable basis for our findings and conclusions based on our engagement objectives.
We believe that the evidence obtained provides a reasonable basis for our findings and conclusions
based on our engagement objectives.
The period audited was January 2017 through to February 2021 and included the following
procedures:
Process and Procedure Review
• Reviewed documentation to determine whether L3Harris complied with key provisions
stipulated in their contracts.
• Reviewed the noise data capturing and reporting process to determine if controls exist to
facilitate complete and accurate data.
• Reviewed the complaint management process to evaluate how complaints are managed.

Data Analysis
• Obtained noise data for a two-year period to assess completeness and accuracy of the data.
Management Interviews
• Conducted interviews with management and staff, from various departments, to gain a better
understanding of the various processes within the Noise program.

Noise Monitor Data Accuracy
Schedule of Findings and Recommendations
1) RATING: MEDIUM
Our review of the noise monitoring program identified an opportunity to enhance the
monitoring process so that the information provided to the public can be relied on. We
identified instances where data was missing or appeared to be inconsistent, which could
indicate that the monitoring terminals were not working as intended or that internal processes
didn’t process the data completely or accurately. This information is provided as a public
service and no regulatory requirement exists for inaccurate or incomplete data. However, as a
public agency, information that can be relied on should be available regardless of how it is
used.
The Port owns 24 data monitoring terminals that capture airplane noise. The Port is also responsible
for monitoring the information that the terminals produce and notifying L3Harris of the need for repairs
or replacement. L3Harris was selected by the Port to perform a daily download of the noise data
captured from the noise monitors and correlates that information to flight tracking data. Once the data
is correlated, it is provided to the Port’s Business Intelligence team, who then uses Tableau to
generate reports for public viewing.
To assess the reasonableness of the data provided for public consumption, we obtained public
testimony and traced them to source reports. We also identified anomalies, for the 26-month period
ending February 2021, and inquired with management and L3Harris as to the cause and whether
remediation efforts were performed.
Below are three examples that illustrate inconsistencies in the data:
• Monitor 9 did not remit LEQ aircraft noise data for approximately three months, from April 30 to
August 08, 2020. On July 27, L3Harris identified the missing data and concluded the monitor was
not functioning and needed to be replaced.
• The table below identifies the terminal and days where no aircraft noise data was captured
for the two-year period ending December 2020. While explanations may exist for some,
others could represent a more pervasive issue requiring proactive steps to address.

No of Days with No Aircraft Noise Recorded
Noise Monitoring Terminal 2019 2020
SEA03 4 7
SEA04 1 3
SEA05 1 N/A
SEA06 6 21
SEA07 1 1
SEA08 93 128
SEA09 25 99
SEA15 1 6
SEA16 8 23
SEA23 3 34
SEA25 11 24

Noise Monitor Data Accuracy
• The schedule below contains various examples that compromised the integrity of the
information. The causes are varied and include a blown fuse, power outages, and
communication issues.
Year NMT # Issue Issue Date Issue Root Cause Resolution Date
Noted By Noted
2020 SEA09 L3Harris The monitor was not 07/27/2020 There was a breakage 08/08/2020
recording any data and the noise monitor
from May until early was replaced. Issue
August appears to have started
on 04/30/2020
2019 SEA09 N/A Missing data from N/A. No Not determined as the As per L3Harris, this
13th to 26th May ticket was issues were not reported was a temporary issue
raised to L3Harris that resolved itself

SEA15 N/A Missing data from N/A. No As per L3Harris, there
07th to 31st March ticket was was no issue and the
raised missing data is
available in the L3Harris
system

2018 SEA07 L3 Harris Missing data on the 4/24/2018 L3Harris performed a site NMT operational on
following dates visit to confirm power and 05/09/2018
• April 23-30 communication
• May 1-8
SEA12 Port of Missing data from 5/17/2018 Blown fuse in the monitor NMT operational on
Seattle May 17th to 31st hand-hole. Thinking it 05/31/2018
was possibly a
connection problem, we
had Seattle City Light
troubleshoot first. Seattle
City Light confirmed that
it wasn’t their issue and
found that there as a
blown fuse in the handhole.
POS Electric Shop
replaced the blown fuse
SEA19 L3 Harris Missing data on the 5/19/2018 L3Harris performed a site NMT operational
following dates; visit to reset power, re- 07/03/2018
• May 17-31 seat connector on bottom
• June 1-30 unit and finally to replace
• July 1-2 LD831 monitor after
spare was available

Recommendation:
Management should develop a process to proactively identify and resolve system outages. This
process should also include establishing criteria (i.e. frequency of breakdowns, variance analysis) to
trigger servicing and replacement decisions, in line with the service response goals outlined in the
agreement with L3Harris.

Noise Monitor Data Accuracy

Management Response/Action Plan
The Noise Programs Office has taken many steps in recent years to provide as much information as
possible to the public via our website. This includes (among other items) monthly complaint reports,
late night operations information and monthly noise monitor data. It is very important to our team to
provide the most complete noise data possible.
This audit was a great opportunity to have a team that may not be familiar with noise monitoring
systems evaluate our data and processes. This exercise, and the auditors’ recommendations, have
highlighted ways in which we can improve the reliability and completeness of our noise data. As a
result, we have put into place measures that will greatly limit the possibility of data gaps and prolonged
interruptions in the future.
Action Items
A system report email to Noise Programs staff now arrives each morning from L3Harris displaying the
status and performance of each monitor. This report is reviewed daily by Noise Program staff to more
proactively address any data interruptions. If flight track to noise correlations are missing from a
normally busy monitor or if any electrical interruptions are shown, Noise Program staff will immediately
contact L3Harris support to initiate investigation and resolution.
L3Harris has also instituted a process for daily internal checks of the monitor system and a more
proactive response. We have seen this proactive process come to fruition in recent weeks as we have
established our portable noise monitoring program.
The Noise Programs staff will work with the Port’s Business Intelligence staff to identify ways to
improve the transfer of noise data to the external web Tableau site and establish checks for data set
completeness. As part of the auditor’s recommendations, the Tableau site is now being updated to
include public messaging on data gaps and system issues.
For the comprehensive management response, refer to Appendix B.

DUE DATE: Immediately

Noise Monitor Data Accuracy
Appendix A: Risk Ratings
Findings identified during the audit are assigned a risk rating, as outlined in the table below. Only one
of the criteria needs to be met for a finding to be rated High, Medium, or Low. Findings rated Low will
be evaluated and may or may not be reflected in the final report.
Financial Internal Commission/
Rating Compliance Public
Stewardship Controls Management
High probability
Non-compliance
Missing or not for external audit Requires
with Laws, Port
High Significant followed issues and / or immediate
Policies,
negative public attention
Contracts
perception
Partial Potential for
Partial controls
compliance with external audit
Requires
Medium Moderate Laws, Port issues and / or
Not functioning attention
Policies negative public
effectively
Contracts perception
Functioning as
Low probability
intended but Mostly complies Does not
for external audit
could be with Laws, Port require
Low Minimal issues and/or
enhanced to Policies, immediate
negative public
improve Contracts attention
perception
efficiency

1 0

Noise Monitor Data Accuracy
Appendix B: Comprehensive Management Response
The Noise Programs Office welcomes the opportunity to improve the process for addressing monitor
maintenance issues, as well as the completeness of noise data reporting to the community. Based
upon the maintenance issue at one of the monitors in 2020, and early feedback from the auditing
team, our office has already taken steps to improve these processes. Those changes, as well as
responses to the specific issues referenced in the report, are discussed below.
Overall Assessment
Overall, the noise monitoring system has operated exceptionally well given the challenging
environments and weather conditions that they often operate under. Total yearly days without pairing
noise events with aircraft overflights (known as correlation, which is needed to interpret the results) is
less than 3% for all monitors in 2020.1 The yearly percent of operational time for all monitors remained
high at 98.5%. Only 3 of the 24 permanent monitors had mechanical issues that resulted in downtime
in 2020.2
Noise Monitor SEA09 outage in 2020
In late April 2020 Noise monitor SEA09 began to display a constant dB reading while appearing to be
operational on the PublicVue site. However, the monitor was not correlating noise events to aircraft
overflights and this was not noted until July when L3Harris was investigating a different issue at the
site. This monitor issue was unusual in nature and never encountered before therefore checks were
not in place to identify the issue at the time by either L3Harris or Port staff. An L3Harris technician
visited the site in early August and found that the monitor was inoperable and needed replacement.
The replacement monitor was then put into service.
Multiple days in 2019 and 2020 noted in report without noise event correlations
The 24 permanent noise monitors are sited to capture aircraft noise from various operational
conditions and flight paths. A large portion of the monitors are placed directly under established flight
paths to the north and south of the runways, while others are placed in further-out locations to capture
noise during specific operational conditions. One of the factors that influences correlations at these
monitor locations is the direction of SEA operations – south-flow or north-flow. As such, it is not
unusual for many of the monitors noted to have days with no aircraft noise events, as shown in the
report. This is typically not a mechanical or software issue but rather a lack of aircraft in the vicinity at
the time and is a condition that applies to many of monitors noted in the report. We have included
additional related information and maps at the end of this document.
Various maintenance issues noted in 2018-2020
Noise monitors are sensitive instruments that are exposed to the elements and the uncertainties of
1 The system can never operate at 100% correlation, because it includes noise monitors that are designed to capture only
specific conditions.
2 The system experienced downtime at Monitors #25 with a minor communications issue, #17 which was struck by a car,
and #9 with a complete failure of the unit.

1 1

Noise Monitor Data Accuracy
their immediate environments, all while operating 24 hours per day. As the maintenance issues that
are listed indicate, many challenges can and will arise in the operation of an extensive system. It has
been beneficial to Noise Programs staff to compile and research these maintenance items from the
past as part of this audit process.
Action Items
A system report email to Noise Programs staff now arrives each morning from L3Harris displaying the
status and performance of each monitor. This report is reviewed daily by Noise Program staff to more
proactively address any data interruptions. If flight track to noise correlations are missing from a
normally busy monitor or if any electrical interruptions are shown, Noise Program staff will immediately
contact L3Harris support to initiate investigation and resolution.
L3Harris has also instituted a process for daily internal checks of the monitor system and a more
proactive response. We have seen this proactive process come to fruition in recent weeks as we have
established our portable noise monitoring program.
The Noise Programs staff will work with the Port’s Business Intelligence staff to identify ways to
improve the transfer of noise data to the external web Tableau site and establish checks for data set
completeness. As part of the auditor’s recommendations, the Tableau site is now being updated to
include public messaging on data gaps and system issues.
Summary
The Noise Programs Office has taken many steps in recent years to provide as much information as
possible to the public via our website. This includes (among other items) monthly complaint reports,
late night operations information and monthly noise monitor data. It is very important to our team to
provide the most complete noise data possible.
This audit was a great opportunity to have a team that may not be familiar with noise monitoring
systems evaluate our data and processes. This exercise, and the auditors’ recommendations, have
highlighted ways in which we can improve the reliability and completeness of our noise data. As a
result, we have put into place measures that will greatly limit the possibility of data gaps and prolonged
interruptions in the future.
Additional Monitor Information:
Monitor SEA03 – The northernmost of the monitors located in the Maple Leaf neighborhood. This
monitor is positioned to capture south-flow arrivals. The high altitudes and varied flight paths can make
this challenging on some days. It may not capture any noise events on north-flow days.
Monitor SEA04 – Located in Magnolia. This monitor captures a portion of south-flow arrivals and the
number of overflights can vary from day to day. It may not have noise events on some north-flow days.
Monitor SEA05 – Located in Medina. This monitor is intended to capture some north-flow departures
and perhaps a few south-flow arrivals. In both instances, the aircraft are at high altitude which can
make correlations challenging.
Monitor SEA06 – Located at the north end of West Seattle. This monitor is sited to capture noise
events from south-flow arrivals and north-flow departures. However, in accordance with SEA noise

1 2

Noise Monitor Data Accuracy
abatement flight procedures, these aircraft are required to stay to the north of West Seattle over the
water. Therefore, the monitor will likely only capture a noise event if an aircraft fails to adhere to the
procedure or passes very close to shore. On many days, aircraft will remain distant from the monitor
over the water and not cause any correlations to occur. This is a preferred outcome.
Monitor SEA07 – Located in central Seattle. This monitor is sited to capture north-flow departure
aircraft as they begin to turn east. It can also monitor south-flow arrivals; however, it is sited east of the
path and may not correlate on some days.
Monitor SEA08 – Located on the northern end of Mercer Island. This monitor is intended specifically
to capture north-flow departure aircraft as they fly eastbound over Lake Washington. However, it may
not capture any aircraft events on south-flow days.
Monitor SEA09 – Located on Beacon Hill. The days of missing noise events are due to maintenance
issues previously noted.
Monitor SEA15 – Located west of the runways in Burien. This monitor has very few direct overflights
– some days none. It is intended to capture the noise from aircraft on the ground from landing, taxiing
and takeoff roll. Direct correlations can be sporadic from day to day.
Monitor SEA16 – Located east of the runways in SeaTac. This monitor has very few direct overflights
– some days none. It is intended to capture the noise from aircraft on the ground from landing, taxiing
and takeoff roll. Direct correlations can be sporadic from day to day.
Monitor SEA23 – Located in eastern Federal Way. This monitor is sited to capture south-flow
departures as they turn east. Due to high altitudes and varied flight paths, correlations can be
challenging on certain days. It may not have any correlations on north-flow days.
Monitor SEA25 – Located in western Federal Way, it is the southernmost monitor from the airport.
This monitor is sited to capture south-flow departures flying west. Due to high altitudes and varied
flight paths, correlations can be challenging on certain days. It may not have any correlations on northflow
days. A few of the days indicated may be due to maintenance issues.

1 3

Noise Monitor Data Accuracy

Example of South Flow Flight Tracks and Noise Monitor Locations

1 4

Noise Monitor Data Accuracy

Example of North Flow Flight Tracks and Noise Monitor Locations

1 5

Limitations of Translatable Documents

PDF files are created with text and images are placed at an exact position on a page of a fixed size.

Web pages are fluid in nature, and the exact positioning of PDF text creates presentation problems.

PDFs that are full page graphics, or scanned pages are generally unable to be made accessible, In these cases, viewing whatever plain text could be extracted is the only alternative.