Introduction: The Center of Excellence (CoE) requested the Colorado Broadband Office (CBO) to facilitate network providers, device manufactures and application vendors to provide first responders with technologies such as a cache of Android Public Safety Broadband provisioned devices running ATAK (Android Tactical Assault Kit) that would assist public safety personnel to use capabilities including, but not limited to, situational awareness including responder locations, chat, data transfer, whiteboarding and more.
The Issue: Country Jam takes place in Mack, Colorado and features over thirty bands on three stages over the course of four days with over 90,000 attendees. With such a large number of people using broadband devices to receive and stream information, users often encounter reduced or no service. Traditionally, public safety users would have to compete with event attendees for bandwidth. With the emergence of prioritized networks for public safety, the challenge is to ensure that public safety communications are reliable and high quality.
The Solution: The overall objective of the CBO in this endeavor was to ensure that public safety had high-bandwidth products paired with public safety grade priority access transport networks and would not experience diminished performance during high congestion events. Additionally, we hope to document, understand, and plan for the mitigation of the challenges experienced at future events.
Country Jam was a great opportunity to leverage various public safety LTE capabilities in order to develop a reusable Team Awareness Kit (TAK) server (Local and/or Cloud based) along with end-user devices running TAK in a pre-planned operational setting with priority access to public safety grade priority service. With memories of the mass shooting at the Route 91 Harvest Music Festival in 2017 fresh on the minds of the Country Jam planners, it was imperative that public safety have the capability to provide robust communications (voice and data) to make quicker decisions, decrease response times, and mitigate threats.
Lessons Learned: We learned many lessons during Country Jam that increased our understanding of communications technology usage during large events and the issues that must be addressed before and during the event, including:
- Testing All Applications on All Devices to be Used
-
We’ve all done it, assume that once the app is loaded and tested on one device that it should work the same on all devices. Unfortunately this event proved this assumption will always come back to haunt you. On the first day of operations we experienced issues with TAK devices including latency issues and the TAK application for unknown reasons kept crashing multiple times on multiple devices. Our team began questioning what could be causing these issue; Is there a problem with the application? Or with the device? Is it the operating system? Could it be the network?
We made a call to Verizon to look into what could cause these issues. Verizon refreshed the devices with Mobile Broadband Priority (MBP) and by the second day of operations, the service was dramatically better and the latency issues experienced on day one seemed to be resolved. To troubleshoot the issue of the application crashing on the devices, an application patch was created.
It was also determined that certain device models worked better with the current configuration of the TAK app. Those devices were used as first out whenever possible.
- COWS without priority updates will negatively impact performance
-
COWS without priority updates will negatively impact performance
Cells on Wheels (COWs) are often utilized to supplement the cellular capacity at large events. In the vicinity of the concert grounds in Mack, Colorado there was a fixed COW as well as a Macro Tower that can facilitate the use of public safety Mobile Broadband Priority (MBP). During the event,it was observed that as a user moved closer to the macro site, speeds and resources on the devices increased. As the same user entered the vicinity of the COW the reverse was obvious – data and services seemed to decrease. During conversations with the carrier, we determined that the COW had not been updated and would not give the user MBP priority. Since the same frequencies are used on both the macro site and the COW there is no way to give priority on a COW that has not been updated with MBP. The general principle in LTE infrastructure is that the device will gravitate to the most dominant site. As we look toward special events with the usage of a COW it will be imperative have the necessary updates made to the Cell on Wheels to avoid the same issues. - Applications allow responders to compartmentalize vital information
-
An important feature of situational awareness applications, such as TAK, is the ability to know where resources are in real time, including the exact location of personnel. This is quite frankly an important feature public safety is excited about as it embraces LTE technology on a greater scale for response. Unfortunately, as we have seen across the country, public safety personnel can become targets. Today with the usage of Land Mobile Radio, many private citizens monitor radio activity to gain key information about incidents in their communities that law enforcement personnel are mitigating and where they are at. Most citizens who monitor radio traffic are driven by curiosity and not something nefarious, however, law enforcement (and other public safety officials) must take precautions. We must consider what information we share, and how we share it, taking the necessary steps to protect sensitive information. It is imperative that we think through all the implications and protect the location of personnel as well as specific elements of operations.
During the Country Jam, Federal law enforcement personnel became concerned that a private security agency would have access to real-time officer location data from the application and moved to cease the usage of TAK during the event. We were able to quickly remedy this by creating a different user group in the TAK application and re-assigning security operations into a compartmentalized group. This limited the access of the security agency to allow them to only view the locations of other security personnel and not law enforcement personnel. This new group was created and operational in less than a minute in the TAK server and was available on devices with a simple refresh.
The ability to customize operational details and give access on a need to know basis is one of the many reasons why Long Term Evolution (LTE) situational awareness platforms are an intriguing option to get an operational snapshot in real-time.
- GIS layers make first responders more efficient
-
One of the unique abilities of using situational technologies is it allows the agency to use Geographic Information Systems (GIS) base layers and incorporate comprehensive overlays of information about the operational arena. The following screen shot shows the overlay created from pre-existing ESRI product that gives users the ability to map key information to enhance situational awareness, such as patterns of traffic, parking locations, vendor locations, VIP section, stages and campground information. For events like Country Jam, it is vital to be able to create a GIS layer with this information, since prior to the event the area was simply an open field with no street signs, landmarks or any identifiable information to know where incidents occur. The mapping applications help give context that is imperative to operational success.
The Mesa County 911 dispatchers used Win TAK which is a Windows IC/OC version of TAK to share calls for service with responding personnel and incident command (IC). Dispatch was operating out of the Grand Junction Regional Communications Center’s Mobile Communications Vehicle located (pictured below) next to the main stage area.
Left: Mobile Communications Vehicle positioned near the stage. Right: a dispatcher workstation with WinTAK. On the second night of the event, dispatch received a call of a gunshot wound (GSW) to the stomach . Dispatch was able to relay information about the general area of where the incident occurred in the campground by dropping a pin on the map using the overlay of campground information. This application provides quick access and detailed information of an incident location and gives real time data as units respond.
- Responders can receive streaming video feeds on smart devices
-
In the planning for Country Jam it was determined the event would use video surveillance. Cameras were positioned throughout the campgrounds and in the concert venue with an air asset with thermal capabilities. Cameras that were positioned around the area of operation allowed command the ability to train one of those cameras into that area where the GSW was reported and follow resources via the TAK application as personnel responded to the incident. The team responsible for TAK is known as PAR TAK/Team Connect. They provided Wowza Video Management Service (VMS) support which allowed users to view streamed video to the TAK devices from any one of the cameras including the air asset with thermal capabilities.
Left: Department of Homeland Security (DHS) helicopter streaming thermal images. Right: Image from the static MainStage1 camera. - Priority access to transport networks is vital to public safety
-
To get a level set before the event we captured a speed test before the event to then compare it to a speed test conducted during the event as the network is inundated at peak times during the concert. It is important to understand in normal day to day operations, the networks performance to give one a better understanding of how that same network performs under intense usage and whether public safety grade priority access truly makes an impact allowing public safety, the voice and data priority they need.
Speed tests conducted two days prior show fast speeds while during peak usage (without prioritization) speeds were very slow.
Left: Speed test taken two days prior to the event. Right: Speed test taken during peak network usage times during the event. We also tested whether the Legacy TAK server might work better than the Team Connect Server. Our testing showed that there was no difference in situational awareness reporting. The performance increase experience on Friday, June 15 was due to priority access to the Verizon network, this was later confirmed in on the Team Connect and the TAK server. Based on this testing, it is clear that having priority access to a transport network is imperative to keep responders and the public safe.
The following image show me carrying two devices through dense crowds 
Left: The pictured “test1” device is connected to the Legacy TAK server. Right: The pictured “OIT Mills” device is connected Team Connect server.
Conclusions: What we learned at the 2018 Country Jam is invaluable to our understanding of public safety communications using priority voice and data on robust application platforms like Team Awareness Kit (TAK). We were able to take many of the lessons learned at Country Jam and apply them the following week in Telluride, Colorado at the 45th Annual Bluegrass Festival. Look for our next blog, which will discuss lessons from the Bluegrass Festival including the continued usage of the TAK application, usage of ESChat and Visual Labs, and discussion of tests conducted comparing Verizon priority and FirstNet enabled devices on their respective transport networks.