Transportation Object and Sound Detection

Dale Smith

Dale Smith

Peachtree Corners, Georgia

0 0
  • 0 Collaborators

Detect vehicle crashes and route first responders without the dispacher's involvement via audio detection on traffic cameras. Restart the traffic camera video feed if it goes down, count vehicle types, warm reboot the camera, provide one-click remote firmware upgrades for camera deployments. ...learn more

Project status: Under Development

oneAPI, Mobile, Networking, Internet of Things, Artificial Intelligence

Intel Technologies
Movidius NCS, OpenVINO, oneAPI

Links [3]

Overview / Usage

Vallum Halo Manager is a Network Monitoring and Management (NMM) solution designed to allow organizations to monitor network equipment (SNMP), servers, workstations, mobile devices, applications, databases, Docker containers, security cameras, and other embedded Linux devices for availability and health. Halo Manager’s functionality is completely customizable with specialized reusable embedded microservice applications called Halo Applications. It’s unique architecture does not have a complex central server install that is found with competing solutions, which allows it to be deployed and up and running in a few of minutes. Once deployed, it will immediately initiate an auto-discovery for devices and begin to monitor them.

Traffic cameras are not currently utilized to their fullest capabilities. Often, a truck is sent out when the video feed is down.

We are an Axis Communications partner. Our Agent runs on the camera, and our embedded microservice App, which runs in the Agent context, detects when the video feed goes down, and restart the video feed. We can also warm reboot the camera, if necessary.

This eliminates sending a bucket truck out for this problem.

We can also collect information from the camera operation and preserve it across reboots. This allows operations staff to review and find problems with cameras. We generate alerts for such situations, and allow customers to automate solutions on the device, whenever possible, using our rShell App. We can gather information such as packets in/out of the device and cpu utilization, and detect slow bandwidth from the camera as well as externally via our Polling App.

We are performing all processing on-device. We can easily support low bandwidth and interrupted bandwidth situations, storing json on the device and delivering it when connectivity is re-established. We do not transmit the video.

We plan to introduce traffic accident detection via an on-device sound detector. We can then send an alert, bypassing the dispatcher, so the nearest police vehicle is directed to the site as quickly as feasible. For critical injuries, we can cut response time in that critical First Hour.

Currently, we support the armv7l and mips32 architecture used by Axis Communications, and the armv7l Raspberry PI 3 device, as well as x86_64. Our rShell app currently supports bash scripts, and we plan to support PowerShell. We will bring on other architectures as needed, as long as we can obtain a cross-compiler for the device and the manufacturer's SDK, if available.

We are a developer partner with Security and Safety Things, a Bosch startup.

Our solution does not require

  • Configuration files

  • Training classes

  • Complicated database installations

When the Halo front-end is fully converted to React, it may be utilized to do single click deploy of Apps to hundreds of devices, as well as create dashboards for monitoring purposes.

Methodology / Approach

Our patent-pending technology is our distributed architecture.

We have broken apart the traditional network monitoring software agent. The Halo Agent runs as a systemd service on Linux directly on a device, server, or cloud instance. It does the following, and ONLY the following, very well.

  • Install an App
  • Deliver data an App collects to the remote Polling App upon request
  • Upgrade an App
  • Remove an App

All intelligence is contained in the embedded microservice App, which is installed from a remote site. We can write Apps to meet customer needs without re-compiling the Agent.

A Polling App, running on a separate device, is responsible for communicating with up to 255 devices. From this Polling App, another Polling App is used to collect and forward data to the customer's data ingestion workflow or to our Front End.

The Polling App does not require root access, since we are not simply pinging the remote devices, but performing a more sophisticated auto-discovery and validation process, as well as periodically polling discovered devices.

All platforms have a Cpu stats and Network stats App.

We supply camera Apps for the Axis line of security cameras, which includes restarting the video feed, warm reboot of the camera, and health monitoring of Western Digital industrial (not consumer) SD cards. Plans are to extend these capabilities to allow customers to upgrade firmware, apply PTZ information if the camera is inadvertently moved, and dirty lens detection.

We also have a Docker monitoring app for x86_64, for the cloud or on-premise data center, and plan to introduce container orchestration.

Additionally, we are planning an App based on the OpenIPMI library. This allows us to monitor on-premise data center server metrics such as inlet temperature, fan speed, and temperature inside the server cabinet. This opens up the industrial device market for us, which enables us to send alerts and perform mitigations on the device, on the factory floor.

Our goal is to move alerts and subsequent actions to the device for the fastest possible mitigation of known problems and solutions. We plan to support Bash and Powershell scripts thru our rShell App.

Technologies Used

Currently we are relying on the camera manufacturer or device for compute capabilities.

Axis Communications provided us with several cameras so we can do testing. We are deploying these cameras on the self-driving vehicle track via Curiosity Lab on Technology Parkway in Peachtree Corners, GA, in collaboration with Georgia Power. Technology Parkway is a 1.5 mile AV test track, repurposed from the existing roadway, to demonstrate smart traffic applications.

We are also using a Raspberry PI outside the Atlanta Tech Park startup incubator to do our own object detection on live traffic.

Since our Agent and Apps are written in C/C, we are using the OpenCV and Tensorflow C API.

Currently, AI and machine learning models are deployed to remote devices as part of the relevant App, and are then stored locally on the device.

We plan to use OpenVINO for deep learning and other types of model deployment, at least on the x86_64 platform. We are still unsure whether we can use OpenVINO on the arm, mips and other architectures, so we may make some adjustments.

We were delighted to see the introduction of the Neural Compute Stick 2, and promptly emptied out the penny jar to purchase one (the low cost point is very attractive for us). We plan to employ transfer learning to enhance deep learning models. We want to use a combination of cloud resources, the NCS 2, and on-site GPU resources for the transfer learning, but we are just beginning to work on these workflows.

Comments (0)