By David Camilo, Edited by Nima Schei– Computer vision faces a processing challenge, in order to get results as fast as possible. Usually, the algorithmic processes, such as machine learning and deep learning processing take place at the cloud, where all the data is analyzed, processed, and then the results is sent to the final user; This is because processing requires a large amount of hardware resources that are not easily accessible in the traditional infrastructures.
For example, in the traditional surveillance systems with vision capabilities, the cameras are recording all the events that occur, then the data is sent to the cloud to be analyzed; But with the delay in response, and without a person constantly monitoring the cameras, most of the information could be lost or arrive after one event has occurred: For instance, in a residential building it’s common that a guard checks the camera system all the time, but if he or she is distracted, an intruder could enter the building without being seen, and if the systems has computer vision capabilities, the analysis of the data could take time and may arrive once the intruder is already inside the building.
For solving this, there are systems that have been developed recently using IoT solutions and Edge computing. In the article “Edge computing: vision and challenges” [1], the authors describe video analytics as one of the main challenges for the Edge Computing. In this article, we present a few solutions and proposals that involve Edge Computing and IoT for computer vision.
IoT and Edge Computing
First a little introduction on IoT (Internet of Things) and Edge Computing. Internet of Things is an emergent technology in which the common objects are connected to the internet in order to simplify our lives and to automate various tasks. According to statista[2], in 2016 there was 15.41 billion devices connected to the internet and for 2021 there will be 42.62 billion devices.
The IoT technology is used for example in smart homes: Think about having your coffee maker, your alarm clock, and your water heater, connected in an IoT environment. You program your alarm clock to ring at 6 am, at 5:55 am the alarm clock sends a message to the coffee maker, and it starts to make the coffee, then, when the alarm clock rings at 6, you’ll have your coffee made and warm ready to be drunk. You usually take 5 minutes drinking your coffee, then, while you’re enjoying the coffee, the coffee maker sends a message to the water heater, which starts to warm up the water for you to take a shower when you end your coffee.
IoT has also industrial applications, for example in an agricultural farm. Instead of the sprinklers having programmed to work at a specific time, it’s possible to have sensors that measure the humidity of the soil, and the crops hydration condition. Then, when the sensors measure that the crop needs water, it activates the sprinklers at the needed time. This could save water, and therefore money, and improves harvest quality.
The IoT’s common architecture includes sensors and actuators, which recollect the data, and send it to data centers to process and analyze it.
With the Edge Computing, there is no necessity to send all the data to the cloud to be processed and analyzed. Using an Edge Node, that is a piece of hardware with high capability of processing, the information and data produced by the IoT devices could be processed at the edge, improving response time and reducing the delays on a system [As shown in figure 2].
Real-Time Video Analytics: The Killer App for Edge Computing
In this paper, the authors present what could be the future of the video analytics involving edge computing. With the growth of technologies like the edge computing, and the necessity of applications with small latency times and fast responses, Ananthanarayanan et al. propose that a geographically distributed architecture of public clouds and edges is the only possible solution to achieve the real-time requirements of video analytics applications. They also propose Rocket, a video analytics software stack which can handle different type of video applications like surveillance and security systems, self-driving cars or vision zero for traffic.
One of the proposed applications that Rocket can handle, is now under operation, in Bellevue, Washington, using cameras to process all the information on traffic intersections [5]
REVAMP2T [6]
REVAMP2T is a solution for pedestrian tracking, using Edge video analytics for multi camera. It was presented in 2019 by Christopher Neff, Matías Mendieta, Shrey Mohan, Mohammedreza Baharani, Samuel Rogers and our CTO, Dr. Hamed Tabkhi. They developed an end-to-end IoT system and deep learning algorithms that allows to detect, re-identificate and track pedestrians across multiple cameras without storing the streamed data.
For its IoT system, the edge nodes act as the IoT devices, and contain cameras equipped with Nvidia AGX Xavier, a potent device in which they perform their deep learning-based video analytics algorithms. The edge servers connect the edge nodes of their respective area, and perform the processing, communications and storage needed for the app to work. It is important to notice that REVAMP2T doesn’t store any personal information of the pedestrians detected.
ParkMaster [7]
ParkMaster was presented in 2015. Its purpose is to exploit the smartphones’ cameras inside vehicles, in order to find an on-roadside parking spot, using image recognition algorithms running on the smartphones.
In this case, the smartphones’ cameras work as IoT devices and at the same time as edge nodes, making the computing vision processing. After the smartphone ends the visual analytic algorithm, uploads the data to the cloud, where is stored, and gets available for other drivers, working as a crowdsourcing system.
Conclusion
In order to achieve the computer vision real-time requirements, edge computing and IoT are starting to be researched and used in both, academic and industrial fields. The necessity of high processing for the deep learning or machine learning algorithms used in these solutions are the perfect excuse to exploit this new technology. We briefly presented 3 different solutions that mark the path to follow in the visual analytics field. This also brings new concerns about privacy that must be boarded by the developers, regarding the facial recognition and storage of private information. Finally, as the time passes, the developers are more exiting the cloud and more getting involved in the edge.
References:
[1] W. Shi, J. Cao, Q. Zhang, Y. Li and L. Xu, “Edge Computing: Vision and Challenges,” in IEEE Internet of Things Journal, vol. 3, no. 5, pp. 637-646, Oct. 2016, doi: 10.1109/JIOT.2016.2579198.
[2] Statista (2016) Internet of Things (IoT) connected devices installed base worldwide from 2015 to 2025 [Online] Available in: www.statista.com/statistics/471264/iot-number-of-connected-devices-worldwide/. Consulted: 06-21-2020.
[3] Image taken from: Kristiani E., Yang CT., Wang Y.T., Huang CY. (2019) Implementation of an Edge Computing Architecture Using OpenStack and Kubernetes. In: Kim K., Baek N. (eds) Information Science and Applications 2018. ICISA 2018. Lecture Notes in Electrical Engineering, vol 514. Springer, Singapore
[4] G. Ananthanarayanan et al., “Real-Time Video Analytics: The Killer App for Edge Computing,” in Computer, vol. 50, no. 10, pp. 58-67, 2017, doi: 10.1109/MC.2017.3641638.
[5] Michael Martinez, “Real-Time Video Analytics: The Killer App for Edge Computing” [Online] Available at: https://www.computer.org/publications/tech-news/research/real-time-video-analytics-for-camera-surveillance-in-edge-computing Consulted: 06-22-2020
[6] Neff, Christopher & Mendieta, Matias & Mohan, Shrey & Baharani, Mohammadreza & Rogers, Samuel & Tabkhi, Hamed. (2019). REVAMP2T: Real-time Edge Video Analytics for Multi-camera Privacy-aware Pedestrian Tracking. IEEE Internet of Things Journal. PP. 1-1. 10.1109/JIOT.2019.2954804.
[7] Sammarco, Matteo & Grassi, Giulio & Pau, Giovanni & Bahl, Victor & Jamieson, Kyle. (2015). ParkMaster: Leveraging Edge Computing in Visual Analytics.
