Unlocking AI at the edge with new tools from Deci

Edge devices must be able to process delivered data quickly, and in real time. And, edge AI applications are effective and scalable only when they can make highly accurate imaging predictions. 

Take the complex and mission critical task of autonomous driving: All relevant objects in the driving scene must be taken into account — be it pedestrians, lanes, sidewalks, other vehicles or traffic signs and lights.

“For example, an autonomous vehicle driving through a crowded city must maintain high accuracy while also operating in real time with very low latency; otherwise, drivers’ and pedestrians’ lives can be in danger,” said Yonatan Geifman, CEO and cofounder of deep learning company Deci. 

Key to this is semantic segmentation, or image segmentation. But, there’s a quandary: Semantic segmentation models are complex, often slowing their performance. 

“There is often a trade-off between the accuracy and the speed and size of these models,” said Geifman, whose company this week released a set of semantic segmentation models, DeciSeg, to help solve this complex problem.

“This can be a barrier to real-time edge applications,” said Geifman. “Creating accurate and computational-efficient models is a true pain point for deep learning engineers, who are making great attempts to achieve both the accuracy and speed that will satisfy the task at hand.”

The power of the edge

According to Allied Market Research, the global edge AI (artificial intelligence) market size will reach nearly $39 billion by 2030, a compound annual growth rate (CAGR) of close to 19% over 10 years. Meanwhile, Astute Analytica reports that the global edge AI software market will reach more than $8 billion by 2027, a CAGR of nearly 30% from 2021.

“Edge computing with AI is a powerful combination that can bring promising applications to both consumers and enterprises,” said Geifman. 

For end users, this translates to more speed, improved reliability and overall better experience, he said. Not to mention better data privacy, as the data used for processing remains on the local device — mobile phones, laptops, tablets — and doesn’t have to be uploaded into third-party cloud services. For enterprises with consumer applications, this means a significant reduction in cloud compute costs, said Geifman. 

Another reason edge AI is so important: Communication bottlenecks. Many machine vision edge devices require heavy-duty analysis for video streams in high resolution. But, if the communication requirements are too large relative to network capacity, some users will not obtain the required analysis. “Therefore, moving the computation to the edge, even partially, will allow for operation at scale,” said Geifman. 

No critical trade-offs

Semantic segmentation is key to edge AI and is one of the most widely-used computer vision tasks across many business verticals: automotive, healthcare, agriculture, media and entertainment, consumer applications, smart cities, and other image-intensive implementations. 

Many of these applications “are critical in the sense that obtaining the correct and real-time segmentation prediction can be a matter of life or death,” said Geifman. 

Autonomous vehicles, for one; another is cardiac semantic segmentation. For this critical task in MRI analysis, images are partitioned into several anatomically meaningful segments that are used to estimate criticalities such as myocardial mass and wall thickness, explained Geifman. 

There are, of course, examples beyond mission-critical situations, he said, such as video conferencing virtual background features or intelligent photography. 

Unlike image classification models — which are designed to determine and label one object in a given image — semantic segmentation models assign a label to each pixel in an image, explained Geifman. They are typically designed using encoder/decoder architecture structure. The encoder progressively downsamples the input while increasing the number of feature maps, thus constructing informative spatial features. The decoder receives these features and progressively upsamples them into a full-resolution segmentation map. 

And, while it is often required for many edge AI applications, there are significant barriers to running semantic segmentation models directly on edge devices. These include high latency and the inability to deploy models due to their size. 

Very accurate segmentation models are not only much larger than classification models, explained Geifman, they are also often applied on larger input images, which “quadratically increases” their computational complexity. This translates into slower inference performance. 

As an example: Defect-inspection systems running on manufacturing lines that must maintain high accuracy to reduce false alarms, but can’t sacrifice speed in the process, said Geifman. 

Lower latency, higher accuracy

The DeciSeg models were automatically generated by Deci’s Automated Neural Architecture Construction (AutoNAC) technology. The Tel Aviv-based company says these “significantly outperform” existing publicly-available models, including Apple’s MobileViT and Google’s DeepLab.

As Geifman explained, the AutoNAC engine considers a large search space of neural architectures. While searching this space, it takes into account parameters such as baseline accuracy, performance targets, inference hardware, compilers and quantization. AutoNAC attempts to solve a constrained optimization problem while completing several objectives at once — that is, preserving the baseline accuracy with a model that has a certain memory footprint.

The models deliver more than 2 times lower latency and 3 to 7% higher accuracy, said Geifman. This allows companies to develop new use cases and applications on edge AI devices, reduce inference costs (as AI practitioners will no longer need to run tasks in expensive cloud environments), open new markets and shorten development times, said Geifman. AI teams can resolve deployment challenges while obtaining the desired accuracy, speed, and model size.

“DeciSeg models enable semantic segmentation tasks that previously could not be carried out on edge applications because they were too resource intensive,” said Geifman. The new set of models “have the potential to transform industries at large.”