Neural Magic, which offers software for growing edge AI market, gets $30M boost

Neural Magic, which provides software to facilitate deep learning deployment in edge locations, today announced a $30 million series A funding round.

The market for edge AI is exploding as more companies deploy the technology in a variety of applications across industries — including in areas like ​​asset maintenance and monitoring, factory automation, and telehealth. The market is expected to be worth $1.83 billion by 2026, according to a report by Markets and Markets.

Keeping pace

But customer accelerator chips made by companies like Google and Nvidia for inference at the edge are increasingly unable to keep up with the improvements in efficiency, speed, and cost offered by additional software approaches, like the one pushed by Neural Magic.

Many companies and developers prefer the simplicity of using basic CPU chips and optimizing with software. A company like Target has racks of hardware in every store and might prefer to optimize with software made by Neural Magic rather than making complex, bespoke investments into specialized accelerator chips like Google’s Tensor Processing Unit (TPU), Neural Magic CEO Brian Stevens said in an interview with VentureBeat.

“That’s the world we’re trying to create. We want to give developers the flexibility to deploy AI on commodity processors already located in the edge location,” he said.

This financing, which brings the company’s total amount raised to $50 million, was led by existing investor NEA, with participation from Andreessen Horowitz, Amdocs, Comcast Ventures, Pillar VC, and Ridgeline Ventures.

Neural Magic will use the new capital to invest in the open source inference models it has built, as well as the proprietary engine the company offers to help developers deploy the models.

Industry expansion

Since the company launched two years ago, a host of other startups have emerged to offer some sort of AI software at the edge, including NeuReality, Deci, CoCoPie, OctoML, and DeepCube. However, Neural Magic is the only company offering free open source modeling, matched with a software deployment engine optimized for speed, Stevens said.

Players like AMD and Intel are also working on optimization software layers for their hardware. For example, Intel has released OpenVino, a free toolkit for optimizing deep learning models. However, Neural Magic offers what it calls “recipes” that can be plugged into machine learning libraries like PyTorch to make models more sparse and speed up its engine, Stevens said.

The company’s open source offering launched quietly in February and now has upwards of 1,000 unique installations per week, according to Stevens, who took over as CEO this year. The traction got NEA’s Greg Papadopoulos excited enough about the company to lead the latest investment and join Neural Magic’s board, Stevens said. Papadopoulos is the former CTO of Sun Microsystems and has done work at MIT on parallel data flow computing architectures. Papadopoulos came to believe that hardware brings too much friction to inference, meaning companies like Nvidia won’t be able to own the market with GPU hardware optimizations alone.