Band of AI startups launch ‘rebel alliance’ for interoperability

More than 20 AI startups have banded together to create the AI Infrastructure Alliance in order to build a software and hardware stack for machine learning and adopt common standards. The alliance brings together companies like Algorithmia; Determined AI, which works with deep learning; data monitoring startup WhyLabs; and Pachyderm, a data science company that raised $16 million last year in a round led by M12, formerly Microsoft Ventures. A spokesperson for the alliance said partner organizations have raised about $200 million in funding from investors.

Dan Jeffries, chief tech evangelist at Pachyderm, will serve as director of the alliance. He said the group began to form from conversations that started over a year ago. Participants include a number of companies whose founders have experience running systems at scale within Big Tech companies. For example, WhyLabs CEO and cofounder Alessya Visnjic worked on fixing machine learning issues at Amazon, and Jeffries previously worked with machine learning at Red Hat.

But in a conversation with VentureBeat, Jeffries referred to the endeavor for small to medium-size businesses in AI as a “rebel alliance against the empire” that will serve as an alternative to offerings from Big Tech cloud providers, which he characterized as “building an infrastructure just to lock you in.”

“Don’t get me wrong: There’s nothing wrong with a big proprietary tool if you’re all in, but a true canonical stack is one that’s portable across environments,” he said. “To become part of a truly foundational stack of the future, you’ve got to run in multiple environments. And you’ve got to play nice with others in the sandbox, and you have to have interoperability in that market.”

“Not everyone in the group will survive. But we’ve talked about this. Like we’re in this Cambrian explosion period, and the Alliance at this point, it will serve where we are in the adoption curve. Some of these companies will go away or fold into whoever the eventual winner is,” he said.

The alliance initially plans to focus on things like small partnerships between developers working on tools and frameworks, facilitating joint documentation, and creating test software for integration. How to eliminate bias in algorithms before being deployed will not be considered as part of what Jeffries refers to as the canonical stack.

Examples of alliances formed in the AI space to tackle include the Open Source Neural Exchange (ONNX), created by Facebook and Microsoft in 2017, and open source projects like MLFlow, TensorFlow, and Apache Spark, which cofounders of Determined AI contributed to while at UC Berkeley.