AI2’s new model aims to be open and powerful yet cost effective
September 10, 2024
The Allen Institute for AI (AI2) released a new open-source model that hopes to answer the need for a large language model (LLM) that is both a strong performer and cost-effective. The new model, which it calls OLMoE, leverages a sparse mixture of experts (MoE) architecture. It has 7 billion parameters but uses only 1 billion parameters per input token. It has two versions: OLMoE-1B-7B, which is…