Apple has Unveiled an Open-Source LLM Model

Despite being a little late to the AI game, it looks like Apple is planning to go all in. Following the WWDC event, it had been announced that nearly all of Apple’s products will have AI solutions available through Apple Intelligence. Now, Apple is reportedly delving deeper into language models powered by AI.

A 7 billion parameter language model called DCLM-Baseline-7B was recently made available by Apple on Hugging Face. The model is a part of the DataComp for Language Models (DCLM) benchmark, which aims to raise the standard of language model training datasets. 2.5 trillion tokens from public datasets were used to train this model, which also includes weights, training code, and dataset. It has a 2048-token context window and uses English data primarily.

This model has attributes comparable to other well-known models like Gemma and Llama 2. Tested against popular models of similar size on the Massive Multitask Language Understanding (MMLU) benchmark, DCLM-Baseline-7B was outperformed by Microsoft’s Phi-3 but still performed competitively, surpassing Mistral 7B.

One of the most significant features of DCLM-Baseline-7B is that it is fully open-sourced, with “open data, open weight models, and open training code,” as affirmed by Apple research scientist Vaishaal Shankar.

This is not Apple’s first experimentation with AI models; the company has previously released models like the multimodal large language model (MLLM) Ferret-UI. The AI model was created with accurate task execution in mind, specifically for handling open-ended language instructions and user interfaces. Ferret-UI’s primary focus is its multimodal capabilities, which combine a sophisticated language understanding with visual recognition designed for mobile user interface screens. It also includes reasoning, grounding, and referring capabilities.

We’ll be able to witness Apple compete in the AI space and get a better idea of the potential for success of its AI initiatives once iOS 18 and Apple Intelligence are made available later this year.