Since its conception, The goal of each new AI model has always been to be bigger and better than its predecessors. However, recently there has been a shift in the industry. Tech companies have started looking into smaller AI systems that are way more cost-effective to develop.
Microsoft has contributed to this trend with its recent release of three compact AI models that are a part of the Phi-3 technology family.
Phi-3-Mini, the smallest of them all, is a 3.8 billion parameter language model that was trained on 3.3 trillion tokens in a large dataset. Despite being compact, Phi-3-Mini has performance levels comparable to larger models like Mixtral 8x7B and GPT-3.5.
This Phi-3 model can fit on a smartphone and can be used without an internet connection. Moreover, it is capable of operating on standard computer chips rather than the more pricey Nvidia processors.
“One of the things that makes Phi-2 better than Meta’s Llama 2 7B and other models is that its 2.7 billion parameter size is very well suited for fitting on a phone,” said Harkirat Behl, one of the creators of Phi-3.
In December, the company unveiled Phi-2, which outperformed larger models like Llama 2. According to Microsoft, Phi-3 outperforms the previous iteration and can produce results that are comparable to those of a model ten times larger.
Large tech companies can lower the prices by using smaller models as they require less processing power. These companies hope that more users will be able to use artificial intelligence in areas where the larger, more sophisticated models have proven to be prohibitively expensive.
Microsoft has not provided more details as of yet, apart from stating that utilizing the new models would be “substantially cheaper” than using larger models like GPT-4.
Phi-3-Mini is not without limitations, though. Its size essentially limits it for some tasks, even though it shows a comparable level of language understanding and reasoning ability to much larger models. For instance, its inability to store large amounts of “factual knowledge” causes it to perform worse on tests like TriviaQA.
Microsoft thinks a search engine added to the model can help with these shortcomings. Furthermore, the model’s language proficiency is limited to English. This emphasizes the necessity of looking into multilingual capabilities for Small Language Models.
Additionally, Microsoft has released the Phi-3-Medium and Phi-3-Small models, which are both considerably more powerful than the Phi-3-Mini.
- Apple iOS 18.3 to Have Exciting New Features! - January 13, 2025
- Google is Assembling an Advanced ‘world modeling’ AI team - January 9, 2025
- The Year in AI: Top Breakthroughs that Redefined Technology in 2024 - January 7, 2025
