OpenAI launched the GPT-4o mini model to replace GPT-3.5, which is the most cost-effective small model on the market.
The model scored 82% on MMLU and outperformed GPT-4 on the chat preference test on the LMSYS leaderboard. GPT-4o mini is priced at 15 cents per million input tokens and 60 cents per million output tokens, an order of magnitude cheaper than the previous cutting-edge model and more than 60% cheaper than GPT-3.5 Turbo.
GPT-4o mini supports text and visual input, and will support image, video, and audio input and output in the future. The model has a context window of 128K tokens, supports up to 16K output tokens per request, and has knowledge coverage until October 2023. The improved tagger makes processing non-English text more cost-effective.
GPT-4o mini surpasses GPT-3.5 Turbo and other small models in text intelligence and multimodal reasoning, supports multiple languages, and performs well on function calls, mathematical reasoning, and coding tasks.
Starting today, Free, Plus, and Team users will have access to GPT-4o mini, replacing GPT-3.5. Enterprise users will also have access next week.
Evaluation Results of GPT-4o mini
GPT-4o mini was evaluated on multiple key benchmarks, showing strong performance in text intelligence, multimodal reasoning, mathematics, and coding tasks. The following are the specific evaluation results:
Text Intelligence and Reasoning Tasks
MMLU (Text Intelligence and Reasoning Benchmark): GPT-4o mini scored 82.0%, better than Gemini Flash’s 77.9% and Claude Haiku’s 73.8%.
Math and Coding Tasks
MGSM (Mathematical Reasoning): GPT-4o mini scored 87.0%, higher than Gemini Flash’s 75.5% and Claude Haiku’s 71.7%.
HumanEval (coding performance): GPT-4o mini scored 87.2%, surpassing Gemini Flash’s 71.5% and Claude Haiku’s 75.9%.
Multimodal Reasoning Tasks
MMMU (Multimodal Reasoning Evaluation): GPT-4o mini scored 59.4%, higher than Gemini Flash’s 56.1% and Claude Haiku’s 50.2%.
Specific Evaluation Data of GPT-4o mini
Specific Evaluation Data of GPT-4o mini
Model Evaluation Scores comparison
Overall performance : GPT-4o mini performs well on multiple evaluation benchmarks, especially in text intelligence, multimodal reasoning, mathematical reasoning, and coding tasks, significantly outperforming similar small models.
Cost-effectiveness : Compared with previous models, GPT-4o mini provides stronger performance at a lower cost and is suitable for large-scale applications.
Multimodal capabilities : Supports multimodal input (text and visual), with plans to expand to more input and output forms (images, video, and audio), giving it greater flexibility in a wide range of applications.
Cost of GPT-4o mini
GPT-4o mini features significant cost-effectiveness, with a price significantly lower than previous models, making it more cost-effective to develop and run AI applications. Here are the detailed costs:
Input Cost
15 cents per million input tokens (equivalent to about 2500 pages in a standard book).
This is an order of magnitude reduction in cost from previous cutting-edge models.
Output Cost
60 cents per million output tokens.
Compared with GPT-3.5 Turbo, the cost is reduced by more than 60%.
Cost-effectiveness advantage
Significant price reduction : The GPT-4o mini is priced significantly lower than previous models, especially the GPT-3.5 Turbo, making the development and operation of AI applications more affordable.
Suitable for large-scale applications : Low cost makes it more attractive for large-scale applications, especially those that require a lot of contextual input and fast real-time response, such as customer support chatbots.
High performance : Despite the reduced cost, GPT-4o mini still performs well on multiple evaluation benchmarks, providing a high-performance AI solution.
💡
Future-Proof Your Business. Invest in growth with our servers
