Mistral AI, a Paris-based artificial intelligence startup, today unveiled its latest advanced AI model capable of processing both images and text.
The new model, called Pixtral 12B, employs about 12 billion parameters and is the first of its models capable of vision encoding, making it possible for it to “see” images alongside text.
The new model is based on Mistral’s Nemo 12B, an AI model previously released by the company capable of understanding text, with the addition of a 400 million-parameter vision adapter. The adapter allows users to add images through URLs or encode them via base64 within the inputted text.
Many other AI large language models have also added multimodal capabilities that allow users to input images such as Anthropic PBC’s Claude family, OpenAI’s GPT-4o and Google LLC’s Gemini. The addition of image reasoning capabilities to Pixtral 12B should provide it the ability similarly to answer questions about images, provide captioning, count objects and more.
The company released the parameters and code via a torrent link on GitHub and the AI distribution platform Hugging Face. The company has encouraged developers to start downloading and using it.
Now that the model is available for download, developers will be able to fine-tune and train the model for their own purposes. The company offers some of its models open-source under the Apache 2.0 license without restrictions. For others, Mistral offers a dev license that is free for development, but requires a paid license for commercial applications, but not for research uses. The company has not clarified what license Pixtral 12B will fall under.
Sophia Yang, head of Mistral developer relations, said in a post on X, that the model will soon be available for testing on Mistral’s chatbot and application programming interface platforms, Le Chat and Le Platforme.