Enhancing mission-critical enterprise collaboration with multi-LLM support for Mattermost Copilot

Mattermost is excited to announce the release of v10.0, bringing with it a groundbreaking enhancement to our Mattermost Copilot plugin: support for multiple large language models (multi-LLM). 

This feature, designed to empower mission-critical enterprises, adds a new layer of flexibility, privacy, and control to your AI-driven workflows. 

In this article, we’ll dive into what multi-LLM support means for you and how it can be a key component in accelerating your enterprise workflows and bolstering resilience. For more information, please refer to the product documentation.

For a broader understanding of where we started with Mattermost Copilot for workplace collaboration, we encourage you to read our previous blog post.

For Mattermost administrators looking to get started with Mattermost Copilot multi-LLM support, check out our new Academy course that shows how to connect OpenAI and Azure OpenAI Service’s GPT-4o, Anthropic’s Claude Sonnet 3.5, and a local Meta Llama 3.1 model to Mattermost, each with its own custom prompt instructions to create specialized assistants.

Introducing Multi-LLM support 

Multi-LLM support allows users to integrate and utilize multiple language models within their Mattermost Enterprise environment. Whether you’re leveraging third-party models for specific tasks or deploying private, self-hosted or on-prem models for sensitive operations, multi-LLM support provides unparalleled versatility and control. And if you utilize Azure’s OpenAI Service, this new functionality allows you to extract further value from your Microsoft investment by connecting specialized AI tools to your Mattermost platform.

Key differentiators

• Enhanced flexibility: Seamlessly switch between different language models based on your specific needs or remix them by customizing their individual prompts via the System Console.
• Increased control: Tailor your AI workflows with a mix of third-party and private models for the different scenarios your team may encounter.
• Improved security: Maintain the highest standards of privacy by using self-hosted or on-prem models for sensitive data.
• Optimized performance: Choose the best model for each task, ensuring optimal performance and accuracy.

What can you do with multi-LLM support?

Multi-LLM support delivers a world of possibilities. Here are a few ways this feature can transform your enterprise workflows.

Customizable assistants with unique prompts

One of the most powerful features of multi-LLM support is the ability to customize each language model with its own unique prompt. This means you can even configure the same back-end (like Azure OpenAI’s GPT-4o) multiple times with different prompts, easily creating a variety of specialized assistants available for your Mattermost deployment.

For mission-critical use cases such as SecOps, incident response, and command and control, this capability is particularly valuable. Here are some specific examples:

• SecOps assistant: Utilize OpenAI’s GPT-4o with a prompt tailored to threat intelligence and anomaly detection. This model can help security teams quickly identify potential threats by analyzing log data, flagging unusual activity, and providing insights into emerging vulnerabilities. OpenAI’s GPT-4o is chosen for its robust language understanding and ability to process complex security data.
• Incident response assistant: Deploy a model like Anthropic’s Claude 3.5 Sonnet, known for its safety and alignment features, with a prompt focused on incident triage and response coordination. This assistant can guide your team through the initial steps of incident response, suggest best practices, and generate incident reports. Claude’s emphasis on safe and reliable outputs makes it ideal for handling sensitive incident response tasks.
• Command and Control (C2) assistant: Implement a local model like Meta’s Llama 3 with a prompt that enhances real-time decision-making and operational coordination. Deployed air-gapped with supporting hardware, this model can assist in drafting and disseminating commands, tracking task progress, and maintaining situational awareness during critical operations.
• Compliance and audit assistant: Use Azure OpenAI Service’s GPT-4o with a prompt designed for regulatory compliance and audit processes. This assistant can help your organization stay compliant by reviewing policies, generating compliance reports, and identifying areas of risk. GPT-4o’s strength in enterprise-level data analysis combined with Azure’s tools for compliance management make this a suitable choice for the role.
• Disaster recovery assistant: Leverage Azure OpenAI Service’s GPT-4o with a prompt focused on disaster recovery planning and execution. This assistant can provide guidelines for disaster recovery protocols, coordinate recovery tasks, and ensure that critical systems are restored efficiently. Azure’s cloud infrastructure and AI capabilities offer the scalability and reliability needed for effective disaster recovery management.

This level of customization allows each team within your organization to have an AI assistant specifically tuned to their needs, maximizing efficiency and enhancing productivity across all departments. By leveraging different prompts, you ensure that each interaction is contextually relevant and aligned with the specific requirements of the task at hand.

Context-specific responses

Utilize specialized language models for different scenarios. For instance, a customer support team can use a model fine-tuned for customer interactions while a development team might employ a model with access to technical documentation. Because Mattermost Copilot can connect with source control (GitHub) and project management (Jira), it’s easy to equip your virtual assistants with this information.

Privacy and security with multi-LLM support

Privacy and security are paramount for mission-critical enterprises. Multi-LLM support enhances these aspects by allowing organizations to choose between third-party and private, on-prem, or self-hosted models based on the sensitivity of the data and the task at hand.

Using third-party models

Third-party models can be advantageous for general tasks and scenarios where data sensitivity is low. These models are often highly optimized and can provide quick, efficient results for everyday operations.

Deploying self-hosted or on-prem models

For tasks involving sensitive or proprietary information, enterprises can deploy private, self-hosted models on-premises. This setup ensures that no data leaves your secure environment, maintaining the highest level of privacy and compliance with regulatory standards. By balancing the use of third-party and private models, organizations can optimize both performance and security.

Conclusion

Mattermost v10.0’s multi-LLM support is a significant step forward in our commitment to providing advanced, flexible, and secure AI solutions for mission-critical enterprises. This feature not only enhances the functionality of Mattermost Copilot but also empowers users to tailor their AI interactions to meet their unique needs.

We extend our heartfelt thanks to our vibrant open source community, whose contributions have been instrumental in bringing this feature to life. The Copilot plugin, like all Mattermost offerings, thrives on community collaboration and innovation.

Stay tuned for more updates and enhancements as we continue to push the boundaries of what’s possible with AI in enterprise collaboration. If you’re a Mattermost administrator looking to get started with Mattermost Copilot multi-LLM support, be sure to check out our new Academy course to learn how to connect OpenAI and Azure OpenAI Service’s GPT-4o, Anthropic’s Claude Sonnet 3.5, and a local Meta Llama 3.1 model to Mattermost and see custom prompts in action.

Ready to elevate your Mattermost experience? Get started with Mattermost v10.0.

Thank you for joining us on this exciting journey. We look forward to seeing how you leverage multi-LLM support to transform your workflows and drive innovation in your organization. As always, please reach out to [email protected] with any questions or feedback.