CAIDA chooses Mattermost to streamline global research collaboration through better project management

CAIDA hackathon

CAIDA hackathon

Executive Summary

By moving to Mattermost, the Center for Applied Internet Data Analysis (CAIDA)—an internet research group with high security requirements and a distributed team—was able to easily coordinate projects across the globe and cultivate a stronger, more engaged community.

For 20 years, CAIDA had been using an internally developed IRC-like messaging solution to coordinate project management and execution, facilitate research collaboration and conduct day-to-day conversations. Unfortunately, the platform had its limitations. It was hard for new users to get up to speed. It lacked file-sharing functionality. It required compiling and installing client software or providing new users an account on an internal machine where the software was already installed, which meant a time-consuming vetting process.

As many enterprises moved their chat logs “into the cloud,” CAIDA searched for a solution that allowed them to keep their logs local—on their own internal server. Fortunately, a computer science researcher on campus discovered Mattermost, the open source enterprise messaging workspace built for organizations that place a premium on privacy, security and open software ecosystems. CAIDA has been using it ever since. The power, performance and extensibility of Mattermost has made this scientific research group more productive than ever.

Organization

Founded in 1997, CAIDA is a research group based at the University of California’s San Diego Supercomputer Center, which is primarily funded by the National Science Foundation. CAIDA conducts network and security research and builds research infrastructure to support large-scale data collection, curation and data distribution to the scientific research community.

CAIDA’s mission includes the pursuit of macroscopic insights into internet structure, dynamics and evolution; fostering a collaborative environment in which data can be acquired, analyzed and shared; and informing internet-related public policy debates with the best available data and objective analyses. Their publications, presentations, software, data and tools are available at http://www.caida.org.

Challenge

For 20 years, CAIDA’s researchers—many of whom are scattered across the globe—relied on a text-based, IRC-like messaging solution to stay connected and collaborate on various projects. While the solution was extremely lightweight and partially developed in-house, it had a steep learning curve.

Though CAIDA relied on this “old-fashioned” solution for a long time, there were a few downsides to using it. There was no graphical user interface and it was hard for new users to learn all the keyboard commands and shortcuts. It lacked file-sharing functionality. Perhaps most challengingly, CAIDA had to provide new users with an account on an internal machine to relieve them of having to install and maintain their own version of the client software. Account access required a thorough, time-consuming vetting process.

“We needed to create a UNIX account, giving them access to a shell,” explains Josh Polterock, manager of scientific projects for CAIDA. “It’s a major security issue.”

Like most organizations that deal with sensitive information, security and privacy are major concerns for CAIDA.

“We have similar concerns for protecting proprietary and sensitive data one would expect from any enterprise,” Polterock says. “We conduct large-scale active and passive data experiments. In some cases, we collect data that contains sensitive information. Security is important to protecting the integrity of our scientific work and the privacy of users whose resources might appear in our data.”

To this end, CAIDA continuously strives to make steady progress on many fronts—like algorithms, approaches, methods and tools—to increase security.

After deciding that its internal messaging tool no longer met its standards, CAIDA began its search for a substitute solution that would enable its researchers to seamlessly collaborate across the globe—without forcing the organization to compromise on security or usability.

Solution

CAIDA—which hosts its own email, domain and web services—needed a messaging platform that could live in their own data center. When a research collaborator recommended Mattermost, the open source enterprise messaging workspace built for privacy-conscious organizations, the team began vetting the solution.

“It’s a bit of an oddity: In a time where Slack is gaining so much ground, we were looking for something better that could live on our own infrastructure,” Polterock explains.

Mattermost—which can be hosted on any cloud or in the data center—met that criteria. After testing the solution and liking what they saw, CAIDA deployed Mattermost in June 2017. A month later, they shut off their older chat system and migrated all of their messaging to Mattermost.

“In the end, the main driver was ease of use and quick onboarding,” Polterock continues. “Really, we see no competitors. There’s been no turning back.”

Results

The move to Mattermost was quick; the system was up and running in a few days.

“We actually kept the old system running for a while in case folks struggled to convert, but people basically loved it,” Polterock says. “Issues in the group—whether technical, administrative or otherwise—get resolved more quickly.”

Mattermost protects the research group’s privacy, self-determination and commitment to an open source ecosystem, which were the most important criteria for CAIDA during the decision-making process. The team also enjoys the fact that Mattermost enables them to display complex math equations in a visually engaging manner. The platform has proven surprisingly easy to use; CAIDA even gives out accounts to workshop participants to facilitate communication during those events.  

“I think the fact that employees are not saying much speaks to the fact that it already fills the primary requirements,” Polterock explains. “Like great software infrastructure should be—it just works.”

CAIDA’s most popular Mattermost use cases are as follows:

  • Day-to-day communication. Mattermost has become CAIDA’s go-to method of communication. The team creates channels for each research project. They also have channels for discussions about their website, monitoring, visualization, routing and more. Because their team is distributed, Mattermost is also used as a punch clock of sorts. “It’s a way to tell everyone else, ‘Hey, I’m around’ or ‘Leaving for the day,’” Polterock says.
  • Project management. CAIDA relies extensively on Mattermost to coordinate research processes across the globe—both with employees, including those who travel a lot or work remotely, and students who support research efforts. Mattermost is also used to collaborate with external researchers. “We might have someone come and present at a workshop and something interesting comes out of it,” Polterock continues. “All of a sudden, we have an external collaborator who then brings one of their grad students on board. We send these people invites to our Mattermost team that supports collaborators. This kind of infrastructure is critical for the success of our globally growing research group.”
  • Workshops. Each year, CAIDA hosts three to four workshops with anywhere between 25 and 100 attendees; topics covered include cybersecurity, internet economics and network complexity. Previously, the organization tried to use Jabber as a communications tool that workshop participants could use to collaborate with one another. But that meant attendees either needed to have a Jabber account or they had to create one on the spot. Now, a few weeks before a workshop, CAIDA sends out Mattermost invites to participants en masse. People sign up and get to know each other before an event. They then rely on Mattermost during the workshop—and, in many instances, after its conclusion. “There’s zero training,” Polterock says. “People recognize the model and they’re immediately productive.”

As a result, Mattermost provides CAIDA with a number of benefits, including:

  • Increased productivity. Employees, many of whom work remotely, can easily track down colleagues to find the information they need when they need it. Additionally, staff productivity is not hampered by time zone differences; Mattermost’s notifications remain until employees have the chance to view them, which supports asynchronous collaboration. This capability can dramatically accelerate the pace of research—and any other endeavor.
  • More research. Coordinating research with global teams has never been easier for CAIDA and their colleagues. Thanks to Mattermost, research that ultimately influences important policy moves forward much faster because conversations happen in real time and no one is slowed down by a clogged email inbox.
  • A stronger community. Since Mattermost enables workshop attendees to get to know each other before events and stay in touch afterward, the platform helps CAIDA build an even stronger community.

CAIDA is now exploring the power of Mattermost’s extensibility features (e.g., integrations with Asana and Jenkins). They’re also eyeing webhooks for real-time alerting. In the meantime, the organization is happy to enjoy a modern and secure collaboration platform that makes it easier to fulfill its mission.  

“Anything that speeds up the process of integrating collaborators, sharing ideas and increasing bandwidth amongst group members increases our competitive advantage,” Polterock concludes.

Learn More

Are you looking to improve process coordination and streamline project management across your organization? Get in touch with the Mattermost customer team to learn how.

About Mattermost

Mattermost provides enterprise-grade collaboration solutions for the world’s leading organizations on a vibrant open source platform. Our private, hybrid and public cloud solutions offer secure, configurable, highly scalable messaging across web, mobile and PC with archiving, search, and deep integration across in-house systems.

About CAIDA

Founded in 1997, the Center for Applied Internet Data Analysis (CAIDA) conducts network research and builds research infrastructure to support large-scale data collection, curation, and data distribution to the scientific research community. For more information, visit www.caida.org.

SHARE THIS ARTICLE:

mm
Justin Reynolds

Justin Reynolds is a copywriter, editor and content strategist based in Connecticut who joined Mattermost in June 2017.

Subscribe for articles & tutorials

To get future blog posts to your inbox, subscribe below.

Bring your messaging and tools together to get more done, faster