An Introduction to Mattermost’s Product Documentation Tech Stack
As I approach my first anniversary at Mattermost as a Senior Technical Writer, it’s fun to look back on what has been an empowering year of daily technical learning.
When I joined Mattermost, I was new to many of the processes and tools used by the team:
- reStructuredText and Markdown source file syntax
- Documentation output engine called Sphinx
- Python development environment
- Search engine optimization (SEO) requirements
- Source control, pull request (PR) processes, branching strategies, and review workflows using Git and GitHub
A year in, I’ve now introduced, led, and supported enhancements across all of these areas, and more! I’d like to introduce you to our product documentation tech stack and share some key learnings we’ve adopted. I’ll start with a brief high-level overview of the documentation tools we use every day to build, contribute to, improve, and maintain the Mattermost product documentation at docs.mattermost.com.
RST and MD Content Files
We write our content in reStructuredText (RST) files and Markdown (MD) files. RST files work well for technical product documentation because this declarative format is verbose enough to guide authors towards content that’s machine parsable. RST and MD source files become many of the individual pages of our product documentation site.
Sphinx Generates HTML Output
We use a Python-based open source tool called Sphinx to generate static HTML files from our RST and MD source files. Sphinx was initially created for Python documentation to read in formatted source files, render them into an abstract source tree, then transform that rendered data into generated static HTML output. We customize the majority of Sphinx input and output behavior through a conf.py build configuration file.
Because Sphinx extracts, transforms, and loads, you can also think of Sphinx as an ETL documentation tool. Other open source static site generation web framework tools similar to Sphinx include Jekyll, Gatsby, and Hugo.
Pipenv – A Python Environment Management Tool
For our local development environments, we use an environment management tool for Python called Pipenv. It consolidates and simplifies the development process within a single command line tool that includes two key components:
- A pipfile file used by the Pipenv virtual environment to manage project dependencies and their versions.
- A pipfile.lock file that generates checksums for each dependency.
SEO Support Using Extensions
We use two open source Sphinx extensions to add SEO support to Sphinx (*):
The Sphinx reredirects extension manages redirects for moved documentation pages. It generates HTML output pages containing refresh redirects metadata to the new page location. This ensures that documentation visitors don’t hit a dead end by following a link.
The Sphinx sitemap extension generates a sitemap.xml file as part of each docs build. Search engines use the sitemap.xml file to index Mattermost’s Product Documentation website and return search results.
(*) We’re no longer using the publicly released versions of these Sphinx extensions. Instead, we’re maintaining custom implementations with help from our Mattermost Community.
Aren’t these just development tools?!
Yes, the tools we use every day to manage Mattermost Product Documentation are development tools! Mattermost Technical Writers approach daily work like our Developer coworkers in Engineering do!
Our repositories are cloned locally, and our local development environments must be set up with our build dependencies like any other development project.
Writers aren’t required to use a local development environment since GitHub has robust web tools that work well for documentation corrections and updates. However, I personally enjoy working locally in an integrated development environment (IDE), such as Visual Studio Code, because it’s considerably faster for developing new feature documentation. I can try something, build quickly, review my changes, then iterate.
Development-Centric Challenges
With development-centric tools come challenges that require Mattermost writers to have development-centric knowledge.
Remember the pipfile that manages project dependencies? We’ve only started managing those dependencies in recent months. Earlier this year, we were barely aware that we had development dependencies and a sitemap.
Prior to our inaugural Mattermost Docathon in July 2021, our pipfile listed each dependency with an asterisk (which pulls down the latest version available) and not a specific release version. This meant that, under specific circumstances, updates to our dependencies could easily make their way into our source code, and into our Production instance, whether we wanted them or not.
Our open dependency configuration and our inexperience left us wide open to unintentional whoops! moments. Like that time in April 2021, when we accidentally pulled down a breaking change to our sitemap extension.
One unintentional update upgraded our sitemap extension to a release that appended version details to the sitemap output. When search engines crawled the latest build of the product documentation site, hundreds of files were missing as far as search engines were concerned because the sitemap was suddenly full of changed URLs.
Our first attempt to fix this included naively commenting out the version details specified in the conf.py configuration file, then regenerating the documentation. Our sitemap was restored with this change. After learning that the version and release fields in the conf.py file provided support for generating multiple versions of product documentation, we realized that commenting out these fields to restore our sitemap would restrict our ability to generate multiple version documentation in the future.
Mattermost Community to the Rescue!
The right fix came through one amazing Mattermost Docathon 2021 contribution that addressed our dependency lockdown issues, our sitemap woes, as well as slow performance when generating previews and builds.
This community-driven documentation contribution PR:
- Removed extension code from the pipfile to stop relying on the publicly published extension.
- Locked down existing dependencies to specific versions to match our Production environment so that upgrades won’t happen again by mistake.
- Integrated a custom version of the reredirects and sitemap extensions based on the publicly available open source projects.
- Updated both custom extensions to take advantage of multi-core processing to enable faster builds and previews.
- Taught us how to customize the output of the sitemap.xml file within our custom sphinx-sitemap extension code to remove the version details we didn’t want.
Thanks to ongoing technical guidance and support from our Mattermost Community, we understand the capabilities and features of our documentation tech stack, our extensions are now custom implementations that we control and maintain, and we know to lock down all future dependencies to avoid unintentional updates.
Most importantly, we understand the development-centric nature of the docs ecosystem we work in every day which leads to fewer unintentional mistakes and more time for writing innovation!
Want to Get Involved with Mattermost Product Documentation?
Are you interested in contributing to Mattermost product documentation or Mattermost developer documentation?
We welcome your contributions, and getting started is easy!
Stop by our DWG: Documentation Working Group Community channel to say hello, then head over to GitHub to browse our open help wanted tickets for Mattermost product documentation and developer documentation.
