how to create a chatops bot

How to create your ChatOps bot

September 8, 2022

Engineering

Communication within organizations has evolved from email threads to real-time chats. With the vast potential of real-time messaging, organizations are beginning to explore how they could make these chat applications do more. We now even have a name for this phenomenon — ChatOps. To get a taste of the power of ChatOps in action, let’s build a ChatOps bot that sits in a Mattermost channel with Errbot.

What is ChatOps?

ChatOps is an approach to collaboration. Without ChatOps, this is what teamwork looks like in a typical engineering team:

teamwork without chatops

In the model above, people often switch between applications and struggle to tell what every team member is doing, which in turn creates knowledge silos. With ChatOps, the model changes to the image below:

teamwork with chatops

With ChatOps, the chat room also becomes a shared console where team members initiate tasks with commands issued to the ChatOps bots, which are just scripts. The ChatOps bot then executes those commands in some development infrastructure on behalf of the user. For example, a user could issue a command to the integrated ChatOps bot to trigger a deployment to Amazon Web Services (AWS). That exemplifies what ChatOps is — people, conversations, bots, and tools gathered in a chat room. 

For DevOps practitioners, having bots automate software releases, deployments, incident reporting, and other essential functions increases the team’s productivity. And with everything happening in a shared space, it’s easier for teams to respond to incidents collaboratively and swiftly. A shared space also reduces context switching.

Building a chatbot with Errbot

The software domain frowns on repetition. That’s why there are frameworks for almost everything these days — think web development frameworks like Django and Node. Similarly, some frameworks make writing a ChatOps bot a snap. Errbot is one such framework.

Errbot has built-in support for multiple server backends like Mattermost, Telegram, and others. Though Errbot comes with pre-built security and access control features, we need only focus on extending it to suit our use case. For example, we could extend Errbot and have it pull server logs into a Mattermost channel. With Errbot, the possibilities of setting up a productive ChatOps workflow are endless.

To give you an idea of what’s possible with Errbot, we will build a bot that notifies members of a channel about events related to GitHub issues and retrieves details of a repository when issued the command to do so.

Prerequisites

  • Python greater than or equivalent to 3.4 installed. Check out the Python site to download the appropriate version.

Git installed and a GitHub account. Check out this tutorial on the Git website to learn more.

Step 1: Setting up our development environment

Go to your terminal and execute the following commands:

  • Run python3 -m venv env-name on Unix and macOS or python -m venv env-name on Windows to create a virtual environment where we install the dependencies for our project. Replace env-name with whatever name you chose for your virtual environment.
  • Run source env-name/bin/activate on Unix and macOS or .\env-name\Scripts\activate on Windows to activate the virtual environment.
  • Install Errbot with pip install errbot.
  • Run git clone https://github.com/errbotio/errbot-mattermost-backend.git.
  • Enter the directory with cd errbot-mattermost-backend.
  • Install dependencies with pip install -r requirements.txt.
  • Run errbot --init to create a bot in development mode. This command should also generate a config.py file and a plugin folder in the errbot-mattermost-backend directory.

Next, let’s set things up on Mattermost.

Step 2: Creating a bot account on the Mattermost server

Visit Mattermost and click Get Started to spin up a Mattermost instance.

Sign up with your work or school email. When done, you should see an interface like the one below:

creating a chatbot account

Mattermost would automatically create a team in your workspace based on the company name you provide at the time of signup. For me, the default team is called py-universe. Yours could be anything. However, keep in mind that under the hood, Mattermost would name the auto-created team main (we will be using this name later). Furthermore, the auto-created team comes with the Off-Topic and Town Square channels out of the box.

Next, we create a Mattermost bot account.

Follow the steps described in the section titled User Interface (UI) on the Mattermost guide. Set the bot’s Role to System Admin and keep the generated Token somewhere as we will need it soon. Go to your team’s menu as shown in the image below and select Invite People. Type your bot’s username in the dialog box and then click Invite.

add chatbot to team

Our bot is now part of the py-universe team and the channels. Next, we connect our local bot with the Mattermost bot account.

Step 3: Connecting Errbot to Mattermost

Open the config.py file in errbot-mattermost-backend and update the following:

  • Update BACKEND = 'Text' to BACKEND = 'Mattermost'. This tells Errbot we’ll connect to Mattermost.
  • Replace @CHANGE_ME assigned to BOT_ADMINS with your Mattermost username.

Next, add the following to your config.py file. If these configs already exist, override them:

    import os
    local_dir_path = os.path.dirname(__file__) 
    BOT_DATA_DIR = os.path.join(local_dir_path, "data") 
    BOT_EXTRA_PLUGIN_DIR = os.path.join(local_dir_path, "plugins")
    BOT_EXTRA_BACKEND_DIR = os.path.join(local_dir_path, "..")
    BOT_LOG_LEVEL = logging.DEBUG

With the BOT_DATA_DIR, we’ve specified the directory where we want our bot to store all configuration data. The BOT_EXTRA_PLUGIN_DIR config tells our bot where to find the custom code we’ll be writing and the BOT_EXTRA_BACKEND_DIR config tells our bot where to find the code for the Mattermost BACKEND we specified earlier. Finally setting the BOT_LOG_LEVEL to DEBUG would log events to the console; this would come in handy when debugging.

Next, we tell our bot what Mattermost bot account to use. Add the below to your config.py file:

BOT_IDENTITY = {
  'team': 'main',
  'server': 'py-universe.cloud.mattermost.com',  # Grab this from your browser's URL field       
  'token': '4xmh',
 
  # Optional 
  'scheme': "https", 
  'port': 443,
  'timeout': 30
    }

Make sure to replace the values of server, and token with your own values. Note: leave the team as main.

At this point leave all the values you haven’t updated in the config.py file untouched.

Spin up your local bot with the command errbot. 

Type !tryme in any of the channels on Mattermost. Your bot should respond with “It works!” as shown in the image below:

tryme command

Moving forward, let’s add custom features to the bot.

Step 4: Adding features to our ChatOps bot

We want to have our bot:

  • Listen using a webhook for issues opened or closed on a GitHub repo and update the Mattermost channel.
  • Receive a command from the Mattermost channel that grabs the details of a GitHub repo and posts it in the channel.

Adding the webhook feature

To extend Errbot, we have to create our own plugin. Creating a plugin entails creating a *.plug file with an accompanying *.py file in the auto-generated plugin folder. Errbot then picks up the logic defined in the added *.py file. 

Let’s create a plugin that interacts with GitHub through a webhook.

Create github_mattermost/github_mattermost.plug in the plugin folder and add this content to it:

  [Core]
  Name = GithubMattermostPlugin
  Module = github_mattermost

  [Documentation]
  Description = Notify Mattermost channel of new issues opened/closed, and retrieve list of issues.

  [Python]
  Version = 3

The Module variable points to the accompanying .py file we create in that folder. The Name variable specifies the name of our custom plugin class. We create this in the .py file.

Create github_mattermost/github_mattermost.py in the plugin folder and add the following content:

from errbot import BotPlugin, botcmd, webhook 


class GithubMattermostPlugin(BotPlugin):
    @webhook 
    def github_issues(self, payload):
        if type(payload) != str: 
            for room in self.rooms(): 
                self.send( 
                    self.build_identifier(f"~{room.name}"), 
                    'Title: Issue {0}!\n Issue URL:  {1}'.format(payload['action'], payload['issue']['url']),
                )

The @webhook decorator marks github_issues as a webhook handler, allowing Errbot to hook up an endpoint to our function. To enable that, we must first configure the web server that comes with Errbot.

Go to Direct Messages then click <your bot> on Mattermost as shown in the image below:

add webhooks to chatbot

In your bot’s Direct Messages module, issue the following command:

!plugin config Webserver {
    'HOST': '0.0.0.0',
    'PORT': 3141,
    'SSL': {
    'certificate': '',
    'enabled': False,
    'host': '0.0.0.0',
    'key': '',
    'port': 3142
    }
}

Errbot configures the web server with the credentials above and responds with Plugin configuration done. if everything goes well, as shown in the image below:

how to configure plugin1

Go to any of the channels on Mattermost and test your endpoint with the command !webhook test /github_issues as shown in the image below:

configure plugin2

With the endpoint set, we connect it to a repo on GitHub. But, our endpoint is on the localhost. However, we can expose our localhost to the internet using ngrok.

Download and install ngrok via the application website, then create an account. On Windows, expose the localhost by running ngrok.exe http 3141. You can follow the instructions on the link above to run the same command for a different operating system. You should see an output that ends with something like this: Forwarding http://7e9ea9dc.ngrok.io -> 127.0.0.1:3141. Alternatively, you could use a service that doesn’t require a signup, like localhost.run or localtunnel.

Take note of the forwardingURL (*.ngrok.io in my case). We use it to set things up on GitHub.

To finish setting up our webhook, go to one of your existing GitHub repositories or create a new one and follow the steps in this GitHub guide. Following these steps should bring you to a page like the one in the image below: 

finish setting up webhook

While on that page, do the following:

  • Set the Payload URL to your own ngrok URL followed by /github_issues
  • Set content-type to application/json
  • Do not add a Secret or Enable SSL Verification
  • For events that would trigger the webhook, only select Let me select individual events then check issues

The testing payload will fail with a 500 error, but only because it sends content that Errbot isn’t set up to handle. To test our setup, create a new issue or close and reopen an existing issue in the repository where you added the webhook. If the setup works, it posts the URL of the issue to all the channels that our bot is in, as shown in the image below:

test chatbot

Getting a GitHub repository’s details

First, install requests with pip install requests.

Next (if you don’t have one) create a personal access token on GitHub following this GitHub guide. We use it to authenticate our API calls.

Go to plugin/github_mattermost/github_mattermost.py and add import requests to the top of the file. Then add the function defined below to the class in that file. Be mindful of indentation—the first line below should be inline with @webhook above:

@botcmd(split_args_with=None)
def get_repo(self, mess, args):
    token = 'your personal access token here'
    repo_owner = args[0]
    repo_name = args[1]
    headers = {'Authorization': 'token' + token}
    url =  "https://api.github.com/repos/{0}/{1}".format(repo_owner, repo_name)
    repo_details = requests.get(url, headers=headers)
    return(repo_details.json())

The snippet above updates a Mattermost channel with the details of a repo. The @botcmd decorator tells Errbot to treat get_repo as a bot command. Don’t forget to update token with your own personal access token.

Note: Errbot does not auto-restart its server when you make changes to the code. Always kill and restart the errbot each time you make a change.

To test the command, go to any of the channels where we added our bot and type:

!get_repo Your_Github_Username Repo_Name

The command should update the channel with details of the requested repo in JSON format, as shown in the image below:

test chatbot 2

Even though we are interacting mostly with GitHub in this piece, we could have our bot connect to any service that has an API or a webhook. For example, we could pull incident data from a monitoring system like Datadog. In fact, we can even replace our GitHub webhook with CircleCI and have our bot notify members of a channel when a workflow is completed. With a ChatOps bot, there is a diverse range of possibilities that we can achieve.

Build Your ChatOps workflows with Mattermost

In this piece, we’ve seen how ChatOps is fundamentally about automating workflows with bots. With ChatOps powered by these bots, code deployments, system monitoring, and more could be automated right from the chat room. ChatOps has created a new way of collaboration with the chat room becoming the central place where powerful automated workflows are initiated. 

While building custom bots from scratch could require a significant amount of time for complex workflows, Errbot and similar frameworks were created to streamline the task of building ChatOps bots. Errbot’s built-in support for multiple chat applications makes creating a bot that interacts with popular chat applications a breeze.

If you’re interested in getting started with ChatOps for your team, why not try building a bot like this one in a free Mattermost Cloud server?

Read More Engineering Articles