Note

The documentation you're currently reading is for version 2.3.0. Click here to view documentation for the latest stable version.

Packs

What is a Pack?

Pack is the unit of deployment for integrations and automations that extend BWC. Typically a pack is organized along service or product boundaries e.g. AWS, Docker, Sensu etc. A pack can contain Actions, Workflows, Rules, Sensors, and Aliases. StackStorm content is always part of a pack, so it’s important to understand how to create packs and work with them.

Some packs extend BWC to integrate it with external systems — like AWS, GitHub, or JIRA. We call them “integration packs”. Some packs capture automation patterns — they contain workflows, rules, and actions for a specific automation process — like st2 demo pack. We call them “automation packs”. This naming is mostly a convention: StackStorm makes no distinction between the two.

Integration packs can be shared and reused by anyone who uses a service the pack is meant for, and you will find a lot of those on StackStorm Exchange. Automation packs are often very specific and have little use outside of a particular team or company; they are usually shared internally.

Managing Packs

Note

Everything about packs got better and easier in BWC 2.1! There are new API endpoints, CLI commands, repository structure for hosting packs, and an awesome pack collection on StackStorm Exchange. To make it happen, a few things had to be deprecated. Check the upgrade notes and Pack Management Transition.

BWC packs are managed through st2 pack <...> commands: st2 pack -h will give you a useful overview.

A few packs (such as the core pack for basic StackStorm actions) come pre-installed with StackStorm. list and get are the primary commands to get information about local packs:

# List all installed packs
st2 pack list

# Get detailed information about an installed pack
st2 pack get core

When using BWC and pack management actions, all the packs are installed into the system packs directory which defaults to /opt/stackstorm/packs.

Discovering Packs

There’s over a hundred StackStorm packs already available to you! StackStorm Exchange is a collection of ready-made packs submitted and maintained by the StackStorm community. There are packs for most of the popular cloud providers and DevOps tools, as well as more peculiar integrations (hello, tesla and hue).

You can browse the pack listing at exchange.stackstorm.org, or search the pack index in CLI with st2 pack search and st2 pack show:

# Search query is applied across all pack parameters.

# It will search through pack names:
st2 pack search sensu
# And keywords:
st2 pack search monitoring
# And description (use quotes for multi-word search):
st2 pack search "Amazon Web Services"
# And even pack author:
st2 pack search "Jon Middleton"

# Show an index entry for the pack
# with the exact name match
st2 pack show sensu

Installing a Pack

Installing a pack is simple:

# Install from the Exchange by pack name
st2 pack install sensu

# You can also install multiple packs:
st2 pack install datadog github

This command will download packs from the StackStorm Exchange organization on GitHub, place them under /opt/stackstorm/packs, and register them with BWC.

Essentially, st2 pack install works with git repositories: there is one for every pack in the Exchange, and you can install your own packs from git just as easily.

# Install your own pack from git
st2 pack install https://github.com/emedvedev/chatops_tutorial

By default, the latest version of a pack will be installed, but you can specify a particular version, branch, tag, or even a commit hash. Exact version with = is required.

# Fetch a specific commit
st2 pack install cloudflare=776b9a4

# Or a version tag
st2 pack install cloudflare=0.1.0

# Or a branch
st2 pack install https://github.com/emedvedev/chatops_tutorial=testing

Running st2 pack install on an already installed pack will replace it with the requested version or upgrade to latest if the version is not specified. Your config file will not be overwritten, so you can revert to an older version just as easily, but for production deployments we recommend to always specify versions in case there are major changes in latest.

To uninstall a pack, use remove:

st2 pack remove sensu

Configuring a Pack

Integration packs often require configuration to adjust to the environment. For example, you need to specify an SMTP server to use the email pack, a puppet master URL to use the Puppet pack, or a Keystone endpoint and tenant credentials for OpenStack.

Most packs that require configuration can be configured interactively:

st2 pack config cloudflare

You will be prompted for configuration parameters in an interactive tool with descriptions, suggestions, and defaults. You will also be asked to verify your final config file in a text editor before saving it; it’s optional, and most packs don’t require more than two or three fields, but we have to comply with Health and Safety. The generated file will be placed in /opt/stackstorm/configs/<pack>.yaml and loaded.

Warning

NB: BWC loads pack configuration into MongoDB. This is automatically loaded when you use st2 pack config. But if you manually edit your pack configuration, or use configuration management tools to manage those files, you must tell BWC to load the updated config.

You can do this with: `sudo st2ctl reload --register-configs. Otherwise there will be much head-scratching as you wonder why BWC seems to be completely ignoring your updated configuration.

Trust us. We’ve all been there.

For more nice tricks on pack configuration, see Pack Configuration.

Developing a Pack

See Create and Contribute a Pack for details on how to package your integrations and automations in a pack, how to fork a pack for development or create your own, how to publish it, and how to contribute it to the BWC community. If you’re planning to develop any BWC content, we would strongly suggest getting yourself familiar with that page: every piece of content in StackStorm has to belong to a pack, and a good understanding of pack workflow will make your development process much easier!

What’s Next?

Advanced Topics

Under the Hood: Pack Basics

The st2 pack commands described above is a convenience layer on top of pack basics. Once you understand the basics, you can work with the pack content directly, which might be preferred if you work with configuration management or have a very specific deployment workflow.

Packs are placed in a system pack directory which defaults to /opt/stackstorm/packs. A virtualenv needs to be created for each pack containing Python actions/sensors under /opt/stackstorm/virtualenv. Python dependencies are installed inside the virtualenv with pip -r requirements.txt.

When BWC loads the content, it looks into the system packs directory (/opt/stackstorm/packs) and any additional directories listed in packs_base_paths in st2.conf (typically in /etc/st2/st2.conf). The value must be a string of directory paths, separated by a colon (think $PATH). For example:

[content]
packs_base_paths=/home/myuser1/packs:/home/myuser2/packs

Directories are always searched from left to right in the order they are specified, with the system packs directory always searched first.

A pack configuration file must be stored as /opt/stackstorm/configs/<pack_ref>.yaml and follow a schema defined in /opt/stackstorm/packs/<pack_dir>/config.schema.yaml. See the Pack Configuration section for details.

When the pack content changes, it has to be registered again (reloaded). To register individual packs, use st2 pack register --packs=pack1,pack2. To register everything at once, use st2ctl reload (run it with -h to explore the fine-tuning flags).

Installing packs from private repositories

If you’re installing a pack from a private repository on GitHub, you can use HTTPS auth with a personal access token, or create a deploy key to use SSH.

Access tokens are used with HTTPS URLs: st2 pack install https://<user>:<token>@github.com/username/repo.git. Be advised that your token will be logged in StackStorm, git, your shell history, and probably some other log files, including git error logs if anything fails. Using SSH auth is a much better idea most of the time.

For SSH (URLs starting with git@) auth you have to create a deploy key, and require the system user running the command (stanley or root, depending on your configuration) to have a private key. Deploy keys are more secure than personal access tokens and can be configured on the per-repo basis.

Other git hosting services should also support either SSH or HTTPS auth, and would be configured in a similar fashion.

Hosting your own pack index

When you run pack management commands like st2 pack install sensu or st2 pack search git, BWC uses a pack index file to resolve short names and perform searches. A pack index is, essentially, a JSON object: it contains metadata and URLs for all available packs.

The StackStorm Exchange index file is a default file used by all BWC instances, and a good example of the index format. The file is hosted on GitHub (StackStorm-Exchange/index) and proxied through CloudFlare CDN.

The index path is specified in st2.conf as content.index_url. You can replace the default index, or even use more than one with a comma-separated list:

[content]
index_url=https://my-super-index.org/index.json,https://exchange.stackstorm.org/v1/index.json

Contents from all specified indexes will merge with descending priority (left to right). In the example above, sensu pack in your own index would override sensu pack from the Exchange.

There are multiple reasons to consider hosting your own index, especially with HA deployments or multi-server setups:

  • mirroring: in case the main index is not available, your mirror will be used;
  • forking: if you fork Exchange packs often, you can create an index that is going to list your forks;
  • enterprise restrictions: if you need pack names to resolve, but can’t install from github, you can specify your own index as the only source;
  • a centralized resolver: in a multi-server deployment, you can host an index to keep repo URLs in a centralized location;
  • bragging rights: get your own packs resolvable by short names because the cool kids are doing it.

In most cases there are many other ways to solve the same problem, but sometimes a pack index is a viable alternative. Create your index file and make it accessible over HTTPS, change st2.conf—and you’re good!

When you have to monitor index health, the /packs/index/health API endpoint will show you the state of all indexes used by your BWC instance.

Questions? Problems? Suggestions? Engage!