The documentation you're currently reading is for version 2.4.1. Click here to view documentation for the latest stable version.

Upgrade Notes

BWC v2.4

BWC v2.3

  • The dest_server parameter has been removed from the linux.scp action and replaced with the destination parameter.

    This offers more flexibility. source and destination parameters can now contain a local path or a full source/destination which includes the server part (e.g. server.fqdn:/etc/hosts).

  • The API endpoint for searching or showing packs has been updated to return an empty list instead of None when the pack was not found in the index. This is technically a breaking change, but a necessary one because returning None caused the client to throw an exception.

  • Notifier now consumes the ActionExecution RabbitMQ exchange with queue name Notifier used to scan the LiveAction exchange with queue name. When you upgrade from BWC versions older than v2.3, make sure the queue size is 0 before upgrading. If you upgrade when it’s non-empty, you might miss notifications. Post-upgrade, please delete the queue manually using rabbitmqadmin delete queue If this is not done, the queue will grow indefinitely and RabbitMQ will consume large amounts of disk space. See issue 3622 for details.

  • Introduced a backward incompatible change (PR #3514) in the st2client API query() method. It returns a tuple of (result, total_number_of_items) instead of result. This is fixed in v2.3.2. Upgrade to v2.3.2 if you are seeing errors similar to those in issue #3606 or if you are using the st2client API’s query() method.

BWC v2.2

  • Additional validation has been introduced for triggers.

    1. Trigger payload is now validated against the trigger payload_schema schema when dispatching a trigger inside the sensor.

      Validation is only performed if the system.validate_trigger_payload config option is enabled and if the trigger object defines a payload_schema attribute.

    2. Trigger parameters are now validated for non-system (user-defined) triggers when creating a rule.

      Validation is only performed if the system.validate_trigger_parameters config option is enabled and if the trigger object defines parameters_schema attribute.

    Both of these configuration options are disabled by default with v2.2. In future they will be enabled by default.

  • The database schema for Mistral has changed. The executions_v2 table is no longer used. The table has been broken down into workflow_executions_v2, task_executions_v2, and action_executions_v2. After upgrade, using the Mistral CLI commands such as mistral execution-list will return an empty table. The records in executions_v2 have not been deleted. The commands are reading from the new tables. There is currently no migration script to move existing records from executions_v2 into the new tables. To read from executions_v2, either use psql or install an older version of the python-mistralclient in a separate Python virtual environment.

  • If you’re seeing an error event_triggers_v2 already exists when running mistral-db-manage upgrade head, this means the mistral services started before the mistral-db-manage commands were run. Refer to this procedure to recover the system.

  • Jinja notations {{user.key}} and {{system.key}} to access datastore items under user and system scopes are now unsupported. Please use {{st2kv.user.key}} and {{st2kv.system.key}} notations instead. Also, please update your BWC content (actions, rules and workflows) to use the new notation.

  • When installing StackStorm using the installer script a random password is generated for MongoDB and PostgreSQL. This means you now need to explicitly pass the --config-file /etc/st2/st2.conf argument to all st2 CLI scripts (e.g. st2-apply-rbac-definitions) which need access to the database (MongoDB). If you don’t do that, “access denied” error will be returned, because it will try to use a default password when connecting to the database.

    st2-apply-rbac-defintions --config-file /etc/st2/st2.conf

    If you need access to the plain-text version of the password used by StackStorm services to talk to MongoDB and PostgreSQL, you can find it in /etc/st2/st2.conf ([database] section) /etc/mistral/mistral.conf ([database] section) files.

BWC v2.1

  • WARNING: The following changes may require you to update your custom packs during the upgrade.

    • The version attribute in pack.yaml metadata must now contain a valid semver version string (<major>.<minor>.<patch>, e.g. 1.0.1). In addition, the email attribute must be a valid email address.
    • Pack ref and action parameter names can now only contain valid word characters (a-z, 0-9 and _). No dashes! hpe_icsp is ok, but hpe-icsp is not.

    The st2ctl and st2-register-content scripts are now doing additional validation. If you happen to have a pack which doesn’t satisfy these new validation criteria, it will fail to load. Therefore, to upgrade BWC from v2.0.* to 2.1.*, follow these steps:

    1. Use yum or apt-get to upgrade to the newest version.
    2. Update community packs to the latest version from StackStorm Exchange with st2 pack install <pack>.
    3. Reload the content with st2ctl reload --register-all.
    4. If you have packs that don’t satisfy the rules above, validation fails and the pack load will throw errors. Fix the packs to conform to the rules above, and reload the content again.

    In 2.1.0, BWC attempts to auto-correct some validation failures and display a warning. In a future release this auto-correction will be removed. Please update your packs ASAP.

  • st2contrib is now deprecated and replaced by StackStorm Exchange . All the packs from st2contrib have been migrated to StackStorm Exchange. For more information see Pack Management Transition.

  • Pack “subtree” repositories (repositories containing multiple packs inside the packs/ subdir) are no longer supported. The subtree parameter in packs.install is removed. The new convention is one pack per git/GitHub repo. If you happen to use subtrees with your private packs, they will have to be split into multiple single-pack repositories in order for st2 pack install to be able to install the packs.

  • The packs pack is deprecated starting from 2.1; in future versions it will be completely replaced with the st2 pack <...> commands and API endpoints.

  • Pack metadata file (pack.yaml) can now contain a new ref attribute, in addition to name. ref acts as a unique identifier; it offers for a more readable name. For example, if a pack name is Travis CI, a repo containing it is stackstorm-travis_ci, and ref is travis_ci. Previously the pack files would live in travis_ci/ directory and pack directory name served as a unique identifier for a pack.

  • Support for .gitinfo file has been removed and as such the action has also been removed. All the pack directories at /opt/stackstorm/packs are now direct git checkouts of the corresponding pack repositories from Exchange or your own origin, so this file is not needed anymore.

  • Datastore scopes are now st2kv.system and st2kv.user as opposed to system and user. If you are accessing datastore items in your content, you should now use the Jinja expressions {{}} and {{}}. The older Jinja expressions {{}} and {{}} are still supported for backward compatibility but will be removed in future releases.

  • Runners are now pluggable. With this version, we are piloting an ability to register runners just like other BWC content. You can register runners by simply running st2ctl reload --register-runners. This feature is in beta. No backward compatibility is guaranteed. Please wait for a release note indicating general availability of this feature.

  • Config schemas now also support nested objects. Previously config schema and configuration files needed to be fully flat to be able to utilize default values from the config schema and dynamic configuration values.

    The config schema file can now contain arbitrary levels of nesting of the attributes and it will still work as expected.

    Old approach (flat schema):

        description: "API server host."
        type: "string"
        required: true
        secret: false
        description: "API server port."
        type: "integer"
        required: true
        description: "API server token."
        type: "string"
        required: true
        secret: true
        description: "Auth server host."
        type: "string"
        required: true
        secret: false
        description: "Auth server port."
        type: "integer"
        required: true

    New approach (nested schemas are supported):

        description: "API related configuration options."
        type: "object"
        required: false
        additionalProperties: false
            description: "API server host."
            type: "string"
            required: true
            secret: false
            description: "API server port."
            type: "integer"
            required: true
            description: "API server token."
            type: "string"
            required: true
            secret: true
        description: "Auth API related configuration options."
        type: "object"
        required: false
        additionalProperties: false
            description: "Auth server host."
            type: "string"
            required: true
            secret: false
            description: "Auth server port."
            type: "integer"
            required: true

BWC v2.0

  • st2ctl reload now also registers rules by default. Prior to this release actions, aliases, sensors, triggers and configs were registered. Now rules are also registered by default.

BWC v1.6

  • Python runner actions can now return execution status (success, failure) by returning a tuple from the Python action class run() method. The first item in this tuple is a boolean flag indicating success or failure and the second one is the result. For example:

    def run(self):
        # Code to do something awesome
        if something_awesome_working == True
            return (True, result)  #  Succeeded is True and the result from action on success
        return (False, result)  #  Succeeded is False and the result from action on failure

    This allows users to also return a result from a failing action. This result can then be used in workflows, etc. Previously this was not possible since the only way for action to be considered as failed was to throw an exception or exit with a non-zero exit code.

    Note: This change is fully backward compatible unless you have an existing action which returns a tuple with two items.

    For existing actions which don’t return a status flag, the same rules apply as before - an action is considered successful unless it throws an exception or exits with a non-zero exit code.

    If you have an existing action which returns a tuple with two items such as the one shown in the example below, you have two options:

    def run(self):
        result = ('item1', 'item2')
        return result
    1. Update action to return a list instead of a tuple.

      def run(self):
          result = ('item1', 'item2')
          return list(result)


      def run(self):
          result = ['item1', 'item2']
          return result
    2. Update action to also return a status.

      def run(self):
          result = ('item1', 'item2')
          return (True, result)

BWC v1.5

  • The previously deprecated Fabric-based remote runner has been removed. This means ssh_runner.use_paramiko_ssh_runner config option is now obsolete.

  • Underscore (_) prefix has been removed from the sensor_service and config variable available on the Sensor and PollingSensor class. Those variables are now available via self.sensor_service and self.config respectively.

    For backward compatibility reasons and ease of migration, the old approach will still work, but you are encouraged to upgrade your sensors to use the new way of referencing those variables.

  • Support for loading content (sensors, actions and rules) from .json files has been removed. Support for JSON was deprecated a long time ago and now the only supported format is YAML files with .yaml extension).

    If you want to directly save content which you retrieve from the API using CLI on disk, you can now use the --yaml flag with the list and get CLI commands (e.g. st2 rule get <rule ref> --yaml > packs/<my pack>/my_rule.yaml).

  • Pack config files located inside the pack directory (config.yaml) have been deprecated in favor of the new pack configuration v2. This new configuration approach offers more flexibility. In addition, those new config files are located outside the pack directory, in the /opt/stackstorm/configs/ directory. This makes it easier to follow an infrastructure as code approach. Updating packs is also easier, as users don’t need to directly manipulate pack content anymore.

    For more information about the new pack configuration, please see Pack Configuration.

  • New log attribute has been added to the action execution object. This attribute is a list and contains all the state (status) transitions for executions (e.g. requested -> scheduled -> running -> complete, etc.).

    Keep in mind that this attribute will only be populated for new execution objects (those created after the upgrade to v1.5).

  • The datastore data model has changed. We’ve introduced the notion of scope and secret. See Scoping items in datastore and storing secrets in datastore for details.

    A migration tool is provided (/opt/stackstorm/st2/bin/ if you are upgrading from older versions.

BWC v1.4

  • matchregex rule criteria operator has been updated so now the dot character (.) also matches a new line. This makes the existing criteria patterns which use dot character more greedy. Previously, it didn’t match new lines so some of the existing matchregex criteria patterns which operate on multi line strings might be affected.

    For example, let’s say we have the following criteria pattern - .*stackstorm.*. Previously, the following string - test\nstackstorm\ntest would not match, but now it does.

    If you are affected and you want to revert to the old behavior (less greedy matches), you can do so by modifying the criteria pattern regular expression so it’s less greedy (e.g. by adding ^ and/or $ character or similar).

    matchregex is now deprecated in favor of regex and iregex operators.

  • regex and iregex been added to the rule criteria operators list. These behave like'pattern', trigger_value) and'pattern',trigger_value, re.IGNORECASE) in Python. They do not have the DOTALL modifier. To match newline characters, they must be explicit in the search pattern.

  • To make working with non-string positional parameters in the local and remote runner script actions easier, simple new rules for parameter value serialization have been established. Previously all the values were serialized as Python literals which made all the parameters with type other than string very hard to parse and use in the script actions.

    More information about new positional parameter serialization rules can be found in the documentation.

  • The list of required and optional configuration arguments for the LDAP authentication backend has changed. The LDAP authentication backend supports other login name such as sAMAccountName. This requires a separate service account for the LDAP backend to query for the DN related to the login name for bind to validate the user password. Also, users must be in one or more groups specified in group_dns to be granted access.

  • Mistral has deprecated the use of task name (i.e. $.task1) to reference task result. It is replaced with a task function that returns attributes of the task such as id, state, result, and additional information (i.e. task(task1).result).

BWC v1.3

  • New abandoned action execution status has been introduced. State is applied to action execution when an actionrunner currently running some executions quits or is killed via TERM. This is therefore effectively a failure state as BWC can no longer validate the state of this execution. Being a failure state, any code that checks for an action failure should be updated to check for abandoned state in addition to failed and timeout.

BWC v1.2

  • Refactor retries in the Mistral action runner to use exponential backoff. Configuration options for Mistral have changed. The options max_attempts and retry_wait are deprecated. Please refer to the configuration section of docs for more details.

  • Change headers and params parameters in the core.http action from string to object. If you have any code or rules that call this action, you need to update it to pass in a new and correct type.

  • Local runner has been updated so all the commands which are executed as a different user and result in using sudo set $HOME variable to the home directory of the target user. Previously, the $HOME variable reflected the home directory of the user which executed sudo and under which action runner is running.

    Keep in mind that this condition is only met if action runner is running as root and/or if action runner is running a system user (stanley) and a different user is requested when running a command using user parameter.

  • Support of default values is added to the API model. As a result, input parameters defined in the action metadata that is of type string no longer supports None or null.

  • New timeout action execution status has been introduced. This status is a special type of a failure and implies an action timeout.

All the existing runners (local, remote, python, http, action chain) have been updated to utilize this new status when applicable. Previously, if an action timed out, status was set to failed and the timeout could only be inferred from the error message in the result object.

If you have code which checks for an action failure you need to update it to also check for timeout in addition to failed status.

Upgrading from 1.1

To upgrade a pre-1.2.0 StackStorm instance provisioned with the All-in-one Installer, you will need to perform the following steps:

  1. Back up /opt/puppet/hieradata/answers.json.
  2. Update (or insert) the following lines in /opt/puppet/hieradata/answers.yaml:
st2::version: 1.2.0
st2::revision: 8
st2::mistral_git_branch: st2-1.2.0
hubot::docker: true

If answers.yaml does not exist, create it. If you changed any install parameters manually (e.g. password, ChatOps token, SSH user), put these values into answers.yaml as well, otherwise they’ll be overwritten.

  1. If you’re running ChatOps, stop the Hubot service with service hubot stop.
  2. Remove /etc/facter/facts.d/st2web_bootstrapped.txt and execute update-system:
sudo rm /etc/facter/facts.d/st2web_bootstrapped.txt
sudo update-system
  1. After the update is done, restart BWC and hubot:
sudo st2ctl restart
sudo service docker-hubot restart

To verify the upgrade, please follow the link to run the self-verification script.

BWC v1.1

Migrating to v1

The st2_deploy scripted installer will upgrade v0.13 to v1.1. However we encourage you to switch to All-in-one Installer. To migrate to new All-in-one deployment from existing pre-v1.1 installations:

  1. Install BWC on a new clean box with All-in-one Installer.
  2. Copy the content from the previous installation to /opt/stackstorm/packs and reload it with st2ctl reload --register-all.
  3. Adjust the content according to upgrade notes below. Test and ensure your automations work.
  4. Save the audit log files from /var/log/st2/*.audit.log for future reference. We do not migrate execution history to the new installation, but all the execution data is kept in these structured logs for audit purpose.


Don’t run the All-in-one installer over an existing BWC deployment.


  • Triggers now have a ref_count property which must be included in Trigger objects created in previous versions of BWC. A migration script is provided at ${dist_packages}/st2common/bin/ The migration script is run as part of when you upgrade from versions >= 0.13 to v1.1.

  • Messaging queues are now exclusive and in some cases renamed from previous versions. To remove old queues run the migration script ${dist_packages}/st2common/bin/ after installation. The migration script is run as part of when you upgrade from versions >= 0.13 to v1.1.

  • Mistral is updated to YAQL v1.0. Earlier versions of YAQL are deprecated. Expect some minor syntax changes to YAQL expressions.

  • Mistral has implemented new YAQL function for referencing environment variables in the data context. The env() function replaces $.__env when referencing the environment variables. For example, $.__env.st2_execution_id becomes env().st2_execution_id.

    WARNING: Referencing $.__env will lead to YAQL evaluation errors! Please update your workflows accordingly.

  • Mistral has implemented new YAQL function for referencing task result. Given task1, the function call task(task1).result, replaces $.task1 when referencing the result of task1. The old reference style will be fully deprecated in the next major release of Mistral (the OpenStack Mitaka release cycle).

BWC v 0.11

  • Rules now have to be part of a pack. If you don’t specify a pack, the pack name is assumed to be default. A migration script is installed at ${dist_packages}/st2common/bin/ This migration script is run as part of when you upgrade from versions < 0.9 to 0.11.

BWC v0.9

  • Process names for all BWC services now start with st2. sensor_container now runs as st2sensorcontainer, rules_engine runs as st2rulesengine, actionrunner now runs as st2actionrunner. st2ctl has been updated to handle the name change seamlessly. If you have tools that rely on the old process names, upgrade them to use the new names.

  • All BWC tools now use the st2 prefix as well. rule_tester is now st2-rule-tester, registercontent is now st2-register-content.

  • Authentication is now enabled by default for production (package based) deployments. For information on how to configure this, see Authentication.

  • For consistency reasons, the runners have been renamed:

    • run-local -> local-shell-cmd
    • run-local-script -> local-shell-script
    • run-remote -> remote-shell-cmd
    • run-remote-script -> remote-shell-script
    • run-python -> python-script
    • run-http -> http-request

    Note: For backward compatibility reasons, those runners are still available and can be referenced through their old names, but you are encouraged to update your actions to use the new names.