Note

The documentation you're currently reading is for version 2.3.0. Click here to view documentation for the latest stable version.

Traces

Traces are tracking entities that serve to gather all BWC entities like ActionExecution, TriggerInstance and Rule that originate from an event. In the BWC context an event could be one of the following:

  • Events from an external system sent to BWC via a Sensor or Webhook.
  • Action executed via UI, CLI or API.
  • Action executed via ChatOps.

Examples

Let us walk through a few canonical examples:

External events

Sensors dispatch TriggerInstances into BWC and Webhooks are also translated to TriggerInstances when posted to BWC. Rules are written to match specific Triggers and compared against TriggerInstances.

In the canonical case, TriggerInstance(ti1) dispatched by Sensor to BWC, matches a Rule(r1) leading to an ActionExecution(ae1). On completion of ae1 an ActionTrigger TriggerInstance(ti2) is dispatched by BWC.

The trace created in this case contains all the entities from above since they originate from the same event i.e. TriggerInstance(ti1) dispatched into BWC.

Trace
  |- ti1
  |- r1
  |- ae1
  |- ti2

Connected flows

BWC raises an internal Trigger called the ActionTrigger. It is possible for rules to be used in conjunction with those on completion of executions.

ActionExecution(ae1) started by user, on completion of ae1 an ActionTrigger TriggerInstance(ti1) is dispatched, Rule(r1) matches and leads to ActionExecution(ae2) another ActionTrigger TriggerInstance(ti2) is dispatched but no rule matched.

The trace created in this case contains all the entities from above since they cascade from the same origin i.e. ActionExecution(ae1) dispatched into the system.

Trace
  |- ae1
  |- ti1
  |- r1
  |- ae2
  |- ti2

Tracing Triggers and Executions

It is possible for users to define identifying information for a Trace at event injection points. The injection points for BWC where a Trace can start are:

  • Dispatch a Trigger (more precisely this is dispatching a TriggerInstance) by a Sensor.
  • Webhook posted to BWC.
  • Execute an Action (aka creation of an ActionExecution) via UI, CLI, API or Chat.

What is a trace_tag and trace_id?

  • trace-tag : User specified and therefore friendly way to tag or identify a Trace. There is no requirement for this value to be unique and BWC will not enforce this either. Whenever only a trace-tag is provided at one of the injection points a new Trace is started if one does not already exist.
  • trace-id : This is a BWC defined value and is guaranteed to be unique. Users can specify this value at the injection points as all but a Trace with the specified trace-id must already exist.

Dispatch a Trigger

TriggerInstance dispatch most often happens from a Sensor. The authoring a sensor page contains information on how to introduce a Trace.

A brief snippet is included here to explain some trace specific constructs. A sensor would inject such triggers by using the sensor_service passed into the sensor on instantiation:

self.sensor_service.dispatch(trigger=trigger, payload=payload, trace_tag=trace_tag)

Here the Sensor is expected to supply a meaningful value for trace_tag e.g.:

  • Commit SHA of a git commit for a git commit hook trigger.
  • ID of the event from a monitoring system, like Sensu or Nagios, that is relayed to BWC.

Webhook

Both custom webhooks and generic BWC webhooks support supplying a trace-tag via a header.

  • Header : St2-Trace-Tag

In case of a custom webhook the curl command will be:

curl -X POST http://127.0.0.1:9101/v1/webhooks/sample -H "X-Auth-Token: matoken" -H "Content-Type: application/json" -H "St2-Trace-Tag: webhook-1" --data '{"key1": "value1"}'

Execute an Action

Execution of an Action can also be associated with a Trace. Here is how this could be done from the CLI:

To start a new trace use trace-tag:

$ st2 run core.local date --trace-tag TraceDateAction

To associate with an existing trace use trace-id:

$ st2 run core.local uname --trace-id 55d505fd32ed35711522c4c8

Viewing Traces

BWC CLI provides the ability to list and get traces.

List

  • All traces in the system:
$ st2 trace list
  • Filter by trace-id:
$ st2 trace list --trace-tag <trace-tag>
  • Filter by execution:
$ st2 trace list --execution 55d505fd32ed35711522c4c7
  • Filter by rule:
$ st2 trace list --rule 55d5064432ed35711522c4ca
  • Filter by trigger-instance:
$ st2 trace list --trigger-instance 55d5069832ed35711cc4b08e

Get

  • Get a specific trace:
$ st2 trace get <trace-id>
  • View the causation chain in a trace for an action execution. Similarly for rule and trigger-instance:
$ st2 trace get <trace-id> -e
  • View specific type in a trace:
$ st2 trace get <trace-id> --show-executions
  • Hide no-op trigger instances. These are trigger instances which do not lead to a rule enforcement:
$ st2 trace get <trace-id> --hide-noop-triggers

Is everything traced?

By default all ActionExecutions and TriggerInstances are traced. If no trace-tag is provided by a user then BWC automatically generate a trace-tag to provide tracking.