Monitoring TIBCO BusinessWorks with Prometheus

Author: MacroNova

November 1, 2018

Purpose of hereby tutorial is to show case how to monitor performance of simple REST service implemented in TIBCO BusinessWorks Container Edition using Prometheus. As a key component enabling Prometheus server to periodically pull metrics from BWCE container, we will leverage BW6 Prometheus extension available on GitHub. Mentioned tool hooks into BusinessWorks event bus and exposes common JVM statistics, as well as configured process and activity-level metrics via HTTP endpoint.

Let us take a quick look at the architecture of our system. REST service persists information about money transfers into relational database. BWCE container exposes HTTP endpoint which is being periodically accessed by Prometheus server. End user interacts with Grafana to view pre-defined dashboards populated with data from Prometheus.

Monitoring BWCE with Prometheus architecture.
Figure 1: Architecture of BWCE monitoring solution with Prometheus.

Download tutorial archive which contains Docker Compose manifest setting up complete environment within seconds. Unzip the archive and import BusinessWorks project into TIBCO BusinessStudio. Take your time to navigate through processes. The purpose of Report job is to show how to adjust Prometheus histogram buckets for long running processes.

bwce-prometheus-tutorial.zip
├── Dockerfile                                              # Docker file resopnsible for building BWCE application image
├── docker-compose.yaml                                     # Docker Compose manifest that provisions MySQL, BWCE, Prometheus and Grafana
├── ear
│   └── samples.bwce.prometheus.application_1.0.0.ear       # BWCE Application EAR
├── resources
│   ├── addons
│   │   ├── config
│   │   │   └── prometheus-reporter.json                    # BWCE Prometheus Exporter configuration
│   │   └── jars
│   │       ├── com.tibco.bw.jdbc.datasourcefactory.mysql   # Standard way to install MySQL JDBC driver inside BWCE image
│   │       └── io.macronova.tibco.bw6.prometheus.event.subscriber_1.0.0.201811090250.jar   # BWCE Prometheus Exporter binary
│   ├── grafana                                             # Grafana scripts that create sample dashboard and Prometheus datasource
│   ├── mysql                                               # MySQL scripts provisioning schema for BWCE application
│   └── prometheus                                          # Prometheus configuration
├── samples.bwce.prometheus                                 # Sample BWCE application module
└── samples.bwce.prometheus.application                     # Sample BWCE application

Listing 1: Structure of tutorial archive with description.

Open file resources/addons/config/Prometheus-reporter.json in your favourite text editor. Configuration instructs Prometheus exporter to bind to port 1234 and expose metrics of all BW processes matching regular expression samples.bwce.prometheus.*. Moreover, we would like to instrument only JDBC activities (matched by name, not type). Histogram buckets of long running processes samples.bwce.prometheus.batch.* have been explicitly overridden to reflect expected execution duration more accurately.

{
   "httpPort": "1234",
   "includeProcesses": [
      "samples.bwce.prometheus.*"
   ],
   "includeActivities": [
      "samples.bwce.prometheus.*#JDBC.*"
   ],
   "processHistogramBuckets": [
      0.005, 0.01, 0.025, 0.05, 0.075, 0.1, 0.25, 0.5, 0.75, 1, 2.5, 5, 7.5, 10
   ],
   "activityHistogramBuckets": [
      0.002, 0.005, 0.01, 0.025, 0.05, 0.075, 0.1, 0.25, 0.5, 0.75, 1
   ],
   "processHistogramOverrides": [
      { "name": "samples.bwce.prometheus.batch.*", "buckets": [ 0.5, 1, 2, 5, 10, 60, 300, 600 ] }
   ]
}

Listing 2: Configuration of BW6 Prometheus Exporter.

Note

Further steps assume that you have basic BWCE image available as bwce:latest. If not, follow TIBCO documentation.

Let us run a simulation! Navigate to root folder of decompressed tutorial archive and execute docker build -t samples.bwce.prometheus:latest . to build BWCE application container, followed by docker-compose up which starts complete environment as described above. Open Swagger UI at http://localhost:8080 and trigger few requests to REST service. You can observe exposed Prometheus metrics accessing http://localhost:1234 or http://localhost:9090.

Alternatively, compose simple script and run it periodically as shown in the listing below.

tibco@localhost:/tmp$ cat test.sh 
curl -X POST \
    --header 'Content-Type: application/json' \
    --header 'Accept: application/json' \
    -d '{"Debtor": "Mark","Creditor": "Anna","Amount": 1.2}' \
    http://localhost:8080/v1/transfer
tibco@localhost:/tmp$ chmod a+x test.sh
tibco@localhost:/tmp$ watch -n1 ./test.sh

Listing 3: Generate one request per second to BWCE API.

After a while open Grafan dashboard called BWCE Performance (htttp://localhost:3000) to review performance of your microservice. Example dashboard displays total number of requests and errors per minute, as well as latency of REST endpoint.

BWCE Grafana Dashboard.
Figure 2: Example Grafana dashboard presenting BW6 process metrics.

Once you finish experimenting, shutdown Docker Compose environment (hit Ctrl+C) and clean persistent volumes by executing docker-compose down --volumes. Make sure you read documentation of BusinessWorks Prometheus Exporter for detailed description of all features and installation guide. Contributions, ideas and pull requests are welcome!