Kubernetes Wake-up Operator
December 10, 2018
The majority of digital companies plan nowadays their cloud journey. The triggering factor is the promise of reduced cost of IT infrastructure ownership and day-to-day maintenance, backed up by nearly infinite scaling possibilities. To fully laverage cloud flexibility, applications need to scale dynamically adapting to real-time traffic. Companies tend to improve mission ciritcal, high volume applications enabling them to scale automatically. They tend to forget about low hanging fruit to reduce overall AWS bill - components that run sporadically, for example once per week.
Let us imagine a reporting process which is triggered by JMS message. On average CEO requests hereby report once per month. It would be a costly waste of resources to keep the container running in 24/7 mode. MacroNova team developed Kubernetes Wake-up operator which can start or stop any container when given event occurs. Currently we support trigger of TIBCO EMS message, but Apache Kafka is next on the roadmap!
Below diagram presents sample JMS service deployed in Kubernetes cluster. Mentioned service listens on EMS queue queue.sample for requests. JMS client has been configured to use client acknowledge mode not to loose any not successfully processed message.
Figure 1: Sample JMS service deployed in Kubernetes.
First step would be to deploy Wake-up Operator.
$ kubectl apply -f operator.yaml
Listing 1: Deploy Wake-up Operator.
Since Wake-up operator will establish new connection to EMS server, let us configure new account with limited permissions. Provided user needs to be able to view server details (connect only to active instance from fault-tolerant pair), view destination statistics, and optionally list topic durable subscriptions. Connect using tibemsadmin64 tool to EMS server and execute below commands.
tcp://localhost:7222> create user wakeup-operator password=secret tcp://localhost:7222> grant admin user=wakeup-operator view-server,view-destination,view-durable
Listing 2: Create dedicated EMS user for Wake-up Operator.
Finally, we need to create custom Kubernetes resource that will tell Wake-up operator which container to start (or stop) upon message on given EMS destination. Of course single operator deployment can manage multiple applications, connected to different EMS instances and listening on dedicated queues.
$ cat cr.yaml apiVersion: com.supernova/v1 kind: Wakeup metadata: name: sample-jms-service-wakeup spec: objectName: sample-jms-service objectKind: Deployment emsTrigger: serverUrl: tcp://ems-server1:7222,tcp://ems-server2:7222 username: wakeup-operator password: secret destinationNames: [ "queue.sample" ] $ kubectl apply -f cr.yaml wakeup.com.supernova/sample-jms-service-wakeup created
Listing 3: Deploy custom resource to manage sample JMS service.
After a minute you should see that sample JMS service has been scaled down, because there was no message pending on queue.sample.
$ kubectl logs -l name=wakeup-operator 2018-11-24 17:43:42 INFO WakeupOperator:53 - Scaling application [sample-jms-service] down $ kubectl get deployments NAME READY UP-TO-DATE AVAILABLE AGE hello-minikube 1/1 1 1 5d20h sample-jms-service 0/0 0 0 31m wakeup-operator 1/1 1 1 19m
Listing 4: Operator terminating idle application.
Next, let us send dummy message to EMS queue using GEMS utility. Wait for a minute or so and you should see that container with sample JMS service has been started.
$ kubectl logs -l name=wakeup-operator 2018-11-24 17:43:42 INFO WakeupOperator:53 - Scaling application [sample-jms-service] down 2018-11-24 17:47:42 INFO WakeupOperator:53 - Scaling application [sample-jms-service] up $ kubectl get deployments NAME READY UP-TO-DATE AVAILABLE AGE hello-minikube 1/1 1 1 5d20h sample-jms-service 1/1 1 1 33m wakeup-operator 1/1 1 1 22m
Listing 5: Operator starting the container.
Once the message is successfully processed and acknowledged, Wake-up operator scales down the service again.
$ kubectl logs -l name=wakeup-operator 2018-11-24 17:43:42 INFO WakeupOperator:53 - Scaling application [sample-jms-service] down 2018-11-24 17:47:42 INFO WakeupOperator:53 - Scaling application [sample-jms-service] up 2018-11-24 17:49:42 INFO WakeupOperator:53 - Scaling application [sample-jms-service] down $kubectl get deployments NAME READY UP-TO-DATE AVAILABLE AGE hello-minikube 1/1 1 1 5d20h sample-jms-service 0/0 0 0 35m wakeup-operator 1/1 1 1 23m
Listing 6: Operator terminating idle application.
Full reduction of infrastructure costs requires not only scaling down idle applications, but also setup of K8s auto-scaler which is part of standard Kubernetes installation. Utility removes unused nodes from the cluster and therefore releases computing resources back to cloud provider. Auto-scaler is also capable of adding servers if new pod cannot fit on any node in the cluster.
If you would like to try beta preview of Wake-up operator, do not hesitate to contact us at email@example.com.