Monitoring Custom Applications Within the BMC Performance Manager Framework

BMC Performance Manager framework offers various options for monitoring custom applications and components or technologies:

When an IT infrastructure states that it is using the BMC Performance Manager framework for monitoring, it automatically implies that it is using BMC Performance Manager for Servers (since it is the main product) and then, in addition, there would be other different KMs according to their specific needs.

Acquisition of BMC Performance Manager for Servers brings not only the monitoring framework, but also a few monitoring tools to help monitor custom applications. These features come for free and include:

PROS —

Most applications generate log files to record operations and errors. Checking the presence of the various processes and parsing log files for errors is often considered as an alternative way of monitoring an application without developing a custom KM dedicated to it. This feature is free.

CONS —

Relying only on information gained through processes and log files monitoring features of BMC Performance Manager for Servers is more than often considered as a non-ideal solution because of several limitations:

•

General: Inability to group the monitoring of processes, services and logs application-wise under a single icon in the Console prevents rapid problem-identification. Several alerts could be raised, but the operators have to study each alert in thorough detail to know which applications they stem from.

•

Log: Inability to analyze files with multi-lines entries or search for complex regular expressions. It has also great difficulties dealing with very large log files. Another limitation of this log file monitoring feature is that it does not allow the administrator to raise alerts to the operators with the actual content of the log file in the event, thus leaving the operator with a meaningless event stating that “an error” was reported in the log file. The only way to find the root cause is for the operator to sift through the log file line by line to see what the problem is.

•	Log: Relying only on log files can be dangerous: for instance if nothing is recorded in the log file, it could either imply that everything is working properly or that the application is completely frozen.

•

General: Relying on processes and log files monitoring leaves out more recent applications composed of a Web front-end, a database back-end and often a Java Application Server in the middle. Such applications do not have specific processes and most often do not produce any log files. Ensuring the availability of the Web server and the database server does not necessarily mean that the application based on those infrastructure components is running properly (a dropped table in a database, for example, can cause an entire application to fail while the Web and database servers are just running fine).

•	General: There is no easy way to duplicate the monitoring setup for one application on one serve to another server running the same application, especially with the monitoring of processes. Things get even worse if the said application is installed in another directory than the original one.

•	General: Log files and processes monitoring do not allow administrators to monitor performance metrics and response times related to an application, which means that even a huge slow-down of an application or other IT component being monitored will not be reported.

These limitations in BMC Performance Manager for Servers often lead IT administrators to consider the development of custom monitoring solutions in order to be able to properly monitor applications for which there no KMs are available in the BMC Software’s catalog. This option is described in the following section.

The BMC PATROL framework provides the ability to the administrators to create custom monitoring solutions to cover their specific needs. This can be achieved with the BMC PATROL Developer Console that lets the user develop what is called a custom KM written in PATROL’s own scripting language: PSL.

PROS —

Developing a custom KM with PSL is logically the most flexible solution in order to cover a custom application because the developer has the liberty to tailor a monitoring solution that is specific to his custom needs. It can be done in-house and at any time.

CONS —

Developing a custom KM cannot really be considered as THE obvious choice because of several issues, besides, although at first it may seem more economically viable, on a deeper project planning, the hidden costs are revealed. In addition, a large and complex environment may require many such custom KMs….Here are some of the commonly stated difficulties encountered by customers who have ventured on this path, and no longer wish to repeat the experience.

•

Human resources: Developing a custom KM with PSL obviously requires developer skills, deep knowledge of the PATROL Agent’s internals and PSL skills, which means having people dedicated to the task. Apart from the cost of man-hours spent on the development, it is a fact that people with good PSL skills are hard to find as PSL is now considered as an aging technology and hence few developers invest in learning PSL.

•

Re-inventing the wheel with repetitive basic coding: Gathering the information needed to check the health of an application or other IT component often requires the KM developer to re-create code that has already been done many times by other professional developers. Having to re-invent the wheel to collect monitoring data is the most pitiful time wasting issue when developing a custom KM. For example: Executing an SQL query, sending an HTTP request or listening to SNMP traps, all generally considered as basic tasks, but which nevertheless require hundreds of well-thought lines of code, especially when using PSL.

•

GUI and Packaging: The pure PSL coding part is sometimes not the most time-consuming step when developing a custom KM. For proper use in production, a custom KM needs to be packaged for the various components of the PATROL framework (consoles, agents, Console Servers, Web Console, etc.). The packaging of a KM often implies dealing with time-consuming tasks such as creation of suitable graphical user interface, the icons, online help, proper platform targeting, etc. None of these tasks are automated in the PATROL framework and require a thorough knowledge of the packaging and installation systems.

•

Testing: The testing of a custom KM is extremely important before deploying it into production. To be valid, testing and debugging need to be done in a proper test environment once the package has been created and installed on the test servers. A poorly designed KM can cause the targeted application to crash or can consume 100% of system resources. It can also simply fail because the packaging was done incorrectly. Once problems have been detected in the test environment and bugs have been identified and resolved, the whole fixing, packaging, testing and deployment process cycle starts all over again. This needs to be repeated as many times as necessary in order to get a fully functional yet non-problematic custom monitoring solution.

•

Deployment: Once a custom KM has been successfully validated, it needs to be deployed into production, which again is not an easy task depending on the deployment tools being used, and which involve time and risks. The main issue with deployment is that it’s a recurring issue: each new custom KM needs to be packaged and deployed into production while other standard solutions (like the Log Management KM described above) need to be deployed only once on a server.

•

Solution upgrades and release management: The development process is an interminable cycle. Once a custom KM is released, the developers continually face requests from users and administrators: new bugs to be fixed, small feature improvements, performance improvements, and updates to keep in step with the application’s releases, as well as other brand new features that were not planned during the development process. Once these requests are legitimated, the development cycle restarts all over again. In addition, the alteration of the code of an existing custom KM is possible or easier when done by the team that worked on its creation… however it seems that PSL developers rarely spend more than 2 years in the same PSL development position. So it’s back to square one – trying to find the suitable human resources for the project.

Although on one hand custom KMs seem like the most flexible and viable solution at the outset, it is proven in today’s world of fast-changing technologies, that in the long run, it turns out it is economically more viable to purchase a ready KM specifically designed to deal with complex and diverse monitoring needs; one that is regularly upgraded and maintained by professionals bound by terms of contract.

BMC Performance Manager Monitoring Studio is a KM for PATROL and is also available as an agent-less solution for BMC Portal. It enables administrators to set up the monitoring without coding, of almost any application for which no standard monitoring solution is available. In a few clicks, and thanks to intelligent wizards, it can cover up to 100% of critical applications within your BMC Software monitoring environment. This is a monitoring toolbox that offers diverse monitoring tools.

Centralized monitoring management: BMC Performance Manager Monitoring Studio not only allows consolidation and centralization of diverse monitoring needs through one solution, but it also offers the ability to group the monitored objects application-wise or as best suited to the organizations specific monitoring-management needs. This renders the monitoring management far more efficient and productive.

Monitoring basic yet critical system elements: Effective monitoring of an application or any part of an IT infrastructure cannot ignore monitoring the basic system elements: processes or Windows services, file systems used by the application (often critical), its folders (directories) and its most critical files. BMC Performance Manager Monitoring Studio allows administrators to set up the monitoring of these "basic system elements" in a very easy, yet effective way.

Retrieving valuable information using standard but diverse technologies: Getting valuable information through diverse means such as JMX, SNMP, WBEM, Windows performance counters and Windows event logs is often essential, especially if there is no monitoring solution for a non-standard or customized software application. Monitoring Studio enables the administrators to gather information using the SNMP protocol (polling and trap listening), WBEM and WMI queries, Windows performance counters and event logs. The JMX polling feature of this product facilitates monitoring of JBoss, JOnAS, WebLogic, WebSphere or any generic JMX -enabled Java application under a single icon. There is no need to interrogate these diverse application servers through their respective interfaces. Also, just as for any “information source”, advanced string searches and numeric values extraction can be performed easily.

Fine-tune the monitoring with powerful string and numeric value searches: The intelligent wizards of the product ask for details on the information sources you would like to monitor: multi-line LOG files, scripts and commands output, HTTP requests, database queries or WBEM and WMI queries. Administrators then specify what they are looking for within these sources, strings to be found or not, numeric values to be extracted, etc. Once an “information source” is defined, a detailed string search can be set up using the following options:

BMC Performance Manager Monitoring Studio offers one of the fastest and most advanced string searches available on the market. The product is proven to be able to handle 4 gigabytes of XML-formatted log files at once in less than 10 minutes, without even taking the monitored system down.

Graphical visualization of numeric values monitored: Problems with an application are not always as simply represented as a sentence stating “an error has occurred.” At times, an application reports its health by providing critical numbers, like those of the number of waiting customers, queued requests, processing time, etc. Perhaps it is helpful to know and be alerted on the number of clients are waiting to connect to an application or how many jobs still need to be processed. Number searches allow administrators to extract actual values in a given text and build graphs from these values with alert thresholds. This can be done with most of the Monitoring Studio tools such as file-analysis, database queries, SNMP polling, etc.

Automated alert acknowledgement: Event-driven monitoring (like SNMP traps, LOG files or Windows events) rarely integrates with icons that are supposed to show the current state of a monitored object. This is what acknowledgment is all about. On detection of a failure, the icon of the monitored object should ideally reflect an alarm state, but then reverting to the normal state often necessitates a manual action. This is annoying if the average frequency of alerts is relatively high. BMC Performance Manager Monitoring Studio features automatic acknowledgement capabilities. Alerts triggered by an event can be automatically cleared by either another event or a timeout. This way, no manual operation is required. More intelligent automation translates into optimization of resources and higher productivity.

Specific automated recovery actions: The administrator can select the type of specific actions to be taken when an alert is triggered (application failure occurs etc.). With Alert Actions, it is possible to either customize the way a notification is performed for an application alert, or specify execution of a recovery action when a problem occurs:

PROS —

Monitoring Studio KM for PATROL offers an impressive panoply of monitoring tools and offers a fairly good amount of flexibility to the administrator during set-up of the monitoring of a custom application without all the difficulties related to custom KM development.

CONS —

This ease and large range of tools comes with a price, i.e. the cost of the license and the flexibility level cannot compare to that offered through a custom developed KM.

However, in terms of ROI, it turns out BMC Performance Manager Monitoring Studio fares very well as compared to the difficulties that custom KM development entails. The next section compares the 3 available options in term of features and cost.