The Latest NetApp News
Product and Solution Information, Press Releases, Announcements
|Introducing a Framework to Monitor NetApp E-Series Solutions|
|Posted: Mon Jun 01, 2020 03:42:52 PM|
Good software development considers functionality from the perspective of the user. For example, as a systems administrator, I want to easily monitor the systems in my environment to ensure that they perform and function as expected. But software is rarely described by a single user story, so let’s add another. As a systems administrator, I want to easily monitor the systems in my environment from a single monitoring solution.
New Feature of E-Series Performance Analyzer
That user story forms the foundation of our vision for the newest feature of the NetApp® E-Series Performance Analyzer: plug-ins. If you have ever played PC games, you might be familiar with the concept of a mod. Mods are community-developed content that extends the original functionality of a game. Mods can be as simple as new character skins, or they can create entirely new game modes. A key takeaway is that players are able to build upon core functionality to add features that the original developers might not have envisioned.
Before we go on, if you’re unfamiliar with the E-Series Performance Analyzer, let me briefly introduce you to it. There are a few key components to monitoring any environment:
Rather than reinvent the wheel, we picked the best (in our opinion) open-source components for each task, and developed related automation for seamless deployment of an end-to-end monitoring stack that uses Docker. For the E-Series Performance Analyzer, we use Python scripts to collect performance metrics from desired E-Series storage systems by using their RESTful API. We store those statistics in InfluxDB and use Grafana to visualize the data. Each component runs as a Docker container; after it’s set up, you can access Grafana using your web browser. This open-source project is available on GitHub under an MIT license.
Why Add Plug-ins?
Plug-ins came about after we tried to implement BeeGFS support into this monitoring stack for ourselves. Originally, that process was entirely too convoluted, and thus the new plug-in framework was born. We wanted a simpler mechanism to extend the core functionality of the E-Series Performance Analyzer beyond simply monitoring E-Series storage systems.
Starting with the 2.1 release of the E-Series Performance Analyzer, you can create your own plug-ins by defining new collection mechanisms to store metrics in InfluxDB and add dashboards to Grafana. As of version 2.1, InfluxDB and Grafana are considered core components, and all collection methods (including collecting metrics from E-Series) are considered plug-ins.
One of the reasons we created this new plug-in framework is to be able to add support for monitoring new NetApp solutions. For example, NetApp offers support for BeeGFS, and we wanted you to have a simple way to monitor BeeGFS alongside your E-Series storage systems. There is a BeeGFS monitoring service (beegfs-mon) that can feed metrics into InfluxDB. It provides several predefined Grafana dashboards but does not include InfluxDB or Grafana. The E-Series Performance Analyzer provides a convenient “batteries included” approach to monitoring BeeGFS, and the plug-in for BeeGFS is available on GitHub.
At this point, you might be wondering about the limits of this plug-in approach. A key requirement is to get metrics from the monitored endpoint into the containerized InfluxDB instance. If what you’re monitoring can push metrics into a database, you simply need to point it at the InfluxDB instance, and your plug-in needs only the Grafana data source and dashboards that you want to include. For other devices, you might need a “middleman” that can pull metrics from the endpoint and route them into InfluxDB. For E-Series systems, this was accomplished with a Python script. Some software might, like BeeGFS, provide its own method of retrieving and storing data (for example, beegfs-mon). If this piece is external to the monitored endpoint, it might make sense to containerize it as part of your plug-in for the E-Series Performance Analyzer, but this is not a requirement.