We have a few processes that run on our servers which utilize "temporary" disk space for updates/definitions/etc... Unfortunately on a number of servers this causes our volume monitor alerts to trigger due to crossing the 95% utilization threshold we have set. Of course, minutes (in fairness sometimes hours) later the temporary space is cleared up and the volume falls below the threshold clearing the alerts.
I'd like to add some more intelligence to our alert that might avoid at least the majority of these scenarios from happening.
I know it's not possible to use an "APM: Component" monitor for a suppression condition when using a "Volume" monitor for the Trigger Conditions...yeah I tried it, sue me. I feel like there has to be a way to do this, or at least we cant be the only people who experience this same scenario.
So how have others done it? Is there a way to perhaps set some "Node Variable" within APM, and then look to that variable within Alert manager for a suppression condition?
Thanks in advance.