In Ch4 you used the UFM event log GUI to inspect InfiniBand fabric events after a fault. That GUI is backed by a REST API -- the same API you use to build automated polling, alerting, and integration with external monitoring systems.

The UFM REST API runs on the UFM management host (typically a separate server or the UFM appliance) on port 443. Authentication is HTTP Basic over TLS. Every response is JSON.

The four endpoints you need

GET /ufmRest/app/ufmVersion -- health check. Returns UFM version and fabric summary. If this call fails, UFM itself has a problem.

GET /ufmRest/resources/ports -- all fabric ports with current state, speed, and counters. This is the bulk-polling endpoint: one call returns every port in the fabric. Filter by guid for a specific HCA, by peer_guid to trace a link, or by logical_state != Active to find anything non-healthy.

GET /ufmRest/resources/alarms -- all active alarms. This is your first-stop dashboard call. An empty alarm list means UFM sees no current issues. A non-empty list gives you the alarm type, severity, and affected GUIDs.

GET /ufmRest/events -- fabric event stream. Ordered by timestamp. Use ?from_event_id= to poll incrementally. This is how you stream events into Prometheus or a SIEM.

# Get all active alarms -- formatted
curl -sk -u admin:password \
  https://ufm-host/ufmRest/resources/alarms \
  | python3 -m json.tool

# Poll ports filtered to non-Active state
curl -sk -u admin:password \
  "https://ufm-host/ufmRest/resources/ports?logical_state=Down" \
  | jq '.[].guid'

# Incremental event stream -- last 500 events
curl -sk -u admin:password \
  "https://ufm-host/ufmRest/events?from_event_id=0&max_results=500" \
  | jq '.[].type'

Polling frequency

Counter polling at less than 60 seconds produces mostly noise -- most error counters increment slowly enough that sub-minute deltas are meaningless. The exception is SymbolErrors during active link degradation, which can increment by thousands per second. If you detect a rising SymbolErrors count, switch to 10-second polling for that specific port.

What UFM REST does not expose

UFM REST gives you InfiniBand fabric state. It does not give you:

• GPU metrics (use DCGM)
• Ethernet fabric counters from Spectrum switches (use Prometheus + SNMP/gNMI or Telegraf)
• Application-layer NCCL timing (use NCCL logs or DCGM NVLink counters)

A healthy monitoring stack uses all three. UFM REST is the IB fabric layer.