Skip to content

[v2] refactor; Prometheus to handle null values and clean up #2705

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
svenvg93 wants to merge 3 commits into
base: 2.x
Choose a base branch
from
+131 −206
Open

[v2] refactor; Prometheus to handle null values and clean up #2705

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a88098a
refactor; promtheus to handle null values and clean up
svenvg93 Feb 12, 2026
ed9d1b6
chore: add breaking change tracking doc.
svenvg93 Feb 12, 2026
3b7f502
chore: fix import in tests
svenvg93 Feb 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions BREAKING_CHANGES.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Breaking Changes

This document tracks breaking changes that may affect existing users.

## v2.x - Prometheus Metrics Refactor

**Date**: 2026-02-12
**PR**: #XXXX
**Impact**: High - Affects all users with Prometheus dashboards/alerts

### Overview

The Prometheus metrics implementation has been significantly refactored to follow Prometheus best practices and naming conventions.

### Metric Name Changes

All speed metrics now have explicit `_per_second` suffix to indicate they are rates:

| Old Metric | New Metric | Migration |
|------------|------------|-----------|
| `speedtest_tracker_download_bytes` | `speedtest_tracker_download_bytes_per_second` | Update all dashboard queries and alert rules |
| `speedtest_tracker_upload_bytes` | `speedtest_tracker_upload_bytes_per_second` | Update all dashboard queries and alert rules |
| `speedtest_tracker_download_bits` | `speedtest_tracker_download_bits_per_second` | Update all dashboard queries and alert rules |
| `speedtest_tracker_upload_bits` | `speedtest_tracker_upload_bits_per_second` | Update all dashboard queries and alert rules |
| `speedtest_tracker_downloaded_bytes` | `speedtest_tracker_test_downloaded_bytes_total` | Update all dashboard queries and alert rules |
| `speedtest_tracker_uploaded_bytes` | `speedtest_tracker_test_uploaded_bytes_total` | Update all dashboard queries and alert rules |
| `speedtest_tracker_result_id` | `speedtest_tracker_info` | Update queries - value is now always `1` |

### Label Changes

Some labels have been removed to reduce cardinality:

**Removed Labels**:
- `server_id` - Use `server_name` instead for filtering
- `server_country` - Not essential for most queries
7 changes: 5 additions & 2 deletions app/Listeners/ProcessSpeedtestDataIntegrations.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
use App\Events\SpeedtestCompleted;
use App\Events\SpeedtestFailed;
use App\Jobs\Influxdb\v2\WriteResult;
use App\Services\PrometheusMetricsService;
use App\Settings\DataIntegrationSettings;
use Illuminate\Support\Facades\Cache;

class ProcessSpeedtestDataIntegrations
{
Expand All @@ -15,6 +15,7 @@ class ProcessSpeedtestDataIntegrations
*/
public function __construct(
public DataIntegrationSettings $settings,
public PrometheusMetricsService $prometheusService,
) {}

/**
Expand All @@ -27,7 +28,9 @@ public function handle(SpeedtestCompleted|SpeedtestFailed $event): void
}

if ($this->settings->prometheus_enabled) {
Cache::forever('prometheus:latest_result', $event->result->id);
// Update Prometheus metrics cache when speedtest completes/fails
// This prevents rebuilding metrics on every scrape
$this->prometheusService->updateMetrics($event->result);
}
}
}
266 changes: 69 additions & 197 deletions app/Services/PrometheusMetricsService.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@

namespace App\Services;

use App\Enums\ResultStatus;
use App\Models\Result;
use App\Settings\DataIntegrationSettings;
use Illuminate\Support\Facades\Cache;
Expand All @@ -18,237 +17,110 @@ public function __construct(

public function generateMetrics(): string
{
$registry = new CollectorRegistry(new InMemory);

$resultId = Cache::get('prometheus:latest_result');

if (! $resultId) {
return $this->emptyMetrics();
}

$lastResult = Result::find($resultId);
// Return cached metrics if available
// This avoids rebuilding the registry and querying the DB on every scrape
return Cache::get('prometheus:metrics', $this->emptyMetrics());
}

if (! $lastResult) {
return $this->emptyMetrics();
}
public function updateMetrics(Result $result): void
{
// Build metrics only when data changes (speedtest completes/fails)
$registry = new CollectorRegistry(new InMemory);

$this->registerMetrics($registry, $lastResult);
$this->registerMetrics($registry, $result);

$renderer = new RenderTextFormat;
$metrics = $renderer->render($registry->getMetricFamilySamples());

return $renderer->render($registry->getMetricFamilySamples());
// Cache the rendered metrics so scrapes don't rebuild every time
Cache::forever('prometheus:metrics', $metrics);
}

protected function registerMetrics(CollectorRegistry $registry, Result $result): void
{
$labels = $this->buildLabels($result);
$labelNames = array_keys($labels);
$labelValues = array_values($labels);
$timestamp = $result->updated_at?->timestamp;

// Info metric - always exported so users can see test status (including failures)
$infoGauge = $registry->getOrRegisterGauge(
'speedtest_tracker',
'result_id',
'Speedtest result id',
$labelNames
);
$infoGauge->set($result->id, $labelValues);

// Only export numeric metrics for completed tests
// Failed/incomplete tests won't have valid measurements
if ($result->status !== ResultStatus::Completed) {
return;
}

// Download speed in bytes
$downloadBytesGauge = $registry->getOrRegisterGauge(
'speedtest_tracker',
'download_bytes',
'Download speed in bytes per second',
$labelNames
);
$downloadBytesGauge->set($result->download, $labelValues);

// Upload speed in bytes
$uploadBytesGauge = $registry->getOrRegisterGauge(
'speedtest_tracker',
'upload_bytes',
'Upload speed in bytes per second',
$labelNames
);
$uploadBytesGauge->set($result->upload, $labelValues);
// Standard 'up' metric - exporter is responding
$up = $registry->getOrRegisterGauge('speedtest_tracker', 'up', 'Exporter is responding');
$up->set(1, []);

// Download speed in bits per second
$downloadBitsGauge = $registry->getOrRegisterGauge(
'speedtest_tracker',
'download_bits',
'Download speed in bits per second',
$labelNames
);
$downloadBitsGauge->set(toBits($result->download), $labelValues);
// Build info metric - application version
$buildInfo = $registry->getOrRegisterGauge('speedtest_tracker', 'build_info', 'Application version information', ['version']);
$buildInfo->set(1, [config('speedtest.build_version')]);

// Upload speed in bits per second
$uploadBitsGauge = $registry->getOrRegisterGauge(
'speedtest_tracker',
'upload_bits',
'Upload speed in bits per second',
$labelNames
);
$uploadBitsGauge->set(toBits($result->upload), $labelValues);
// Info metric - always set to 1, metadata in labels
// Exported for both completed and failed tests so Prometheus can track all test attempts
$infoGauge = $registry->getOrRegisterGauge('speedtest_tracker', 'info', 'Speedtest metadata and status', $labelNames);
$infoGauge->set(1, $labelValues, $timestamp);

// Ping latency in milliseconds
$pingGauge = $registry->getOrRegisterGauge(
'speedtest_tracker',
'ping_ms',
'Ping latency in milliseconds',
$labelNames
);
$pingGauge->set($result->ping, $labelValues);
// Register all speed/latency metrics
// Failed tests will have null values, which registerGaugeIfNotNull automatically skips

// Ping jitter
$pingJitterGauge = $registry->getOrRegisterGauge(
'speedtest_tracker',
'ping_jitter_ms',
'Ping jitter in milliseconds',
$labelNames
);
$pingJitterGauge->set($result->ping_jitter, $labelValues);
// Speed metrics (rates)
$this->registerGaugeIfNotNull($registry, 'download_bytes_per_second', 'Download speed in bytes per second', $labelNames, $labelValues, $result->download, $timestamp);
$this->registerGaugeIfNotNull($registry, 'upload_bytes_per_second', 'Upload speed in bytes per second', $labelNames, $labelValues, $result->upload, $timestamp);
$this->registerGaugeIfNotNull($registry, 'download_bits_per_second', 'Download speed in bits per second', $labelNames, $labelValues, $result->download ? toBits($result->download) : null, $timestamp);
$this->registerGaugeIfNotNull($registry, 'upload_bits_per_second', 'Upload speed in bits per second', $labelNames, $labelValues, $result->upload ? toBits($result->upload) : null, $timestamp);

// Download jitter
$downloadJitterGauge = $registry->getOrRegisterGauge(
'speedtest_tracker',
'download_jitter_ms',
'Download jitter in milliseconds',
$labelNames
);
$downloadJitterGauge->set($result->download_jitter, $labelValues);
// Ping metrics
$this->registerGaugeIfNotNull($registry, 'ping_ms', 'Ping latency in milliseconds', $labelNames, $labelValues, $result->ping, $timestamp);
$this->registerGaugeIfNotNull($registry, 'ping_low_ms', 'Ping low latency in milliseconds', $labelNames, $labelValues, $result->ping_low, $timestamp);
$this->registerGaugeIfNotNull($registry, 'ping_high_ms', 'Ping high latency in milliseconds', $labelNames, $labelValues, $result->ping_high, $timestamp);

// Upload jitter
$uploadJitterGauge = $registry->getOrRegisterGauge(
'speedtest_tracker',
'upload_jitter_ms',
'Upload jitter in milliseconds',
$labelNames
);
$uploadJitterGauge->set($result->upload_jitter, $labelValues);
// Jitter metrics
$this->registerGaugeIfNotNull($registry, 'ping_jitter_ms', 'Ping jitter in milliseconds', $labelNames, $labelValues, $result->ping_jitter, $timestamp);
$this->registerGaugeIfNotNull($registry, 'download_jitter_ms', 'Download jitter in milliseconds', $labelNames, $labelValues, $result->download_jitter, $timestamp);
$this->registerGaugeIfNotNull($registry, 'upload_jitter_ms', 'Upload jitter in milliseconds', $labelNames, $labelValues, $result->upload_jitter, $timestamp);

// Packet loss
$packetLossGauge = $registry->getOrRegisterGauge(
'speedtest_tracker',
'packet_loss_percent',
'Packet loss percentage',
$labelNames
);
$packetLossGauge->set($result->packet_loss, $labelValues);

// Ping latency low/high
$pingLowGauge = $registry->getOrRegisterGauge(
'speedtest_tracker',
'ping_low_ms',
'Ping low latency in milliseconds',
$labelNames
);
$pingLowGauge->set($result->ping_low, $labelValues);

$pingHighGauge = $registry->getOrRegisterGauge(
'speedtest_tracker',
'ping_high_ms',
'Ping high latency in milliseconds',
$labelNames
);
$pingHighGauge->set($result->ping_high, $labelValues);
$this->registerGaugeIfNotNull($registry, 'packet_loss_percent', 'Packet loss percentage', $labelNames, $labelValues, $result->packet_loss, $timestamp);

// Download latency metrics (IQM = Interquartile Mean - more reliable than average)
$downloadLatencyIqmGauge = $registry->getOrRegisterGauge(
'speedtest_tracker',
'download_latency_iqm_ms',
'Download latency interquartile mean in milliseconds',
$labelNames
);
$downloadLatencyIqmGauge->set($result->downloadlatencyiqm, $labelValues);

$downloadLatencyLowGauge = $registry->getOrRegisterGauge(
'speedtest_tracker',
'download_latency_low_ms',
'Download latency low in milliseconds',
$labelNames
);
$downloadLatencyLowGauge->set($result->downloadlatency_low, $labelValues);

$downloadLatencyHighGauge = $registry->getOrRegisterGauge(
'speedtest_tracker',
'download_latency_high_ms',
'Download latency high in milliseconds',
$labelNames
);
$downloadLatencyHighGauge->set($result->downloadlatency_high, $labelValues);
$this->registerGaugeIfNotNull($registry, 'download_latency_iqm_ms', 'Download latency interquartile mean in milliseconds', $labelNames, $labelValues, $result->downloadlatencyiqm, $timestamp);
$this->registerGaugeIfNotNull($registry, 'download_latency_low_ms', 'Download latency low in milliseconds', $labelNames, $labelValues, $result->downloadlatency_low, $timestamp);
$this->registerGaugeIfNotNull($registry, 'download_latency_high_ms', 'Download latency high in milliseconds', $labelNames, $labelValues, $result->downloadlatency_high, $timestamp);

// Upload latency metrics
$uploadLatencyIqmGauge = $registry->getOrRegisterGauge(
'speedtest_tracker',
'upload_latency_iqm_ms',
'Upload latency interquartile mean in milliseconds',
$labelNames
);
$uploadLatencyIqmGauge->set($result->uploadlatencyiqm, $labelValues);
$this->registerGaugeIfNotNull($registry, 'upload_latency_iqm_ms', 'Upload latency interquartile mean in milliseconds', $labelNames, $labelValues, $result->uploadlatencyiqm, $timestamp);
$this->registerGaugeIfNotNull($registry, 'upload_latency_low_ms', 'Upload latency low in milliseconds', $labelNames, $labelValues, $result->uploadlatency_low, $timestamp);
$this->registerGaugeIfNotNull($registry, 'upload_latency_high_ms', 'Upload latency high in milliseconds', $labelNames, $labelValues, $result->uploadlatency_high, $timestamp);

$uploadLatencyLowGauge = $registry->getOrRegisterGauge(
'speedtest_tracker',
'upload_latency_low_ms',
'Upload latency low in milliseconds',
$labelNames
);
$uploadLatencyLowGauge->set($result->uploadlatency_low, $labelValues);

$uploadLatencyHighGauge = $registry->getOrRegisterGauge(
'speedtest_tracker',
'upload_latency_high_ms',
'Upload latency high in milliseconds',
$labelNames
);
$uploadLatencyHighGauge->set($result->uploadlatency_high, $labelValues);

// Bytes transferred during test
$downloadedBytesGauge = $registry->getOrRegisterGauge(
'speedtest_tracker',
'downloaded_bytes',
'Total bytes downloaded during test',
$labelNames
);
$downloadedBytesGauge->set($result->downloaded_bytes, $labelValues);

$uploadedBytesGauge = $registry->getOrRegisterGauge(
'speedtest_tracker',
'uploaded_bytes',
'Total bytes uploaded during test',
$labelNames
);
$uploadedBytesGauge->set($result->uploaded_bytes, $labelValues);
// Bytes transferred during test (cumulative totals)
$this->registerGaugeIfNotNull($registry, 'test_downloaded_bytes_total', 'Total bytes downloaded during test', $labelNames, $labelValues, $result->downloaded_bytes, $timestamp);
$this->registerGaugeIfNotNull($registry, 'test_uploaded_bytes_total', 'Total bytes uploaded during test', $labelNames, $labelValues, $result->uploaded_bytes, $timestamp);

// Test duration
$downloadElapsedGauge = $registry->getOrRegisterGauge(
'speedtest_tracker',
'download_elapsed_ms',
'Download test duration in milliseconds',
$labelNames
);
$downloadElapsedGauge->set($result->download_elapsed, $labelValues);
$this->registerGaugeIfNotNull($registry, 'download_elapsed_ms', 'Download test duration in milliseconds', $labelNames, $labelValues, $result->download_elapsed, $timestamp);
$this->registerGaugeIfNotNull($registry, 'upload_elapsed_ms', 'Upload test duration in milliseconds', $labelNames, $labelValues, $result->upload_elapsed, $timestamp);
}

$uploadElapsedGauge = $registry->getOrRegisterGauge(
'speedtest_tracker',
'upload_elapsed_ms',
'Upload test duration in milliseconds',
$labelNames
);
$uploadElapsedGauge->set($result->upload_elapsed, $labelValues);
protected function registerGaugeIfNotNull(
CollectorRegistry $registry,
string $name,
string $help,
array $labelNames,
array $labelValues,
mixed $value,
?int $timestamp = null
): void {
if ($value !== null) {
$gauge = $registry->getOrRegisterGauge(
'speedtest_tracker',
$name,
$help,
$labelNames
);
$gauge->set($value, $labelValues, $timestamp);
}
}

protected function buildLabels(Result $result): array
{
return [
'server_id' => (string) ($result->server_id ?? ''),
'server_name' => $result->server_name ?? '',
'server_country' => $result->server_country ?? '',
'server_location' => $result->server_location ?? '',
'isp' => $result->isp ?? '',
'scheduled' => $result->scheduled ? 'true' : 'false',
Expand Down
1 change: 0 additions & 1 deletion composer.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@
"filament/filament": "^5.2",
"filament/spatie-laravel-settings-plugin": "^5.2",
"influxdata/influxdb-client-php": "^3.8",
"laravel-notification-channels/telegram": "^6.0",
"laravel/framework": "^12.50.0",
"laravel/prompts": "^0.3.11",
"laravel/sanctum": "^4.3.0",
Expand Down
2 changes: 1 addition & 1 deletion composer.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading