RUM-7102 feat: Add Time To Network Settled metric in RUM #2125

ncreated · 2024-11-29T07:55:33Z

What and why?

📦⏱️ This PR introduces the Time-to-Network-Settled (TTNS) metric to measure the time from when a view becomes visible until all initial network resources have finished loading. This metric helps gauge the time it takes for a view to be fully displayed with all required resources.

The TTNS value is reported under these conditions:

Resources starting within 100ms of the view becoming visible are considered "initial network resources."
The TTNS value is available only after all initial resources load, and at least 100ms after the view starts.
The value is not tracked if the view is stopped before resources finish loading.
The value is tracked only if the app was in the "active" state during view loading.

Example: In following scenario, resources 1, 2, and 3 are "initial," and the completion of the last resource determines TTNS. Resource 4 is not "initial" as it started after the 100ms threshold.

The TTNS value is reported in the view.network_settled_time attribute, in nanoseconds.

This is the counterpart to Android PR #2392.

How?

The metric is modeled as an object implementing the TTNSMetricTracking interface, which tracks key events related to TTNS (resource start, stop, and view stop) and provides a method to retrieve the metric value:

internal protocol TTNSMetricTracking {
    func trackResourceStart(at startDate: Date, resourceID: RUMUUID)
    func trackResourceEnd(at endDate: Date, resourceID: RUMUUID, resourceDuration: TimeInterval?)
    func trackResourceDropped(resourceID: RUMUUID)
    func trackViewWasStopped()
    func value(at time: Date, appStateHistory: AppStateHistory) -> TimeInterval?
}

The TTNS object is created within each RUMViewScope at the start of the view and passed to RUMResourceScope to track resources associated with that view.

This PR includes both unit and integration tests for TTNS, with integration tests located in ViewLoadingMetricsTests. These tests will also serve as a foundation for upcoming view loading metrics.

Review checklist

Feature or bugfix MUST have appropriate tests (unit, integration)
Make sure each commit and the PR mention the Issue number or JIRA reference
Add CHANGELOG entry for user facing changes
Add Objective-C interface for public APIs (see our guidelines [internal]) and run make api-surface)

datadog-datadog-prod-us1 · 2024-11-29T08:00:44Z

Datadog Report

Branch report: ncreated/RUM-7102/vl-ttns-basics
Commit report: fd4e916
Test service: dd-sdk-ios

✅ 0 Failed, 3617 Passed, 0 Skipped, 2m 21.76s Total Time
🔻 Test Sessions change in coverage: 5 decreased, 4 increased, 5 no change

🔻 Code Coverage Decreases vs Default Branch (5)

test DatadogSessionReplayTests iOS 33.12% (-1.2%) - Details
test DatadogCoreTests tvOS 78.47% (-0.13%) - Details
test DatadogInternalTests iOS 80.3% (-0.05%) - Details
test DatadogTraceTests tvOS 54.24% (-0.02%) - Details
test DatadogInternalTests tvOS 80.4% (-0.02%) - Details

mariedm

Well done! 💪 Just a couple of non-blocking questions and comments, but overall, it's nicely written, well-documented, and the test suite is robust.

DatadogRUM/Sources/RUMMetrics/TTNSMetric.swift

mariedm · 2024-12-04T18:21:56Z

DatadogRUM/Sources/DataModels/RUMDataModels.swift

@@ -1827,6 +1830,7 @@ public struct RUMResourceEvent: RUMDataModel {
            case transferSize = "transfer_size"
            case type = "type"
            case url = "url"
+            case worker = "worker"


Do you know if the explicit key mappings are required to match a backend schema?

They are required as schema mixes multiple conventions (mainly snake_case and kebab-case), e.g.:

case analytics = "analytics" // ... case reactNative = "react-native" // ... case sessionSampleRate = "session_sample_rate"

DatadogRUM/Sources/RUMMetrics/TTNSMetric.swift

mariedm · 2024-12-04T18:28:50Z

DatadogRUM/Sources/RUMMetrics/TTNSMetric.swift

+    ///   - endDate: The end time of the resource (device time, no NTP offset).
+    ///   - resourceID: The unique identifier for the resource.
+    ///   - resourceDuration: The resource duration, if available.
+    func trackResourceEnd(at endDate: Date, resourceID: RUMUUID, resourceDuration: TimeInterval?) {


Why is resourceDuration provided at all? Shouldn't the method rely solely on endDate to calculate the duration? I can see that in the RUMResourceScope we provide it when we send a resource event, but not when tracking an error.

The "start" and "end" times for RUM resource are approximations based on network request timings derived from the resume() and completion {} in URLSession.

For a more accurate representation, RUM listens to URLSessionTaskMetrics delivered to the session delegate. In addition to various temporal metrics such as DNS, TCP, and TLS timings, URLSessionTaskMetrics also provides the fetch duration. This fetch duration serves as a more precise alternative for the RUM resource "duration," which is now tracked here for the TTNS metric.

Got it, thanks for the explanation!

DatadogRUM/Tests/RUMMetrics/TTNSMetricTests.swift

ncreated · 2024-12-10T10:56:55Z

DatadogRUM/Tests/RUMMonitor/Scopes/RUMViewScopeTests.swift

@@ -2595,7 +2595,7 @@ class RUMViewScopeTests: XCTestCase {
        XCTAssertEqual(event.view.resource.count, 1, "After dropping 1 Resource event (out of 2), View should record 1 Resource")
        XCTAssertEqual(event.view.action.count, 0, "After dropping a User Action event, View should record no actions")
        XCTAssertEqual(event.view.error.count, 0, "After dropping an Error event, View should record 0 Errors")
-        XCTAssertEqual(event.dd.documentVersion, 3, "After starting the application, stopping the view, starting/stopping one resource out of 2, discarding a user action and an error, the View scope should have sent 3 View events.")
+        XCTAssertEqual(event.dd.documentVersion, 4, "It should create 4 view update.")


👍 This is expected as now we also cause view updates when events are dropped. This doesn't change what is sent to RUM intake as redundant updates are filtered before upload.

ncreated · 2024-12-10T10:59:03Z

@mariedm Thanks for review! All feedback addressed 👌.

I'm requesting another approval because there was small addition of new logic in the last commit (fd4e916) following recent spec changes. We now only count resources that are sent to RUM, excluding any resources that are dropped by the user with event mapper API.

mariedm

LGTM! 🚀

ncreated added 4 commits November 29, 2024 08:54

RUM-7102 Update RUM schema

e179f52

RUM-7102 Implement TTNS metric for RUM

c38e7b4

RUM-7102 Integrate TTNS metric into RUM

3ab544b

RUM-7102 Add integration tests for TTNS metric

62f355b

ncreated self-assigned this Nov 29, 2024

RUM-7102 Refine & update CHANGELOG.md

63dc7f8

ncreated changed the base branch from develop to feature/view-loading-times November 29, 2024 08:13

ncreated marked this pull request as ready for review November 29, 2024 09:32

ncreated requested review from a team as code owners November 29, 2024 09:32

mariedm approved these changes Dec 5, 2024

View reviewed changes

ncreated changed the title ~~RUM-7102 feat: Add Time To Network Setled metric in RUM~~ RUM-7102 feat: Add Time To Network Settled metric in RUM Dec 10, 2024

ncreated added 2 commits December 10, 2024 10:03

RUM-7102 CR feedback

ab55dec

RUM-7102 Exclude dropped resources and errors from TTNS computation

fd4e916

ncreated commented Dec 10, 2024

View reviewed changes

ncreated requested a review from mariedm December 10, 2024 10:57

mariedm approved these changes Dec 10, 2024

View reviewed changes

ncreated merged commit 14c6904 into feature/view-loading-times Dec 10, 2024
16 checks passed

ncreated mentioned this pull request Dec 10, 2024

RUM-6501 feat: RUM View Loading Time metrics #2139

Merged

4 tasks

ncreated mentioned this pull request Dec 18, 2024

RUM-7107 feat: Add Interaction To Next View metric in RUM #2153

Merged

4 tasks

maxep mentioned this pull request Feb 5, 2025

Release 2.23.0 #2184

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

RUM-7102 feat: Add Time To Network Settled metric in RUM #2125

RUM-7102 feat: Add Time To Network Settled metric in RUM #2125

ncreated commented Nov 29, 2024 •

edited

Loading

datadog-datadog-prod-us1 bot commented Nov 29, 2024 •

edited

Loading

mariedm left a comment

mariedm Dec 4, 2024

ncreated Dec 10, 2024

mariedm Dec 4, 2024

ncreated Dec 10, 2024

mariedm Dec 10, 2024

ncreated Dec 10, 2024

ncreated commented Dec 10, 2024

mariedm left a comment

RUM-7102 feat: Add Time To Network Settled metric in RUM #2125

RUM-7102 feat: Add Time To Network Settled metric in RUM #2125

Conversation

ncreated commented Nov 29, 2024 • edited Loading

What and why?

How?

Review checklist

datadog-datadog-prod-us1 bot commented Nov 29, 2024 • edited Loading

Datadog Report

🔻 Code Coverage Decreases vs Default Branch (5)

mariedm left a comment

Choose a reason for hiding this comment

mariedm Dec 4, 2024

Choose a reason for hiding this comment

ncreated Dec 10, 2024

Choose a reason for hiding this comment

mariedm Dec 4, 2024

Choose a reason for hiding this comment

ncreated Dec 10, 2024

Choose a reason for hiding this comment

mariedm Dec 10, 2024

Choose a reason for hiding this comment

ncreated Dec 10, 2024

Choose a reason for hiding this comment

ncreated commented Dec 10, 2024

mariedm left a comment

Choose a reason for hiding this comment

ncreated commented Nov 29, 2024 •

edited

Loading

datadog-datadog-prod-us1 bot commented Nov 29, 2024 •

edited

Loading