feat(workflows): implement metrics for fdb driver #2057

MasterPtato · 2025-02-20T02:38:09Z

Changes

cloudflare-workers-and-pages · 2025-02-20T02:38:11Z

Deploying rivet with Cloudflare Pages

Latest commit:	`6f890ba`
Status:	🚫 Build failed.

View logs

MasterPtato · 2025-02-20T02:38:43Z

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

How to use the Graphite Merge Queue

Add the label merge-queue to this PR to add it to the merge queue.

You must have a Graphite account in order to use the merge queue. Sign up using this link.

_{An organization admin has enabled the Graphite Merge Queue in this repository.} _{Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.}

This stack of pull requests is managed by Graphite. Learn more about stacking.

greptile-apps

PR Summary

This PR implements metrics collection for the FDB (FoundationDB) driver in the workflow system, focusing on workflow state tracking and performance monitoring. Here's a concise summary of the key changes:

Added DataSubspaceKey and workflow state tracking structures to monitor workflow lifecycle states (Complete, Running, Sleeping, Dead)
Implemented metrics collection in publish_metrics() to track total, active, dead, and sleeping workflows with proper reset functionality
Made PendingSignalSubspaceKey::new() public and improved key handling for better metrics access
Added workflow state detection based on workflow keys and improved error handling for metrics collection
Implemented proper locking mechanism for metrics publishing to prevent concurrent updates

The changes provide a robust foundation for monitoring workflow performance and state transitions in the FDB driver.

_{4 file(s) reviewed, 1 comment(s)}
_{Edit PR Review Bot Settings | Greptile}

greptile-apps · 2025-02-20T08:03:53Z

packages/common/chirp-workflow/core/src/db/fdb_sqlite_nats/mod.rs

+						} else if let WorkflowState::Dead = current_state {
+							if let Ok(_) = self
+								.subspace
+								.unpack::<keys::workflow::OutputChunkKey>(entry.key())
+							{
+								current_state = WorkflowState::Complete;
+							} else if let Ok(_) = self
+								.subspace
+								.unpack::<keys::workflow::WorkerInstanceIdKey>(entry.key())
+							{
+								current_state = WorkflowState::Running;
+							} else if let Ok(_) = self
+								.subspace
+								.unpack::<keys::workflow::HasWakeConditionKey>(entry.key())
+							{
+								current_state = WorkflowState::Sleeping;
+							}
+						}


logic: State transition logic doesn't handle the case where multiple state indicators exist simultaneously - could lead to incorrect metrics

graphite-app · 2025-03-06T03:33:08Z

Merge activity

Mar 5, 10:33 PM EST: A user added this pull request to the Graphite merge queue.
Mar 5, 10:35 PM EST: CI is running for this PR on a draft PR: #2125
Mar 5, 10:37 PM EST: A user merged this pull request with the Graphite merge queue via draft PR: #2125.

## Changes

MasterPtato mentioned this pull request Feb 20, 2025

fix: move actor state to fdb #2056

Closed

NathanFlurry force-pushed the 02-20-feat_workflows_implement_metrics_for_fdb_driver branch from c521a04 to 729d516 Compare February 20, 2025 03:50

NathanFlurry force-pushed the 02-20-fix_move_actor_state_to_fdb branch from 8412313 to 1b5874e Compare February 20, 2025 03:50

NathanFlurry marked this pull request as ready for review February 20, 2025 08:02

This was referenced Feb 20, 2025

fix(docker): fix corepack not enabled error #2058

Closed

feat: add support for JSON5/JSONC config files #2059

Closed

greptile-apps bot reviewed Feb 20, 2025

View reviewed changes

NathanFlurry changed the base branch from 02-20-fix_move_actor_state_to_fdb to graphite-base/2057 February 20, 2025 21:34

NathanFlurry mentioned this pull request Feb 20, 2025

chore: add basic load test #2060

Closed

MasterPtato force-pushed the graphite-base/2057 branch from 1b5874e to 4b4e6ae Compare February 20, 2025 22:02

MasterPtato mentioned this pull request Mar 5, 2025

fix: update actor kv keys #2120

Closed

MasterPtato force-pushed the 02-20-fix_move_actor_state_to_fdb branch from f02e69d to b3d7bc6 Compare March 5, 2025 20:55

MasterPtato force-pushed the 02-20-feat_workflows_implement_metrics_for_fdb_driver branch from c6712a2 to 29cc6ae Compare March 5, 2025 20:55

feat(workflows): implement metrics for fdb driver

6f890ba

MasterPtato mentioned this pull request Mar 5, 2025

fix: fix system test #2121

Closed

NathanFlurry force-pushed the 02-20-feat_workflows_implement_metrics_for_fdb_driver branch from 29cc6ae to 6f890ba Compare March 5, 2025 23:56

NathanFlurry mentioned this pull request Mar 5, 2025

chore: bump version from 5.1.2 to 25.1.3 #2122

Closed

MasterPtato force-pushed the 02-20-fix_move_actor_state_to_fdb branch from 454f741 to b3d7bc6 Compare March 6, 2025 00:50

MasterPtato force-pushed the 02-20-feat_workflows_implement_metrics_for_fdb_driver branch from 6f890ba to 29cc6ae Compare March 6, 2025 00:50

NathanFlurry force-pushed the 02-20-feat_workflows_implement_metrics_for_fdb_driver branch from 29cc6ae to 6f890ba Compare March 6, 2025 01:32

NathanFlurry force-pushed the 02-20-fix_move_actor_state_to_fdb branch from b3d7bc6 to 454f741 Compare March 6, 2025 01:32

This was referenced Mar 6, 2025

docs: add helm rollback troubleshooting guide #2123

Closed

fix(system-test): fix isolates use of hono #2124

Closed

MasterPtato force-pushed the 02-20-fix_move_actor_state_to_fdb branch from 454f741 to b3d7bc6 Compare March 6, 2025 02:38

MasterPtato force-pushed the 02-20-feat_workflows_implement_metrics_for_fdb_driver branch from 6f890ba to 29cc6ae Compare March 6, 2025 02:38

NathanFlurry force-pushed the 02-20-feat_workflows_implement_metrics_for_fdb_driver branch from 29cc6ae to 6f890ba Compare March 6, 2025 03:30

graphite-app bot closed this Mar 6, 2025

graphite-app bot deleted the 02-20-feat_workflows_implement_metrics_for_fdb_driver branch March 6, 2025 03:37

github-actions bot mentioned this pull request Mar 6, 2025

chore(main): release 25.2.0 #2079

Merged

This was referenced Mar 6, 2025

chore: add some logs #2133

Closed

fix: fix wf change bug with client wf #2134

Closed

fix: fix topology units, core actor endpoints #2135

Closed

fix: fix pb draining #2136

Closed

fix(workflows): remove tagged signals from fdb driver #2137

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(workflows): implement metrics for fdb driver #2057

feat(workflows): implement metrics for fdb driver #2057

MasterPtato commented Feb 20, 2025

cloudflare-workers-and-pages bot commented Feb 20, 2025 •

edited

Loading

MasterPtato commented Feb 20, 2025 •

edited by NathanFlurry

Loading

greptile-apps bot left a comment

greptile-apps bot Feb 20, 2025

graphite-app bot commented Mar 6, 2025 •

edited

Loading

feat(workflows): implement metrics for fdb driver #2057

feat(workflows): implement metrics for fdb driver #2057

Conversation

MasterPtato commented Feb 20, 2025

Changes

cloudflare-workers-and-pages bot commented Feb 20, 2025 • edited Loading

Deploying rivet with Cloudflare Pages

MasterPtato commented Feb 20, 2025 • edited by NathanFlurry Loading

How to use the Graphite Merge Queue

greptile-apps bot left a comment

Choose a reason for hiding this comment

PR Summary

greptile-apps bot Feb 20, 2025

Choose a reason for hiding this comment

graphite-app bot commented Mar 6, 2025 • edited Loading

Merge activity

cloudflare-workers-and-pages bot commented Feb 20, 2025 •

edited

Loading

MasterPtato commented Feb 20, 2025 •

edited by NathanFlurry

Loading

graphite-app bot commented Mar 6, 2025 •

edited

Loading