Workflow status STUCK as "running" with weChat Cloud functions, while the result is already successfully generated in fact. please help fix~

[Zerglingzl]

Self Checks

• This is only for bug report, if you would like to ask a question, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• Please do not modify this template :) and fill in all the required fields.

Dify version

0.15.3

Cloud or Self Hosted

Cloud

Steps to reproduce

I encountered the same issue “thread #12798” mentioned and sincerely hope that Dify can resolve it as soon as possible; otherwise, it will significantly impact our product’s user experience.

My situation might be somewhat unique, yet it makes reproducing the issue even easier—basically, it occurs over 50% of the time.

The business logic is as follows:
1. I use WeChat cloud functions to call Dify’s workflow API. Once the workflow starts running, to avoid waiting too long for a result, the cloud function exits and is terminated.
2. I use WeChat cloud functions to poll the workflow status periodically. If the status becomes “succeed,” I retrieve the result.

As a result, the problem mentioned in “thread #12798” keeps occurring, causing the workflow status to remain stuck on “running.” In reality, however, the workflow has completed (log analysis shows that the final step has produced results), and the tokens for the large model have already been consumed, which is extremely frustrating.

for Debug, there are some workflow run IDs (which are still "running") of mine:

1f692007-1cb1-4dc5-8a61-7b2637309513
1666d166-623f-4c60-9d82-e0d1b2ced4bd
f7f184e0-460a-4dab-9849-75b528f16e99
72f73925-372c-422b-b547-9c93a49bf2dc

BTW, I'm using SaaS Dify 0.15.3.

@kazuhisa-wada @WeakX @takatost

✔️ Expected Behavior

No response

❌ Actual Behavior

No response

[dosubot]

The issue you're experiencing, where the workflow status remains stuck as "running" despite the result being successfully generated, is related to network disconnections during workflow execution. This causes the main thread responsible for database operations to be terminated. To resolve this in Dify version 0.15.3, you should:

1. Decouple the database operations from the main thread by moving them to a separate thread.
2. Create a new dedicated thread for database operations.
3. Move the database update operations from the main thread to this new thread.

This approach ensures that database updates continue even if the main thread is killed due to network disconnection, particularly in streaming mode. This solution aligns with the architecture changes recommended in Dify v0.15.1, where the main thread handles execution threads and message listening, while a new dedicated thread manages database updates ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[Zerglingzl]

@dosu I can't apply your solution since:
1, I'm using Cloud version of Dify
2, I can't prevent weChat cloud functions from termination.

For my understanding, Workflow should have an “asynchronous execution” mechanism (especially for workflows that run for an long period):
1. The caller initiates the workflow.
2. The workflow runs independently of the caller.
3. The caller retrieves the results as needed.

This mechanism should be relatively simple and is particularly well-suited for third-party platforms that require batch processing.

[dosubot]

To implement an asynchronous execution mechanism in the Cloud version of Dify, you can use a queue system to handle tasks independently of the caller. This approach allows the workflow to run asynchronously, enabling the caller to retrieve results as needed without being affected by the termination of WeChat cloud functions.

Here's a basic outline of how you can achieve this:

1. Initiate the Workflow: When the caller initiates the workflow, add the task to a queue instead of processing it immediately.

2. Process the Workflow Asynchronously: Use a worker service, such as Celery, to process tasks from the queue. This allows the workflow to run independently of the caller. You can start a Celery worker with the following command:

   celery -A app.celery worker -P gevent -c 1 -Q dataset,generation,mail,ops_trace --loglevel INFO

   This command sets up a Celery worker to handle tasks asynchronously, ensuring that the workflow continues to run even if the caller disconnects ^[1].

3. Retrieve Results: The caller can periodically check the status of the workflow and retrieve results once they are available. This can be done by polling the workflow status or using a callback mechanism to notify the caller when the workflow is complete.

By using this approach, you can ensure that workflows run independently and asynchronously, which is particularly useful for integrating with third-party platforms like WeChat that require batch processing.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}