Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: max_tokens set to 1000 for "auto_generate_name" #14247

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

yzztin
Copy link

@yzztin yzztin commented Feb 24, 2025

Summary

Close #14246

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Tip

Close issue syntax: Fixes #<issue number> or Resolves #<issue number>, see documentation for more details.

Screenshots

Before After
... ...

Checklist

Important

Please review the checklist below before submitting your pull request.

  • This change requires a documentation update, included: Dify Document
  • I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
  • I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
  • I've updated the documentation accordingly.
  • I ran dev/reformat(backend) and cd web && npx lint-staged(frontend) to appease the lint gods

@dosubot dosubot bot added size:XS This PR changes 0-9 lines, ignoring generated files. 🐞 bug Something isn't working labels Feb 24, 2025
@yzztin
Copy link
Author

yzztin commented Feb 24, 2025

I actually don't konw is it reasonable to set max_tokens be 1000, just in my case, it worked.

@crazywoola
Copy link
Member

I actually don't konw is it reasonable to set max_tokens be 1000, just in my case, it worked.

I don't think this is a good idea, because you have more token consumption later.

@MrWangChong
Copy link

我实际上不知道这样设置是否合理max_tokens1000但就我的情况而言,它有效。

May I ask how the code modification will take effect? I will use it in sequence:
docker compose down
docker compose pull
docker compose up -d
The result still doesn't work

I actually don't konw is it reasonable to set max_tokens be 1000, just in my case, it worked.

@yzztin
May I ask how the code modification will take effect? I will use it in sequence:
docker compose down
docker compose pull
docker compose up -d
The result still doesn't work

@yzztin
Copy link
Author

yzztin commented Feb 26, 2025

@MrWangChong @littlebox692
If you want to resolve the problem not gracefully, you can change the max_token like me.
Steps for changing code in container:

  1. use GUI tools go into the dify api container like vscode docker plugin
  2. find the code in app/api/core/llm_generator/llm_generator.py, the functions is generate_conversation_name()
  3. change the code nearly line 51: (just make the max_tokens bigger, a num which is enough for you)
original code: prompt_messages=prompts, model_parameters={"max_tokens": 100, "temperature": 1}, stream=False
changed code: prompt_messages=prompts, model_parameters={"max_tokens": 1000, "temperature": 1}, stream=False
  1. restart the container, but not down, also you can docker commit it.

But, attention, it's not a good idea like @crazywoola said. What I recommend is waiting the official solution.

@MrWangChong
Copy link

@MrWangChong @littlebox692 If you want to resolve the problem not gracefully, you can change the max_token like me. Steps for changing code in container:

  1. use GUI tools go into the dify api container like vscode docker plugin
  2. find the code in app/api/core/llm_generator/llm_generator.py, the functions is generate_conversation_name()
  3. change the code nearly line 51: (just make the max_tokens bigger, a num which is enough for you)
original code: prompt_messages=prompts, model_parameters={"max_tokens": 100, "temperature": 1}, stream=False
changed code: prompt_messages=prompts, model_parameters={"max_tokens": 1000, "temperature": 1}, stream=False
  1. restart the container, but not down, also you can docker commit it.

But, attention, it's not a good idea like @crazywoola said. What I recommend is waiting the official solution.

@yzztin
Thank you for your response
I am certain that I have modified the max_token and restarted Docker. But it still hasn't taken effect.
This problem should only occur with deepseek-r1. I have tried switching to qwen2.5 and deepseek-v2, but still have this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐞 bug Something isn't working size:XS This PR changes 0-9 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

The auto_generate_name param doesn't work correctly when r1 is used as the default model.
3 participants