Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Tool calling kwargs #7910

Open
ButterflyAtHeart opened this issue Mar 5, 2025 · 3 comments
Open

[Bug] Tool calling kwargs #7910

ButterflyAtHeart opened this issue Mar 5, 2025 · 3 comments
Labels
bug Something isn't working

Comments

@ButterflyAtHeart
Copy link

What happened?

I am working with llama3.1 70 b and when calling tools in my react agent, the model sometimes predicts the next_tool_args to be {"foo": "bar"} instead of {"kwargs":{"foo":"bar"}.
This raises a ValueError: Arg foo is not in the tool's args.

I am not sure if this is a design choice, but if not please let me know, and I could try to implement a fix for this.

Steps to reproduce

import dspy
f = dspy.Tool(
            func=lambda **kwargs: "Completed.",
            name="finish",
            desc=f"Signals that the final outputs are now available and marks the task as complete.",
            args={}
        )
f(foo="bar")

DSPy version

2.6.10

@ButterflyAtHeart ButterflyAtHeart added the bug Something isn't working label Mar 5, 2025
@okhat
Copy link
Collaborator

okhat commented Mar 5, 2025

Thank you @ButterflyAtHeart ! This makes sense; let's figure out how to fix it.

First, does this actually cause a problem? I guess it shows up in the trajectory as an exception, which is confusing to the user and to the model? Either way, yes we do want to fix this.

@ButterflyAtHeart
Copy link
Author

You are right, it does not pose a real problem except showing up in trajectory, and maybe confusing the user as well as maybe saving another round of where the LLM has to correct this.
My current idea is to test for _ParameterKind.VAR_KEYWORD in the signature after line 88 of tools.py. and then setting a new instance variable of Tool, to look out for that. This we can then use to adjust Tool.__call__ .
If this sounds confusing I am sorry for that. I will open a PR tomorrow and you can check it out.

@okhat
Copy link
Collaborator

okhat commented Mar 5, 2025

This makes sense but it sounds like it might add a bunch of code. I wonder how we can "fix" this by simply ignoring the args to finish entirely... I want to avoid making the code have a lot of special cases.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants