Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Tool signature error for Anthropic when tool parameter includes a class #1091

Closed
jk10001 opened this issue Feb 22, 2025 · 0 comments · Fixed by #1101
Closed

[Bug]: Tool signature error for Anthropic when tool parameter includes a class #1091

jk10001 opened this issue Feb 22, 2025 · 0 comments · Fixed by #1101
Labels
bug Something isn't working

Comments

@jk10001
Copy link
Contributor

jk10001 commented Feb 22, 2025

Describe the bug

With anthropic models there is an issue with the tool signature if the function registered has a parameter that is defined by a class.

The code below results in the error:
anthropic.BadRequestError: Error code: 400 - {'type': 'error', 'error': {'type': 'invalid_request_error', 'message': 'tools.1.custom.input_schema: JSON schema is invalid. It must match JSON Schema draft 2020-12 (https://json-schema.org/draft/2020-12). Learn more about tool use at https://docs.anthropic.com/en/docs/tool-use.'}}

Steps to reproduce

Code below produces the error. Replace config_claude_haiku_35 with your config.

from autogen import register_function
from typing import List, TypedDict
import os
from autogen import ConversableAgent
import pprint
from dotenv import load_dotenv
import json


class class_test(TypedDict):
    item1: str
    item2: str


def tool_call_test(information: str, class_test_list: List[class_test]) -> str:
    return "TERMINATE"


assistant = ConversableAgent(
    name="Assistant",
    system_message="You are a helpful AI assistant. "
    "Return 'TERMINATE' when the task is done.",
    llm_config={
        "cache_seed": False,
        "config_list": config_claude_haiku_35,
    },
)


user_proxy = ConversableAgent(
    name="User",
    llm_config={
        "cache_seed": False,
    },
    human_input_mode="ALWAYS",
)


register_function(
    tool_call_test,
    caller=assistant,
    executor=user_proxy,
    name="tool_call_test",
    description="A function to add record some information.",
)

pprint.pprint(assistant.llm_config["tools"])

message1 = "Make a call to tool_call_test. Make up the data."

chat_result = user_proxy.initiate_chat(assistant, message=message1)
`

The tool signature that is registered automatically is:
`
[
    {
        "type": "function",
        "function": {
            "description": "A function to add record some information.",
            "name": "tool_call_test",
            "parameters": {
                "type": "object",
                "properties": {
                    "information": {
                        "type": "string",
                        "description": "information"
                    },
                    "class_test_list": {
                        "$defs": {
                            "class_test": {
                                "properties": {
                                    "item1": {
                                        "title": "Item1",
                                        "type": "string"
                                    },
                                    "item2": {
                                        "title": "Item2",
                                        "type": "string"
                                    }
                                },
                                "required": [
                                    "item1",
                                    "item2"
                                ],
                                "title": "class_test",
                                "type": "object"
                            }
                        },
                        "items": {
                            "$ref": "#/$defs/class_test"
                        },
                        "type": "array",
                        "description": "class_test_list"
                    }
                },
                "required": [
                    "information",
                    "class_test_list"
                ]
            }
        }
    }
]

The error is caused by
"$ref": "#/$defs/class_test".
If you manually change the tool signature with update_tool_signature so this is
"$ref": "#/properties/class_test_list/$defs/class_test"
Claude uses the function correctly.

Other LLMs (e.g. llama-3.3-70b) seem to be able to use the tool correctly with either tool signature.

Model Used

Claude Haiku 3.5, Sonnet 3.5

Expected Behavior

No response

Screenshots and logs

No response

Additional Information

No response

@jk10001 jk10001 added the bug Something isn't working label Feb 22, 2025
marufaytekin added a commit to marufaytekin/ag2 that referenced this issue Feb 24, 2025
github-merge-queue bot pushed a commit that referenced this issue Feb 25, 2025
* [Bug Fix] Tool signature error for Anthropic #1091

* tests added

* updated tests to reflect test coverage

* pre-commit checks

---------

Co-authored-by: Davor Runje <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant