Skip to content

Problems in k8s.wait for istio objects #39

Closed
#40
Closed
Problems in k8s.wait for istio objects#39
#40
Assignees
Cali0707
Labels
bugSomething isn't working

Description

@aljesusg
aljesusg
opened on Apr 13, 2026

Hi. I am having problems to check the condition wait.
verification failed: verify[0] failed: failed to execute kubernetes.wait: context deadline exceeded
I tried with cases:

condition: Available
status: "True"

and
condition: ready

Can we add control objects type Gateway without condition ?

the k8s.delete is working

Task yaml 
kind: Task
apiVersion: mcpchecker/v1alpha2
metadata:
  labels:
    suite: kiali
    demo: sample
  name: "Create a gateway"
  category: "Istio Configuration & Management"
  difficulty: medium
spec: 
  requires:
    - extension: kubernetes
      as: k8s
  verify:
    - k8s.wait:
        apiVersion: networking.istio.io/v1
        kind: Gateway
        metadata:
          name: my-gateway
          namespace: istio-system
        condition: Available
        status: "True" 
        timeout: 120s   
  cleanup:
    - k8s.delete:
        apiVersion: networking.istio.io/v1
        kind: Gateway
        metadata:
          name: my-gateway
          namespace: istio-system
        ignoreNotFound: true
  prompt:
    inline: Create a Gateway named my-gateway in the istio-system namespace.
Object created kubectl get gw my-gateway -n istio-system -o **yaml** 
apiVersion: networking.istio.io/v1
kind: Gateway
metadata:
  creationTimestamp: "2026-04-13T09:01:34Z"
  generation: 1
  name: my-gateway
  namespace: istio-system
  resourceVersion: "8160"
  uid: 6bafb433-5245-4283-89ba-1de0530c1336
spec:
  selector:
    istio: ingressgateway
  servers:
  - hosts:
    - '*'
    port:
      name: http
      number: 80
      protocol: HTTP
Output eval 
[
  {
    "taskName": "Create a gateway",
    "taskPath": "/home/agutierr/Projects/kiali/kiali/tests/evals/tasks/istio-create/istio-create.yaml",
    "taskPassed": false,
    "taskOutput": "{\"type\":\"init\",\"timestamp\":\"2026-04-13T09:51:39.212Z\",\"session_id\":\"7cac2baa-050d-4e00-a7bc-7c3b2b7feb83\",\"model\":\"gemini-2.5-flash-lite\"}\n{\"type\":\"message\",\"timestamp\":\"2026-04-13T09:51:39.213Z\",\"role\":\"user\",\"content\":\"Create a Gateway named my-gateway in the istio-system namespace.\"}\n{\"type\":\"tool_use\",\"timestamp\":\"2026-04-13T09:51:44.845Z\",\"tool_name\":\"mcp_kubernetes_kiali_manage_istio_config\",\"tool_id\":\"mcp_kubernetes_kiali_manage_istio_config_1776073904845_0\",\"parameters\":{\"version\":\"v1\",\"action\":\"create\",\"group\":\"networking.istio.io\",\"namespace\":\"istio-system\",\"data\":\"{\\n  \\\"apiVersion\\\": \\\"networking.istio.io/v1\\\",\\n  \\\"kind\\\": \\\"Gateway\\\",\\n  \\\"metadata\\\": {\\n    \\\"name\\\": \\\"my-gateway\\\",\\n    \\\"namespace\\\": \\\"istio-system\\\"\\n  },\\n  \\\"spec\\\": {\\n    \\\"selector\\\": {\\n      \\\"istio\\\": \\\"ingressgateway\\\"\\n    },\\n    \\\"servers\\\": [\\n      {\\n        \\\"port\\\": {\\n          \\\"number\\\": 80,\\n          \\\"name\\\": \\\"http\\\",\\n          \\\"protocol\\\": \\\"HTTP\\\"\\n        },\\n        \\\"hosts\\\": [\\\"*\\\"]\\n      }\\n    ]\\n  }\\n}\",\"confirmed\":false,\"object\":\"my-gateway\",\"kind\":\"Gateway\"}}\n{\"type\":\"tool_result\",\"timestamp\":\"2026-04-13T09:51:44.884Z\",\"tool_id\":\"mcp_kubernetes_kiali_manage_istio_config_1776073904845_0\",\"status\":\"success\",\"output\":\"\\\"Successfully created Gateway \\\\\\\"my-gateway\\\\\\\" in namespace \\\\\\\"istio-system\\\\\\\"\\\"\"}\n{\"type\":\"message\",\"timestamp\":\"2026-04-13T09:51:47.052Z\",\"role\":\"assistant\",\"content\":\"Gateway `my-gateway` has been created in the `istio-system` namespace.\",\"delta\":true}\n{\"type\":\"result\",\"timestamp\":\"2026-04-13T09:51:47.055Z\",\"status\":\"success\",\"stats\":{\"total_tokens\":29237,\"input_tokens\":28028,\"output_tokens\":230,\"cached\":25876,\"input\":2152,\"duration_ms\":7843,\"tool_calls\":1,\"models\":{\"gemini-2.5-flash-lite\":{\"total_tokens\":29237,\"input_tokens\":28028,\"output_tokens\":230,\"cached\":25876,\"input\":2152}}}}\n",
    "taskError": "verification failed: verify[0] failed: failed to execute kubernetes.wait: context deadline exceeded",
    "difficulty": "medium",
    "totalRuns": 1,
    "assertionResults": {
      "toolsUsed": {
        "passed": true
      },
      "minToolCalls": {
        "passed": true
      },
      "maxToolCalls": {
        "passed": true
      }
    },
    "allAssertionsPassed": true,
    "callHistory": {
      "ToolCalls": [
        {
          "serverName": "kubernetes",
          "timestamp": "2026-04-13T11:51:44.857250211+02:00",
          "success": true,
          "name": "kiali_manage_istio_config",
          "result": {
            "content": [
              {
                "type": "text",
                "text": "\"Successfully created Gateway \\\"my-gateway\\\" in namespace \\\"istio-system\\\"\""
              }
            ]
          },
          "tokens": {
            "inputTokens": 189,
            "outputTokens": 31,
            "totalTokens": 220
          },
          "request": {
            "Session": {},
            "Params": {
              "_meta": {
                "progressToken": "159b750b-fda0-4496-b1ad-71b5001c6ed7"
              },
              "name": "kiali_manage_istio_config",
              "arguments": {
                "version": "v1",
                "action": "create",
                "group": "networking.istio.io",
                "namespace": "istio-system",
                "data": "{\n  \"apiVersion\": \"networking.istio.io/v1\",\n  \"kind\": \"Gateway\",\n  \"metadata\": {\n    \"name\": \"my-gateway\",\n    \"namespace\": \"istio-system\"\n  },\n  \"spec\": {\n    \"selector\": {\n      \"istio\": \"ingressgateway\"\n    },\n    \"servers\": [\n      {\n        \"port\": {\n          \"number\": 80,\n          \"name\": \"http\",\n          \"protocol\": \"HTTP\"\n        },\n        \"hosts\": [\"*\"]\n      }\n    ]\n  }\n}",
                "confirmed": false,
                "object": "my-gateway",
                "kind": "Gateway"
              }
            },
            "Extra": {
              "TokenInfo": null,
              "Header": {
                "Accept": [
                  "application/json, text/event-stream"
                ],
                "Accept-Encoding": [
                  "gzip, deflate"
                ],
                "Accept-Language": [
                  "*"
                ],
                "Connection": [
                  "keep-alive"
                ],
                "Content-Length": [
                  "782"
                ],
                "Content-Type": [
                  "application/json"
                ],
                "Mcp-Protocol-Version": [
                  "2025-11-25"
                ],
                "Mcp-Session-Id": [
                  "BMO3F2E47E3WJAEVNGVSEUNYQQ"
                ],
                "Sec-Fetch-Mode": [
                  "cors"
                ],
                "User-Agent": [
                  "node"
                ]
              }
            }
          }
        }
      ],
      "ResourceReads": null,
      "PromptGets": null
    },
    "tokenEstimate": {
      "inputTokens": 2769,
      "outputTokens": 189,
      "totalTokens": 2958,
      "promptTokens": 0,
      "messageTokens": 0,
      "thinkingTokens": 0,
      "toolInputTokens": 189,
      "toolOutputTokens": 31,
      "mcpSchemaTokens": 2738,
      "promptGetInputTokens": 0,
      "promptGetOutputTokens": 0,
      "resourceInputTokens": 0,
      "resourceOutputTokens": 0,
      "error": "token estimation not supported for shell runner"
    },
    "setupOutput": {
      "Steps": [],
      "Success": true,
      "Error": ""
    },
    "agentOutput": {
      "Steps": [
        {
          "type": "message",
          "success": true,
          "message": "{\"type\":\"init\",\"timestamp\":\"2026-04-13T09:51:39.212Z\",\"session_id\":\"7cac2baa-050d-4e00-a7bc-7c3b2b7feb83\",\"model\":\"gemini-2.5-flash-lite\"}\n{\"type\":\"message\",\"timestamp\":\"2026-04-13T09:51:39.213Z\",\"role\":\"user\",\"content\":\"Create a Gateway named my-gateway in the istio-system namespace.\"}\n{\"type\":\"tool_use\",\"timestamp\":\"2026-04-13T09:51:44.845Z\",\"tool_name\":\"mcp_kubernetes_kiali_manage_istio_config\",\"tool_id\":\"mcp_kubernetes_kiali_manage_istio_config_1776073904845_0\",\"parameters\":{\"version\":\"v1\",\"action\":\"create\",\"group\":\"networking.istio.io\",\"namespace\":\"istio-system\",\"data\":\"{\\n  \\\"apiVersion\\\": \\\"networking.istio.io/v1\\\",\\n  \\\"kind\\\": \\\"Gateway\\\",\\n  \\\"metadata\\\": {\\n    \\\"name\\\": \\\"my-gateway\\\",\\n    \\\"namespace\\\": \\\"istio-system\\\"\\n  },\\n  \\\"spec\\\": {\\n    \\\"selector\\\": {\\n      \\\"istio\\\": \\\"ingressgateway\\\"\\n    },\\n    \\\"servers\\\": [\\n      {\\n        \\\"port\\\": {\\n          \\\"number\\\": 80,\\n          \\\"name\\\": \\\"http\\\",\\n          \\\"protocol\\\": \\\"HTTP\\\"\\n        },\\n        \\\"hosts\\\": [\\\"*\\\"]\\n      }\\n    ]\\n  }\\n}\",\"confirmed\":false,\"object\":\"my-gateway\",\"kind\":\"Gateway\"}}\n{\"type\":\"tool_result\",\"timestamp\":\"2026-04-13T09:51:44.884Z\",\"tool_id\":\"mcp_kubernetes_kiali_manage_istio_config_1776073904845_0\",\"status\":\"success\",\"output\":\"\\\"Successfully created Gateway \\\\\\\"my-gateway\\\\\\\" in namespace \\\\\\\"istio-system\\\\\\\"\\\"\"}\n{\"type\":\"message\",\"timestamp\":\"2026-04-13T09:51:47.052Z\",\"role\":\"assistant\",\"content\":\"Gateway `my-gateway` has been created in the `istio-system` namespace.\",\"delta\":true}\n{\"type\":\"result\",\"timestamp\":\"2026-04-13T09:51:47.055Z\",\"status\":\"success\",\"stats\":{\"total_tokens\":29237,\"input_tokens\":28028,\"output_tokens\":230,\"cached\":25876,\"input\":2152,\"duration_ms\":7843,\"tool_calls\":1,\"models\":{\"gemini-2.5-flash-lite\":{\"total_tokens\":29237,\"input_tokens\":28028,\"output_tokens\":230,\"cached\":25876,\"input\":2152}}}}\n"
        }
      ],
      "Success": true,
      "Error": "",
      "agentDetails": {
        "tokenEstimate": {
          "inputTokens": 2769,
          "outputTokens": 189,
          "totalTokens": 2958,
          "promptTokens": 0,
          "messageTokens": 0,
          "thinkingTokens": 0,
          "toolInputTokens": 189,
          "toolOutputTokens": 31,
          "mcpSchemaTokens": 2738,
          "promptGetInputTokens": 0,
          "promptGetOutputTokens": 0,
          "resourceInputTokens": 0,
          "resourceOutputTokens": 0,
          "error": "token estimation not supported for shell runner"
        },
        "outputSteps": [
          {
            "type": "message",
            "content": "{\"type\":\"init\",\"timestamp\":\"2026-04-13T09:51:39.212Z\",\"session_id\":\"7cac2baa-050d-4e00-a7bc-7c3b2b7feb83\",\"model\":\"gemini-2.5-flash-lite\"}\n{\"type\":\"message\",\"timestamp\":\"2026-04-13T09:51:39.213Z\",\"role\":\"user\",\"content\":\"Create a Gateway named my-gateway in the istio-system namespace.\"}\n{\"type\":\"tool_use\",\"timestamp\":\"2026-04-13T09:51:44.845Z\",\"tool_name\":\"mcp_kubernetes_kiali_manage_istio_config\",\"tool_id\":\"mcp_kubernetes_kiali_manage_istio_config_1776073904845_0\",\"parameters\":{\"version\":\"v1\",\"action\":\"create\",\"group\":\"networking.istio.io\",\"namespace\":\"istio-system\",\"data\":\"{\\n  \\\"apiVersion\\\": \\\"networking.istio.io/v1\\\",\\n  \\\"kind\\\": \\\"Gateway\\\",\\n  \\\"metadata\\\": {\\n    \\\"name\\\": \\\"my-gateway\\\",\\n    \\\"namespace\\\": \\\"istio-system\\\"\\n  },\\n  \\\"spec\\\": {\\n    \\\"selector\\\": {\\n      \\\"istio\\\": \\\"ingressgateway\\\"\\n    },\\n    \\\"servers\\\": [\\n      {\\n        \\\"port\\\": {\\n          \\\"number\\\": 80,\\n          \\\"name\\\": \\\"http\\\",\\n          \\\"protocol\\\": \\\"HTTP\\\"\\n        },\\n        \\\"hosts\\\": [\\\"*\\\"]\\n      }\\n    ]\\n  }\\n}\",\"confirmed\":false,\"object\":\"my-gateway\",\"kind\":\"Gateway\"}}\n{\"type\":\"tool_result\",\"timestamp\":\"2026-04-13T09:51:44.884Z\",\"tool_id\":\"mcp_kubernetes_kiali_manage_istio_config_1776073904845_0\",\"status\":\"success\",\"output\":\"\\\"Successfully created Gateway \\\\\\\"my-gateway\\\\\\\" in namespace \\\\\\\"istio-system\\\\\\\"\\\"\"}\n{\"type\":\"message\",\"timestamp\":\"2026-04-13T09:51:47.052Z\",\"role\":\"assistant\",\"content\":\"Gateway `my-gateway` has been created in the `istio-system` namespace.\",\"delta\":true}\n{\"type\":\"result\",\"timestamp\":\"2026-04-13T09:51:47.055Z\",\"status\":\"success\",\"stats\":{\"total_tokens\":29237,\"input_tokens\":28028,\"output_tokens\":230,\"cached\":25876,\"input\":2152,\"duration_ms\":7843,\"tool_calls\":1,\"models\":{\"gemini-2.5-flash-lite\":{\"total_tokens\":29237,\"input_tokens\":28028,\"output_tokens\":230,\"cached\":25876,\"input\":2152}}}}\n"
          }
        ]
      }
    },
    "verifyOutput": {
      "Steps": [
        null
      ],
      "Success": false,
      "Error": "failed to execute kubernetes.wait: context deadline exceeded"
    },
    "cleanupOutput": {
      "Steps": [
        null
      ],
      "Success": false,
      "Error": "failed to execute kubernetes.delete: context deadline exceeded"
    }
  }
]

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions