API Reference
!!! note Full Genkit documentation is available at genkit.dev
genkit
Genkit: generation, tools, structured output, flows and agents for Python.
Register plugins, define flows and tools, and run generation.
Example
from genkit import Genkit from genkit_google_genai import GoogleAI
ai = Genkit(plugins=[GoogleAI()], model=GoogleAI.gemini('gemini-flash-latest'))
@ai.flow async def suggest_dish(cuisine: str) -> str: res = await ai.generate(prompt=f'Suggest one {cuisine} dish.') return res.text
async def main() -> None: print(await suggest_dish('Thai')) # => Green curry with chicken
if name == 'main': ai.run_main(main())
ToolChoice = Literal['auto', 'required', 'none']
module-attribute
Whether the model may call tools (auto), must call one (required), or must not (none).
Document
Bases: genkit._core._base.GenkitModel
Multi-part document that can be embedded, indexed, or retrieved.
Source code in packages/genkit/src/genkit/_core/_model.py
1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 | |
media
cached
property
All media parts.
text
cached
property
Concatenate all text parts.
__init__(content, metadata=None)
Initialize with content parts and optional metadata.
Source code in packages/genkit/src/genkit/_core/_model.py
1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 | |
from_media(url, content_type=None, metadata=None)
classmethod
Create a document from a media URL. metadata goes on the document.
Source code in packages/genkit/src/genkit/_core/_model.py
1261 1262 1263 1264 1265 1266 1267 1268 1269 | |
from_text(text, metadata=None)
classmethod
Create a document from a text string. metadata goes on the document.
Source code in packages/genkit/src/genkit/_core/_model.py
1256 1257 1258 1259 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
Embedding
Bases: genkit._core._base.GenkitModel
Model for embedding data.
Source code in packages/genkit/src/genkit/_core/_typing.py
325 326 327 328 329 330 331 332 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
ErrorReason
Bases: genkit._core._compat.StrEnum
Why a failure happened, beyond its status.
Genkit sets a reason when it classifies its own failures, and plugins set
one when they classify a provider failure (provider_error(reason=...)),
so app code can match on it whichever plugin raised. Read it from
GenkitError.reason or Failure.reason. It travels in
details['reason']; the other SDKs don't have it.
Source code in packages/genkit/src/genkit/_core/_error.py
89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 | |
Failure
Bases: genkit._core._typing._RuntimeErrorWire
A failure value on response.error, Operation.error, AgentOutput.error, or SessionSnapshot.error.
Not an exception: generate and agents return failures as values. reason
is set when the framework classified the failure.
response = await ai.generate(prompt='Refund order A-17.', tools=['refund_order'])
response.error.status # => 'INTERNAL'
response.error.message # => 'internal error'
Source code in packages/genkit/src/genkit/_core/_error.py
141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
FinishReason
Bases: genkit._core._compat.StrEnum
FinishReason data type class.
Source code in packages/genkit/src/genkit/_core/_typing.py
88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 | |
is_abnormal()
True when generation stopped abnormally (blocked, aborted, failed, interrupted, other).
Source code in packages/genkit/src/genkit/_core/_typing.py
100 101 102 103 104 105 106 107 108 | |
Flow
Bases: genkit._core._action.Action[genkit._core._action.InputT, genkit._core._action.OutputT, genkit._core._action.ChunkT, None]
A typed, traced, streamable flow action.
@ai.flow constructs and registers a Flow on the Genkit instance.
Constructing Flow(fn) directly builds a standalone (unregistered) flow.
Source code in packages/genkit/src/genkit/_core/_flow.py
48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 | |
config_schema
property
The config class set at definition time, from config_schema= or the fn annotation.
input_class
property
The action's input annotation as a concrete class, when it is one.
params
property
The function's input and context parameters, as Genkit passes them.
__call__(input=NO_INPUT, *, context=None)
async
__call__(
input: None = None,
*,
context: dict[str, Any] | None = None
) -> OutputT
__call__(
input: InputT, *, context: dict[str, Any] | None = None
) -> OutputT
Call the action directly, returning just the response value.
Source code in packages/genkit/src/genkit/_core/_action.py
1350 1351 1352 1353 1354 1355 1356 1357 | |
__init__(func, *, name=None, description=None)
__init__(
func: Callable[
[FlowContext[ChunkT], object], Awaitable[OutputT]
],
*,
name: str | None = None,
description: str | None = None
) -> None
__init__(
func: DefaultInputCtxFn[
FlowContext[_FlowChunkT], _FlowInputT, _FlowOutputT
],
*,
name: str | None = None,
description: str | None = None
) -> None
__init__(
func: Callable[
[InputT, FlowContext[ChunkT]], Awaitable[OutputT]
],
*,
name: str | None = None,
description: str | None = None
) -> None
__init__(
func: CtxDefaultInputFn[
FlowContext[_FlowChunkT], _FlowInputT, _FlowOutputT
],
*,
name: str | None = None,
description: str | None = None
) -> None
__init__(
func: Callable[
[FlowContext[ChunkT], InputT], Awaitable[OutputT]
],
*,
name: str | None = None,
description: str | None = None
) -> None
__init__(
func: Callable[
[FlowContext[ChunkT]], Awaitable[OutputT]
],
*,
name: str | None = None,
description: str | None = None
) -> None
__init__(
func: DefaultInputFn[_FlowInputT, _FlowOutputT],
*,
name: str | None = None,
description: str | None = None
) -> None
__init__(
func: Callable[[InputT], Awaitable[OutputT]],
*,
name: str | None = None,
description: str | None = None
) -> None
__init__(
func: Callable[[], Awaitable[OutputT]],
*,
name: str | None = None,
description: str | None = None
) -> None
Initialize a Flow from func.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
func
|
collections.abc.Callable[..., collections.abc.Awaitable[typing.Any]]
|
Async flow handler taking at most one input and an optional
|
required |
name
|
str | None
|
Optional flow name. Defaults to |
None
|
description
|
str | None
|
Optional description. Defaults to |
None
|
Source code in packages/genkit/src/genkit/_core/_flow.py
141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 | |
run(input=NO_INPUT, *, on_chunk=None, context=None, telemetry_labels=None, init=None)
async
run(
input: None = None,
*,
on_chunk: Callable[[ChunkT], None] | None = None,
context: dict[str, Any] | None = None,
telemetry_labels: dict[str, object] | None = None,
init: InitT | None = None
) -> ActionResponse[OutputT]
run(
input: InputT,
*,
on_chunk: Callable[[ChunkT], None] | None = None,
context: dict[str, Any] | None = None,
telemetry_labels: dict[str, object] | None = None,
init: InitT | None = None
) -> ActionResponse[OutputT]
Run the action and return its result with trace ids.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
input
|
genkit._core._action.InputT | None
|
The input, validated against the input schema. Required unless the action takes no input or its function gives the input a default; omit it to use that default. On a flow or tool, an explicit None is a value and is validated like any other. |
genkit._core._action.NO_INPUT
|
on_chunk
|
collections.abc.Callable[[genkit._core._action.ChunkT], None] | None
|
Called with each chunk the action streams. |
None
|
context
|
dict[str, typing.Any] | None
|
Request context (auth, headers, ...), read as |
None
|
telemetry_labels
|
dict[str, object] | None
|
Custom labels set as span attributes. |
None
|
init
|
genkit._core._action.InitT | None
|
Session init value for a bidi action (e.g. a served agent's session identity). Validated against the init schema; plain actions ignore it. |
None
|
Returns:
| Type | Description |
|---|---|
genkit._core._action.ActionResponse[genkit._core._action.OutputT]
|
ActionResponse with the result, trace id and span id. |
Raises:
| Type | Description |
|---|---|
genkit._core._error.GenkitError
|
If input validation fails (INVALID_ARGUMENT status). |
Source code in packages/genkit/src/genkit/_core/_action.py
1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 | |
stream(input=NO_INPUT, *, context=None, telemetry_labels=None, init=None)
stream(
input: None = None,
*,
context: dict[str, Any] | None = None,
telemetry_labels: dict[str, object] | None = None,
init: InitT | None = None
) -> StreamResponse[ChunkT, OutputT]
stream(
input: InputT,
*,
context: dict[str, Any] | None = None,
telemetry_labels: dict[str, object] | None = None,
init: InitT | None = None
) -> StreamResponse[ChunkT, OutputT]
Return a StreamResponse: iterate .stream for chunks, await .response for the result.
Takes the same keyword arguments as run, minus on_chunk. If the
action raises, both the loop and .response raise.
Source code in packages/genkit/src/genkit/_core/_action.py
1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 | |
FlowContext
Bases: genkit._core._action.ActionContext[genkit._core._action.ChunkT]
Execution context passed to flow functions.
Provides ctx.send_chunk(chunk) for streaming ChunkT chunks and
ctx.context for caller context data.
FlowContext and ToolContext are siblings, so neither annotation
accepts the other. A helper shared by a flow and a tool takes the union:
# 1. Read the caller's uid from either context
def user_id(ctx: FlowContext | ToolContext) -> str:
return ctx.context['auth']['uid']
# 2. Call it from a flow and from a tool
@ai.flow
async def reorder_last(dish: str, ctx: FlowContext) -> str:
return f'{dish} for {user_id(ctx)}'
@ai.tool
async def check_allergies(dish: str, ctx: ToolContext) -> str:
return f'{dish}: nut-free for {user_id(ctx)}'
# 3. Both see the same request context
await reorder_last('tartine', context={'auth': {'uid': 'diner-42'}})
# => 'tartine for diner-42'
await check_allergies('tartine', context={'auth': {'uid': 'diner-42'}})
# => 'tartine: nut-free for diner-42'
Source code in packages/genkit/src/genkit/_core/_action.py
1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 | |
context
property
The request context (auth, headers, ...) the caller attached.
is_streaming
property
True if a streaming callback is registered.
streaming_callback
property
The streaming callback, if any.
Use this when you need to pass the callback to another action. For sending chunks directly, use send_chunk() instead.
send_chunk(chunk)
Send a streaming chunk to the client.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
chunk
|
genkit._core._action.ChunkT
|
The chunk data to stream. |
required |
Source code in packages/genkit/src/genkit/_core/_action.py
1053 1054 1055 1056 1057 1058 1059 1060 | |
Genkit
The main entry point for building AI-powered applications.
Registers plugins, defines flows and tools, and runs generation. Both
Genkit(model=...) and ai.generate(model=...) accept a typed
ModelRef from a plugin constructor (recommended, e.g.
GoogleAI.gemini('gemini-flash-latest')) or a plugin-prefixed
string ('googleai/gemini-flash-latest'). Plugins resolve models lazily
on demand when ai.generate() is called, without needing pre-registration
or SDK updates.
Example
from genkit import Genkit from genkit_google_genai import GoogleAI
ai = Genkit(plugins=[GoogleAI()], model=GoogleAI.gemini('gemini-flash-latest'))
@ai.tool async def current_weather(city: str) -> str: return f'Sunny in {city}'
@ai.flow async def weather_report(prompt: str) -> str: res = await ai.generate(prompt=prompt, tools=[current_weather]) return res.text
async def main() -> None: print(await weather_report('Weather in Paris?')) # => It's sunny in Paris.
if name == 'main': ai.run_main(main())
Source code in packages/genkit/src/genkit/_ai/_aio.py
205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 1790 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802 1803 1804 1805 1806 1807 1808 1809 1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 1820 1821 1822 1823 1824 1825 1826 1827 1828 1829 1830 1831 1832 1833 1834 1835 1836 1837 1838 1839 1840 1841 1842 1843 1844 1845 1846 1847 1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 1864 1865 1866 1867 1868 1869 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 2079 2080 2081 2082 2083 2084 2085 2086 2087 2088 2089 2090 2091 2092 2093 2094 2095 2096 2097 2098 2099 2100 2101 2102 2103 2104 2105 2106 2107 2108 2109 2110 2111 2112 2113 2114 2115 2116 2117 2118 2119 2120 2121 2122 2123 2124 2125 2126 2127 2128 2129 2130 2131 2132 2133 2134 2135 2136 2137 2138 2139 2140 2141 2142 2143 2144 2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2157 2158 2159 2160 2161 2162 2163 2164 2165 2166 2167 2168 2169 2170 2171 2172 2173 2174 2175 2176 2177 2178 2179 2180 2181 2182 2183 2184 2185 2186 2187 2188 2189 2190 2191 2192 2193 2194 2195 2196 2197 2198 2199 2200 2201 2202 2203 2204 2205 2206 2207 2208 2209 2210 2211 2212 2213 2214 2215 2216 2217 2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228 2229 2230 2231 2232 2233 2234 2235 2236 2237 2238 2239 2240 2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 2253 2254 2255 2256 2257 2258 2259 2260 2261 2262 2263 2264 2265 2266 2267 2268 2269 2270 2271 2272 2273 2274 2275 2276 2277 2278 2279 2280 2281 2282 2283 2284 2285 2286 2287 2288 2289 2290 2291 2292 2293 2294 2295 2296 2297 2298 2299 2300 2301 2302 2303 2304 2305 2306 2307 2308 2309 2310 2311 2312 2313 2314 2315 2316 2317 2318 2319 2320 2321 2322 2323 2324 2325 2326 2327 2328 2329 2330 2331 2332 2333 2334 2335 2336 2337 2338 2339 2340 2341 2342 2343 2344 2345 2346 2347 2348 2349 2350 2351 2352 2353 2354 2355 2356 2357 2358 2359 2360 2361 2362 2363 2364 2365 2366 2367 2368 2369 2370 2371 2372 2373 2374 2375 2376 2377 2378 2379 2380 2381 2382 2383 2384 2385 2386 2387 2388 2389 2390 2391 2392 2393 2394 2395 2396 2397 2398 2399 2400 2401 2402 2403 2404 2405 2406 2407 2408 2409 2410 2411 2412 2413 2414 2415 2416 2417 2418 2419 2420 2421 2422 2423 2424 2425 2426 2427 2428 2429 2430 2431 2432 2433 2434 2435 2436 2437 2438 2439 2440 2441 2442 2443 2444 2445 2446 2447 2448 2449 2450 2451 2452 2453 2454 2455 2456 2457 2458 2459 2460 2461 2462 2463 2464 2465 2466 2467 2468 2469 2470 2471 2472 2473 2474 2475 2476 2477 2478 2479 2480 2481 2482 2483 2484 2485 2486 2487 2488 2489 2490 2491 2492 2493 2494 2495 2496 2497 2498 2499 2500 2501 2502 2503 2504 2505 2506 2507 2508 2509 2510 2511 2512 2513 2514 2515 2516 2517 2518 2519 2520 2521 2522 2523 2524 2525 2526 2527 2528 2529 2530 2531 2532 2533 2534 2535 2536 2537 2538 2539 2540 2541 2542 2543 2544 2545 2546 2547 2548 2549 2550 2551 2552 2553 2554 2555 2556 2557 2558 2559 2560 2561 2562 2563 2564 2565 2566 2567 2568 2569 2570 2571 2572 2573 2574 2575 2576 2577 2578 2579 2580 2581 2582 2583 2584 2585 2586 2587 2588 2589 2590 2591 2592 2593 2594 2595 2596 2597 2598 2599 2600 2601 2602 2603 2604 2605 2606 2607 2608 2609 2610 2611 2612 2613 2614 2615 2616 2617 2618 2619 2620 2621 2622 2623 2624 2625 2626 2627 2628 2629 2630 2631 2632 2633 2634 2635 2636 2637 2638 2639 2640 2641 2642 2643 2644 2645 2646 2647 2648 2649 2650 2651 2652 2653 2654 2655 2656 2657 2658 2659 2660 2661 2662 2663 2664 2665 2666 2667 2668 2669 2670 2671 2672 2673 2674 2675 2676 2677 2678 2679 2680 2681 2682 2683 2684 2685 2686 2687 2688 2689 2690 2691 2692 2693 2694 2695 2696 2697 2698 2699 2700 2701 2702 2703 2704 2705 2706 2707 2708 2709 2710 2711 2712 2713 2714 2715 2716 2717 2718 2719 2720 2721 2722 2723 2724 2725 2726 2727 2728 2729 2730 2731 2732 2733 2734 2735 2736 2737 2738 2739 2740 2741 2742 2743 2744 2745 2746 2747 2748 2749 2750 2751 2752 2753 2754 2755 2756 2757 2758 2759 2760 2761 2762 2763 2764 2765 2766 2767 2768 2769 2770 2771 2772 2773 2774 2775 2776 2777 2778 2779 2780 2781 2782 2783 2784 2785 2786 2787 2788 2789 2790 2791 2792 2793 2794 2795 2796 2797 2798 2799 2800 2801 2802 2803 2804 2805 2806 2807 2808 2809 2810 2811 2812 2813 2814 2815 2816 2817 2818 2819 2820 2821 2822 2823 2824 2825 2826 2827 2828 2829 2830 2831 2832 2833 2834 2835 2836 2837 2838 2839 2840 2841 2842 2843 2844 2845 2846 2847 2848 2849 2850 2851 2852 2853 2854 2855 2856 2857 2858 2859 2860 2861 2862 2863 2864 2865 2866 2867 | |
__init__(*args, plugins=None, model=None, prompt_dir=None)
Initialize a Genkit instance.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
plugins
|
list[genkit._core._plugin.Plugin] | None
|
Plugins to register (e.g. |
None
|
model
|
genkit._ai._model.ModelArg | None
|
Default model for |
None
|
prompt_dir
|
str | pathlib.Path | None
|
Optional directory for loading |
None
|
Source code in packages/genkit/src/genkit/_ai/_aio.py
260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 | |
cancel_operation(operation, *, context=None, config=None)
async
cancel_operation(
operation: Operation[T],
*,
context: dict[str, Any] | None = None,
config: Mapping[str, Any] | None = None
) -> Operation[T]
cancel_operation(
operation: Mapping[str, Any],
*,
context: dict[str, Any] | None = None,
config: Mapping[str, Any] | None = None
) -> Operation[Any]
Cancel a background job.
Same input and return types as check_operation: Operation[T]
in, Operation[T] out; a stored dict in, Operation[Any] out.
Same context / config pockets.
Example
# 1. Reload the stored handle and cancel it
op = await ai.cancel_operation(json.loads(saved))
# 2. The returned handle reflects the cancel attempt
print(op.id)
# => operations/risotto-123
Source code in packages/genkit/src/genkit/_ai/_aio.py
2721 2722 2723 2724 2725 2726 2727 2728 2729 2730 2731 2732 2733 2734 2735 2736 2737 2738 2739 2740 2741 2742 2743 2744 | |
check_operation(operation, *, context=None, config=None)
async
check_operation(
operation: Operation[T],
*,
context: dict[str, Any] | None = None,
config: Mapping[str, Any] | None = None
) -> Operation[T]
check_operation(
operation: Mapping[str, Any],
*,
context: dict[str, Any] | None = None,
config: Mapping[str, Any] | None = None
) -> Operation[Any]
Poll a background job.
Generic like JS checkOperation<T>: an Operation[T] comes back
as Operation[T], a stored dict as Operation[Any]. The handle's
action key finds the job. For a background model the handle is
validated as Operation[ModelResponse] before anything runs: an
unknown key, a missing id, or an output that isn't a
ModelResponse is INVALID_ARGUMENT.
Pass context={'secrets': {'api_key': ...}} again when start used a
per-request key. config is client knobs (base_url,
location, api_version), not video settings.
Example
# 1. Start a job and store the handle as JSON
op = await ai.generate_operation(
model='googleai/veo-3.1-generate-preview',
prompt='A chef plating risotto.',
)
saved = op.model_dump_json()
# 2. Later, reload it typed and poll
op = await ai.check_operation(Operation[ModelResponse].model_validate_json(saved))
print(op.done)
# => False
# 3. Poll until done; output is a ModelResponse
while not op.done:
op = await ai.check_operation(op)
if op.output and op.output.media:
print(op.output.media.url)
# => https://storage.googleapis.com/.../risotto.mp4
Source code in packages/genkit/src/genkit/_ai/_aio.py
2659 2660 2661 2662 2663 2664 2665 2666 2667 2668 2669 2670 2671 2672 2673 2674 2675 2676 2677 2678 2679 2680 2681 2682 2683 2684 2685 2686 2687 2688 2689 2690 2691 2692 2693 2694 2695 2696 2697 2698 2699 2700 2701 | |
current_context()
staticmethod
Get the current execution context, or None if not in an action.
Source code in packages/genkit/src/genkit/_ai/_aio.py
2596 2597 2598 2599 | |
define_background_model(name, *, start, check, cancel=None, info=None, config_schema=None, metadata=None, description=None)
Register a background model for long-running AI operations.
Source code in packages/genkit/src/genkit/_ai/_aio.py
834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 | |
define_batch_evaluator(name, fn, *, display_name, definition, is_billed=False, config_schema=None, metadata=None, description=None)
Register a batch evaluator.
The function takes one EvaluatorRequest and returns list[EvaluatorResponse].
Read options from req.options. A second parameter raises TypeError when defined.
Source code in packages/genkit/src/genkit/_ai/_aio.py
765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 | |
define_dynamic_action_provider(name, fn, *, description=None, cache_ttl_millis=None, metadata=None)
Register a Dynamic Action Provider (DAP).
Source code in packages/genkit/src/genkit/_ai/_aio.py
504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 | |
define_embedder(name, fn, *, info=None, metadata=None, description=None, config_schema=None)
define_embedder(
name: str,
fn: EmbedderFn,
*,
info: EmbedderInfo | None = None,
metadata: dict[str, object] | None = None,
description: str | None = None,
config_schema: (
type[BaseModel] | dict[str, object] | None
) = None
) -> Embedder
define_embedder(
name: str,
fn: CtxFirstEmbedderFn,
*,
info: EmbedderInfo | None = None,
metadata: dict[str, object] | None = None,
description: str | None = None,
config_schema: (
type[BaseModel] | dict[str, object] | None
) = None
) -> Embedder
Register a custom embedder action.
The options class comes from config_schema or the fn's
EmbedRequest[Cfg] annotation; given both, they must be the same class.
Source code in packages/genkit/src/genkit/_ai/_aio.py
883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 | |
define_evaluator(name, fn, *, display_name, definition, is_billed=False, config_schema=None, metadata=None, description=None)
Register an evaluator that scores one dataset row at a time.
Example
# 1. Score whether the answer mentions the expected dish
async def mentions_dish(row: BaseDataPoint, options: object | None) -> EvaluatorResponse:
hit = row.reference.lower() in str(row.output).lower()
return EvaluatorResponse(test_case_id=row.test_case_id or '', evaluation=[Score(score=hit)])
# 2. Register it under a name
ai.define_evaluator(
'mentions_dish',
mentions_dish,
display_name='Mentions dish',
definition='Whether the answer names the expected dish.',
)
# 3. Run it over a dataset
rows = await ai.evaluate(evaluator='mentions_dish', dataset=dataset)
# => [EvaluatorResponse(test_case_id='...', evaluation=[Score(score=True)]), ...]
Source code in packages/genkit/src/genkit/_ai/_aio.py
718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 | |
define_format(formatter)
Register a custom output format. See genkit.formats.Formatter.
Source code in packages/genkit/src/genkit/_ai/_aio.py
900 901 902 | |
define_helper(name, fn)
Register a Handlebars helper for prompt templates.
Dotprompt calls the helper as fn(params, options): params is a
list of the positional arguments from the template, and options
carries the named arguments in options.hash. A one-argument
function such as lambda s: s.upper() raises TypeError at
render time. Return a str.
Example
# 1. Register a helper that takes one positional and one named argument
def menu_item(params: list[Any], options: Any) -> str:
dish = str(params[0])
return dish.upper() if options.hash.get('loud') else dish
ai.define_helper('menu_item', menu_item)
# 2. Use it in a prompt template
special = ai.define_prompt(
name='special',
model='googleai/gemini-flash-latest',
prompt="Describe today's special, {{menu_item dish loud=true}}, in one line.",
)
# 3. Render to check the expanded text
request = await special.render(input={'dish': 'smoked salmon tartine'})
print(request.messages[0].content[0].text)
# => Describe today's special, SMOKED SALMON TARTINE, in one line.
Source code in packages/genkit/src/genkit/_ai/_aio.py
456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 | |
define_interrupt(name, *, input_schema=None, description=None)
Register an interrupt tool that always pauses for user input.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str
|
Tool name |
required |
input_schema
|
type[pydantic.BaseModel] | dict[str, object] | None
|
Optional input schema (Pydantic model or JSON schema dict) |
None
|
description
|
str | None
|
Tool description |
None
|
Returns:
| Type | Description |
|---|---|
genkit._core._tool.Tool[typing.Any, typing.NoReturn]
|
The registered interrupt tool. Awaiting it directly raises |
Example
ask_user = ai.define_interrupt( name='ask_user', input_schema=Question, description='Ask the user a question', )
Source code in packages/genkit/src/genkit/_ai/_aio.py
687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 | |
define_json_schema(name, json_schema)
Register a JSON schema for use in prompts.
Source code in packages/genkit/src/genkit/_ai/_aio.py
499 500 501 502 | |
define_middleware(cls, *, name=None, description=None)
Register a middleware class on this app's registry.
Source code in packages/genkit/src/genkit/_ai/_aio.py
645 646 647 648 649 650 651 652 653 654 655 | |
define_model(name, fn, *, config_schema=None, metadata=None, info=None, description=None)
define_model(
name: str,
fn: ModelFn,
*,
config_schema: (
type[BaseModel] | dict[str, object] | None
) = None,
metadata: dict[str, object] | None = None,
info: ModelInfo | None = None,
description: str | None = None
) -> Model
define_model(
name: str,
fn: CtxFirstModelFn,
*,
config_schema: (
type[BaseModel] | dict[str, object] | None
) = None,
metadata: dict[str, object] | None = None,
info: ModelInfo | None = None,
description: str | None = None
) -> Model
Register a custom model action.
Source code in packages/genkit/src/genkit/_ai/_aio.py
821 822 823 824 825 826 827 828 829 830 831 832 | |
define_partial(name, source)
Register a Handlebars partial template.
Source code in packages/genkit/src/genkit/_ai/_aio.py
490 491 492 | |
define_prompt(name=None, *, variant=None, model=None, config=None, description=None, system=None, prompt=None, messages=None, output_format=None, output_content_type=None, output_instructions=None, output_constrained=None, max_turns=None, return_tool_requests=None, metadata=None, tools=None, tool_choice=None, use=None, docs=None, input_schema=None, output_schema=None)
define_prompt(
name: str | None = None,
*,
variant: str | None = None,
model: ModelRef[Any] | Action | str | None = None,
config: None = None,
description: str | None = None,
system: str | list[Part] | None = None,
prompt: str | list[Part] | None = None,
messages: str | list[Message] | None = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
max_turns: int | None = None,
return_tool_requests: bool | None = None,
metadata: dict[str, object] | None = None,
tools: Sequence[str | Tool] | None = None,
tool_choice: ToolChoice | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None,
input_schema: type[InputT],
output_schema: type[OutputT]
) -> Prompt[InputT, OutputT]
define_prompt(
name: str | None = None,
*,
variant: str | None = None,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
config: ModelConfigDict,
description: str | None = None,
system: str | list[Part] | None = None,
prompt: str | list[Part] | None = None,
messages: str | list[Message] | None = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
max_turns: int | None = None,
return_tool_requests: bool | None = None,
metadata: dict[str, object] | None = None,
tools: Sequence[str | Tool] | None = None,
tool_choice: ToolChoice | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None,
input_schema: type[InputT],
output_schema: type[OutputT]
) -> Prompt[InputT, OutputT]
define_prompt(
name: str | None = None,
*,
variant: str | None = None,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
config: ModelRefConfigT | Mapping[str, Any] | None,
description: str | None = None,
system: str | list[Part] | None = None,
prompt: str | list[Part] | None = None,
messages: str | list[Message] | None = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
max_turns: int | None = None,
return_tool_requests: bool | None = None,
metadata: dict[str, object] | None = None,
tools: Sequence[str | Tool] | None = None,
tool_choice: ToolChoice | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None,
input_schema: type[InputT],
output_schema: type[OutputT]
) -> Prompt[InputT, OutputT]
define_prompt(
name: str | None = None,
*,
variant: str | None = None,
model: ModelRef[Any] | Action | str | None = None,
config: None = None,
description: str | None = None,
system: str | list[Part] | None = None,
prompt: str | list[Part] | None = None,
messages: str | list[Message] | None = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
max_turns: int | None = None,
return_tool_requests: bool | None = None,
metadata: dict[str, object] | None = None,
tools: Sequence[str | Tool] | None = None,
tool_choice: ToolChoice | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None,
input_schema: type[InputT],
output_schema: dict[str, object] | str | None = None
) -> Prompt[InputT, Any]
define_prompt(
name: str | None = None,
*,
variant: str | None = None,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
config: ModelConfigDict,
description: str | None = None,
system: str | list[Part] | None = None,
prompt: str | list[Part] | None = None,
messages: str | list[Message] | None = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
max_turns: int | None = None,
return_tool_requests: bool | None = None,
metadata: dict[str, object] | None = None,
tools: Sequence[str | Tool] | None = None,
tool_choice: ToolChoice | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None,
input_schema: type[InputT],
output_schema: dict[str, object] | str | None = None
) -> Prompt[InputT, Any]
define_prompt(
name: str | None = None,
*,
variant: str | None = None,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
config: ModelRefConfigT | Mapping[str, Any] | None,
description: str | None = None,
system: str | list[Part] | None = None,
prompt: str | list[Part] | None = None,
messages: str | list[Message] | None = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
max_turns: int | None = None,
return_tool_requests: bool | None = None,
metadata: dict[str, object] | None = None,
tools: Sequence[str | Tool] | None = None,
tool_choice: ToolChoice | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None,
input_schema: type[InputT],
output_schema: dict[str, object] | str | None = None
) -> Prompt[InputT, Any]
define_prompt(
name: str | None = None,
*,
variant: str | None = None,
model: ModelRef[Any] | Action | str | None = None,
config: None = None,
description: str | None = None,
system: str | list[Part] | None = None,
prompt: str | list[Part] | None = None,
messages: str | list[Message] | None = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
max_turns: int | None = None,
return_tool_requests: bool | None = None,
metadata: dict[str, object] | None = None,
tools: Sequence[str | Tool] | None = None,
tool_choice: ToolChoice | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None,
input_schema: dict[str, object] | str | None = None,
output_schema: type[OutputT]
) -> Prompt[Any, OutputT]
define_prompt(
name: str | None = None,
*,
variant: str | None = None,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
config: ModelConfigDict,
description: str | None = None,
system: str | list[Part] | None = None,
prompt: str | list[Part] | None = None,
messages: str | list[Message] | None = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
max_turns: int | None = None,
return_tool_requests: bool | None = None,
metadata: dict[str, object] | None = None,
tools: Sequence[str | Tool] | None = None,
tool_choice: ToolChoice | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None,
input_schema: dict[str, object] | str | None = None,
output_schema: type[OutputT]
) -> Prompt[Any, OutputT]
define_prompt(
name: str | None = None,
*,
variant: str | None = None,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
config: ModelRefConfigT | Mapping[str, Any] | None,
description: str | None = None,
system: str | list[Part] | None = None,
prompt: str | list[Part] | None = None,
messages: str | list[Message] | None = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
max_turns: int | None = None,
return_tool_requests: bool | None = None,
metadata: dict[str, object] | None = None,
tools: Sequence[str | Tool] | None = None,
tool_choice: ToolChoice | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None,
input_schema: dict[str, object] | str | None = None,
output_schema: type[OutputT]
) -> Prompt[Any, OutputT]
define_prompt(
name: str | None = None,
*,
variant: str | None = None,
model: ModelRef[Any] | Action | str | None = None,
config: None = None,
description: str | None = None,
system: str | list[Part] | None = None,
prompt: str | list[Part] | None = None,
messages: str | list[Message] | None = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
max_turns: int | None = None,
return_tool_requests: bool | None = None,
metadata: dict[str, object] | None = None,
tools: Sequence[str | Tool] | None = None,
tool_choice: ToolChoice | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None,
input_schema: (
type | dict[str, object] | str | None
) = None,
output_schema: (
type | dict[str, object] | str | None
) = None
) -> Prompt[Any, Any]
define_prompt(
name: str | None = None,
*,
variant: str | None = None,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
config: ModelConfigDict,
description: str | None = None,
system: str | list[Part] | None = None,
prompt: str | list[Part] | None = None,
messages: str | list[Message] | None = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
max_turns: int | None = None,
return_tool_requests: bool | None = None,
metadata: dict[str, object] | None = None,
tools: Sequence[str | Tool] | None = None,
tool_choice: ToolChoice | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None,
input_schema: (
type | dict[str, object] | str | None
) = None,
output_schema: (
type | dict[str, object] | str | None
) = None
) -> Prompt[Any, Any]
define_prompt(
name: str | None = None,
*,
variant: str | None = None,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
config: ModelRefConfigT | Mapping[str, Any] | None,
description: str | None = None,
system: str | list[Part] | None = None,
prompt: str | list[Part] | None = None,
messages: str | list[Message] | None = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
max_turns: int | None = None,
return_tool_requests: bool | None = None,
metadata: dict[str, object] | None = None,
tools: Sequence[str | Tool] | None = None,
tool_choice: ToolChoice | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None,
input_schema: (
type | dict[str, object] | str | None
) = None,
output_schema: (
type | dict[str, object] | str | None
) = None
) -> Prompt[Any, Any]
Register a prompt template.
Example
joke = ai.define_prompt(name='joke', prompt='Tell a joke about {{topic}}.') res = await joke(input={'topic': 'cats'}) print(res.text)
=> Why did the cat sit on the keyboard? To keep an eye on the mouse.
Source code in packages/genkit/src/genkit/_ai/_aio.py
1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 | |
define_schema(name, schema)
Register a Pydantic schema for use in prompts.
Source code in packages/genkit/src/genkit/_ai/_aio.py
494 495 496 497 | |
define_value(*, kind, name, value)
Store value under kind and name for later lookup_value.
Defining the same kind and name again replaces the value, so a re-run notebook cell does not fail.
Raises:
| Type | Description |
|---|---|
ValueError
|
|
Source code in packages/genkit/src/genkit/_ai/_aio.py
946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 | |
embed(*, embedder, content, metadata=None, config=None, context=None)
async
embed(
*,
embedder: str | EmbedderRef[Any],
content: str | Document,
metadata: dict[str, object] | None = None,
config: None = None,
context: dict[str, Any] | None = None
) -> list[Embedding]
embed(
*,
embedder: str | EmbedderRef[EmbedderRefConfigT],
content: str | Document,
metadata: dict[str, object] | None = None,
config: EmbedderRefConfigT | Mapping[str, Any] | None,
context: dict[str, Any] | None = None
) -> list[Embedding]
Generate vector embeddings for a single document or string.
config is layered over the EmbedderRef's version and config
(the call wins per key) and reaches the embedder as
request.options. A config object must be the embedder's options
class. An embedder name that isn't registered raises GenkitError
with NOT_FOUND.
metadata is attached to a string content. A Document
already carries its own metadata, so passing both raises TypeError.
context reaches the embedder as ctx.context, as on
generate. Omitted, the embedder sees the enclosing action's
context. The GoogleAI, VertexAI and OpenAI embedders read a
per-request key from context={'secrets': {'api_key': ...}}, so
the tenant, not the plugin key, is billed. Ollama and Bedrock
embedders always use the plugin's client.
Example
from genkit_google_genai import GoogleAI
embeddings = await ai.embed( embedder=GoogleAI.embedder('gemini-embedding-001'), content='Hello world', ) vector = embeddings[0].embedding
Source code in packages/genkit/src/genkit/_ai/_aio.py
2419 2420 2421 2422 2423 2424 2425 2426 2427 2428 2429 2430 2431 2432 2433 2434 2435 2436 2437 2438 2439 2440 2441 2442 2443 2444 2445 2446 2447 2448 2449 2450 2451 2452 2453 2454 2455 2456 2457 2458 2459 2460 2461 2462 2463 2464 2465 2466 2467 2468 2469 2470 2471 | |
embed_many(*, embedder, content, metadata=None, config=None, context=None)
async
embed_many(
*,
embedder: str | EmbedderRef[Any],
content: Sequence[str | Document],
metadata: dict[str, object] | None = None,
config: None = None,
context: dict[str, Any] | None = None
) -> list[Embedding]
embed_many(
*,
embedder: str | EmbedderRef[EmbedderRefConfigT],
content: Sequence[str | Document],
metadata: dict[str, object] | None = None,
config: EmbedderRefConfigT | Mapping[str, Any] | None,
context: dict[str, Any] | None = None
) -> list[Embedding]
Generate vector embeddings for multiple documents in a single batch call.
content is any sequence (list, tuple, ...) of strings or
Documents; one embedding comes back per item, in order. A bare
str raises TypeError; use embed for one item.
config, metadata and context work the same as on embed.
Source code in packages/genkit/src/genkit/_ai/_aio.py
2497 2498 2499 2500 2501 2502 2503 2504 2505 2506 2507 2508 2509 2510 2511 2512 2513 2514 2515 2516 2517 2518 2519 2520 2521 2522 2523 2524 2525 2526 2527 2528 | |
evaluate(*, evaluator, dataset, config=None, eval_run_id=None)
async
Evaluate a dataset using the specified evaluator.
Returns a list of rows. A per-row evaluator gives one row per
datapoint, in dataset order. A batch evaluator gives the rows its
function returned, as returned. Each row's evaluation is a list of
scores.
config is merged over the EvaluatorRef's config (the call
wins per key) and handed to the evaluator as its second argument. When
neither sets anything, the evaluator gets None. An evaluator name
that isn't registered raises GenkitError with NOT_FOUND.
Example
from genkit.evaluator import BaseDataPoint
results = await ai.evaluate( evaluator='my_eval', dataset=[BaseDataPoint(input='What is 2+2?', output='4')], ) for row in results: for score in row.evaluation: print(row.test_case_id, score.score)
Source code in packages/genkit/src/genkit/_ai/_aio.py
2530 2531 2532 2533 2534 2535 2536 2537 2538 2539 2540 2541 2542 2543 2544 2545 2546 2547 2548 2549 2550 2551 2552 2553 2554 2555 2556 2557 2558 2559 2560 2561 2562 2563 2564 2565 2566 2567 2568 2569 2570 2571 2572 2573 2574 2575 2576 2577 2578 2579 2580 2581 2582 2583 2584 2585 2586 2587 2588 2589 2590 2591 2592 2593 2594 | |
flow(name=None, *, description=None)
flow(
name: str | None = None,
*,
description: str | None = None
) -> _FlowDecorator
flow(
func: Callable[
[FlowContext[ChunkT], object], Awaitable[OutputT]
],
) -> Flow[Any, OutputT, ChunkT]
flow(
func: DefaultInputCtxFn[
FlowContext[ChunkT], InputT, OutputT
],
) -> DefaultedFlow[InputT, OutputT, ChunkT]
flow(
func: Callable[
[InputT, FlowContext[ChunkT]], Awaitable[OutputT]
],
) -> Flow[InputT, OutputT, ChunkT]
flow(
func: CtxDefaultInputFn[
FlowContext[ChunkT], InputT, OutputT
],
) -> DefaultedFlow[InputT, OutputT, ChunkT]
flow(
func: Callable[
[FlowContext[ChunkT], InputT], Awaitable[OutputT]
],
) -> Flow[InputT, OutputT, ChunkT]
flow(
func: Callable[[object], Awaitable[OutputT]],
) -> Flow[Any, OutputT]
flow(
func: Callable[
[FlowContext[ChunkT]], Awaitable[OutputT]
],
) -> Flow[None, OutputT, ChunkT]
flow(
func: DefaultInputFn[InputT, OutputT],
) -> DefaultedFlow[InputT, OutputT]
flow(
func: Callable[[InputT], Awaitable[OutputT]],
) -> Flow[InputT, OutputT]
flow(
func: Callable[[], Awaitable[OutputT]],
) -> Flow[None, OutputT]
Decorator to register an async function as a flow.
Can be used bare (@ai.flow) or with arguments (@ai.flow(),
@ai.flow('custom_name', description='...')).
A flow takes at most one input. To read the request context or stream
chunks, add a parameter annotated FlowContext, in any position.
Any other second parameter raises TypeError when the flow is defined.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str | collections.abc.Callable[..., collections.abc.Awaitable[typing.Any]] | None
|
Optional name for the flow (or the function itself when used
as |
None
|
description
|
str | None
|
Optional description for the flow. |
None
|
Example
from genkit import FlowContext, Genkit from genkit_google_genai import GoogleAI
ai = Genkit(plugins=[GoogleAI()], model=GoogleAI.gemini('gemini-flash-latest'))
@ai.flow async def my_flow(prompt: str) -> str: res = await ai.generate(prompt=prompt) return res.text
@ai.flow async def streaming_flow(x: int, ctx: FlowContext[str]) -> str: ctx.send_chunk('progress') return 'done'
Source code in packages/genkit/src/genkit/_ai/_aio.py
416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 | |
generate(*, model=None, prompt=None, system=None, messages=None, tools=None, return_tool_requests=None, tool_choice=None, resume_respond=None, resume_restart=None, resume_metadata=None, config=None, max_turns=None, context=None, output_schema=None, output_format=None, output_content_type=None, output_instructions=None, output_constrained=None, use=None, docs=None)
async
generate(
*,
model: ModelRef[Any] | Action | str | None = None,
prompt: str | list[Part] | None = None,
system: str | list[Part] | None = None,
messages: list[Message] | None = None,
tools: Sequence[str | Tool] | None = None,
return_tool_requests: bool | None = None,
tool_choice: ToolChoice | None = None,
resume_respond: Part | list[Part] | None = None,
resume_restart: Part | list[Part] | None = None,
resume_metadata: dict[str, Any] | None = None,
config: None = None,
max_turns: int | None = None,
context: dict[str, object] | None = None,
output_schema: type[OutputT],
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None
) -> ModelResponse[OutputT]
generate(
*,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
prompt: str | list[Part] | None = None,
system: str | list[Part] | None = None,
messages: list[Message] | None = None,
tools: Sequence[str | Tool] | None = None,
return_tool_requests: bool | None = None,
tool_choice: ToolChoice | None = None,
resume_respond: Part | list[Part] | None = None,
resume_restart: Part | list[Part] | None = None,
resume_metadata: dict[str, Any] | None = None,
config: ModelConfigDict,
max_turns: int | None = None,
context: dict[str, object] | None = None,
output_schema: type[OutputT],
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None
) -> ModelResponse[OutputT]
generate(
*,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
prompt: str | list[Part] | None = None,
system: str | list[Part] | None = None,
messages: list[Message] | None = None,
tools: Sequence[str | Tool] | None = None,
return_tool_requests: bool | None = None,
tool_choice: ToolChoice | None = None,
resume_respond: Part | list[Part] | None = None,
resume_restart: Part | list[Part] | None = None,
resume_metadata: dict[str, Any] | None = None,
config: ModelRefConfigT | Mapping[str, Any] | None,
max_turns: int | None = None,
context: dict[str, object] | None = None,
output_schema: type[OutputT],
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None
) -> ModelResponse[OutputT]
generate(
*,
model: ModelRef[Any] | Action | str | None = None,
prompt: str | list[Part] | None = None,
system: str | list[Part] | None = None,
messages: list[Message] | None = None,
tools: Sequence[str | Tool] | None = None,
return_tool_requests: bool | None = None,
tool_choice: ToolChoice | None = None,
resume_respond: Part | list[Part] | None = None,
resume_restart: Part | list[Part] | None = None,
resume_metadata: dict[str, Any] | None = None,
config: None = None,
max_turns: int | None = None,
context: dict[str, object] | None = None,
output_schema: (
type[Any] | dict[str, Any] | str | None
) = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None
) -> ModelResponse[Any]
generate(
*,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
prompt: str | list[Part] | None = None,
system: str | list[Part] | None = None,
messages: list[Message] | None = None,
tools: Sequence[str | Tool] | None = None,
return_tool_requests: bool | None = None,
tool_choice: ToolChoice | None = None,
resume_respond: Part | list[Part] | None = None,
resume_restart: Part | list[Part] | None = None,
resume_metadata: dict[str, Any] | None = None,
config: ModelConfigDict,
max_turns: int | None = None,
context: dict[str, object] | None = None,
output_schema: (
type[Any] | dict[str, Any] | str | None
) = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None
) -> ModelResponse[Any]
generate(
*,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
prompt: str | list[Part] | None = None,
system: str | list[Part] | None = None,
messages: list[Message] | None = None,
tools: Sequence[str | Tool] | None = None,
return_tool_requests: bool | None = None,
tool_choice: ToolChoice | None = None,
resume_respond: Part | list[Part] | None = None,
resume_restart: Part | list[Part] | None = None,
resume_metadata: dict[str, Any] | None = None,
config: ModelRefConfigT | Mapping[str, Any] | None,
max_turns: int | None = None,
context: dict[str, object] | None = None,
output_schema: (
type[Any] | dict[str, Any] | str | None
) = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None
) -> ModelResponse[Any]
Generate text or structured data using a language model.
model accepts a typed ModelRef from a plugin constructor
(recommended, e.g. GoogleAI.gemini('gemini-flash-latest'),
which takes the bare model ID, stamps the 'googleai/' namespace
prefix, and type-checks config), a plugin-prefixed string
('googleai/gemini-flash-latest'), a registered model Action, or
None to use the default model= passed to Genkit(...). When
passing a plain str for a plugin model, the <plugin>/ prefix
(such as 'googleai/' or 'vertexai/') is required; passing an
unprefixed string raises GenkitError with NOT_FOUND. Any model
ID served by the provider (including newly released models) is resolved
lazily on demand at call time without needing pre-registration or SDK
updates.
tools is typed as Sequence rather than list because Sequence
is covariant: list[Tool] or list[str] are both assignable to
Sequence[str | Tool], but not to list[str | Tool].
Example
from pydantic import BaseModel
class Weather(BaseModel): city: str forecast: str
res = await ai.generate( prompt='Weather in Paris?', tools=[current_weather], output_schema=Weather, ) if res.output: print(res.output)
=> city='Paris' forecast='Sunny'
max_turns caps tool rounds (default 50, matching Go): the model may
call tools that many times and then reply, so a run uses at most
max_turns + 1 model calls. Past the cap the response is ABORTED
with reason MAX_TURNS_EXCEEDED.
prompt and system strings are sent exactly as written. Braces
are content (JSON, code, or another template), not Handlebars.
Partials ({{> persona}}), define_helper helpers and
{{media url=...}} only render in define_prompt or a .prompt
file; otherwise build the text or Part.from_media(...) yourself.
Source code in packages/genkit/src/genkit/_ai/_aio.py
1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 | |
generate_operation(*, model=None, prompt=None, system=None, messages=None, config=None, context=None, use=None, docs=None)
async
generate_operation(
*,
model: (
ModelRef[Any] | BackgroundModel | str | None
) = None,
prompt: str | list[Part] | None = None,
system: str | list[Part] | None = None,
messages: list[Message] | None = None,
config: None = None,
context: dict[str, object] | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None
) -> Operation[ModelResponse]
generate_operation(
*,
model: (
ModelRef[ModelRefConfigT]
| BackgroundModel
| str
| None
) = None,
prompt: str | list[Part] | None = None,
system: str | list[Part] | None = None,
messages: list[Message] | None = None,
config: ModelConfigDict,
context: dict[str, object] | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None
) -> Operation[ModelResponse]
generate_operation(
*,
model: (
ModelRef[ModelRefConfigT]
| BackgroundModel
| str
| None
) = None,
prompt: str | list[Part] | None = None,
system: str | list[Part] | None = None,
messages: list[Message] | None = None,
config: ModelRefConfigT | Mapping[str, Any] | None,
context: dict[str, object] | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None
) -> Operation[ModelResponse]
Generate content using a long-running model, returning an Operation to poll.
A background start is one request that returns a job handle. There is
no tool loop and no output parsing on check_operation, so tools,
max_turns, and the output_* options are not accepted here.
Example
op = await ai.generate_operation( model='googleai/veo-3.1-generate-preview', prompt='A timelapse of a flower blooming.', ) while not op.done: op = await ai.check_operation(op)
Source code in packages/genkit/src/genkit/_ai/_aio.py
2789 2790 2791 2792 2793 2794 2795 2796 2797 2798 2799 2800 2801 2802 2803 2804 2805 2806 2807 2808 2809 2810 2811 2812 2813 2814 2815 2816 2817 2818 2819 2820 2821 2822 2823 2824 2825 2826 2827 2828 2829 2830 2831 2832 2833 2834 2835 2836 2837 2838 2839 2840 2841 2842 2843 2844 2845 2846 2847 2848 2849 2850 2851 2852 2853 2854 2855 2856 2857 2858 2859 2860 2861 2862 2863 2864 2865 2866 2867 | |
generate_stream(*, model=None, prompt=None, system=None, messages=None, tools=None, return_tool_requests=None, tool_choice=None, resume_respond=None, resume_restart=None, resume_metadata=None, config=None, max_turns=None, context=None, output_schema=None, output_format=None, output_content_type=None, output_instructions=None, output_constrained=None, use=None, docs=None)
generate_stream(
*,
model: ModelRef[Any] | Action | str | None = None,
prompt: str | list[Part] | None = None,
system: str | list[Part] | None = None,
messages: list[Message] | None = None,
tools: Sequence[str | Tool] | None = None,
return_tool_requests: bool | None = None,
tool_choice: ToolChoice | None = None,
resume_respond: Part | list[Part] | None = None,
resume_restart: Part | list[Part] | None = None,
resume_metadata: dict[str, Any] | None = None,
config: None = None,
max_turns: int | None = None,
context: dict[str, object] | None = None,
output_schema: type[OutputT],
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None
) -> ModelStreamResponse[OutputT]
generate_stream(
*,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
prompt: str | list[Part] | None = None,
system: str | list[Part] | None = None,
messages: list[Message] | None = None,
tools: Sequence[str | Tool] | None = None,
return_tool_requests: bool | None = None,
tool_choice: ToolChoice | None = None,
resume_respond: Part | list[Part] | None = None,
resume_restart: Part | list[Part] | None = None,
resume_metadata: dict[str, Any] | None = None,
config: ModelConfigDict,
max_turns: int | None = None,
context: dict[str, object] | None = None,
output_schema: type[OutputT],
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None
) -> ModelStreamResponse[OutputT]
generate_stream(
*,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
prompt: str | list[Part] | None = None,
system: str | list[Part] | None = None,
messages: list[Message] | None = None,
tools: Sequence[str | Tool] | None = None,
return_tool_requests: bool | None = None,
tool_choice: ToolChoice | None = None,
resume_respond: Part | list[Part] | None = None,
resume_restart: Part | list[Part] | None = None,
resume_metadata: dict[str, Any] | None = None,
config: ModelRefConfigT | Mapping[str, Any] | None,
max_turns: int | None = None,
context: dict[str, object] | None = None,
output_schema: type[OutputT],
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None
) -> ModelStreamResponse[OutputT]
generate_stream(
*,
model: ModelRef[Any] | Action | str | None = None,
prompt: str | list[Part] | None = None,
system: str | list[Part] | None = None,
messages: list[Message] | None = None,
tools: Sequence[str | Tool] | None = None,
return_tool_requests: bool | None = None,
tool_choice: ToolChoice | None = None,
resume_respond: Part | list[Part] | None = None,
resume_restart: Part | list[Part] | None = None,
resume_metadata: dict[str, Any] | None = None,
config: None = None,
max_turns: int | None = None,
context: dict[str, object] | None = None,
output_schema: (
type[Any] | dict[str, Any] | str | None
) = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None
) -> ModelStreamResponse[Any]
generate_stream(
*,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
prompt: str | list[Part] | None = None,
system: str | list[Part] | None = None,
messages: list[Message] | None = None,
tools: Sequence[str | Tool] | None = None,
return_tool_requests: bool | None = None,
tool_choice: ToolChoice | None = None,
resume_respond: Part | list[Part] | None = None,
resume_restart: Part | list[Part] | None = None,
resume_metadata: dict[str, Any] | None = None,
config: ModelConfigDict,
max_turns: int | None = None,
context: dict[str, object] | None = None,
output_schema: (
type[Any] | dict[str, Any] | str | None
) = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None
) -> ModelStreamResponse[Any]
generate_stream(
*,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
prompt: str | list[Part] | None = None,
system: str | list[Part] | None = None,
messages: list[Message] | None = None,
tools: Sequence[str | Tool] | None = None,
return_tool_requests: bool | None = None,
tool_choice: ToolChoice | None = None,
resume_respond: Part | list[Part] | None = None,
resume_restart: Part | list[Part] | None = None,
resume_metadata: dict[str, Any] | None = None,
config: ModelRefConfigT | Mapping[str, Any] | None,
max_turns: int | None = None,
context: dict[str, object] | None = None,
output_schema: (
type[Any] | dict[str, Any] | str | None
) = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None
) -> ModelStreamResponse[Any]
Stream generated text, returning a ModelStreamResponse with .stream and .response.
With output_schema=Recipe, each chunk.output is a partial of
that type: same attributes, any field may still be None or a
prefix. Guard the field you are about to use. The finished
Recipe is only (await result.response).output, and it's None
when the reply isn't a Recipe.
If the model fails partway through, the async for still ends
normally. The final response has finish_reason == FAILED,
error set, text == '', message is None, and messages
ending at the last complete turn, so it's safe to send back. The
chunks you already received are the record of what was shown.
Use async with result.stream as stream: so that leaving the loop
early (break, an exception, a disconnected client) closes the
stream and stops the model call.
Example
# 1. Start streaming
result = ai.generate_stream(prompt='Write a haiku about rain.')
# 2. Print chunks as they arrive; the context manager closes the stream
async with result.stream as stream:
async for chunk in stream:
print(chunk.text, end='')
print()
# => Soft rain on the roof...
# 3. Await the final response
final = await result.response
print(final.finish_reason)
# => stop
Source code in packages/genkit/src/genkit/_ai/_aio.py
2207 2208 2209 2210 2211 2212 2213 2214 2215 2216 2217 2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228 2229 2230 2231 2232 2233 2234 2235 2236 2237 2238 2239 2240 2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 2253 2254 2255 2256 2257 2258 2259 2260 2261 2262 2263 2264 2265 2266 2267 2268 2269 2270 2271 2272 2273 2274 2275 2276 2277 2278 2279 2280 2281 2282 2283 2284 2285 2286 2287 2288 2289 2290 2291 2292 2293 2294 2295 2296 2297 2298 2299 2300 2301 2302 | |
lookup_model(name)
async
Return the model action registered under name, or None.
Plugin models that haven't been used yet are resolved through the
plugin, so 'googleai/gemini-flash-latest' works before any
generate call. Background models aren't included; they run through
generate_operation.
Source code in packages/genkit/src/genkit/_ai/_aio.py
904 905 906 907 908 909 910 911 912 913 | |
lookup_tool(name)
async
Return the tool registered under name, or None.
The same handle @ai.tool returns, so a tool defined in one module
can be called or passed in tools=[...] from another without
importing it.
Example
# 1. Define a tool in one module
@ai.tool
async def menu_price(dish: str) -> float:
return 14.5
# 2. Find it by name somewhere else
price_tool = await ai.lookup_tool('menu_price')
response = await ai.generate(prompt='How much is the ramen?', tools=[price_tool])
print(response.text)
# => The ramen is $14.50.
Source code in packages/genkit/src/genkit/_ai/_aio.py
915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 | |
lookup_value(*, kind, name)
Return the value defined under kind and name, or None.
Source code in packages/genkit/src/genkit/_ai/_aio.py
942 943 944 | |
middleware(cls=None, *, name=None, description=None)
middleware(cls: type[MiddlewareT]) -> type[MiddlewareT]
middleware(
*,
name: str | None = None,
description: str | None = None
) -> Callable[[type[MiddlewareT]], type[MiddlewareT]]
Decorator that registers a custom middleware on this app's registry.
Source code in packages/genkit/src/genkit/_ai/_aio.py
671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 | |
prompt(name, *, variant=None, input_schema=None, output_schema=None)
prompt(
name: str,
*,
variant: str | None = None,
input_schema: None = None,
output_schema: None = None
) -> Prompt[Any, Any]
prompt(
name: str,
*,
variant: str | None = None,
input_schema: type[InputT],
output_schema: None = None
) -> Prompt[InputT, Any]
prompt(
name: str,
*,
variant: str | None = None,
input_schema: None = None,
output_schema: type[OutputT]
) -> Prompt[Any, OutputT]
prompt(
name: str,
*,
variant: str | None = None,
input_schema: type[InputT],
output_schema: type[OutputT]
) -> Prompt[InputT, OutputT]
Look up a prompt by name and optional variant.
Source code in packages/genkit/src/genkit/_ai/_aio.py
1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 | |
run(name, fn, *, metadata=None)
async
Run a function as a discrete traced step within a flow.
fn takes no arguments and returns an awaitable, so async def
functions, lambda: fetch(i), and callables with an async def
__call__ all work. Same shape as JS ai.run('step', fn).
Example
# 1. Each fan-out step gets its own span in the trace
menus = await asyncio.gather(*[ai.run(f'fetch-{i}', lambda i=i: fetch_menu(i)) for i in range(3)])
# 2. Keyword form still works
total = await ai.run(name='sum', fn=lambda: sum_prices(menus))
Source code in packages/genkit/src/genkit/_ai/_aio.py
2601 2602 2603 2604 2605 2606 2607 2608 2609 2610 2611 2612 2613 2614 2615 2616 2617 2618 2619 2620 2621 2622 2623 2624 2625 2626 2627 2628 2629 2630 2631 2632 2633 2634 2635 2636 2637 2638 2639 | |
run_main(coro)
Run your main coroutine and return its result, like asyncio.run.
With reflection on (GENKIT_ENV=dev, which genkit start sets, or
GENKIT_REFLECTION_ENABLED=true), it keeps the process alive after
main returns, so the Dev UI can keep running your flows, until
Ctrl+C, SIGTERM, or the reflection server stops on its own. With
reflection off, it returns as soon as main does.
If main raises while reflection is on, the error is logged and the
Dev UI stays up. The error is raised once the process stops. Ctrl+C
after a successful main raises KeyboardInterrupt, and SIGTERM
returns main's result.
Example
from genkit import Genkit
from genkit_google_genai import GoogleAI
# 1. Initialize Genkit and define a flow
ai = Genkit(plugins=[GoogleAI()], model=GoogleAI.gemini('gemini-flash-latest'))
@ai.flow
async def suggest_dish(cuisine: str) -> str:
response = await ai.generate(prompt=f'Suggest one {cuisine} dish.')
return response.text
# 2. Run a quick check from your script's entry point
async def main() -> None:
print(await suggest_dish('Thai'))
# 3. Start it
ai.run_main(main())
# => Green curry with chicken
# `python main.py` exits here. Under `genkit start`, the process
# stays up for the Dev UI until you press Ctrl+C.
Source code in packages/genkit/src/genkit/_ai/_aio.py
1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 1724 1725 1726 1727 1728 1729 | |
tool(name=None, *, description=None, input_schema=None)
tool(
name: str | None = None,
*,
description: str | None = None,
input_schema: (
type[BaseModel] | dict[str, object] | None
) = None
) -> _ToolDecorator
tool(
func: Callable[
[ToolContext[ChunkT], object], Awaitable[OutputT]
],
) -> Tool[Any, OutputT, ChunkT]
tool(
func: DefaultInputCtxFn[
ToolContext[ChunkT], InputT, OutputT
],
) -> DefaultedTool[InputT, OutputT, ChunkT]
tool(
func: Callable[
[InputT, ToolContext[ChunkT]], Awaitable[OutputT]
],
) -> Tool[InputT, OutputT, ChunkT]
tool(
func: CtxDefaultInputFn[
ToolContext[ChunkT], InputT, OutputT
],
) -> DefaultedTool[InputT, OutputT, ChunkT]
tool(
func: Callable[
[ToolContext[ChunkT], InputT], Awaitable[OutputT]
],
) -> Tool[InputT, OutputT, ChunkT]
tool(
func: Callable[[object], Awaitable[OutputT]],
) -> Tool[Any, OutputT]
tool(
func: Callable[
[ToolContext[ChunkT]], Awaitable[OutputT]
],
) -> Tool[None, OutputT, ChunkT]
tool(
func: DefaultInputFn[InputT, OutputT],
) -> DefaultedTool[InputT, OutputT]
tool(
func: Callable[[InputT], Awaitable[OutputT]],
) -> Tool[InputT, OutputT]
tool(
func: Callable[[], Awaitable[OutputT]],
) -> Tool[None, OutputT]
Decorator to register a function as a tool.
Can be used bare (@ai.tool) or with arguments (@ai.tool(),
@ai.tool('custom_name', description='...')).
The return annotation is what the model binds as outputSchema.
Awaiting the tool directly returns what the function returned.
A tool takes at most one input, and that input's type is the schema the
model fills in. For several fields, use one Pydantic model. To read the
request context or interrupt, add a parameter annotated
ToolContext, in any position. Any other second parameter raises
TypeError when the tool is defined.
Example
@ai.tool async def current_weather(city: str) -> str: return f'Sunny in {city}'
class Forecast(BaseModel): city: str days: int = 3
@ai.tool async def forecast(input: Forecast, ctx: ToolContext) -> str: user = ctx.context.get('user_id') return f'{input.days}-day forecast for {input.city} ({user})'
res = await ai.generate(prompt='Weather in Paris?', tools=['current_weather', 'forecast'])
Source code in packages/genkit/src/genkit/_ai/_aio.py
604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 | |
GenkitError
Bases: Exception
Base error class for Genkit errors.
Source code in packages/genkit/src/genkit/_core/_error.py
519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 | |
__init__(*, message, status=None, cause=None, details=None, reason=None, source=None, retry_after_ms=None)
Initialize a GenkitError.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
message
|
str
|
The error message. |
required |
status
|
genkit._core._error.StatusName | None
|
The status name for this error. |
None
|
cause
|
BaseException | None
|
The underlying exception; set as |
None
|
details
|
typing.Any
|
Optional detail information. |
None
|
reason
|
genkit._core._error.ErrorReason | None
|
Extra why when we classified the failure. |
None
|
source
|
str | None
|
Optional prefix for |
None
|
retry_after_ms
|
float | None
|
How long the provider asked callers to wait before retrying. |
None
|
Source code in packages/genkit/src/genkit/_core/_error.py
522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 | |
GenkitLogger
Bases: typing.Protocol
The calls get_logger promises. Nothing here names the logging backend.
Each method takes an event string, optional printf-style args, and
keyword fields that are recorded as structured attributes.
Source code in packages/genkit/src/genkit/_core/_logger.py
374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 | |
debug(event, *args, **fields)
Log at DEBUG.
Source code in packages/genkit/src/genkit/_core/_logger.py
381 382 383 | |
error(event, *args, **fields)
Log at ERROR.
Source code in packages/genkit/src/genkit/_core/_logger.py
393 394 395 | |
exception(event, *args, **fields)
Log at ERROR with the active exception's traceback.
Source code in packages/genkit/src/genkit/_core/_logger.py
397 398 399 | |
info(event, *args, **fields)
Log at INFO.
Source code in packages/genkit/src/genkit/_core/_logger.py
385 386 387 | |
warning(event, *args, **fields)
Log at WARNING.
Source code in packages/genkit/src/genkit/_core/_logger.py
389 390 391 | |
Interrupt
Bases: BaseException
Pause a tool so the caller can approve, reply, or restart.
Raise Interrupt(metadata) from a tool or from tool middleware.
It is a control-flow signal like asyncio.CancelledError, so it
subclasses BaseException: an except Exception in tool code can't
swallow it and skip the approval. Catch Interrupt by name if a tool
needs to clean up, then re-raise. Raised anywhere else (a flow, an
agent turn body), the action fails with GenkitError INTERNAL.
In the trace the tool's own span shows as interrupted; the generate
that paused on it stays green.
Source code in packages/genkit/src/genkit/_core/_error.py
420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 | |
Media
Bases: genkit._core._base.GenkitModel
Model for media data.
Source code in packages/genkit/src/genkit/_core/_typing.py
764 765 766 767 768 769 770 771 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
Message
Bases: genkit._core._base.GenkitModel
A single turn in a conversation.
Source code in packages/genkit/src/genkit/_core/_model.py
1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 | |
interrupts
property
Tool requests marked as interrupted.
media
property
The first media part in this message, or None.
reasoning
property
All reasoning (thinking) parts concatenated; '' when the model sent none.
text
property
All text parts concatenated into a single string.
tool_requests
property
All tool request parts in this message.
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
ModelResponse
Bases: genkit._core._base.GenkitModel, typing.Generic[genkit._core._model.OutputT]
Model response with utilities for text extraction, output parsing, and validation.
Source code in packages/genkit/src/genkit/_core/_model.py
1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 1790 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802 1803 1804 1805 1806 1807 1808 1809 1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 1820 1821 1822 1823 1824 1825 1826 1827 1828 1829 1830 1831 1832 1833 1834 1835 1836 1837 1838 1839 1840 1841 1842 1843 1844 1845 1846 1847 1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 1864 1865 1866 1867 1868 1869 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 | |
interrupts
property
Tool requests marked as interrupted.
media
property
The first media part in the response message, or None.
Use media_parts when the model returns more than one.
media_parts
property
All media parts in the response message.
messages
property
All messages including request history and the response message.
Recomputed each read so attaching request later still shows up.
output
property
Parsed structured output, or None when the reply is not that shape.
generate() does not throw when the text is not the schema. If you
asked for a schema and this is not it, read error / .text.
Only complete JSON counts: a reply cut off mid-object is None (check
finish_reason == 'length' to tell a token cap from bad JSON).
reasoning
property
The reply's reasoning (thinking) text, kept out of .text; '' when there is none.
text
property
All text parts concatenated into a single string.
tool_requests
property
All tool request parts in the response message.
Recomputed each read so a later message still shows up.
__eq__(other)
Compare responses by message, finish_reason, and poll snapshot.
Same job id with a later done/error/output is a later check, not the same response. Timing on the handle is not part of the job.
Source code in packages/genkit/src/genkit/_core/_model.py
1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 | |
__hash__()
Return identity-based hash.
Source code in packages/genkit/src/genkit/_core/_model.py
1917 1918 1919 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
model_post_init(__context)
Initialize default usage and custom dict if not provided.
Source code in packages/genkit/src/genkit/_core/_model.py
1805 1806 1807 1808 1809 1810 | |
ModelResponseChunk
Bases: genkit._core._base.GenkitModel, typing.Generic[genkit._core._model.OutputT]
Streaming chunk with text, accumulated text, and output parsing.
Source code in packages/genkit/src/genkit/_core/_model.py
2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 2079 2080 2081 2082 2083 2084 2085 2086 2087 2088 2089 2090 2091 2092 2093 2094 2095 2096 2097 2098 2099 2100 2101 2102 2103 2104 2105 2106 2107 2108 2109 2110 2111 2112 2113 2114 2115 2116 2117 2118 2119 2120 2121 2122 2123 2124 2125 2126 2127 2128 2129 2130 2131 2132 2133 2134 2135 2136 2137 2138 2139 2140 2141 2142 2143 2144 2145 2146 2147 2148 | |
accumulated_text
property
Text from all previous chunks plus this chunk.
media
property
The first media part in this chunk, or None.
output
property
The reply so far, parsed as far as it goes. Never raises.
With output_schema=Recipe, this is a Recipe filled as far as
the JSON goes:
- Optional fields populate from their default or default_factory
(called fresh per chunk, so non-deterministic factories like uuid4
or datetime.now change per chunk until final validation), and
missing required fields are None.
- Closed fields and list items run field validation (@field_validator
and field constraints, while @model_validator waits until the
object closes); closed values that fail validation pass through raw.
- Open str fields stream as prefixes, while open str values for
Enum/datetime/Literal and unresolved discriminated unions
stay at None (or their default) until closed. Open numbers still
pass through raw for datetime fields and for numeric Literal
values that already match a member (even if more digits may follow).
(await stream.response).output is the only fully validated Recipe.
The partial is built without validation, so a required title: str
can read None here even though the type says str. Guard each
field before you use it: if chunk.output and chunk.output.title:.
With no schema class, this is the JSON value so far (dict, list,
or scalar). It's None before an object starts or while the text
can't be parsed.
reasoning
property
Reasoning (thinking) text in this chunk.
text
property
Text content of this chunk.
tool_requests
property
All tool request parts in this chunk.
tool_responses
property
All tool response parts in this chunk.
__eq__(other)
Check equality.
Source code in packages/genkit/src/genkit/_core/_model.py
2052 2053 2054 2055 2056 | |
__hash__()
Return hash.
Source code in packages/genkit/src/genkit/_core/_model.py
2058 2059 2060 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
ModelStreamResponse
Bases: genkit._core._action.StreamResponse[genkit._ai._model.ModelResponseChunk[genkit._ai._prompt.OutputT], genkit._ai._model.ModelResponse[genkit._ai._prompt.OutputT]], typing.Generic[genkit._ai._prompt.OutputT]
Chunks of a model reply, then the full ModelResponse.
Returned by ai.generate_stream() and prompt.stream(). Iteration
and .response follow StreamResponse; failures and early exit don't:
- Before the request (unknown model, bad config, ...): the loop and
.responseboth raise theGenkitError(NOT_FOUND, ...). - Model or tool failure on any turn, including the first: the loop
ends normally and
.responseresolves witherrorset andfinish_reason == FAILED. The chunks already received are the record of what was shown. - Early close (
async withorawait result.aclose()): stops the in-flight model or tool turn and awaitsfinally:cleanup;.responseresolves withfinish_reason == ABORTED,error.status == 'CANCELLED', andmessagesending at the last completed round.
Example
result = ai.generate_stream(prompt='Describe miso soup in one line.')
async for chunk in result.stream:
print(chunk.text)
# => Savory soybean broth
# with tofu and scallions.
response = await result.response
print(response.finish_reason)
# => stop
Source code in packages/genkit/src/genkit/_ai/_prompt.py
160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 | |
response
property
The final result, available once the run ends.
stream
property
The chunks, in order (AsyncIterator). Iterating the response itself reads the same stream.
aclose()
async
Close the stream and release any underlying resources.
Source code in packages/genkit/src/genkit/_core/_action.py
294 295 296 297 298 | |
MultipartToolResponse
Bases: genkit._core._base.GenkitModel, typing.Generic[genkit._core._model.OutputT]
A tool result with structured output plus media the model can see.
Return one from a tool when the reply is more than a JSON value, such as
a caption and a screenshot. A plain return value still works and is
treated as output only. Annotate MultipartToolResponse[ShotOut] so
the tool's output schema is ShotOut.
Example
@ai.tool
async def screenshot(label: str) -> MultipartToolResponse[Shot]:
return MultipartToolResponse(output=Shot(label=label), content=[png])
Source code in packages/genkit/src/genkit/_core/_model.py
2227 2228 2229 2230 2231 2232 2233 2234 2235 2236 2237 2238 2239 2240 2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 2253 2254 2255 2256 2257 2258 2259 2260 2261 2262 2263 2264 2265 2266 2267 2268 2269 2270 2271 2272 2273 2274 2275 2276 2277 2278 2279 2280 2281 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
Operation
Bases: genkit._core._base.GenkitModel, typing.Generic[genkit._core._operation.OutputT]
Poll handle for a background job.
A background action returns an Operation right away. Poll it until
done is true, then read output or error. Generic over the
output type, like JS Operation<O> and Go Operation[Out]. Background
models return Operation[ModelResponse]. The wire shape does not depend
on OutputT, so a stored handle goes back to ai.check_operation as
the dict you saved, or reloads with
Operation[ModelResponse].model_validate_json(...).
Example
# 1. Start a background video generation
op = await ai.generate_operation(model='googleai/veo-3.1-generate-preview', prompt='A chef plating risotto.')
print(op.done)
# => False
# 2. Store the handle; later, check it with the dict you saved
saved = op.model_dump_json()
op = await ai.check_operation(json.loads(saved))
# 3. Poll until done; output is a ModelResponse
while not op.done:
op = await ai.check_operation(op)
print(op.output.media.url)
# => https://storage.googleapis.com/.../risotto.mp4
Source code in packages/genkit/src/genkit/_core/_operation.py
39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
Part
Bases: genkit._core._base.GenkitModel
A single piece of content in a message or document.
Source code in packages/genkit/src/genkit/_core/_model.py
573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 | |
is_partial
property
Whether this part is a partial tool response streamed during tool execution.
from_tool_response(name, output=None, ref=None, metadata=None, content=None)
classmethod
Build a tool-response part, e.g. for a manual tool loop.
content carries media the model should see next to output
(a screenshot, a chart). A MultipartToolResponse passed as
output is unwrapped the way the engine does it: its output,
content, and metadata land in their own slots instead of being
serialized as JSON text. Setting content or metadata both on
the MultipartToolResponse and as a keyword raises GenkitError
(INVALID_ARGUMENT) instead of dropping one.
output and metadata are dumped to JSON and content to wire
parts, the same as a tool return inside generate, so a Pydantic
output reaches the model plugin as JSON. Output that can't be dumped,
or a part with nothing in it (an empty media URL), raises
GenkitError (INVALID_ARGUMENT) here.
Source code in packages/genkit/src/genkit/_core/_model.py
663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
respond(output, *, metadata=None)
Build the tool-response part that answers this interrupt without running the tool.
metadata is stored under interruptResponse and defaults to true when omitted.
Source code in packages/genkit/src/genkit/_core/_model.py
759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 | |
restart(*, resumed_metadata=None, replace_input=None)
Build the tool-request part that runs this interrupt again.
resumed_metadata is what the tool reads as ctx.resumed_metadata.
Omit it and the tool still sees a resume (ctx.is_resumed() is true).
replace_input swaps the tool input and keeps the previous input on
metadata['replacedInput'].
Source code in packages/genkit/src/genkit/_core/_model.py
730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 | |
Prompt
Bases: typing.Generic[genkit._ai._prompt.InputT, genkit._ai._prompt.OutputT]
A callable prompt with typed input/output that generates AI responses.
Source code in packages/genkit/src/genkit/_ai/_prompt.py
278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 | |
ref
property
Reference object with prompt name and metadata.
__call__(input=None, *, model=None, config=None, messages=None, tools=None, tool_choice=None, docs=None, use=None, max_turns=None, context=None, return_tool_requests=None, resume_respond=None, resume_restart=None, resume_metadata=None, on_chunk=None)
async
Render the prompt with input as template variables, run it, and return the response.
Keywords take the same names as ai.generate. An omitted (or None)
keyword keeps the prompt's value; tools=[], use=[] and docs=[]
clear the prompt's list for this call. config merges per key.
Source code in packages/genkit/src/genkit/_ai/_prompt.py
438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 | |
__init__(ai, *, variant=None, model=None, config=None, description=None, input_schema=None, input_default=None, system=None, prompt=None, messages=None, output_format=None, output_content_type=None, output_instructions=None, output_schema=None, output_constrained=None, max_turns=None, return_tool_requests=None, metadata=None, tools=None, tool_choice=None, use=None, docs=None, name=None, ns=None)
Initialize prompt with configuration, templates, and schema options.
Source code in packages/genkit/src/genkit/_ai/_prompt.py
281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 | |
render(input=None, *, model=None, config=None, messages=None, tools=None, tool_choice=None, docs=None, use=None, max_turns=None, context=None, return_tool_requests=None, resume_respond=None, resume_restart=None, resume_metadata=None)
async
Render the prompt without running it. Same keywords as __call__, minus on_chunk.
Source code in packages/genkit/src/genkit/_ai/_prompt.py
573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 | |
stream(input=None, *, model=None, config=None, messages=None, tools=None, tool_choice=None, docs=None, use=None, max_turns=None, context=None, return_tool_requests=None, resume_respond=None, resume_restart=None, resume_metadata=None)
Stream the prompt execution. Same keywords as __call__, minus on_chunk.
Iterate .stream on the returned response for chunks; there's no
callback here so chunks only arrive one way.
Source code in packages/genkit/src/genkit/_ai/_prompt.py
518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 | |
PublicError
Bases: genkit._core._error.GenkitError
Error class for issues to be returned to users.
Using this error allows a web framework handler (e.g. FastAPI, Flask) to know it is safe to return the message, details, and HTTP status in a request. Any other GenkitError is 500 Internal Error on the wire.
Source code in packages/genkit/src/genkit/_core/_error.py
753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 | |
__init__(status, message, details=None, *, reason=None)
Initialize a PublicError.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
status
|
genkit._core._error.StatusName
|
The status name for this error. |
required |
message
|
str
|
The error message. |
required |
details
|
typing.Any
|
Optional details to include. |
None
|
reason
|
genkit._core._error.ErrorReason | None
|
Extra why when the failure has a stable classification;
stored on |
None
|
Source code in packages/genkit/src/genkit/_core/_error.py
761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 | |
Role
Bases: genkit._core._compat.StrEnum
Role data type class.
Source code in packages/genkit/src/genkit/_core/_typing.py
111 112 113 114 115 116 117 | |
StreamResponse
Bases: typing.Generic[genkit._core._action.ChunkT_co, genkit._core._action.OutputT_co]
Chunks from a running action, then its result.
Returned by Action.stream() for every action kind: flows, tools,
models, and the rest.
- Chunks are single-pass. Once a loop has read them, a second
async foryields nothing (or re-raises, if the run failed). .responsecan be awaited any number of times, before or after iterating. Chunks sent before you iterate are buffered.- If the run raises, both the loop and
.responseraise it.ModelStreamResponse(generate_stream,prompt.stream) reports model failures on.responseinstead; see its docstring. - Closing before the run finishes (
async with result:orawait result.aclose()) stops the run and awaitsfinally:cleanup;.responsethen raisesGenkitErrorwith statusCANCELLED(ModelStreamResponseresolves withfinish_reason == ABORTEDinstead). A reader cancelled mid-async for(e.g. a client disconnect) also stops the run, and so does dropping a stream left unfinished afterbreak. Not iterating at all leaves the run going. SeeAction.stream.
Example
@ai.flow
async def track_order(order_id: str, ctx: FlowContext[str]) -> str:
ctx.send_chunk('Out for delivery')
return f'Order {order_id} delivered'
result = track_order.stream('A-1042')
async for update in result.stream:
print(update)
# => Out for delivery
print(await result.response)
# => Order A-1042 delivered
Source code in packages/genkit/src/genkit/_core/_action.py
229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 | |
response
property
The final result, available once the run ends.
stream
property
The chunks, in order (AsyncIterator). Iterating the response itself reads the same stream.
aclose()
async
Close the stream and release any underlying resources.
Source code in packages/genkit/src/genkit/_core/_action.py
294 295 296 297 298 | |
Tool
Bases: genkit._core._action.Action[genkit._core._tool.InputT, genkit._core._tool.OutputT, genkit._core._tool.ChunkT, None]
A tool the model can call: an :class:~genkit._core._action.Action of kind tool.
await tool(x) is (await tool.run(x)).response: what the function
returned. A str comes back as the string, a Pydantic model as the
instance, and response(...) as that MultipartToolResponse.
Each tool also carries a tool.v2 action that returns the
MultipartToolResponse envelope. The registry stores both, like JS and
Go; the Dev UI lists and runs the tool.v2 one.
Source code in packages/genkit/src/genkit/_core/_tool.py
250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 | |
config_schema
property
The config class set at definition time, from config_schema= or the fn annotation.
input_class
property
The action's input annotation as a concrete class, when it is one.
model_output_schema
property
writable
JSON Schema the model binds as this tool's outputSchema.
params
property
The function's input and context parameters, as Genkit passes them.
wire_action
property
The tool.v2 action: same function, returns the MultipartToolResponse envelope.
The registry stores it next to the tool under /tool.v2/<name>; the
Dev UI and other wire consumers run it. None for a Tool viewed
from a plain action.
__call__(input=NO_INPUT, *, context=None)
async
__call__(
input: None = None,
*,
context: dict[str, Any] | None = None
) -> OutputT
__call__(
input: InputT, *, context: dict[str, Any] | None = None
) -> OutputT
Call the action directly, returning just the response value.
Source code in packages/genkit/src/genkit/_core/_action.py
1350 1351 1352 1353 1354 1355 1356 1357 | |
__init__(func, *, name=None, description=None, input_schema=None)
__init__(
func: Callable[
[ToolContext[_C], object], Awaitable[_O]
],
*,
name: str | None = None,
description: str | None = None,
input_schema: (
type[BaseModel] | dict[str, object] | None
) = None
) -> None
__init__(
func: DefaultInputCtxFn[ToolContext[_C], _I, _O],
*,
name: str | None = None,
description: str | None = None,
input_schema: (
type[BaseModel] | dict[str, object] | None
) = None
) -> None
__init__(
func: Callable[[_I, ToolContext[_C]], Awaitable[_O]],
*,
name: str | None = None,
description: str | None = None,
input_schema: (
type[BaseModel] | dict[str, object] | None
) = None
) -> None
__init__(
func: CtxDefaultInputFn[ToolContext[_C], _I, _O],
*,
name: str | None = None,
description: str | None = None,
input_schema: (
type[BaseModel] | dict[str, object] | None
) = None
) -> None
__init__(
func: Callable[[ToolContext[_C], _I], Awaitable[_O]],
*,
name: str | None = None,
description: str | None = None,
input_schema: (
type[BaseModel] | dict[str, object] | None
) = None
) -> None
__init__(
func: Callable[[object], Awaitable[_O]],
*,
name: str | None = None,
description: str | None = None,
input_schema: (
type[BaseModel] | dict[str, object] | None
) = None
) -> None
__init__(
func: Callable[[ToolContext[_C]], Awaitable[_O]],
*,
name: str | None = None,
description: str | None = None,
input_schema: (
type[BaseModel] | dict[str, object] | None
) = None
) -> None
__init__(
func: DefaultInputFn[_I, _O],
*,
name: str | None = None,
description: str | None = None,
input_schema: (
type[BaseModel] | dict[str, object] | None
) = None
) -> None
__init__(
func: Callable[[_I], Awaitable[_O]],
*,
name: str | None = None,
description: str | None = None,
input_schema: (
type[BaseModel] | dict[str, object] | None
) = None
) -> None
__init__(
func: Callable[[], Awaitable[_O]],
*,
name: str | None = None,
description: str | None = None,
input_schema: (
type[BaseModel] | dict[str, object] | None
) = None
) -> None
Build a tool from an async function without registering it.
define_tool / @ai.tool register it. Passed straight to
generate(tools=[...]), it is an ephemeral tool for that call.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
func
|
collections.abc.Callable[..., collections.abc.Awaitable[typing.Any]]
|
Async tool implementation. It takes at most one input,
annotated with a type that has a JSON schema, plus an optional
|
required |
name
|
str | None
|
Tool name for the model. Defaults to |
None
|
description
|
str | None
|
Sent to the model. Defaults to the function docstring. |
None
|
input_schema
|
type[pydantic.BaseModel] | dict[str, object] | None
|
Optional input schema override (Pydantic model or JSON-schema dict). |
None
|
Raises:
| Type | Description |
|---|---|
TypeError
|
If |
ValueError
|
If no |
Example
from genkit import Tool
async def lookup_price(sku: str) -> str: return '12.00'
ephemeral = Tool(lookup_price, name='lookup_price') res = await ai.generate(prompt='Price of SKU-1?', tools=[ephemeral])
Source code in packages/genkit/src/genkit/_core/_tool.py
369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 | |
definition()
Return the wire-format ToolDefinition for this tool.
output_schema is what the model binds: T for -> T and for
-> MultipartToolResponse[T], None for a bare
-> MultipartToolResponse.
Source code in packages/genkit/src/genkit/_core/_tool.py
478 479 480 481 482 483 484 485 486 487 488 489 490 | |
from_action(action)
classmethod
View a tool Action that wasn't built as a Tool (plugin, DAP, hand-registered) as one.
Shares the action's function and metadata; there is no tool.v2 companion.
Source code in packages/genkit/src/genkit/_core/_tool.py
444 445 446 447 448 449 450 451 452 | |
run(input=NO_INPUT, *, on_chunk=None, context=None, telemetry_labels=None, init=None)
async
run(
input: None = None,
*,
on_chunk: Callable[[ChunkT], None] | None = None,
context: dict[str, Any] | None = None,
telemetry_labels: dict[str, object] | None = None,
init: InitT | None = None
) -> ActionResponse[OutputT]
run(
input: InputT,
*,
on_chunk: Callable[[ChunkT], None] | None = None,
context: dict[str, Any] | None = None,
telemetry_labels: dict[str, object] | None = None,
init: InitT | None = None
) -> ActionResponse[OutputT]
Run the action and return its result with trace ids.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
input
|
genkit._core._action.InputT | None
|
The input, validated against the input schema. Required unless the action takes no input or its function gives the input a default; omit it to use that default. On a flow or tool, an explicit None is a value and is validated like any other. |
genkit._core._action.NO_INPUT
|
on_chunk
|
collections.abc.Callable[[genkit._core._action.ChunkT], None] | None
|
Called with each chunk the action streams. |
None
|
context
|
dict[str, typing.Any] | None
|
Request context (auth, headers, ...), read as |
None
|
telemetry_labels
|
dict[str, object] | None
|
Custom labels set as span attributes. |
None
|
init
|
genkit._core._action.InitT | None
|
Session init value for a bidi action (e.g. a served agent's session identity). Validated against the init schema; plain actions ignore it. |
None
|
Returns:
| Type | Description |
|---|---|
genkit._core._action.ActionResponse[genkit._core._action.OutputT]
|
ActionResponse with the result, trace id and span id. |
Raises:
| Type | Description |
|---|---|
genkit._core._error.GenkitError
|
If input validation fails (INVALID_ARGUMENT status). |
Source code in packages/genkit/src/genkit/_core/_action.py
1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 | |
stream(input=NO_INPUT, *, context=None, telemetry_labels=None, init=None)
stream(
input: None = None,
*,
context: dict[str, Any] | None = None,
telemetry_labels: dict[str, object] | None = None,
init: InitT | None = None
) -> StreamResponse[ChunkT, OutputT]
stream(
input: InputT,
*,
context: dict[str, Any] | None = None,
telemetry_labels: dict[str, object] | None = None,
init: InitT | None = None
) -> StreamResponse[ChunkT, OutputT]
Return a StreamResponse: iterate .stream for chunks, await .response for the result.
Takes the same keyword arguments as run, minus on_chunk. If the
action raises, both the loop and .response raise.
Source code in packages/genkit/src/genkit/_core/_action.py
1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 | |
ToolContext
Bases: genkit._core._action.ActionContext[genkit._core._tool.ChunkT]
What a tool sees while it runs: request context, streaming, and resume state.
Annotate a tool parameter with it to receive it:
@ai.tool
async def check_allergens(dish: str, ctx: ToolContext) -> str:
return 'resumed' if ctx.is_resumed() else ctx.context.get('user', 'guest')
ToolContext is not a FlowContext. A helper shared by a tool and a
flow takes the union:
# 1. Read the caller's uid from either context
def user_id(ctx: FlowContext | ToolContext) -> str:
return ctx.context['auth']['uid']
# 2. Call it from the tool
@ai.tool
async def list_allergies(guest: str, ctx: ToolContext) -> str:
return f'{guest} ({user_id(ctx)}): shellfish'
await list_allergies('Ana', context={'auth': {'uid': 'diner-42'}})
# => 'Ana (diner-42): shellfish'
On a restart after an interrupt, is_resumed() is True and
resumed_metadata is the dict passed to
interrupt.restart(resumed_metadata=...), or {} when the restart
carried none. On a first run, resumed_metadata is None. Check
resumed_metadata is None so static type checkers narrow the dict:
@ai.tool
async def refund_order(order_id: str, ctx: ToolContext) -> str:
# Interrupt is exported from genkit
if ctx.resumed_metadata is None:
raise Interrupt({'reason': 'needs approval'})
approved = ctx.resumed_metadata.get('approved_by')
return f'Refunded {order_id} (approved by {approved})'
Source code in packages/genkit/src/genkit/_core/_tool.py
158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 | |
context
property
The request context (auth, headers, ...) the caller attached.
is_streaming
property
True if a streaming callback is registered.
streaming_callback
property
The streaming callback, if any.
Use this when you need to pass the callback to another action. For sending chunks directly, use send_chunk() instead.
__init__(ctx, *, resumed_metadata=None, original_input=None)
Initialize from the tool action's run context.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
ctx
|
genkit._core._action.ActionContext[genkit._core._tool.ChunkT]
|
Parent action context |
required |
resumed_metadata
|
dict[str, typing.Any] | None
|
Metadata from previous interrupt (if resumed) |
None
|
original_input
|
typing.Any
|
Original tool input before replacement (if resumed) |
None
|
Source code in packages/genkit/src/genkit/_core/_tool.py
197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 | |
is_resumed()
Return True if this run is a restart after an interrupt.
Same as self.resumed_metadata is not None. An empty {} still
counts as resumed.
Source code in packages/genkit/src/genkit/_core/_tool.py
218 219 220 221 222 223 224 | |
send_chunk(chunk)
Send a streaming chunk to the client.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
chunk
|
genkit._core._action.ChunkT
|
The chunk data to stream. |
required |
Source code in packages/genkit/src/genkit/_core/_action.py
1053 1054 1055 1056 1057 1058 1059 1060 | |
get_logger(name=None)
Return a logger whose lines also reach the Dev UI trace panel.
The logger has debug, info, warning, error and
exception. Each takes an event string and keyword fields. Under
genkit start, each line is also exported to the Dev UI with the active
trace and span IDs.
Example
from genkit import Genkit, get_logger
# 1. Create a module logger
ai = Genkit()
logger = get_logger(__name__)
# 2. Log inside a flow; lines attach to the flow's trace
@ai.flow
async def lookup_menu(restaurant_id: str) -> int:
logger.info('menu lookup', restaurant_id=restaurant_id, items=12)
# => console: menu lookup restaurant_id=r-42 items=12
# => Dev UI: same line, attached to the lookup_menu trace
logger.debug('cache miss', restaurant_id=restaurant_id)
# => console: nothing unless GENKIT_LOG=debug
# => Dev UI: cache miss, still exported under `genkit start`
return 12
Source code in packages/genkit/src/genkit/_core/_logger.py
402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 | |
genkit.model
Model protocol types for plugin authors; application code should call :class:genkit.Genkit generate.
BackgroundModel
Bases: genkit._core._background.BackgroundAction[genkit._core._model.ModelRequest, genkit._core._model.ModelResponse]
A background model: start returns an Operation[ModelResponse].
Pass the handle to check until it's done. A finished job's output
is a ModelResponse. label and config_schema are read back from
the start action, so a model looked up from the registry has them too.
Source code in packages/genkit/src/genkit/_ai/_background_model.py
89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 | |
actions
property
The start, check and (if any) cancel actions, for registering or listing.
A background action is three registered actions, not one. Register all of them, or a poll fails later with the check action not found.
input_schema
property
The start action's input JSON schema.
kind
property
The action kind of the start action.
metadata
property
The start action's metadata.
name
property
The name of the start action.
output_schema
property
The start action's output JSON schema (the Operation).
supports_cancel
property
Whether this background action supports cancellation.
__init__(name, start, check, *, cancel=None, label=None, info=None, config_schema=None, metadata=None, description=None)
Build a background model without registering it.
Plugin init / resolve return this; define_background_model
registers the start, check, and cancel actions.
Source code in packages/genkit/src/genkit/_ai/_background_model.py
103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 | |
__init_subclass__(**kwargs)
Read InT / OutT off class X(BackgroundAction[In, Out]).
Source code in packages/genkit/src/genkit/_core/_background.py
234 235 236 237 238 239 240 241 242 243 | |
cancel(operation, *, context=None)
async
Cancel a background operation.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
operation
|
genkit._core._operation.Operation | collections.abc.Mapping[str, typing.Any]
|
The operation to cancel, or the dict a stored one dumped to. |
required |
context
|
dict[str, typing.Any] | None
|
Optional run context (secrets, folded client config). |
None
|
Raises:
| Type | Description |
|---|---|
genkit._core._error.GenkitError
|
UNIMPLEMENTED if this action does not implement
cancel, INVALID_ARGUMENT if |
Source code in packages/genkit/src/genkit/_core/_background.py
429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 | |
check(operation, *, context=None)
async
Check the status of a background operation.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
operation
|
genkit._core._operation.Operation | collections.abc.Mapping[str, typing.Any]
|
The operation to check, or the dict a stored one dumped to. |
required |
context
|
dict[str, typing.Any] | None
|
Optional run context (secrets, folded client config). |
None
|
Raises:
| Type | Description |
|---|---|
genkit._core._error.GenkitError
|
INVALID_ARGUMENT if |
Source code in packages/genkit/src/genkit/_core/_background.py
408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 | |
lookup(registry, key)
async
classmethod
Look up a background action by its start action key.
Matches JS lookupBackgroundAction. The key format is
/{actionType}/{name}, e.g. /background-model/video-gen.
Source code in packages/genkit/src/genkit/_core/_background.py
295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 | |
start(input=None, *, context=None)
async
Start a background operation.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
input
|
genkit._core._model.ModelRequest | None
|
The input request. |
None
|
context
|
dict[str, typing.Any] | None
|
Optional run context. Per-request keys go in
|
None
|
Returns:
| Type | Description |
|---|---|
genkit._core._operation.Operation[genkit._core._model.ModelResponse]
|
An Operation with an ID to track the job. |
Source code in packages/genkit/src/genkit/_ai/_background_model.py
169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 | |
ConstrainedSupport
Bases: genkit._core._compat.StrEnum
Constrained generation support (none, all, no-tools).
Source code in packages/genkit/src/genkit/_core/_typing.py
1226 1227 1228 1229 1230 1231 | |
GenerateActionOptions
Bases: genkit._core._base.GenkitModel
Generate options with messages as list[Message] for type-safe use with ai.generate().
Source code in packages/genkit/src/genkit/_core/_model.py
1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
GenerationUsage
Bases: genkit._core._base.GenkitModel
Model for generationusage data.
Source code in packages/genkit/src/genkit/_core/_typing.py
545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 | |
from_messages(messages, response)
classmethod
Character and media counts for the request messages and the model reply.
Source code in packages/genkit/src/genkit/_core/_typing.py
567 568 569 570 571 572 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
Model
Bases: genkit._core._action.Action[genkit._core._model.ModelRequest, genkit._core._model.ModelResponse, genkit._core._model.ModelResponseChunk]
A model action returned from a plugin's init / resolve / list_actions or define_model.
Plugin init / resolve / list_actions return this without registering it;
define_model registers it.
The config class stays on the action so generate(model='name', config=...)
can isinstance-check a Pydantic instance against a string model name.
It comes from config_schema or the fn's ModelRequest[Cfg]
annotation; given both, they must be the same class.
Source code in packages/genkit/src/genkit/_ai/_model.py
481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 | |
config_schema
property
The config class set at definition time, from config_schema= or the fn annotation.
input_class
property
The action's input annotation as a concrete class, when it is one.
params
property
The function's input and context parameters, as Genkit passes them.
__call__(input=NO_INPUT, *, context=None)
async
__call__(
input: None = None,
*,
context: dict[str, Any] | None = None
) -> OutputT
__call__(
input: InputT, *, context: dict[str, Any] | None = None
) -> OutputT
Call the action directly, returning just the response value.
Source code in packages/genkit/src/genkit/_core/_action.py
1350 1351 1352 1353 1354 1355 1356 1357 | |
__init__(name, fn, *, config_schema=None, metadata=None, info=None, description=None)
__init__(
name: str,
fn: ModelFn,
*,
config_schema: (
type[BaseModel] | dict[str, object] | None
) = None,
metadata: dict[str, object] | None = None,
info: ModelInfo | None = None,
description: str | None = None
) -> None
__init__(
name: str,
fn: CtxFirstModelFn,
*,
config_schema: (
type[BaseModel] | dict[str, object] | None
) = None,
metadata: dict[str, object] | None = None,
info: ModelInfo | None = None,
description: str | None = None
) -> None
Build a model action.
Source code in packages/genkit/src/genkit/_ai/_model.py
516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 | |
run(input=NO_INPUT, *, on_chunk=None, context=None, telemetry_labels=None, init=None)
async
run(
input: None = None,
*,
on_chunk: Callable[[ChunkT], None] | None = None,
context: dict[str, Any] | None = None,
telemetry_labels: dict[str, object] | None = None,
init: InitT | None = None
) -> ActionResponse[OutputT]
run(
input: InputT,
*,
on_chunk: Callable[[ChunkT], None] | None = None,
context: dict[str, Any] | None = None,
telemetry_labels: dict[str, object] | None = None,
init: InitT | None = None
) -> ActionResponse[OutputT]
Run the action and return its result with trace ids.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
input
|
genkit._core._action.InputT | None
|
The input, validated against the input schema. Required unless the action takes no input or its function gives the input a default; omit it to use that default. On a flow or tool, an explicit None is a value and is validated like any other. |
genkit._core._action.NO_INPUT
|
on_chunk
|
collections.abc.Callable[[genkit._core._action.ChunkT], None] | None
|
Called with each chunk the action streams. |
None
|
context
|
dict[str, typing.Any] | None
|
Request context (auth, headers, ...), read as |
None
|
telemetry_labels
|
dict[str, object] | None
|
Custom labels set as span attributes. |
None
|
init
|
genkit._core._action.InitT | None
|
Session init value for a bidi action (e.g. a served agent's session identity). Validated against the init schema; plain actions ignore it. |
None
|
Returns:
| Type | Description |
|---|---|
genkit._core._action.ActionResponse[genkit._core._action.OutputT]
|
ActionResponse with the result, trace id and span id. |
Raises:
| Type | Description |
|---|---|
genkit._core._error.GenkitError
|
If input validation fails (INVALID_ARGUMENT status). |
Source code in packages/genkit/src/genkit/_core/_action.py
1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 | |
stream(input=NO_INPUT, *, context=None, telemetry_labels=None, init=None)
stream(
input: None = None,
*,
context: dict[str, Any] | None = None,
telemetry_labels: dict[str, object] | None = None,
init: InitT | None = None
) -> StreamResponse[ChunkT, OutputT]
stream(
input: InputT,
*,
context: dict[str, Any] | None = None,
telemetry_labels: dict[str, object] | None = None,
init: InitT | None = None
) -> StreamResponse[ChunkT, OutputT]
Return a StreamResponse: iterate .stream for chunks, await .response for the result.
Takes the same keyword arguments as run, minus on_chunk. If the
action raises, both the loop and .response raise.
Source code in packages/genkit/src/genkit/_core/_action.py
1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 | |
ModelConfig
Bases: genkit._core._base.GenkitConfig
Settings every model understands, plus extra for provider-only ones.
Unknown keyword arguments raise, so ModelConfig(temprature=0.2) fails
where it was typed instead of being sent or silently dropped. A
per-request API key goes in context={'secrets': {'api_key': ...}}, not
here.
Source code in packages/genkit/src/genkit/_core/_model.py
99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 | |
common_field_bounds = {}
class-attribute
Inclusive (min, max) a provider allows for a numeric common field.
A plugin narrows a common field here instead of re-declaring it. Values
outside the range fail validation, and the Dev UI schema advertises the
range as minimum/maximum.
Example
class OpenAIConfig(ModelConfig):
common_field_bounds = {'temperature': (0.0, 2.0), 'top_p': (0.0, 1.0)}
OpenAIConfig(temperature=3.0)
# => ValidationError: temperature must be between 0.0 and 2.0, got 3.0
extra = None
class-attribute
instance-attribute
Provider settings the model's config class doesn't declare, sent as-is.
Keys are the provider's wire names. The plugin merges them into its
request after the declared fields, so a colliding key wins. Fields Genkit
builds from the request (messages, tools) are rejected rather than
overwritten. Where the map lands depends on the provider: the request
body for Gemini, OpenAI and Anthropic, options for Ollama, and
additionalModelRequestFields for Bedrock chat models (the InvokeModel
body for Bedrock image models).
Keys aren't validated, except that an API key here raises like one at the top level: config travels with the request into traces. Don't put other secrets here either.
max_output_tokens = Field(default=None, description='Maximum number of tokens to generate.')
class-attribute
instance-attribute
Upper bound on tokens in the reply.
stop_sequences = Field(default=None, description='Strings that end the reply when generated.')
class-attribute
instance-attribute
Strings that end the reply when the model emits one.
temperature = Field(default=None, description='Sampling temperature. Higher is more random.')
class-attribute
instance-attribute
Sampling temperature. Higher is more random.
top_k = Field(default=None, description='The maximum number of tokens to consider when sampling.')
class-attribute
instance-attribute
Sample only from the k most likely tokens.
top_p = Field(default=None, description='The maximum cumulative probability of tokens to consider when sampling.')
class-attribute
instance-attribute
Sample from the smallest token set whose probability reaches p.
version = Field(default=None, description='Model version to call, for providers that pin one.')
class-attribute
instance-attribute
Model version to call, for providers that pin one per request.
__get_pydantic_json_schema__(core_schema, handler)
classmethod
Advertise common_field_bounds on the matching schema properties.
Source code in packages/genkit/src/genkit/_core/_model.py
179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 | |
__pydantic_init_subclass__(**kwargs)
classmethod
Reject common_field_bounds entries that name unknown fields or invert (min, max).
Source code in packages/genkit/src/genkit/_core/_model.py
153 154 155 156 157 158 159 160 161 162 163 164 | |
ModelContext
Bases: genkit._core._action.ActionContext[genkit._core._model.ModelResponseChunk]
Execution context passed to model and background-model functions.
ctx.send_chunk(chunk)streams a :class:~genkit.ModelResponseChunkto the caller.ctx.is_streamingisTruewhen the caller requested streaming.ctx.contextholds the caller's context dict (e.g.ctx.context.get('secrets')).
Source code in packages/genkit/src/genkit/_core/_action.py
1097 1098 1099 1100 1101 1102 1103 | |
context
property
The request context (auth, headers, ...) the caller attached.
is_streaming
property
True if a streaming callback is registered.
streaming_callback
property
The streaming callback, if any.
Use this when you need to pass the callback to another action. For sending chunks directly, use send_chunk() instead.
send_chunk(chunk)
Send a streaming chunk to the client.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
chunk
|
genkit._core._action.ChunkT
|
The chunk data to stream. |
required |
Source code in packages/genkit/src/genkit/_core/_action.py
1053 1054 1055 1056 1057 1058 1059 1060 | |
ModelInfo
Bases: genkit._core._base.GenkitModel
Model for modelinfo data.
Source code in packages/genkit/src/genkit/_core/_typing.py
603 604 605 606 607 608 609 610 611 612 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
ModelRef
dataclass
Bases: typing.Generic[genkit._core._model.ModelRefConfigT]
Handle for a model tied to a config schema.
Fields cannot be rebound. config and info are copied at construction so later mutations of the caller's objects don't change the ref; the copies themselves stay ordinary mutable Pydantic models.
Source code in packages/genkit/src/genkit/_core/_model.py
486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 | |
ModelRequest
Bases: genkit._core._base.GenkitModel, typing.Generic[genkit._core._model.ModelRequestConfigT]
Hand-written model request with veneer types and flat output accessors.
Output settings live nested as output: OutputConfig so dump/validate
round-trips the wire shape, while flat properties (output_format etc.)
stay the plugin-author convenience surface. Messages and docs use veneer
types (Message, Document) for helpers like .text.
Example
from genkit.model import ModelConfig
class GeminiConfig(ModelConfig): safety_settings: dict[str, str] | None = None
def gemini_model(request: ModelRequest[GeminiConfig]) -> ModelResponse: temp = request.config.temperature # inherited from ModelConfig for msg in request.messages: print(msg.text) # Message veneer property if request.output_format == 'json': schema = request.output_schema
Note
Pass output settings as output=OutputConfig(...). The flat
names (output_format etc.) are convenience properties you read
and write after construction — they are not constructor arguments,
so passing them there leaves output unset.
Source code in packages/genkit/src/genkit/_core/_model.py
1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 | |
output_constrained
property
writable
Whether constrained decoding is requested; reads output.constrained.
output_content_type
property
writable
Output content type; reads output.content_type.
output_format
property
writable
Output format (e.g. 'json'); reads output.format.
output_schema
property
writable
Output JSON schema; reads output.json_schema.
__class_getitem__(typevar_values)
ModelRequest[Cfg], where Cfg must be a pydantic model class.
Checked here, where the annotation is evaluated, so a TypedDict or dict config fails when the model function is defined, with the same error on every Python version.
Source code in packages/genkit/src/genkit/_core/_model.py
1637 1638 1639 1640 1641 1642 1643 1644 1645 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
ModelStage
Bases: genkit._core._compat.StrEnum
Model stage (featured, stable, unstable, legacy, deprecated).
Source code in packages/genkit/src/genkit/_core/_typing.py
1234 1235 1236 1237 1238 1239 1240 1241 | |
ModelSupports
Bases: genkit._core._base.GenkitModel
Model for modelsupports data.
Source code in packages/genkit/src/genkit/_core/_typing.py
1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
OutputConfig
Bases: genkit._core._base.GenkitModel
Output settings for a model request.
Construct with json_schema=; the serialized key on the wire is
schema.
Source code in packages/genkit/src/genkit/_core/_model.py
1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 | |
schema
property
Read-only alias for json_schema, matching the wire key and JS.
Shadows Pydantic's deprecated BaseModel.schema() classmethod so
request.output.schema returns the schema dict, or None.
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
ToolDefinition
Bases: genkit._core._base.GenkitModel
Model for tooldefinition data.
Source code in packages/genkit/src/genkit/_core/_typing.py
712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
ToolRequest
Bases: genkit._core._base.GenkitModel
Model for toolrequest data.
Source code in packages/genkit/src/genkit/_core/_typing.py
774 775 776 777 778 779 780 781 782 783 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
ToolResponse
Bases: genkit._core._base.GenkitModel
Model for toolresponse data.
Source code in packages/genkit/src/genkit/_core/_typing.py
786 787 788 789 790 791 792 793 794 795 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
genkit.embedder
Embedder namespace module for Genkit.
This module provides embedder-related types and utilities for plugin authors and advanced users who need access to the embedder protocol types.
Example
from genkit.embedder import ( EmbedContext, EmbedRequest, EmbedResponse, Embedder, EmbedderRef, )
EmbedContext
Bases: genkit._core._action.ActionContext[typing_extensions.Never]
Execution context passed to embedder functions.
ctx.context holds the caller's context dict. Embedders return one
response and never emit chunks, so send_chunk accepts nothing.
ctx.is_streaming still reports whether the caller asked to stream
(Embedder.stream(), a Dev UI run with stream=true).
A per-tenant key passed on embed reaches the embedder here:
# 1. Read the tenant's key from the call's context
async def embed_menu(req: EmbedRequest, ctx: EmbedContext) -> EmbedResponse:
api_key = ctx.context['secrets']['api_key']
return await crm_embeddings(req.input, api_key=api_key)
ai.define_embedder(name='bistro/menu-embedder', fn=embed_menu)
# 2. Each call carries its own tenant's key
await ai.embed(
embedder='bistro/menu-embedder',
content=Document.from_text('Smoked Salmon Tartine'),
context={'secrets': {'api_key': tenant_key}},
)
# => [Embedding(embedding=[0.12, -0.03, ...])]
Source code in packages/genkit/src/genkit/_core/_action.py
1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 | |
context
property
The request context (auth, headers, ...) the caller attached.
is_streaming
property
True if a streaming callback is registered.
streaming_callback
property
The streaming callback, if any.
Use this when you need to pass the callback to another action. For sending chunks directly, use send_chunk() instead.
send_chunk(chunk)
Send a streaming chunk to the client.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
chunk
|
genkit._core._action.ChunkT
|
The chunk data to stream. |
required |
Source code in packages/genkit/src/genkit/_core/_action.py
1053 1054 1055 1056 1057 1058 1059 1060 | |
EmbedRequest
Bases: genkit._core._base.GenkitModel, typing.Generic[genkit._core._model.RequestOptionsT]
Embed request whose documents are the public Document type.
What options holds depends on how the embedder annotates request:
- No annotation: Genkit infers
EmbedRequest[Cfg]from the embedder's Pydanticconfig_schema, sooptionsis aCfg. EmbedRequest[CrmEmbedConfig]:optionsis a validatedCrmEmbedConfig.- Bare
EmbedRequest:optionsis adict. An explicit annotation is honored as written, the same asModelRequest.
Bare EmbedRequest means EmbedRequest[Any] to type checkers, so a
helper annotated with it accepts any typed request.
Source code in packages/genkit/src/genkit/_core/_model.py
1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 | |
__class_getitem__(typevar_values)
EmbedRequest[Cfg], where Cfg must be a pydantic model class.
Source code in packages/genkit/src/genkit/_core/_model.py
1363 1364 1365 1366 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
EmbedResponse
Bases: genkit._core._base.GenkitModel
Model for embedresponse data.
Source code in packages/genkit/src/genkit/_core/_typing.py
316 317 318 319 320 321 322 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
Embedder
Bases: genkit._core._action.Action[genkit._core._model.EmbedRequest, genkit._core._typing.EmbedResponse, typing_extensions.Never]
An embedder action that converts documents into vector embeddings.
Plugin init / resolve return this without registering it.
define_embedder constructs and registers it on the registry.
The options class comes from config_schema or the fn's
EmbedRequest[Cfg] annotation; given both, they must be the same class.
An unannotated fn still gets an EmbedRequest, typed by config_schema
when that is a Pydantic class.
Source code in packages/genkit/src/genkit/_ai/_embedding.py
99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 | |
config_schema
property
The config class set at definition time, from config_schema= or the fn annotation.
input_class
property
The action's input annotation as a concrete class, when it is one.
params
property
The function's input and context parameters, as Genkit passes them.
__call__(input=NO_INPUT, *, context=None)
async
__call__(
input: None = None,
*,
context: dict[str, Any] | None = None
) -> OutputT
__call__(
input: InputT, *, context: dict[str, Any] | None = None
) -> OutputT
Call the action directly, returning just the response value.
Source code in packages/genkit/src/genkit/_core/_action.py
1350 1351 1352 1353 1354 1355 1356 1357 | |
run(input=NO_INPUT, *, on_chunk=None, context=None, telemetry_labels=None, init=None)
async
run(
input: None = None,
*,
on_chunk: Callable[[ChunkT], None] | None = None,
context: dict[str, Any] | None = None,
telemetry_labels: dict[str, object] | None = None,
init: InitT | None = None
) -> ActionResponse[OutputT]
run(
input: InputT,
*,
on_chunk: Callable[[ChunkT], None] | None = None,
context: dict[str, Any] | None = None,
telemetry_labels: dict[str, object] | None = None,
init: InitT | None = None
) -> ActionResponse[OutputT]
Run the action and return its result with trace ids.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
input
|
genkit._core._action.InputT | None
|
The input, validated against the input schema. Required unless the action takes no input or its function gives the input a default; omit it to use that default. On a flow or tool, an explicit None is a value and is validated like any other. |
genkit._core._action.NO_INPUT
|
on_chunk
|
collections.abc.Callable[[genkit._core._action.ChunkT], None] | None
|
Called with each chunk the action streams. |
None
|
context
|
dict[str, typing.Any] | None
|
Request context (auth, headers, ...), read as |
None
|
telemetry_labels
|
dict[str, object] | None
|
Custom labels set as span attributes. |
None
|
init
|
genkit._core._action.InitT | None
|
Session init value for a bidi action (e.g. a served agent's session identity). Validated against the init schema; plain actions ignore it. |
None
|
Returns:
| Type | Description |
|---|---|
genkit._core._action.ActionResponse[genkit._core._action.OutputT]
|
ActionResponse with the result, trace id and span id. |
Raises:
| Type | Description |
|---|---|
genkit._core._error.GenkitError
|
If input validation fails (INVALID_ARGUMENT status). |
Source code in packages/genkit/src/genkit/_core/_action.py
1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 | |
stream(input=NO_INPUT, *, context=None, telemetry_labels=None, init=None)
stream(
input: None = None,
*,
context: dict[str, Any] | None = None,
telemetry_labels: dict[str, object] | None = None,
init: InitT | None = None
) -> StreamResponse[ChunkT, OutputT]
stream(
input: InputT,
*,
context: dict[str, Any] | None = None,
telemetry_labels: dict[str, object] | None = None,
init: InitT | None = None
) -> StreamResponse[ChunkT, OutputT]
Return a StreamResponse: iterate .stream for chunks, await .response for the result.
Takes the same keyword arguments as run, minus on_chunk. If the
action raises, both the loop and .response raise.
Source code in packages/genkit/src/genkit/_core/_action.py
1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 | |
EmbedderInfo
Bases: pydantic.BaseModel
Catalog card for an embedder: label, vector width, and input kinds.
Source code in packages/genkit/src/genkit/_ai/_embedding.py
43 44 45 46 47 48 49 50 51 | |
EmbedderRef
dataclass
Bases: typing.Generic[genkit._ai._embedding.EmbedderRefConfigT]
Handle for an embedder, shaped like ModelRef.
config_schema is optional: embedders have no generic base config the
way models have ModelConfig. Without it, config is a plain mapping
and the call is not checked against a class. With it, config must be
an instance, and ai.embed requires the embedder to declare the same
class. version is the lowest caller layer, below config and the
call's config, the same as ModelRef.version.
Fields cannot be rebound. config and info are copied at construction.
Source code in packages/genkit/src/genkit/_ai/_embedding.py
61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 | |
EmbedderSupports
Bases: pydantic.BaseModel
Embedder capability support.
Source code in packages/genkit/src/genkit/_ai/_embedding.py
34 35 36 37 38 39 40 | |
genkit.formats
Custom output formats. Register a Formatter subclass with ai.define_format().
Example
from genkit.formats import FormatHandler, Formatter, FormatterConfig
FormatHandler
Bases: typing.Generic[genkit._ai._formats._types.OutputT, genkit._ai._formats._types.ChunkT]
Per-request parsers and prompt instructions returned by Formatter.handle().
message_parser builds response.output; chunk_parser builds chunk.output.
instructions, when set, is added to the prompt, except on a natively constrained
JSON request, where the model already follows the schema. A request is natively
constrained when format='json', constrained is on, there is a schema, and the
model declares supports.constrained (all, or no-tools with no tools). A caller
overrides it per request with output_instructions: True adds it, False adds
nothing, a string replaces it.
Source code in packages/genkit/src/genkit/_ai/_formats/_types.py
49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 | |
__init__(message_parser, chunk_parser, instructions=None)
Initializes a FormatHandler.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
message_parser
|
collections.abc.Callable[[genkit._ai._model.Message], genkit._ai._formats._types.OutputT]
|
Parses the final model message into the output value. |
required |
chunk_parser
|
collections.abc.Callable[[genkit._ai._model.ModelResponseChunk], genkit._ai._formats._types.ChunkT]
|
Parses a streamed chunk into the chunk output value. |
required |
instructions
|
str | None
|
Text appended to the prompt, or None for no instructions. |
None
|
Source code in packages/genkit/src/genkit/_ai/_formats/_types.py
61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 | |
parse_chunk(chunk)
Parses one streamed chunk.
Source code in packages/genkit/src/genkit/_ai/_formats/_types.py
82 83 84 | |
parse_message(message)
Parses the final model message.
Source code in packages/genkit/src/genkit/_ai/_formats/_types.py
78 79 80 | |
Formatter
Bases: abc.ABC
A named output format. Subclass it, implement handle(), register with ai.define_format().
handle() runs once per generate call with the request's JSON schema (or None).
Example
import csv
import io
from genkit import Genkit
from genkit.formats import FormatHandler, Formatter, FormatterConfig
from genkit_google_genai import GoogleAI
class CsvFormat(Formatter):
def __init__(self) -> None:
super().__init__('csv', FormatterConfig(content_type='text/csv'))
def handle(self, schema: dict[str, object] | None) -> FormatHandler:
return FormatHandler(
message_parser=lambda msg: list(csv.DictReader(io.StringIO(msg.text))),
chunk_parser=lambda chunk: chunk.text,
instructions='Reply only with CSV. First row is the header.',
)
# 1. Register the format
ai = Genkit(plugins=[GoogleAI()], model='googleai/gemini-flash-latest')
ai.define_format(CsvFormat())
# 2. Request it by name
response = await ai.generate(prompt='List 2 dishes with price.', output_format='csv')
# 3. response.output is what message_parser returned
print(response.output)
# => [{'dish': 'Ramen', 'price': '14.50'}, {'dish': 'Pho', 'price': '13.00'}]
Source code in packages/genkit/src/genkit/_ai/_formats/_types.py
87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 | |
__call__(schema)
Same as handle(schema).
Source code in packages/genkit/src/genkit/_ai/_formats/_types.py
146 147 148 | |
__init__(name, config)
Initializes a Formatter.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str
|
Name passed as |
required |
config
|
genkit._ai._formats._types.FormatterConfig
|
Request settings applied when this format is selected. |
required |
Source code in packages/genkit/src/genkit/_ai/_formats/_types.py
127 128 129 130 131 132 133 134 135 | |
handle(schema)
abstractmethod
Returns the FormatHandler for one generate call.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
schema
|
dict[str, object] | None
|
The request's output JSON schema, or None. |
required |
Source code in packages/genkit/src/genkit/_ai/_formats/_types.py
137 138 139 140 141 142 143 144 | |
FormatterConfig
Bases: genkit._core._base.GenkitModel
Request settings a Formatter applies to every generate call that uses it.
format and content_type replace the request's values. constrained asks for the
model's native JSON mode, and the request can override it. It only takes effect when
there is a schema and the model declares supports.constrained.
Source code in packages/genkit/src/genkit/_ai/_formats/_types.py
32 33 34 35 36 37 38 39 40 41 42 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
genkit.plugin_api
Framework primitives for plugin authors.
GENKIT_USER_AGENT = f'genkit-python/{GENKIT_VERSION}'
module-attribute
Header value plugins send in x-goog-api-client and user-agent.
StatusName = Literal['OK', 'CANCELLED', 'UNKNOWN', 'INVALID_ARGUMENT', 'DEADLINE_EXCEEDED', 'NOT_FOUND', 'ALREADY_EXISTS', 'PERMISSION_DENIED', 'UNAUTHENTICATED', 'RESOURCE_EXHAUSTED', 'FAILED_PRECONDITION', 'ABORTED', 'OUT_OF_RANGE', 'UNIMPLEMENTED', 'INTERNAL', 'UNAVAILABLE', 'DATA_LOSS']
module-attribute
Canonical gRPC status name a GenkitError carries, e.g. 'NOT_FOUND'.
Action
Bases: typing.Generic[genkit._core._action.InputT, genkit._core._action.OutputT, genkit._core._action.ChunkT, genkit._core._action.InitT]
A named, traced, remotely callable function.
Source code in packages/genkit/src/genkit/_core/_action.py
1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 1790 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802 1803 1804 1805 1806 1807 1808 1809 1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 1820 1821 1822 1823 1824 1825 1826 1827 1828 1829 1830 1831 1832 1833 1834 1835 1836 1837 1838 1839 1840 1841 1842 1843 1844 1845 1846 1847 1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 1864 1865 1866 1867 1868 1869 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 | |
config_schema
property
The config class set at definition time, from config_schema= or the fn annotation.
input_class
property
The action's input annotation as a concrete class, when it is one.
params
property
The function's input and context parameters, as Genkit passes them.
__call__(input=NO_INPUT, *, context=None)
async
__call__(
input: None = None,
*,
context: dict[str, Any] | None = None
) -> OutputT
__call__(
input: InputT, *, context: dict[str, Any] | None = None
) -> OutputT
Call the action directly, returning just the response value.
Source code in packages/genkit/src/genkit/_core/_action.py
1350 1351 1352 1353 1354 1355 1356 1357 | |
run(input=NO_INPUT, *, on_chunk=None, context=None, telemetry_labels=None, init=None)
async
run(
input: None = None,
*,
on_chunk: Callable[[ChunkT], None] | None = None,
context: dict[str, Any] | None = None,
telemetry_labels: dict[str, object] | None = None,
init: InitT | None = None
) -> ActionResponse[OutputT]
run(
input: InputT,
*,
on_chunk: Callable[[ChunkT], None] | None = None,
context: dict[str, Any] | None = None,
telemetry_labels: dict[str, object] | None = None,
init: InitT | None = None
) -> ActionResponse[OutputT]
Run the action and return its result with trace ids.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
input
|
genkit._core._action.InputT | None
|
The input, validated against the input schema. Required unless the action takes no input or its function gives the input a default; omit it to use that default. On a flow or tool, an explicit None is a value and is validated like any other. |
genkit._core._action.NO_INPUT
|
on_chunk
|
collections.abc.Callable[[genkit._core._action.ChunkT], None] | None
|
Called with each chunk the action streams. |
None
|
context
|
dict[str, typing.Any] | None
|
Request context (auth, headers, ...), read as |
None
|
telemetry_labels
|
dict[str, object] | None
|
Custom labels set as span attributes. |
None
|
init
|
genkit._core._action.InitT | None
|
Session init value for a bidi action (e.g. a served agent's session identity). Validated against the init schema; plain actions ignore it. |
None
|
Returns:
| Type | Description |
|---|---|
genkit._core._action.ActionResponse[genkit._core._action.OutputT]
|
ActionResponse with the result, trace id and span id. |
Raises:
| Type | Description |
|---|---|
genkit._core._error.GenkitError
|
If input validation fails (INVALID_ARGUMENT status). |
Source code in packages/genkit/src/genkit/_core/_action.py
1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 | |
stream(input=NO_INPUT, *, context=None, telemetry_labels=None, init=None)
stream(
input: None = None,
*,
context: dict[str, Any] | None = None,
telemetry_labels: dict[str, object] | None = None,
init: InitT | None = None
) -> StreamResponse[ChunkT, OutputT]
stream(
input: InputT,
*,
context: dict[str, Any] | None = None,
telemetry_labels: dict[str, object] | None = None,
init: InitT | None = None
) -> StreamResponse[ChunkT, OutputT]
Return a StreamResponse: iterate .stream for chunks, await .response for the result.
Takes the same keyword arguments as run, minus on_chunk. If the
action raises, both the loop and .response raise.
Source code in packages/genkit/src/genkit/_core/_action.py
1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 | |
ActionContext
Bases: typing.Generic[genkit._core._action.ChunkT]
Base execution context for a running action: request context and chunk streaming.
Exported in genkit.plugin_api for custom actions and
BackgroundAction handlers, and subclassed publicly as
genkit.FlowContext, genkit.ToolContext,
genkit.model.ModelContext, genkit.embedder.EmbedContext, and
genkit.exp.agent.AgentContext.
ChunkT is the type send_chunk accepts.
@ai.flow
async def plate_dish(dish: str, ctx: FlowContext[str]) -> str:
ctx.send_chunk('searing')
return f'{dish} for {ctx.context.get("table", "walk-in")}'
Source code in packages/genkit/src/genkit/_core/_action.py
1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 | |
context
property
The request context (auth, headers, ...) the caller attached.
is_streaming
property
True if a streaming callback is registered.
streaming_callback
property
The streaming callback, if any.
Use this when you need to pass the callback to another action. For sending chunks directly, use send_chunk() instead.
send_chunk(chunk)
Send a streaming chunk to the client.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
chunk
|
genkit._core._action.ChunkT
|
The chunk data to stream. |
required |
Source code in packages/genkit/src/genkit/_core/_action.py
1053 1054 1055 1056 1057 1058 1059 1060 | |
ActionKind
Bases: genkit._core._compat.StrEnum
Types of actions that can be registered.
Source code in packages/genkit/src/genkit/_core/_action.py
184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 | |
BackgroundAction
Bases: typing.Generic[genkit._core._background.InT, genkit._core._background.OutT]
A handle over a background job's start, check and cancel actions.
A composite, not an Action: start, check and cancel each have their
own registry key. start returns an Operation[OutT]; pass it to
check until done. Subclass with concrete type arguments, e.g.
class ReportJob(BackgroundAction[ReportRequest, Report]), and the
handle's output validates as Report.
Every attribute is either a ClassVar derived from the type arguments or
derived from the three actions in _bind, so a handle rebuilt from the
registry (_from_actions) behaves like the one that was defined.
Example
# 1. Type the job by subclassing with concrete type arguments
class ReportJob(BackgroundAction[RenderRequest, Report]):
pass
async def start(request: RenderRequest, ctx: ActionContext) -> Operation:
return Operation(id=await renderer.submit(request.quarter))
async def check(op: Operation[Report], ctx: ActionContext) -> Operation:
return await renderer.poll(op.id)
# 2. Build it; plugins return it from init() or resolve()
job = ReportJob('render-report', start, check, kind=ActionKind.CUSTOM)
# 3. The handle's output validates as Report
op = await job.check(await job.start(RenderRequest(quarter='q3')))
print(type(op.output).__name__)
# => Report
Source code in packages/genkit/src/genkit/_core/_background.py
185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 | |
actions
property
The start, check and (if any) cancel actions, for registering or listing.
A background action is three registered actions, not one. Register all of them, or a poll fails later with the check action not found.
input_schema
property
The start action's input JSON schema.
kind
property
The action kind of the start action.
metadata
property
The start action's metadata.
name
property
The name of the start action.
output_schema
property
The start action's output JSON schema (the Operation).
supports_cancel
property
Whether this background action supports cancellation.
__init__(name, start, check, *, kind, cancel=None, metadata=None, description=None)
Build a background action without registering it.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str
|
The start action's name. Check and cancel are
|
required |
start
|
genkit._core._background.StartOpFn[genkit._core._background.InT]
|
Starts the job and returns an |
required |
check
|
collections.abc.Callable[[genkit._core._operation.Operation[genkit._core._background.OutT], genkit._core._action.ActionContext], collections.abc.Awaitable[genkit._core._operation.Operation]]
|
Polls an |
required |
kind
|
genkit._core._action.ActionKind
|
The start action's kind; it is the first segment of the
|
required |
cancel
|
collections.abc.Callable[[genkit._core._operation.Operation[genkit._core._background.OutT], genkit._core._action.ActionContext], collections.abc.Awaitable[genkit._core._operation.Operation]] | None
|
Optional; without it |
None
|
metadata
|
dict[str, typing.Any] | None
|
Copied onto all three actions, like JS/Go. |
None
|
description
|
str | None
|
The start action's description. |
None
|
Source code in packages/genkit/src/genkit/_core/_background.py
245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 | |
__init_subclass__(**kwargs)
Read InT / OutT off class X(BackgroundAction[In, Out]).
Source code in packages/genkit/src/genkit/_core/_background.py
234 235 236 237 238 239 240 241 242 243 | |
cancel(operation, *, context=None)
async
Cancel a background operation.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
operation
|
genkit._core._operation.Operation | collections.abc.Mapping[str, typing.Any]
|
The operation to cancel, or the dict a stored one dumped to. |
required |
context
|
dict[str, typing.Any] | None
|
Optional run context (secrets, folded client config). |
None
|
Raises:
| Type | Description |
|---|---|
genkit._core._error.GenkitError
|
UNIMPLEMENTED if this action does not implement
cancel, INVALID_ARGUMENT if |
Source code in packages/genkit/src/genkit/_core/_background.py
429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 | |
check(operation, *, context=None)
async
Check the status of a background operation.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
operation
|
genkit._core._operation.Operation | collections.abc.Mapping[str, typing.Any]
|
The operation to check, or the dict a stored one dumped to. |
required |
context
|
dict[str, typing.Any] | None
|
Optional run context (secrets, folded client config). |
None
|
Raises:
| Type | Description |
|---|---|
genkit._core._error.GenkitError
|
INVALID_ARGUMENT if |
Source code in packages/genkit/src/genkit/_core/_background.py
408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 | |
lookup(registry, key)
async
classmethod
Look up a background action by its start action key.
Matches JS lookupBackgroundAction. The key format is
/{actionType}/{name}, e.g. /background-model/video-gen.
Source code in packages/genkit/src/genkit/_core/_background.py
295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 | |
start(input=None, *, context=None)
async
Start a background operation.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
input
|
genkit._core._background.InT | None
|
The start request. |
None
|
context
|
dict[str, typing.Any] | None
|
Optional run context. Per-request keys go in
|
None
|
Returns:
| Type | Description |
|---|---|
genkit._core._operation.Operation[genkit._core._background.OutT]
|
An Operation with an ID to track the job. |
Source code in packages/genkit/src/genkit/_core/_background.py
387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 | |
DynamicActionProvider
Lazily resolves actions from an external source with TTL caching.
The cached actions are shared by every event loop that lists this provider,
so an action returned by dap_fn must resolve any loop-bound resource of
its own when it is called, not when it is listed. In-flight fetches are
coalesced per loop, because a task cannot be awaited from a loop other than
the one that created it.
The cache is one attribute holding both the value and its expiry, so a reader takes a consistent pair in a single read and an invalidation from another thread cannot land between the two.
Source code in packages/genkit/src/genkit/_core/_dap.py
42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 | |
invalidate_cache()
Drop the cached actions so the next call fetches them again.
Source code in packages/genkit/src/genkit/_core/_dap.py
75 76 77 | |
list_action_metadata(action_type, action_name)
async
List metadata matching pattern: ''=all, 'prefix'=prefix match, else exact.
Source code in packages/genkit/src/genkit/_core/_dap.py
140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 | |
list_action_metadata_by_key(dap_prefix)
async
List every child action's reflection metadata, keyed by its fully-qualified DAP key.
Source code in packages/genkit/src/genkit/_core/_dap.py
162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 | |
GenkitConfig
Bases: pydantic.BaseModel
Base for settings passed to a Genkit action: model and middleware config.
Python callers construct it with field names (snake_case), and plugins hand model_dump() output to provider SDKs. Genkit's tooling sees camelCase JSON: the Developer UI renders the JSON schema as a form, pre-fills it from traces and .prompt files by exact key, and exports .prompt files from it. Those formats are shared with the JS and Go SDKs, so the schema keys, trace keys and .prompt keys must use one spelling.
- alias_generator=to_camel: camelCase wire names and schema properties.
- validate_by_name / validate_by_alias: accept either spelling; all three type checkers take the snake_case kwarg.
- extra='forbid': an unknown key raises instead of being dropped, and so does a second spelling of a key already given (maxOutputTokens and max_output_tokens in one input).
model_dump() is not overridden: standalone dumps return snake_case field names and keep None values. Pass by_alias=True for Genkit JSON. Nested inside a GenkitModel (such as ModelRequest.config), _serialize_in_genkit_model applies wire defaults (by_alias=True, exclude_none=True) automatically, even when the outer dump started from a plain BaseModel.
Use plain BaseModel for flow and tool input/output; that data is the app's, and Genkit does not rename it. An app's own middleware config may also be a plain BaseModel: its keys are its field names everywhere, and unknown keys still raise.
Source code in packages/genkit/src/genkit/_core/_base.py
145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 | |
Plugin
Bases: abc.ABC
Abstract base class for Genkit plugins.
Source code in packages/genkit/src/genkit/_core/_plugin.py
40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 | |
init()
abstractmethod
async
Lazy warm-up called once per plugin; return actions to pre-register.
A BackgroundAction (e.g. a BackgroundModel) registers its start,
check and cancel actions together. Sequence so an override that
returns list[Action] still type-checks.
Return names with or without the {plugin.name}/ prefix. A name that
doesn't start with it is registered under {plugin.name}/{name} with
the whole name kept: endpoints/123 becomes myplug/endpoints/123,
and other/x becomes myplug/other/x. See :meth:resolve.
Source code in packages/genkit/src/genkit/_core/_plugin.py
45 46 47 48 49 50 51 52 53 54 55 56 57 58 | |
list_actions()
abstractmethod
async
Return advertised actions for dev UI/reflection listing.
A BackgroundAction is listed once, under its start action. Each
returned action's name must match resolution keys (typically
{plugin.name}/localName for plugin-backed actions).
Source code in packages/genkit/src/genkit/_core/_plugin.py
106 107 108 109 110 111 112 113 114 | |
list_middleware()
Return middleware classes this plugin registers on the app. Defaults to none.
Runs once while :class:Genkit is constructed, after built-in
middleware. Each returned :class:~genkit.middleware.Middleware
subclass is registered under its cls.name via
:meth:Genkit.define_middleware() <genkit.Genkit.define_middleware>.
Example
from genkit import Genkit
from genkit.middleware import Middleware
from typing import Any
from genkit.plugin_api import Action, ActionKind, BackgroundAction, Plugin
# 1. Define a middleware class with a registry name
class AllergyCheck(Middleware):
name = 'allergy_check'
# 2. A plugin that contributes the middleware class
class AllergyGuard(Plugin):
name = 'allergy-guard'
async def init(self) -> list[Action | BackgroundAction[Any, Any]]:
return []
async def resolve(self, kind: ActionKind, name: str) -> Action | BackgroundAction[Any, Any] | None:
return None
async def list_actions(self) -> list[Action]:
return []
def list_middleware(self) -> list[type[Middleware]]:
return [AllergyCheck]
# 3. Register the plugin and pass the middleware in use=[...]
ai = Genkit(plugins=[AllergyGuard()])
res = await ai.generate(prompt='Suggest a dish.', use=[AllergyCheck()])
# => AllergyCheck runs around the model call
Source code in packages/genkit/src/genkit/_core/_plugin.py
116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 | |
resolve(action_type, name)
abstractmethod
async
Resolve a single action by kind and its id inside this plugin.
Return a BackgroundAction (e.g. a BackgroundModel) for a
background job. Genkit registers its start, check and cancel actions
together, so a later /check-operation/{name}/check lookup finds the
check action without the plugin routing it. If resolve returns
None for a check or cancel key, Genkit asks again with
ActionKind.BACKGROUND_MODEL and the start id.
Naming rule. When an app asks for {plugin.name}/{rest}:
- Genkit removes exactly one
{plugin.name}/from the front and passesrestasname. That's the provider's id, verbatim: send it upstream unchanged and don't strip anything from it. Provider ids may contain slashes (publisher paths, tuned endpoints, ARNs, OpenRouter ids). - The returned action is renamed to exactly what the app asked for and stored there, whatever name the plugin gave it, so the same string always looks up the same action.
Return None to decline; the caller then gets NOT_FOUND.
Example
from genkit import Genkit
from genkit.plugin_api import ActionKind
from genkit_openai import OpenAI
# 1. Point the OpenAI plugin at OpenRouter, whose ids carry a vendor
ai = Genkit(plugins=[OpenAI(base_url='https://openrouter.ai/api/v1')])
# 2. One `openai/` is the plugin; the rest is the OpenRouter id
response = await ai.generate(model='openai/openai/gpt-4o', prompt='Suggest a dish.')
# => resolve(ActionKind.MODEL, 'openai/gpt-4o')
# request sent with model='openai/gpt-4o'
# 3. The action lives under the id the app typed
action = await ai.lookup_model('openai/openai/gpt-4o')
print(action.name)
# => openai/openai/gpt-4o
Source code in packages/genkit/src/genkit/_core/_plugin.py
60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 | |
is_dev_environment()
Check if running in development mode (GENKIT_ENV=dev).
Source code in packages/genkit/src/genkit/_core/_environment.py
34 35 36 | |
loop_local_client(factory)
loop_local_client(
factory: Callable[[], T],
) -> Callable[[], T]
loop_local_client(
factory: Callable[[S], T],
) -> _LoopLocal[S, T]
Cache a long-lived client per event loop. Use it as a decorator on a factory.
Use it for a client you reuse across calls that code on different event loops may call. Async clients (httpx, the OpenAI and Anthropic SDKs) are bound to the loop they first run on, so one shared client breaks when Genkit runs on a second loop. Each loop gets its own client, built on first use there and returned on every later call there.
The client is shared, so don't close it; it lives as long as its loop. For a
client scoped to one call, skip the cache and use
async with httpx.AsyncClient() as client: inside the call.
import httpx
from genkit.plugin_api import loop_local_client
# 1. Decorate a factory
@loop_local_client
def http_client() -> httpx.AsyncClient:
return httpx.AsyncClient(timeout=60.0)
# 2. Call it wherever you need the client
response = await http_client().get('https://example.com/menu.json')
# 3. Same loop, same client
print(http_client() is http_client())
# => True
Where the client lives depends on whether it carries plugin settings:
- Module level (the decorator above): the client is a plain transport and the API key, URL, headers, and timeout go on each request. Every plugin instance shares one connection pool per loop.
- On the plugin instance: the client is built from plugin settings, as
with SDK clients that take
api_keyorbase_url. Decorate a method; each plugin instance gets its own client per loop:
from openai import AsyncOpenAI
class Bistro(Plugin):
def __init__(self, api_key: str) -> None:
self._api_key = api_key
@loop_local_client
def _client(self) -> AsyncOpenAI:
return AsyncOpenAI(api_key=self._api_key)
async def _generate(self, request: ModelRequest) -> ModelResponse:
completion = await self._client().chat.completions.create(...)
Plain callables work too, e.g. loop_local_client(asyncio.Lock).
Source code in packages/genkit/src/genkit/_core/_loop_cache.py
90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 | |
provider_error(error, *, http_status=None, status=None, headers=None, retry_after_ms=None, message=None, reason=None)
Turn a model provider's failure into a GenkitError that Retry and Fallback can act on.
Raise the result from a model or embedder when the provider call fails:
except httpx.HTTPStatusError as e:
raise provider_error(e, http_status=e.response.status_code, headers=e.response.headers) from e
Status: status if given, else mapped from http_status (400 is
INVALID_ARGUMENT, 429 RESOURCE_EXHAUSTED, 503 UNAVAILABLE, any other 5xx
INTERNAL). A 4xx with no matching status (413, 422) or no usable code at
all is UNKNOWN, which default Retry and Fallback don't act on.
Retry delay: retry_after_ms if given, else the Retry-After header
(seconds or an HTTP date). Retry waits at least that long before the next
attempt. A missing or malformed header just means no floor.
The message is message, else str(error), else the error's type
name, and error is kept as the cause; str() appends the cause's
text unless the message already contains it. A served flow still answers
500 Internal Error: a dead API key on the server isn't the caller's 401.
Raise PublicError to show the caller a status.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
error
|
BaseException
|
What the provider SDK raised. |
required |
http_status
|
int | None
|
The provider's HTTP status code, if there was one. |
None
|
status
|
genkit._core._error.StatusName | None
|
A Genkit status to use instead of mapping |
None
|
headers
|
collections.abc.Mapping[str, str] | None
|
The provider's response headers. |
None
|
retry_after_ms
|
float | None
|
How long to wait before retrying, instead of reading headers. |
None
|
message
|
str | None
|
Message to use instead of |
None
|
reason
|
genkit._core._error.ErrorReason | None
|
Set when the plugin can tell why the provider failed, such as
|
None
|
Returns:
| Type | Description |
|---|---|
genkit._core._error.GenkitError
|
A GenkitError to raise. |
Source code in packages/genkit/src/genkit/_core/_error.py
634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 | |
to_json_schema(schema)
Convert a Python type to JSON schema. Pass-through if already a dict.
Source code in packages/genkit/src/genkit/_core/_schema.py
44 45 46 47 48 49 50 51 | |
genkit.evaluator
Evaluator namespace module for Genkit.
Everything needed to call or write an evaluator lives here.
Example
from genkit import Genkit
from genkit.evaluator import BaseDataPoint, EvaluatorResponse, Score
ai = Genkit()
# 1. Score one datapoint
async def allergen_check(datapoint: BaseDataPoint, _options: object | None = None) -> EvaluatorResponse:
ok = 'peanut' not in str(datapoint.output).lower()
return EvaluatorResponse(test_case_id=datapoint.test_case_id or '', evaluation=[Score(score=ok)])
ai.define_evaluator(
name='allergen_check', display_name='Allergen check', definition='Flags peanuts.', fn=allergen_check
)
# 2. Run it over a dataset
results = await ai.evaluate(
evaluator='allergen_check',
dataset=[BaseDataPoint(input='Suggest a dish.', output='Peanut noodles')],
)
print(results[0].evaluation[0].score)
# => False
BaseDataPoint
Bases: genkit._core._base.GenkitModel
Model for basedatapoint data.
Source code in packages/genkit/src/genkit/_core/_typing.py
335 336 337 338 339 340 341 342 343 344 345 346 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
BatchEvaluator
Bases: genkit._core._action.Action[genkit._core._typing.EvaluatorRequest, genkit._core._typing.EvalResponse, typing_extensions.Never]
Batch evaluator action that runs fn once on the whole EvaluatorRequest, without registering it.
fn returns list[EvaluatorResponse]. The action wraps the list so
action.run(...).response is an EvalResponse.
Plugin init / resolve / list_actions return this.
define_batch_evaluator registers it.
Source code in packages/genkit/src/genkit/_ai/_evaluator.py
255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 | |
config_schema
property
The config class set at definition time, from config_schema= or the fn annotation.
input_class
property
The action's input annotation as a concrete class, when it is one.
params
property
The function's input and context parameters, as Genkit passes them.
__call__(input=NO_INPUT, *, context=None)
async
__call__(
input: None = None,
*,
context: dict[str, Any] | None = None
) -> OutputT
__call__(
input: InputT, *, context: dict[str, Any] | None = None
) -> OutputT
Call the action directly, returning just the response value.
Source code in packages/genkit/src/genkit/_core/_action.py
1350 1351 1352 1353 1354 1355 1356 1357 | |
run(input=NO_INPUT, *, on_chunk=None, context=None, telemetry_labels=None, init=None)
async
run(
input: None = None,
*,
on_chunk: Callable[[ChunkT], None] | None = None,
context: dict[str, Any] | None = None,
telemetry_labels: dict[str, object] | None = None,
init: InitT | None = None
) -> ActionResponse[OutputT]
run(
input: InputT,
*,
on_chunk: Callable[[ChunkT], None] | None = None,
context: dict[str, Any] | None = None,
telemetry_labels: dict[str, object] | None = None,
init: InitT | None = None
) -> ActionResponse[OutputT]
Run the action and return its result with trace ids.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
input
|
genkit._core._action.InputT | None
|
The input, validated against the input schema. Required unless the action takes no input or its function gives the input a default; omit it to use that default. On a flow or tool, an explicit None is a value and is validated like any other. |
genkit._core._action.NO_INPUT
|
on_chunk
|
collections.abc.Callable[[genkit._core._action.ChunkT], None] | None
|
Called with each chunk the action streams. |
None
|
context
|
dict[str, typing.Any] | None
|
Request context (auth, headers, ...), read as |
None
|
telemetry_labels
|
dict[str, object] | None
|
Custom labels set as span attributes. |
None
|
init
|
genkit._core._action.InitT | None
|
Session init value for a bidi action (e.g. a served agent's session identity). Validated against the init schema; plain actions ignore it. |
None
|
Returns:
| Type | Description |
|---|---|
genkit._core._action.ActionResponse[genkit._core._action.OutputT]
|
ActionResponse with the result, trace id and span id. |
Raises:
| Type | Description |
|---|---|
genkit._core._error.GenkitError
|
If input validation fails (INVALID_ARGUMENT status). |
Source code in packages/genkit/src/genkit/_core/_action.py
1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 | |
stream(input=NO_INPUT, *, context=None, telemetry_labels=None, init=None)
stream(
input: None = None,
*,
context: dict[str, Any] | None = None,
telemetry_labels: dict[str, object] | None = None,
init: InitT | None = None
) -> StreamResponse[ChunkT, OutputT]
stream(
input: InputT,
*,
context: dict[str, Any] | None = None,
telemetry_labels: dict[str, object] | None = None,
init: InitT | None = None
) -> StreamResponse[ChunkT, OutputT]
Return a StreamResponse: iterate .stream for chunks, await .response for the result.
Takes the same keyword arguments as run, minus on_chunk. If the
action raises, both the loop and .response raise.
Source code in packages/genkit/src/genkit/_core/_action.py
1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 | |
Evaluator
Bases: genkit._core._action.Action[genkit._core._typing.EvaluatorRequest, genkit._core._typing.EvalResponse, typing_extensions.Never]
Evaluator action that runs fn on each dataset sample, without registering it.
Plugin init / resolve / list_actions return this.
define_evaluator registers it.
Source code in packages/genkit/src/genkit/_ai/_evaluator.py
147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 | |
config_schema
property
The config class set at definition time, from config_schema= or the fn annotation.
input_class
property
The action's input annotation as a concrete class, when it is one.
params
property
The function's input and context parameters, as Genkit passes them.
__call__(input=NO_INPUT, *, context=None)
async
__call__(
input: None = None,
*,
context: dict[str, Any] | None = None
) -> OutputT
__call__(
input: InputT, *, context: dict[str, Any] | None = None
) -> OutputT
Call the action directly, returning just the response value.
Source code in packages/genkit/src/genkit/_core/_action.py
1350 1351 1352 1353 1354 1355 1356 1357 | |
run(input=NO_INPUT, *, on_chunk=None, context=None, telemetry_labels=None, init=None)
async
run(
input: None = None,
*,
on_chunk: Callable[[ChunkT], None] | None = None,
context: dict[str, Any] | None = None,
telemetry_labels: dict[str, object] | None = None,
init: InitT | None = None
) -> ActionResponse[OutputT]
run(
input: InputT,
*,
on_chunk: Callable[[ChunkT], None] | None = None,
context: dict[str, Any] | None = None,
telemetry_labels: dict[str, object] | None = None,
init: InitT | None = None
) -> ActionResponse[OutputT]
Run the action and return its result with trace ids.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
input
|
genkit._core._action.InputT | None
|
The input, validated against the input schema. Required unless the action takes no input or its function gives the input a default; omit it to use that default. On a flow or tool, an explicit None is a value and is validated like any other. |
genkit._core._action.NO_INPUT
|
on_chunk
|
collections.abc.Callable[[genkit._core._action.ChunkT], None] | None
|
Called with each chunk the action streams. |
None
|
context
|
dict[str, typing.Any] | None
|
Request context (auth, headers, ...), read as |
None
|
telemetry_labels
|
dict[str, object] | None
|
Custom labels set as span attributes. |
None
|
init
|
genkit._core._action.InitT | None
|
Session init value for a bidi action (e.g. a served agent's session identity). Validated against the init schema; plain actions ignore it. |
None
|
Returns:
| Type | Description |
|---|---|
genkit._core._action.ActionResponse[genkit._core._action.OutputT]
|
ActionResponse with the result, trace id and span id. |
Raises:
| Type | Description |
|---|---|
genkit._core._error.GenkitError
|
If input validation fails (INVALID_ARGUMENT status). |
Source code in packages/genkit/src/genkit/_core/_action.py
1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 | |
stream(input=NO_INPUT, *, context=None, telemetry_labels=None, init=None)
stream(
input: None = None,
*,
context: dict[str, Any] | None = None,
telemetry_labels: dict[str, object] | None = None,
init: InitT | None = None
) -> StreamResponse[ChunkT, OutputT]
stream(
input: InputT,
*,
context: dict[str, Any] | None = None,
telemetry_labels: dict[str, object] | None = None,
init: InitT | None = None
) -> StreamResponse[ChunkT, OutputT]
Return a StreamResponse: iterate .stream for chunks, await .response for the result.
Takes the same keyword arguments as run, minus on_chunk. If the
action raises, both the loop and .response raise.
Source code in packages/genkit/src/genkit/_core/_action.py
1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 | |
EvaluatorRef
Bases: pydantic.BaseModel
Reference to an evaluator.
Source code in packages/genkit/src/genkit/_ai/_evaluator.py
105 106 107 108 109 110 111 | |
EvaluatorRequest
Bases: genkit._core._base.GenkitModel
Model for evaluatorrequest data.
Source code in packages/genkit/src/genkit/_core/_typing.py
383 384 385 386 387 388 389 390 391 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
EvaluatorResponse
Bases: genkit._core._base.GenkitModel
Model for evaluatorresponse data.
Source code in packages/genkit/src/genkit/_core/_typing.py
363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
Score
Bases: genkit._core._base.GenkitModel
Model for score data.
Source code in packages/genkit/src/genkit/_core/_typing.py
394 395 396 397 398 399 400 401 402 403 404 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
ScoreDetails
Bases: genkit._core._base.GenkitModel
Model for scoredetails data.
Source code in packages/genkit/src/genkit/_core/_typing.py
1098 1099 1100 1101 1102 1103 1104 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
ScoreStatus
Bases: genkit._core._compat.StrEnum
ScoreStatus data type class.
Source code in packages/genkit/src/genkit/_core/_typing.py
80 81 82 83 84 85 | |
genkit.telemetry
On-switch for Genkit traces, and the protocol for a recording backend.
Application code should call :class:genkit.Genkit. genkit start
records to the Developer UI. configure_instrumentation is the hook
for a recording backend; reset_instrumentation starts over between tests.
set_custom_metadata_attributes stamps the active span (tenant_id),
and flush_instrumentations drains exports before a serverless shutdown.
DisposableInstrumentation
Bases: typing.Protocol
Optional: a provider that holds a subscription or client.
reset_instrumentation calls dispose so an exporter from the
last run cannot keep posting after tests tear down.
Source code in packages/genkit/src/genkit/_core/_telemetry/_instrumentation.py
101 102 103 104 105 106 107 108 109 | |
FlushableInstrumentation
Bases: typing.Protocol
Optional: a provider that buffers exports.
flush_instrumentations calls flush so buffered telemetry is on the
wire before a serverless process freezes (or in tests).
Source code in packages/genkit/src/genkit/_core/_telemetry/_instrumentation.py
112 113 114 115 116 117 118 119 120 | |
Instrumentation
Bases: typing.Protocol
Pluggable provider. run_in_new_span wraps next like middleware.
Source code in packages/genkit/src/genkit/_core/_telemetry/_instrumentation.py
90 91 92 93 94 95 96 97 98 | |
SpanContext
Bases: typing.Protocol
Handle to a live span. No backend types leak through.
Source code in packages/genkit/src/genkit/_core/_telemetry/_instrumentation.py
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 | |
span_id
property
Span id, or empty when not instrumented.
trace_id
property
Trace id, or empty when not instrumented.
set_metadata(metadata)
Attach custom metadata. Safe to call multiple times.
Source code in packages/genkit/src/genkit/_core/_telemetry/_instrumentation.py
81 82 83 | |
set_state(state, *, message=None)
Override genkit:state. error marks a failed generate that returned.
Source code in packages/genkit/src/genkit/_core/_telemetry/_instrumentation.py
85 86 87 | |
SpanMetadata
dataclass
Description of a span about to be created.
action_type is the action kind ('model', 'flow', 'tool'...)
for action spans, or the span's own type ('util', 'flowStep'...)
for plain spans. Providers decide how to encode values.
Source code in packages/genkit/src/genkit/_core/_telemetry/_instrumentation.py
53 54 55 56 57 58 59 60 61 62 63 64 65 | |
SpanNext
Bases: typing.Protocol[genkit._core._telemetry._instrumentation.T_co]
next() or next(span). A logger that does not mint ids calls next().
Source code in packages/genkit/src/genkit/_core/_telemetry/_instrumentation.py
47 48 49 50 | |
configure_instrumentation(instrumentation)
Turn on a telemetry backend. Call before Genkit() to stack backends.
Each provider wraps the next. genkit start installs the Developer UI
HTTP poster when a collector URL is set. enable_google_cloud_telemetry()
hangs Cloud's exporter; it does not mint spans.
Source code in packages/genkit/src/genkit/_core/_telemetry/_instrumentation.py
189 190 191 192 193 194 195 196 197 198 199 200 201 202 | |
flush_instrumentations()
Wait for in-flight exports on every configured backend.
Source code in packages/genkit/src/genkit/_core/_telemetry/_instrumentation.py
215 216 217 218 219 | |
is_instrumented_by(kind)
True when a configured provider is an instance of kind.
Testers pass the class of the provider they configured.
Source code in packages/genkit/src/genkit/_core/_telemetry/_instrumentation.py
241 242 243 244 245 246 247 248 | |
reset_instrumentation()
Start telemetry over. Tests only.
Disposes and removes every configured provider, stops Dev UI log export, and forgets the open trace so the next span is a new root.
Source code in packages/genkit/src/genkit/_core/_telemetry/_instrumentation.py
222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 | |
run_in_new_span(name, fn, *, action_type=None, input=None, attributes=None, is_action=False)
async
Run fn inside a new span via the configured provider chain.
No providers → fn runs with a no-op span (empty ids). Index 0 is
outermost. The provider list is snapshotted so configure/reset during an
await cannot break the chain.
Source code in packages/genkit/src/genkit/_core/_telemetry/_instrumentation.py
265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 | |
set_custom_metadata_attributes(attributes)
Write metadata on the active span. No-op outside a span.
Source code in packages/genkit/src/genkit/_core/_telemetry/_instrumentation.py
251 252 253 254 255 | |
genkit.middleware
Middleware for Genkit model calls.
Define a subclass of Middleware and register it on your app
with @ai.middleware:
from genkit import Genkit
from genkit.middleware import Middleware, MiddlewareContext
ai = Genkit()
@ai.middleware(name='logging')
class LoggingMiddleware(Middleware):
async def wrap_generate(self, params, ctx: MiddlewareContext, next_fn):
print('before')
result = await next_fn(params, ctx)
print('after')
return result
response = await ai.generate(
model='your-model-here',
prompt='Hello',
use=[LoggingMiddleware()],
)
Once registered, the middleware is visible in the Dev UI. You can play with the Model Runner and mix-and-match your choice of middleware to see its impact on generating the next response.
Order of middleware in use=[...] determines the order in which they are
called. The first middleware in the list is called first, and the last
middleware in the list is called last.
For example, given this call:
use = [A(), B()]
The execution sequence is:
A_before()
B_before()
model_call()
B_after()
A_after()
GenerateHookParams
Bases: pydantic.BaseModel
Params passed to the wrap_generate hook.
Covers one full iteration of the tool loop: a model call plus optional tool
resolution. message_index tracks streaming position for this turn.
Source code in packages/genkit/src/genkit/_core/_middleware.py
204 205 206 207 208 209 210 211 212 213 214 215 | |
Middleware
Bases: typing.Generic[genkit._core._middleware.TConfig]
Base class for generate middleware.
A middleware is defined by its custom configuration (backed by Pydantic), and a set of hooks to inject logic into the generate pipeline.
The base middleware has no custom configuration and noop hooks. The hooks that are not overridden are still called by the engine when the middleware is invoked.
To author a middleware, 1. Declare a config model, e.g. RetryConfig:
class RetryConfig(BaseModel):
max_retries: int = 3
Any pydantic BaseModel works. Its keys (field names, or aliases if it
declares them) are the config keys in .prompt files, the Dev UI form
and traces. A key the class doesn't declare raises instead of being
dropped (a ValidationError, as for a GenkitConfig), unless the class sets
extra='allow' or 'ignore'.
-
Extend Middleware with your config model and give it a registry name: ai = Genkit()
@ai.middleware class Retry(Middleware[RetryConfig]): name = 'retry' description = 'Retries model calls with exponential backoff and jitter.'
async def wrap_model(self, params, ctx, next_fn): for attempt in range(self.config.max_retries + 1): ...
@ai.middleware (or ai.define_middleware(Retry)) registers the class
so the Dev UI lists it and a .prompt file can name it.
@ai.middleware(name=..., description=...) overrides the class attributes.
name and description are not inherited: a subclass of Retry
sets its own name before it can be registered.
-
Use the Retry middleware in your
generatecall: ai.generate( ..., use=[Retry(max_retries=5)] )Or alternatively, for full keyword auto-complete in your preferred IDE:
ai.generate( ..., use=[Retry(config=RetryConfig(max_retries=5))] )
self.config is set once at construction. The attribute is read-only;
treat the config object as read-only too (Pydantic models are not frozen).
Choose where to store state based on its lifetime:
* **Shared across all requests** (e.g., an audit client, rate limiter,
or compiled regex): set on ``self`` in ``__init__``.
* **Scoped to one ``generate()`` call** (e.g., retry counters or loop
budgets): use local variables inside ``wrap_generate``.
* **Scoped to one turn or tool call** (e.g., per-call timing or header
injection): use local variables inside ``wrap_model`` or ``wrap_tool``.
* **Shared between hooks in one ``generate()`` call** (e.g., recording
a tool call in ``wrap_tool`` and reading it in the next ``wrap_model``
turn): pass it through ``ctx.context``, or store a fresh container on
``self`` in ``__copy__``.
Per-request containers via __copy__:
Genkit calls copy.copy(mw) once at the start of each generate()
call. Because a shallow copy shares any list or dict created in
__init__ across concurrent requests, override __copy__ when you
need a fresh per-request container without re-running __init__::
def __copy__(self) -> Self:
clone = type(self).__new__(type(self))
clone.__dict__.update(self.__dict__)
clone._seen_tools = [] # isolated per generate() call
return clone
Source code in packages/genkit/src/genkit/_core/_middleware.py
398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 | |
config
property
The validated config this instance was constructed with (read-only).
tools(ctx)
Return additional tools to expose to the model for this generate call.
Source code in packages/genkit/src/genkit/_core/_middleware.py
552 553 554 | |
wrap_generate(params, ctx, next_fn)
async
Wrap each iteration of the tool loop (model call + optional tool resolution).
Source code in packages/genkit/src/genkit/_core/_middleware.py
556 557 558 559 560 561 562 563 | |
wrap_model(params, ctx, next_fn)
async
Wrap each model API call.
Source code in packages/genkit/src/genkit/_core/_middleware.py
565 566 567 568 569 570 571 572 | |
wrap_tool(params, ctx, next_fn)
async
Wrap each tool execution.
Return a MultipartToolResponse to forward (or substitute) the
tool's result. Raise Interrupt(metadata) to halt this tool call
and surface an interrupt to the caller.
Source code in packages/genkit/src/genkit/_core/_middleware.py
574 575 576 577 578 579 580 581 582 583 584 585 586 | |
MiddlewareContext
Per-generate() runtime services shared by every middleware in use=[...].
ai is the app's Genkit instance, so await ctx.ai.lookup_model(name),
await ctx.ai.lookup_tool(name) and ctx.ai.lookup_value(kind=..., name=...)
find what the app defined. Inside an agent, read the session with
genkit.exp.current_session(). Also carries a copy of the call's
context= (auth, secrets; the same dict flows and tools read as
ctx.context) and streaming hooks.
A callback stored on on_chunk that raises fails the turn; see
replace_on_chunk.
Source code in packages/genkit/src/genkit/_core/_middleware.py
292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 | |
is_streaming
property
True when the caller registered a streaming callback for this generate.
on_chunk
property
writable
Streaming callback for this generate call, guarded against callback raises.
replace_on_chunk(on_chunk)
Swap the streaming callback; returns the previous callback.
Pass the returned callback back in to restore it. Assigning
ctx.on_chunk directly behaves the same.
If the callback you install raises, generate treats it like a raising caller callback, not a model failure:
- the turn ends with
finish_reason == FAILEDanderror.reason == ErrorReason.STREAM_CALLBACK_FAILED; - the model is not called again: Retry and Fallback don't see it;
- the caller's
async foroverstream.streamends without raising, after the chunks sent before the failure.
Example
# 1. A middleware that redacts card numbers in streamed text
class Redact(Middleware):
async def wrap_model(self, params, ctx, next_fn):
previous = ctx.on_chunk
def redact(chunk):
previous(scrub_card_numbers(chunk)) # raises on a bug
ctx.replace_on_chunk(redact)
try:
return await next_fn(params, ctx)
finally:
ctx.replace_on_chunk(previous)
# 2. Stream through it; Retry outside doesn't re-call the model
stream = ai.generate_stream(
prompt='Read back my order.',
use=[Retry(max_retries=2), Redact()],
)
async for chunk in stream.stream:
print(chunk.text)
# => Your order:
# 3. The turn failed on the callback, not the provider
response = await stream.response
print(response.finish_reason, response.error.reason)
# => failed STREAM_CALLBACK_FAILED
Source code in packages/genkit/src/genkit/_core/_middleware.py
343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 | |
send_chunk(chunk)
Stream a chunk to the client when on_chunk is set.
Source code in packages/genkit/src/genkit/_core/_middleware.py
338 339 340 341 | |
ModelHookParams
Bases: pydantic.BaseModel
Params passed to the wrap_model hook (each raw model API call).
Source code in packages/genkit/src/genkit/_core/_middleware.py
218 219 220 221 222 223 | |
ToolHookParams
Bases: pydantic.BaseModel
Params passed to the wrap_tool hook (each individual tool execution).
Source code in packages/genkit/src/genkit/_core/_middleware.py
226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 | |
genkit.web
For serving flows over HTTP or writing a web framework adapter.
read_body turns the parsed JSON body into the flow input: callers send
{"data": ...}, or {} for no input. wants_stream says whether to
answer with server-sent events, and result_json encodes a one-shot return
value as compact {"result": ...} JSON (camelCase aliases, None model
fields omitted, nested models and datetimes serialized). When a flow fails,
send error_json (or the error_body dict) with error_status as the
HTTP status, or yield error_sse if a stream fails before to_sse starts.
A PublicError's text reaches the caller, and bad input names the failing
fields; anything else is a generic Internal Error so server details stay in
your logs. result_json and to_sse apply the same rule to a failed
agent turn's error (an AgentOutput or SessionSnapshot), which goes
out in a 200: it keeps its status, and a non-public message becomes generic.
Under GENKIT_ENV=dev both keep the error's own text, like Go, so a local
curl against a served flow or agent shows the real cause.
A ContextProvider reads the RequestData for one request and returns
what goes into the flow's context.
to_sse frames a streaming handle (flow.stream(),
ai.generate_stream(), prompt.stream(), chat.send_stream()) as the
SSE lines every adapter sends: a message frame per chunk, then result,
or an error frame instead. It's also how a hand-written route, say a
FastAPI one with typed params and Depends(...), streams without writing
frames itself. to_sse sends the handle's result as returned, so
ai.generate_stream(), prompt.stream(), and flows returning a
ModelResponse put request (system prompt, tools, docs, history) on the
wire; stream a flow returning response.text or response.output from a
served route.
Example
from starlette.requests import Request
from starlette.responses import Response, StreamingResponse
from genkit.web import (
RequestData,
error_json,
error_status,
read_body,
result_json,
to_sse,
wants_stream,
)
# 1. Read auth off the request; the result becomes ctx.context in the flow
async def auth(request_data: RequestData) -> dict[str, object]:
return {'token': request_data.headers.get('authorization')}
# 2. One route: parse, build context, then stream or run once
async def serve(request: Request) -> Response:
try:
body = await request.json()
flow_input = read_body(body)
request_data = RequestData(
request=request,
method=request.method,
headers=request.headers.items(),
input=body.get('data'),
)
ctx = await auth(request_data)
if wants_stream(accept=request.headers.get('accept'), stream=request.query_params.get('stream')):
stream = order_flow.stream(input=flow_input, context=ctx)
return StreamingResponse(to_sse(stream), media_type='text/event-stream')
response = await order_flow.run(input=flow_input, context=ctx)
return Response(result_json(response.response), media_type='application/json')
except Exception as e:
return Response(error_json(e), status_code=error_status(e), media_type='application/json')
# POST /order {"data": {"dish": "pad thai"}}
# => {"result":{"text":"1x pad thai"}}
# POST /order?stream=true {"data": {"dish": "pad thai"}}
# => data: {"message":"Plating pad thai"}
# data: {"result":{"text":"1x pad thai"}}
# POST /order {"input": "pad thai"}
# => 400 {"message":"Flow request must be wrapped in {\"data\": ...}","status":"INVALID_ARGUMENT"}
ContextProvider = Callable[[RequestData], dict[str, Any] | Awaitable[dict[str, Any]]]
module-attribute
Middleware can read request data and add information to the context that will be passed to the Action. If middleware throws an error, that error will fail the request and the Action will not be called.
Expected cases should raise a PublicError, which allows the request handler to know what data is safe to return to end users. Middleware can provide validation in addition to parsing. For example, an auth middleware can have policies for validating auth in addition to passing auth context to the Action.
RequestData
dataclass
What a context_provider sees for one HTTP request.
method, headers and input (the parsed {"data": ...}) are the
same on every adapter, so a provider that reads only those works in Flask,
Django and FastAPI. headers keys are lowercase so Authorization and
authorization look the same on every served flow. Repeated names become
one comma-joined string, so headers.get('authorization') sees every
value that arrived. Pass headers as a mapping or as the raw
(name, value) pairs off the wire.
request is the framework's own request object (Django HttpRequest,
Flask Request, Starlette Request) for what the shared fields don't
carry: cookies, sessions, the logged-in user. A provider that reads it works
on that framework only.
Example
from genkit import FlowContext, PublicError
from genkit.web import RequestData
from genkit_django import genkit_django_handler
# 1. Django session auth; auser() because the view is async (Django 5.0+,
# on 4.2 use sync_to_async(lambda: request.request.user)())
async def django_user(request: RequestData) -> dict[str, object]:
user = await request.request.auser()
if not user.is_authenticated:
raise PublicError('UNAUTHENTICATED', 'Sign in first')
return {'user_id': user.pk}
# 2. The flow reads it from ctx.context
@genkit_django_handler(ai, context_provider=django_user)
@ai.flow
async def reorder(order_id: str, ctx: FlowContext) -> str:
return f'reordered {order_id} for {ctx.context["user_id"]}'
# => reordered ORD-9921 for 42
Source code in packages/genkit/src/genkit/_core/_context.py
26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 | |
__init__(request=None, method='', headers=None, input=None)
Build request data; header names are lowercased and repeats comma-joined.
Source code in packages/genkit/src/genkit/_core/_context.py
73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 | |
StreamHandle
Bases: typing.Protocol
Anything with chunks on .stream and a result on .response.
flow.stream(), ai.generate_stream(), prompt.stream() and
chat.send_stream() all return one.
Source code in packages/genkit/src/genkit/web/_sse.py
46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 | |
response
property
The final result, once the stream ends.
stream
property
Chunks, in order.
error_body(error)
JSON body to send back when a served flow fails.
Only a PublicError's message, details, and status go on the wire; it's
the one error whose author said the text is safe for callers. A bad
request to the flow itself keeps its 4xx status and names the failing
fields without their values, such as "Invalid input for flow 'save':
qty: Input should be a valid integer". Any other GenkitError, a provider
error, and anything else become {"message": "Internal Error", "status": "INTERNAL"}.
Under GENKIT_ENV=dev that body carries the error's own text instead,
so a local curl shows the real cause; the status stays INTERNAL.
Mirrors Go's wire.ClientMessage.
Example
from genkit.web import error_body, error_status
try:
response = await my_flow.run(input=data)
except Exception as e:
return JSONResponse(error_body(e), status_code=error_status(e))
Source code in packages/genkit/src/genkit/_core/_error.py
991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 | |
error_json(error)
Encode error_body(error) as compact JSON; pair with error_status(error).
Source code in packages/genkit/src/genkit/web/_sse.py
98 99 100 | |
error_sse(error)
Encode error_body(error) as one data: {"error": ...}\n\n SSE frame.
Source code in packages/genkit/src/genkit/web/_sse.py
103 104 105 106 | |
error_status(error)
HTTP status to send back when a served flow fails; pairs with error_body.
A PublicError keeps its own status (NOT_FOUND is a 404), and a bad request to the flow itself keeps its 4xx. Any other GenkitError, a provider error, a plain exception, or anything else is a 500.
Source code in packages/genkit/src/genkit/_core/_error.py
940 941 942 943 944 945 946 947 948 949 950 | |
read_body(body)
Return the flow input from a parsed JSON request body.
{"data": x} gives x. {} and {"data": null} mean the caller
sent no input, so a flow with a default uses it. Any other body is a 400
so the caller learns to wrap their input instead of the flow seeing a
half-understood value.
Raises:
| Type | Description |
|---|---|
genkit._core._error.PublicError
|
INVALID_ARGUMENT when the body isn't |
Source code in packages/genkit/src/genkit/_core/_web.py
25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 | |
result_json(result)
Encode a one-shot result as compact {"result": ...} JSON, the bytes every adapter sends.
A failed agent turn's error keeps its status but only a PublicError's
message; anything else goes out as a generic message for that status.
Source code in packages/genkit/src/genkit/web/_sse.py
88 89 90 91 92 93 94 95 | |
to_sse(handle)
async
Frame a streaming handle as Genkit SSE lines.
Yields data: {"message": ...} for each chunk, then data: {"result": ...}.
If the run fails, it yields one data: {"error": ...} instead of the rest
and logs the failure with Python's logging module (5xx with a traceback,
4xx as a warning). Only a PublicError's text reaches the client; anything else goes out as
Internal Error. A failed agent turn's error in the result frame
follows the same rule (see result_json). These are the frames serve_flow,
handle_genkit_request and serve_agent send.
The result goes out as the run returned it. ai.generate_stream(),
prompt.stream(), and flows returning a whole ModelResponse send its
request too (system prompt, tools, docs, history), so stream a flow
returning response.text or response.output from a served route.
.response is read only after the stream drains, so a handle that fails
or is never iterated doesn't leave an un-awaited result behind.
Example
from fastapi import Depends
from fastapi.responses import StreamingResponse
from genkit.web import to_sse
@app.post('/deploy/stream')
async def deploy(service: str, user: User = Depends(current_user)):
# 1. FastAPI owns the route: typed params, Depends, OpenAPI
stream = deploy_service.stream(service, context={'user_id': user.id})
# 2. Genkit owns the wire
return StreamingResponse(to_sse(stream), media_type='text/event-stream')
# => data: {"message":"[u_42] building payments"}
# data: {"result":"https://payments.run.app"}
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
handle
|
genkit.web._sse.StreamHandle
|
The streaming handle to frame. |
required |
Yields:
| Type | Description |
|---|---|
collections.abc.AsyncGenerator[str, None]
|
One SSE |
Source code in packages/genkit/src/genkit/web/_sse.py
109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 | |
wants_stream(*, accept, stream)
Whether the caller asked for server-sent events.
Pass the Accept header and the ?stream= query value. Substring
match so text/event-stream, */* still streams.
Source code in packages/genkit/src/genkit/_core/_web.py
45 46 47 48 49 50 51 | |
genkit.exp
Experimental APIs. Agents stay here even after Genkit Python is GA.
from genkit.exp import Genkit
from genkit.exp.agent import InMemorySessionStore
from genkit_google_genai import GoogleAI
ai = Genkit(plugins=[GoogleAI()])
agent = ai.define_agent(
name='weatherAgent',
model=GoogleAI.gemini('gemini-flash-latest'),
system='Weather assistant.',
store=InMemorySessionStore(),
)
Genkit
Genkit plus the agent APIs, which are still experimental.
Importing this class is how you opt into agents. The stable
from genkit import Genkit does not grow these methods.
Source code in packages/genkit/src/genkit/exp/_api.py
49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 | |
__init__(*args, plugins=None, model=None, prompt_dir=None)
Initialize a Genkit instance.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
plugins
|
list[genkit._core._plugin.Plugin] | None
|
Plugins to register (e.g. |
None
|
model
|
genkit._ai._model.ModelArg | None
|
Default model for |
None
|
prompt_dir
|
str | pathlib.Path | None
|
Optional directory for loading |
None
|
Source code in packages/genkit/src/genkit/_ai/_aio.py
260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 | |
agent(name)
async
Look up a registered agent by name.
Source code in packages/genkit/src/genkit/exp/_api.py
56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 | |
cancel_operation(operation, *, context=None, config=None)
async
cancel_operation(
operation: Operation[T],
*,
context: dict[str, Any] | None = None,
config: Mapping[str, Any] | None = None
) -> Operation[T]
cancel_operation(
operation: Mapping[str, Any],
*,
context: dict[str, Any] | None = None,
config: Mapping[str, Any] | None = None
) -> Operation[Any]
Cancel a background job.
Same input and return types as check_operation: Operation[T]
in, Operation[T] out; a stored dict in, Operation[Any] out.
Same context / config pockets.
Example
# 1. Reload the stored handle and cancel it
op = await ai.cancel_operation(json.loads(saved))
# 2. The returned handle reflects the cancel attempt
print(op.id)
# => operations/risotto-123
Source code in packages/genkit/src/genkit/_ai/_aio.py
2721 2722 2723 2724 2725 2726 2727 2728 2729 2730 2731 2732 2733 2734 2735 2736 2737 2738 2739 2740 2741 2742 2743 2744 | |
check_operation(operation, *, context=None, config=None)
async
check_operation(
operation: Operation[T],
*,
context: dict[str, Any] | None = None,
config: Mapping[str, Any] | None = None
) -> Operation[T]
check_operation(
operation: Mapping[str, Any],
*,
context: dict[str, Any] | None = None,
config: Mapping[str, Any] | None = None
) -> Operation[Any]
Poll a background job.
Generic like JS checkOperation<T>: an Operation[T] comes back
as Operation[T], a stored dict as Operation[Any]. The handle's
action key finds the job. For a background model the handle is
validated as Operation[ModelResponse] before anything runs: an
unknown key, a missing id, or an output that isn't a
ModelResponse is INVALID_ARGUMENT.
Pass context={'secrets': {'api_key': ...}} again when start used a
per-request key. config is client knobs (base_url,
location, api_version), not video settings.
Example
# 1. Start a job and store the handle as JSON
op = await ai.generate_operation(
model='googleai/veo-3.1-generate-preview',
prompt='A chef plating risotto.',
)
saved = op.model_dump_json()
# 2. Later, reload it typed and poll
op = await ai.check_operation(Operation[ModelResponse].model_validate_json(saved))
print(op.done)
# => False
# 3. Poll until done; output is a ModelResponse
while not op.done:
op = await ai.check_operation(op)
if op.output and op.output.media:
print(op.output.media.url)
# => https://storage.googleapis.com/.../risotto.mp4
Source code in packages/genkit/src/genkit/_ai/_aio.py
2659 2660 2661 2662 2663 2664 2665 2666 2667 2668 2669 2670 2671 2672 2673 2674 2675 2676 2677 2678 2679 2680 2681 2682 2683 2684 2685 2686 2687 2688 2689 2690 2691 2692 2693 2694 2695 2696 2697 2698 2699 2700 2701 | |
current_context()
staticmethod
Get the current execution context, or None if not in an action.
Source code in packages/genkit/src/genkit/_ai/_aio.py
2596 2597 2598 2599 | |
define_agent(name, *, model=None, system=None, tools=None, use=None, config=None, max_turns=None, description=None, metadata=None, store=None, state_transform=None, chunk_transform=None, state_schema=None)
define_agent(
name: str,
*,
model: ModelRef[Any] | Action | str | None = None,
system: str | list[Part] | None = None,
tools: Sequence[str | Tool] | None = None,
use: Sequence[Middleware] | None = None,
config: None = None,
max_turns: int | None = None,
description: str | None = None,
metadata: dict[str, object] | None = None,
store: SessionStore[StateT] | None = None,
state_transform: StateTransform | None = None,
chunk_transform: ChunkTransform | None = None,
state_schema: type[StateT] | None = None
) -> Agent[StateT]
define_agent(
name: str,
*,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
system: str | list[Part] | None = None,
tools: Sequence[str | Tool] | None = None,
use: Sequence[Middleware] | None = None,
config: ModelConfigDict,
max_turns: int | None = None,
description: str | None = None,
metadata: dict[str, object] | None = None,
store: SessionStore[StateT] | None = None,
state_transform: StateTransform | None = None,
chunk_transform: ChunkTransform | None = None,
state_schema: type[StateT] | None = None
) -> Agent[StateT]
define_agent(
name: str,
*,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
system: str | list[Part] | None = None,
tools: Sequence[str | Tool] | None = None,
use: Sequence[Middleware] | None = None,
config: ModelRefConfigT | Mapping[str, Any] | None,
max_turns: int | None = None,
description: str | None = None,
metadata: dict[str, object] | None = None,
store: SessionStore[StateT] | None = None,
state_transform: StateTransform | None = None,
chunk_transform: ChunkTransform | None = None,
state_schema: type[StateT] | None = None
) -> Agent[StateT]
Define a prompt-backed agent.
Each turn: attaches session history, calls generate with streaming, updates session. Pass resume in AgentInput to resume from an interrupt.
Pass state_schema (a Pydantic model) to type the custom state tools
read and write — the chat's state, response.state, and streamed
chunk.custom come back as that model instead of a dict.
Example
from genkit.exp import Genkit from genkit.exp.agent import InMemorySessionStore from genkit_google_genai import GoogleAI
ai = Genkit(plugins=[GoogleAI()]) agent = ai.define_agent( name='weatherAgent', model=GoogleAI.gemini('gemini-flash-latest'), system='Weather assistant.', tools=[current_weather], store=InMemorySessionStore(), ) chat = agent.chat() res = await chat.send('Weather in Paris?')
Source code in packages/genkit/src/genkit/exp/_api.py
163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 | |
define_background_model(name, *, start, check, cancel=None, info=None, config_schema=None, metadata=None, description=None)
Register a background model for long-running AI operations.
Source code in packages/genkit/src/genkit/_ai/_aio.py
834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 | |
define_batch_evaluator(name, fn, *, display_name, definition, is_billed=False, config_schema=None, metadata=None, description=None)
Register a batch evaluator.
The function takes one EvaluatorRequest and returns list[EvaluatorResponse].
Read options from req.options. A second parameter raises TypeError when defined.
Source code in packages/genkit/src/genkit/_ai/_aio.py
765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 | |
define_custom_agent(name, fn, *, store=None, state_transform=None, chunk_transform=None, state_schema=None, description=None, metadata=None)
Define and register an agent with full control over the turn loop.
fn receives (SessionRunner, AgentContext) and must call sess.run(handle_turn) to process inputs, then return an AgentResult.
Pass state_schema (a Pydantic model) to type the custom state, so the
chat's state, response.state, and streamed chunk.custom come
back as that model instead of a dict.
Source code in packages/genkit/src/genkit/exp/_api.py
72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 | |
define_dynamic_action_provider(name, fn, *, description=None, cache_ttl_millis=None, metadata=None)
Register a Dynamic Action Provider (DAP).
Source code in packages/genkit/src/genkit/_ai/_aio.py
504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 | |
define_embedder(name, fn, *, info=None, metadata=None, description=None, config_schema=None)
define_embedder(
name: str,
fn: EmbedderFn,
*,
info: EmbedderInfo | None = None,
metadata: dict[str, object] | None = None,
description: str | None = None,
config_schema: (
type[BaseModel] | dict[str, object] | None
) = None
) -> Embedder
define_embedder(
name: str,
fn: CtxFirstEmbedderFn,
*,
info: EmbedderInfo | None = None,
metadata: dict[str, object] | None = None,
description: str | None = None,
config_schema: (
type[BaseModel] | dict[str, object] | None
) = None
) -> Embedder
Register a custom embedder action.
The options class comes from config_schema or the fn's
EmbedRequest[Cfg] annotation; given both, they must be the same class.
Source code in packages/genkit/src/genkit/_ai/_aio.py
883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 | |
define_evaluator(name, fn, *, display_name, definition, is_billed=False, config_schema=None, metadata=None, description=None)
Register an evaluator that scores one dataset row at a time.
Example
# 1. Score whether the answer mentions the expected dish
async def mentions_dish(row: BaseDataPoint, options: object | None) -> EvaluatorResponse:
hit = row.reference.lower() in str(row.output).lower()
return EvaluatorResponse(test_case_id=row.test_case_id or '', evaluation=[Score(score=hit)])
# 2. Register it under a name
ai.define_evaluator(
'mentions_dish',
mentions_dish,
display_name='Mentions dish',
definition='Whether the answer names the expected dish.',
)
# 3. Run it over a dataset
rows = await ai.evaluate(evaluator='mentions_dish', dataset=dataset)
# => [EvaluatorResponse(test_case_id='...', evaluation=[Score(score=True)]), ...]
Source code in packages/genkit/src/genkit/_ai/_aio.py
718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 | |
define_format(formatter)
Register a custom output format. See genkit.formats.Formatter.
Source code in packages/genkit/src/genkit/_ai/_aio.py
900 901 902 | |
define_helper(name, fn)
Register a Handlebars helper for prompt templates.
Dotprompt calls the helper as fn(params, options): params is a
list of the positional arguments from the template, and options
carries the named arguments in options.hash. A one-argument
function such as lambda s: s.upper() raises TypeError at
render time. Return a str.
Example
# 1. Register a helper that takes one positional and one named argument
def menu_item(params: list[Any], options: Any) -> str:
dish = str(params[0])
return dish.upper() if options.hash.get('loud') else dish
ai.define_helper('menu_item', menu_item)
# 2. Use it in a prompt template
special = ai.define_prompt(
name='special',
model='googleai/gemini-flash-latest',
prompt="Describe today's special, {{menu_item dish loud=true}}, in one line.",
)
# 3. Render to check the expanded text
request = await special.render(input={'dish': 'smoked salmon tartine'})
print(request.messages[0].content[0].text)
# => Describe today's special, SMOKED SALMON TARTINE, in one line.
Source code in packages/genkit/src/genkit/_ai/_aio.py
456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 | |
define_interrupt(name, *, input_schema=None, description=None)
Register an interrupt tool that always pauses for user input.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str
|
Tool name |
required |
input_schema
|
type[pydantic.BaseModel] | dict[str, object] | None
|
Optional input schema (Pydantic model or JSON schema dict) |
None
|
description
|
str | None
|
Tool description |
None
|
Returns:
| Type | Description |
|---|---|
genkit._core._tool.Tool[typing.Any, typing.NoReturn]
|
The registered interrupt tool. Awaiting it directly raises |
Example
ask_user = ai.define_interrupt( name='ask_user', input_schema=Question, description='Ask the user a question', )
Source code in packages/genkit/src/genkit/_ai/_aio.py
687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 | |
define_json_schema(name, json_schema)
Register a JSON schema for use in prompts.
Source code in packages/genkit/src/genkit/_ai/_aio.py
499 500 501 502 | |
define_middleware(cls, *, name=None, description=None)
Register a middleware class on this app's registry.
Source code in packages/genkit/src/genkit/_ai/_aio.py
645 646 647 648 649 650 651 652 653 654 655 | |
define_model(name, fn, *, config_schema=None, metadata=None, info=None, description=None)
define_model(
name: str,
fn: ModelFn,
*,
config_schema: (
type[BaseModel] | dict[str, object] | None
) = None,
metadata: dict[str, object] | None = None,
info: ModelInfo | None = None,
description: str | None = None
) -> Model
define_model(
name: str,
fn: CtxFirstModelFn,
*,
config_schema: (
type[BaseModel] | dict[str, object] | None
) = None,
metadata: dict[str, object] | None = None,
info: ModelInfo | None = None,
description: str | None = None
) -> Model
Register a custom model action.
Source code in packages/genkit/src/genkit/_ai/_aio.py
821 822 823 824 825 826 827 828 829 830 831 832 | |
define_partial(name, source)
Register a Handlebars partial template.
Source code in packages/genkit/src/genkit/_ai/_aio.py
490 491 492 | |
define_prompt(name=None, *, variant=None, model=None, config=None, description=None, system=None, prompt=None, messages=None, output_format=None, output_content_type=None, output_instructions=None, output_constrained=None, max_turns=None, return_tool_requests=None, metadata=None, tools=None, tool_choice=None, use=None, docs=None, input_schema=None, output_schema=None)
define_prompt(
name: str | None = None,
*,
variant: str | None = None,
model: ModelRef[Any] | Action | str | None = None,
config: None = None,
description: str | None = None,
system: str | list[Part] | None = None,
prompt: str | list[Part] | None = None,
messages: str | list[Message] | None = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
max_turns: int | None = None,
return_tool_requests: bool | None = None,
metadata: dict[str, object] | None = None,
tools: Sequence[str | Tool] | None = None,
tool_choice: ToolChoice | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None,
input_schema: type[InputT],
output_schema: type[OutputT]
) -> Prompt[InputT, OutputT]
define_prompt(
name: str | None = None,
*,
variant: str | None = None,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
config: ModelConfigDict,
description: str | None = None,
system: str | list[Part] | None = None,
prompt: str | list[Part] | None = None,
messages: str | list[Message] | None = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
max_turns: int | None = None,
return_tool_requests: bool | None = None,
metadata: dict[str, object] | None = None,
tools: Sequence[str | Tool] | None = None,
tool_choice: ToolChoice | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None,
input_schema: type[InputT],
output_schema: type[OutputT]
) -> Prompt[InputT, OutputT]
define_prompt(
name: str | None = None,
*,
variant: str | None = None,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
config: ModelRefConfigT | Mapping[str, Any] | None,
description: str | None = None,
system: str | list[Part] | None = None,
prompt: str | list[Part] | None = None,
messages: str | list[Message] | None = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
max_turns: int | None = None,
return_tool_requests: bool | None = None,
metadata: dict[str, object] | None = None,
tools: Sequence[str | Tool] | None = None,
tool_choice: ToolChoice | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None,
input_schema: type[InputT],
output_schema: type[OutputT]
) -> Prompt[InputT, OutputT]
define_prompt(
name: str | None = None,
*,
variant: str | None = None,
model: ModelRef[Any] | Action | str | None = None,
config: None = None,
description: str | None = None,
system: str | list[Part] | None = None,
prompt: str | list[Part] | None = None,
messages: str | list[Message] | None = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
max_turns: int | None = None,
return_tool_requests: bool | None = None,
metadata: dict[str, object] | None = None,
tools: Sequence[str | Tool] | None = None,
tool_choice: ToolChoice | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None,
input_schema: type[InputT],
output_schema: dict[str, object] | str | None = None
) -> Prompt[InputT, Any]
define_prompt(
name: str | None = None,
*,
variant: str | None = None,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
config: ModelConfigDict,
description: str | None = None,
system: str | list[Part] | None = None,
prompt: str | list[Part] | None = None,
messages: str | list[Message] | None = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
max_turns: int | None = None,
return_tool_requests: bool | None = None,
metadata: dict[str, object] | None = None,
tools: Sequence[str | Tool] | None = None,
tool_choice: ToolChoice | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None,
input_schema: type[InputT],
output_schema: dict[str, object] | str | None = None
) -> Prompt[InputT, Any]
define_prompt(
name: str | None = None,
*,
variant: str | None = None,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
config: ModelRefConfigT | Mapping[str, Any] | None,
description: str | None = None,
system: str | list[Part] | None = None,
prompt: str | list[Part] | None = None,
messages: str | list[Message] | None = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
max_turns: int | None = None,
return_tool_requests: bool | None = None,
metadata: dict[str, object] | None = None,
tools: Sequence[str | Tool] | None = None,
tool_choice: ToolChoice | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None,
input_schema: type[InputT],
output_schema: dict[str, object] | str | None = None
) -> Prompt[InputT, Any]
define_prompt(
name: str | None = None,
*,
variant: str | None = None,
model: ModelRef[Any] | Action | str | None = None,
config: None = None,
description: str | None = None,
system: str | list[Part] | None = None,
prompt: str | list[Part] | None = None,
messages: str | list[Message] | None = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
max_turns: int | None = None,
return_tool_requests: bool | None = None,
metadata: dict[str, object] | None = None,
tools: Sequence[str | Tool] | None = None,
tool_choice: ToolChoice | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None,
input_schema: dict[str, object] | str | None = None,
output_schema: type[OutputT]
) -> Prompt[Any, OutputT]
define_prompt(
name: str | None = None,
*,
variant: str | None = None,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
config: ModelConfigDict,
description: str | None = None,
system: str | list[Part] | None = None,
prompt: str | list[Part] | None = None,
messages: str | list[Message] | None = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
max_turns: int | None = None,
return_tool_requests: bool | None = None,
metadata: dict[str, object] | None = None,
tools: Sequence[str | Tool] | None = None,
tool_choice: ToolChoice | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None,
input_schema: dict[str, object] | str | None = None,
output_schema: type[OutputT]
) -> Prompt[Any, OutputT]
define_prompt(
name: str | None = None,
*,
variant: str | None = None,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
config: ModelRefConfigT | Mapping[str, Any] | None,
description: str | None = None,
system: str | list[Part] | None = None,
prompt: str | list[Part] | None = None,
messages: str | list[Message] | None = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
max_turns: int | None = None,
return_tool_requests: bool | None = None,
metadata: dict[str, object] | None = None,
tools: Sequence[str | Tool] | None = None,
tool_choice: ToolChoice | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None,
input_schema: dict[str, object] | str | None = None,
output_schema: type[OutputT]
) -> Prompt[Any, OutputT]
define_prompt(
name: str | None = None,
*,
variant: str | None = None,
model: ModelRef[Any] | Action | str | None = None,
config: None = None,
description: str | None = None,
system: str | list[Part] | None = None,
prompt: str | list[Part] | None = None,
messages: str | list[Message] | None = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
max_turns: int | None = None,
return_tool_requests: bool | None = None,
metadata: dict[str, object] | None = None,
tools: Sequence[str | Tool] | None = None,
tool_choice: ToolChoice | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None,
input_schema: (
type | dict[str, object] | str | None
) = None,
output_schema: (
type | dict[str, object] | str | None
) = None
) -> Prompt[Any, Any]
define_prompt(
name: str | None = None,
*,
variant: str | None = None,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
config: ModelConfigDict,
description: str | None = None,
system: str | list[Part] | None = None,
prompt: str | list[Part] | None = None,
messages: str | list[Message] | None = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
max_turns: int | None = None,
return_tool_requests: bool | None = None,
metadata: dict[str, object] | None = None,
tools: Sequence[str | Tool] | None = None,
tool_choice: ToolChoice | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None,
input_schema: (
type | dict[str, object] | str | None
) = None,
output_schema: (
type | dict[str, object] | str | None
) = None
) -> Prompt[Any, Any]
define_prompt(
name: str | None = None,
*,
variant: str | None = None,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
config: ModelRefConfigT | Mapping[str, Any] | None,
description: str | None = None,
system: str | list[Part] | None = None,
prompt: str | list[Part] | None = None,
messages: str | list[Message] | None = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
max_turns: int | None = None,
return_tool_requests: bool | None = None,
metadata: dict[str, object] | None = None,
tools: Sequence[str | Tool] | None = None,
tool_choice: ToolChoice | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None,
input_schema: (
type | dict[str, object] | str | None
) = None,
output_schema: (
type | dict[str, object] | str | None
) = None
) -> Prompt[Any, Any]
Register a prompt template.
Example
joke = ai.define_prompt(name='joke', prompt='Tell a joke about {{topic}}.') res = await joke(input={'topic': 'cats'}) print(res.text)
=> Why did the cat sit on the keyboard? To keep an eye on the mouse.
Source code in packages/genkit/src/genkit/_ai/_aio.py
1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 | |
define_prompt_agent(name, *, store=None, state_transform=None, chunk_transform=None, state_schema=None, description=None, metadata=None)
Wire an already-registered prompt as an agent.
Looks up the prompt named name from the registry. Use when the prompt
is defined via ai.define_prompt() or loaded from a .prompt file.
Source code in packages/genkit/src/genkit/exp/_api.py
222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 | |
define_schema(name, schema)
Register a Pydantic schema for use in prompts.
Source code in packages/genkit/src/genkit/_ai/_aio.py
494 495 496 497 | |
define_value(*, kind, name, value)
Store value under kind and name for later lookup_value.
Defining the same kind and name again replaces the value, so a re-run notebook cell does not fail.
Raises:
| Type | Description |
|---|---|
ValueError
|
|
Source code in packages/genkit/src/genkit/_ai/_aio.py
946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 | |
embed(*, embedder, content, metadata=None, config=None, context=None)
async
embed(
*,
embedder: str | EmbedderRef[Any],
content: str | Document,
metadata: dict[str, object] | None = None,
config: None = None,
context: dict[str, Any] | None = None
) -> list[Embedding]
embed(
*,
embedder: str | EmbedderRef[EmbedderRefConfigT],
content: str | Document,
metadata: dict[str, object] | None = None,
config: EmbedderRefConfigT | Mapping[str, Any] | None,
context: dict[str, Any] | None = None
) -> list[Embedding]
Generate vector embeddings for a single document or string.
config is layered over the EmbedderRef's version and config
(the call wins per key) and reaches the embedder as
request.options. A config object must be the embedder's options
class. An embedder name that isn't registered raises GenkitError
with NOT_FOUND.
metadata is attached to a string content. A Document
already carries its own metadata, so passing both raises TypeError.
context reaches the embedder as ctx.context, as on
generate. Omitted, the embedder sees the enclosing action's
context. The GoogleAI, VertexAI and OpenAI embedders read a
per-request key from context={'secrets': {'api_key': ...}}, so
the tenant, not the plugin key, is billed. Ollama and Bedrock
embedders always use the plugin's client.
Example
from genkit_google_genai import GoogleAI
embeddings = await ai.embed( embedder=GoogleAI.embedder('gemini-embedding-001'), content='Hello world', ) vector = embeddings[0].embedding
Source code in packages/genkit/src/genkit/_ai/_aio.py
2419 2420 2421 2422 2423 2424 2425 2426 2427 2428 2429 2430 2431 2432 2433 2434 2435 2436 2437 2438 2439 2440 2441 2442 2443 2444 2445 2446 2447 2448 2449 2450 2451 2452 2453 2454 2455 2456 2457 2458 2459 2460 2461 2462 2463 2464 2465 2466 2467 2468 2469 2470 2471 | |
embed_many(*, embedder, content, metadata=None, config=None, context=None)
async
embed_many(
*,
embedder: str | EmbedderRef[Any],
content: Sequence[str | Document],
metadata: dict[str, object] | None = None,
config: None = None,
context: dict[str, Any] | None = None
) -> list[Embedding]
embed_many(
*,
embedder: str | EmbedderRef[EmbedderRefConfigT],
content: Sequence[str | Document],
metadata: dict[str, object] | None = None,
config: EmbedderRefConfigT | Mapping[str, Any] | None,
context: dict[str, Any] | None = None
) -> list[Embedding]
Generate vector embeddings for multiple documents in a single batch call.
content is any sequence (list, tuple, ...) of strings or
Documents; one embedding comes back per item, in order. A bare
str raises TypeError; use embed for one item.
config, metadata and context work the same as on embed.
Source code in packages/genkit/src/genkit/_ai/_aio.py
2497 2498 2499 2500 2501 2502 2503 2504 2505 2506 2507 2508 2509 2510 2511 2512 2513 2514 2515 2516 2517 2518 2519 2520 2521 2522 2523 2524 2525 2526 2527 2528 | |
evaluate(*, evaluator, dataset, config=None, eval_run_id=None)
async
Evaluate a dataset using the specified evaluator.
Returns a list of rows. A per-row evaluator gives one row per
datapoint, in dataset order. A batch evaluator gives the rows its
function returned, as returned. Each row's evaluation is a list of
scores.
config is merged over the EvaluatorRef's config (the call
wins per key) and handed to the evaluator as its second argument. When
neither sets anything, the evaluator gets None. An evaluator name
that isn't registered raises GenkitError with NOT_FOUND.
Example
from genkit.evaluator import BaseDataPoint
results = await ai.evaluate( evaluator='my_eval', dataset=[BaseDataPoint(input='What is 2+2?', output='4')], ) for row in results: for score in row.evaluation: print(row.test_case_id, score.score)
Source code in packages/genkit/src/genkit/_ai/_aio.py
2530 2531 2532 2533 2534 2535 2536 2537 2538 2539 2540 2541 2542 2543 2544 2545 2546 2547 2548 2549 2550 2551 2552 2553 2554 2555 2556 2557 2558 2559 2560 2561 2562 2563 2564 2565 2566 2567 2568 2569 2570 2571 2572 2573 2574 2575 2576 2577 2578 2579 2580 2581 2582 2583 2584 2585 2586 2587 2588 2589 2590 2591 2592 2593 2594 | |
flow(name=None, *, description=None)
flow(
name: str | None = None,
*,
description: str | None = None
) -> _FlowDecorator
flow(
func: Callable[
[FlowContext[ChunkT], object], Awaitable[OutputT]
],
) -> Flow[Any, OutputT, ChunkT]
flow(
func: DefaultInputCtxFn[
FlowContext[ChunkT], InputT, OutputT
],
) -> DefaultedFlow[InputT, OutputT, ChunkT]
flow(
func: Callable[
[InputT, FlowContext[ChunkT]], Awaitable[OutputT]
],
) -> Flow[InputT, OutputT, ChunkT]
flow(
func: CtxDefaultInputFn[
FlowContext[ChunkT], InputT, OutputT
],
) -> DefaultedFlow[InputT, OutputT, ChunkT]
flow(
func: Callable[
[FlowContext[ChunkT], InputT], Awaitable[OutputT]
],
) -> Flow[InputT, OutputT, ChunkT]
flow(
func: Callable[[object], Awaitable[OutputT]],
) -> Flow[Any, OutputT]
flow(
func: Callable[
[FlowContext[ChunkT]], Awaitable[OutputT]
],
) -> Flow[None, OutputT, ChunkT]
flow(
func: DefaultInputFn[InputT, OutputT],
) -> DefaultedFlow[InputT, OutputT]
flow(
func: Callable[[InputT], Awaitable[OutputT]],
) -> Flow[InputT, OutputT]
flow(
func: Callable[[], Awaitable[OutputT]],
) -> Flow[None, OutputT]
Decorator to register an async function as a flow.
Can be used bare (@ai.flow) or with arguments (@ai.flow(),
@ai.flow('custom_name', description='...')).
A flow takes at most one input. To read the request context or stream
chunks, add a parameter annotated FlowContext, in any position.
Any other second parameter raises TypeError when the flow is defined.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str | collections.abc.Callable[..., collections.abc.Awaitable[typing.Any]] | None
|
Optional name for the flow (or the function itself when used
as |
None
|
description
|
str | None
|
Optional description for the flow. |
None
|
Example
from genkit import FlowContext, Genkit from genkit_google_genai import GoogleAI
ai = Genkit(plugins=[GoogleAI()], model=GoogleAI.gemini('gemini-flash-latest'))
@ai.flow async def my_flow(prompt: str) -> str: res = await ai.generate(prompt=prompt) return res.text
@ai.flow async def streaming_flow(x: int, ctx: FlowContext[str]) -> str: ctx.send_chunk('progress') return 'done'
Source code in packages/genkit/src/genkit/_ai/_aio.py
416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 | |
generate(*, model=None, prompt=None, system=None, messages=None, tools=None, return_tool_requests=None, tool_choice=None, resume_respond=None, resume_restart=None, resume_metadata=None, config=None, max_turns=None, context=None, output_schema=None, output_format=None, output_content_type=None, output_instructions=None, output_constrained=None, use=None, docs=None)
async
generate(
*,
model: ModelRef[Any] | Action | str | None = None,
prompt: str | list[Part] | None = None,
system: str | list[Part] | None = None,
messages: list[Message] | None = None,
tools: Sequence[str | Tool] | None = None,
return_tool_requests: bool | None = None,
tool_choice: ToolChoice | None = None,
resume_respond: Part | list[Part] | None = None,
resume_restart: Part | list[Part] | None = None,
resume_metadata: dict[str, Any] | None = None,
config: None = None,
max_turns: int | None = None,
context: dict[str, object] | None = None,
output_schema: type[OutputT],
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None
) -> ModelResponse[OutputT]
generate(
*,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
prompt: str | list[Part] | None = None,
system: str | list[Part] | None = None,
messages: list[Message] | None = None,
tools: Sequence[str | Tool] | None = None,
return_tool_requests: bool | None = None,
tool_choice: ToolChoice | None = None,
resume_respond: Part | list[Part] | None = None,
resume_restart: Part | list[Part] | None = None,
resume_metadata: dict[str, Any] | None = None,
config: ModelConfigDict,
max_turns: int | None = None,
context: dict[str, object] | None = None,
output_schema: type[OutputT],
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None
) -> ModelResponse[OutputT]
generate(
*,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
prompt: str | list[Part] | None = None,
system: str | list[Part] | None = None,
messages: list[Message] | None = None,
tools: Sequence[str | Tool] | None = None,
return_tool_requests: bool | None = None,
tool_choice: ToolChoice | None = None,
resume_respond: Part | list[Part] | None = None,
resume_restart: Part | list[Part] | None = None,
resume_metadata: dict[str, Any] | None = None,
config: ModelRefConfigT | Mapping[str, Any] | None,
max_turns: int | None = None,
context: dict[str, object] | None = None,
output_schema: type[OutputT],
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None
) -> ModelResponse[OutputT]
generate(
*,
model: ModelRef[Any] | Action | str | None = None,
prompt: str | list[Part] | None = None,
system: str | list[Part] | None = None,
messages: list[Message] | None = None,
tools: Sequence[str | Tool] | None = None,
return_tool_requests: bool | None = None,
tool_choice: ToolChoice | None = None,
resume_respond: Part | list[Part] | None = None,
resume_restart: Part | list[Part] | None = None,
resume_metadata: dict[str, Any] | None = None,
config: None = None,
max_turns: int | None = None,
context: dict[str, object] | None = None,
output_schema: (
type[Any] | dict[str, Any] | str | None
) = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None
) -> ModelResponse[Any]
generate(
*,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
prompt: str | list[Part] | None = None,
system: str | list[Part] | None = None,
messages: list[Message] | None = None,
tools: Sequence[str | Tool] | None = None,
return_tool_requests: bool | None = None,
tool_choice: ToolChoice | None = None,
resume_respond: Part | list[Part] | None = None,
resume_restart: Part | list[Part] | None = None,
resume_metadata: dict[str, Any] | None = None,
config: ModelConfigDict,
max_turns: int | None = None,
context: dict[str, object] | None = None,
output_schema: (
type[Any] | dict[str, Any] | str | None
) = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None
) -> ModelResponse[Any]
generate(
*,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
prompt: str | list[Part] | None = None,
system: str | list[Part] | None = None,
messages: list[Message] | None = None,
tools: Sequence[str | Tool] | None = None,
return_tool_requests: bool | None = None,
tool_choice: ToolChoice | None = None,
resume_respond: Part | list[Part] | None = None,
resume_restart: Part | list[Part] | None = None,
resume_metadata: dict[str, Any] | None = None,
config: ModelRefConfigT | Mapping[str, Any] | None,
max_turns: int | None = None,
context: dict[str, object] | None = None,
output_schema: (
type[Any] | dict[str, Any] | str | None
) = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None
) -> ModelResponse[Any]
Generate text or structured data using a language model.
model accepts a typed ModelRef from a plugin constructor
(recommended, e.g. GoogleAI.gemini('gemini-flash-latest'),
which takes the bare model ID, stamps the 'googleai/' namespace
prefix, and type-checks config), a plugin-prefixed string
('googleai/gemini-flash-latest'), a registered model Action, or
None to use the default model= passed to Genkit(...). When
passing a plain str for a plugin model, the <plugin>/ prefix
(such as 'googleai/' or 'vertexai/') is required; passing an
unprefixed string raises GenkitError with NOT_FOUND. Any model
ID served by the provider (including newly released models) is resolved
lazily on demand at call time without needing pre-registration or SDK
updates.
tools is typed as Sequence rather than list because Sequence
is covariant: list[Tool] or list[str] are both assignable to
Sequence[str | Tool], but not to list[str | Tool].
Example
from pydantic import BaseModel
class Weather(BaseModel): city: str forecast: str
res = await ai.generate( prompt='Weather in Paris?', tools=[current_weather], output_schema=Weather, ) if res.output: print(res.output)
=> city='Paris' forecast='Sunny'
max_turns caps tool rounds (default 50, matching Go): the model may
call tools that many times and then reply, so a run uses at most
max_turns + 1 model calls. Past the cap the response is ABORTED
with reason MAX_TURNS_EXCEEDED.
prompt and system strings are sent exactly as written. Braces
are content (JSON, code, or another template), not Handlebars.
Partials ({{> persona}}), define_helper helpers and
{{media url=...}} only render in define_prompt or a .prompt
file; otherwise build the text or Part.from_media(...) yourself.
Source code in packages/genkit/src/genkit/_ai/_aio.py
1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 | |
generate_operation(*, model=None, prompt=None, system=None, messages=None, config=None, context=None, use=None, docs=None)
async
generate_operation(
*,
model: (
ModelRef[Any] | BackgroundModel | str | None
) = None,
prompt: str | list[Part] | None = None,
system: str | list[Part] | None = None,
messages: list[Message] | None = None,
config: None = None,
context: dict[str, object] | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None
) -> Operation[ModelResponse]
generate_operation(
*,
model: (
ModelRef[ModelRefConfigT]
| BackgroundModel
| str
| None
) = None,
prompt: str | list[Part] | None = None,
system: str | list[Part] | None = None,
messages: list[Message] | None = None,
config: ModelConfigDict,
context: dict[str, object] | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None
) -> Operation[ModelResponse]
generate_operation(
*,
model: (
ModelRef[ModelRefConfigT]
| BackgroundModel
| str
| None
) = None,
prompt: str | list[Part] | None = None,
system: str | list[Part] | None = None,
messages: list[Message] | None = None,
config: ModelRefConfigT | Mapping[str, Any] | None,
context: dict[str, object] | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None
) -> Operation[ModelResponse]
Generate content using a long-running model, returning an Operation to poll.
A background start is one request that returns a job handle. There is
no tool loop and no output parsing on check_operation, so tools,
max_turns, and the output_* options are not accepted here.
Example
op = await ai.generate_operation( model='googleai/veo-3.1-generate-preview', prompt='A timelapse of a flower blooming.', ) while not op.done: op = await ai.check_operation(op)
Source code in packages/genkit/src/genkit/_ai/_aio.py
2789 2790 2791 2792 2793 2794 2795 2796 2797 2798 2799 2800 2801 2802 2803 2804 2805 2806 2807 2808 2809 2810 2811 2812 2813 2814 2815 2816 2817 2818 2819 2820 2821 2822 2823 2824 2825 2826 2827 2828 2829 2830 2831 2832 2833 2834 2835 2836 2837 2838 2839 2840 2841 2842 2843 2844 2845 2846 2847 2848 2849 2850 2851 2852 2853 2854 2855 2856 2857 2858 2859 2860 2861 2862 2863 2864 2865 2866 2867 | |
generate_stream(*, model=None, prompt=None, system=None, messages=None, tools=None, return_tool_requests=None, tool_choice=None, resume_respond=None, resume_restart=None, resume_metadata=None, config=None, max_turns=None, context=None, output_schema=None, output_format=None, output_content_type=None, output_instructions=None, output_constrained=None, use=None, docs=None)
generate_stream(
*,
model: ModelRef[Any] | Action | str | None = None,
prompt: str | list[Part] | None = None,
system: str | list[Part] | None = None,
messages: list[Message] | None = None,
tools: Sequence[str | Tool] | None = None,
return_tool_requests: bool | None = None,
tool_choice: ToolChoice | None = None,
resume_respond: Part | list[Part] | None = None,
resume_restart: Part | list[Part] | None = None,
resume_metadata: dict[str, Any] | None = None,
config: None = None,
max_turns: int | None = None,
context: dict[str, object] | None = None,
output_schema: type[OutputT],
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None
) -> ModelStreamResponse[OutputT]
generate_stream(
*,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
prompt: str | list[Part] | None = None,
system: str | list[Part] | None = None,
messages: list[Message] | None = None,
tools: Sequence[str | Tool] | None = None,
return_tool_requests: bool | None = None,
tool_choice: ToolChoice | None = None,
resume_respond: Part | list[Part] | None = None,
resume_restart: Part | list[Part] | None = None,
resume_metadata: dict[str, Any] | None = None,
config: ModelConfigDict,
max_turns: int | None = None,
context: dict[str, object] | None = None,
output_schema: type[OutputT],
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None
) -> ModelStreamResponse[OutputT]
generate_stream(
*,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
prompt: str | list[Part] | None = None,
system: str | list[Part] | None = None,
messages: list[Message] | None = None,
tools: Sequence[str | Tool] | None = None,
return_tool_requests: bool | None = None,
tool_choice: ToolChoice | None = None,
resume_respond: Part | list[Part] | None = None,
resume_restart: Part | list[Part] | None = None,
resume_metadata: dict[str, Any] | None = None,
config: ModelRefConfigT | Mapping[str, Any] | None,
max_turns: int | None = None,
context: dict[str, object] | None = None,
output_schema: type[OutputT],
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None
) -> ModelStreamResponse[OutputT]
generate_stream(
*,
model: ModelRef[Any] | Action | str | None = None,
prompt: str | list[Part] | None = None,
system: str | list[Part] | None = None,
messages: list[Message] | None = None,
tools: Sequence[str | Tool] | None = None,
return_tool_requests: bool | None = None,
tool_choice: ToolChoice | None = None,
resume_respond: Part | list[Part] | None = None,
resume_restart: Part | list[Part] | None = None,
resume_metadata: dict[str, Any] | None = None,
config: None = None,
max_turns: int | None = None,
context: dict[str, object] | None = None,
output_schema: (
type[Any] | dict[str, Any] | str | None
) = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None
) -> ModelStreamResponse[Any]
generate_stream(
*,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
prompt: str | list[Part] | None = None,
system: str | list[Part] | None = None,
messages: list[Message] | None = None,
tools: Sequence[str | Tool] | None = None,
return_tool_requests: bool | None = None,
tool_choice: ToolChoice | None = None,
resume_respond: Part | list[Part] | None = None,
resume_restart: Part | list[Part] | None = None,
resume_metadata: dict[str, Any] | None = None,
config: ModelConfigDict,
max_turns: int | None = None,
context: dict[str, object] | None = None,
output_schema: (
type[Any] | dict[str, Any] | str | None
) = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None
) -> ModelStreamResponse[Any]
generate_stream(
*,
model: (
ModelRef[ModelRefConfigT] | Action | str | None
) = None,
prompt: str | list[Part] | None = None,
system: str | list[Part] | None = None,
messages: list[Message] | None = None,
tools: Sequence[str | Tool] | None = None,
return_tool_requests: bool | None = None,
tool_choice: ToolChoice | None = None,
resume_respond: Part | list[Part] | None = None,
resume_restart: Part | list[Part] | None = None,
resume_metadata: dict[str, Any] | None = None,
config: ModelRefConfigT | Mapping[str, Any] | None,
max_turns: int | None = None,
context: dict[str, object] | None = None,
output_schema: (
type[Any] | dict[str, Any] | str | None
) = None,
output_format: str | None = None,
output_content_type: str | None = None,
output_instructions: bool | str | None = None,
output_constrained: bool | None = None,
use: Sequence[Middleware] | None = None,
docs: list[Document] | None = None
) -> ModelStreamResponse[Any]
Stream generated text, returning a ModelStreamResponse with .stream and .response.
With output_schema=Recipe, each chunk.output is a partial of
that type: same attributes, any field may still be None or a
prefix. Guard the field you are about to use. The finished
Recipe is only (await result.response).output, and it's None
when the reply isn't a Recipe.
If the model fails partway through, the async for still ends
normally. The final response has finish_reason == FAILED,
error set, text == '', message is None, and messages
ending at the last complete turn, so it's safe to send back. The
chunks you already received are the record of what was shown.
Use async with result.stream as stream: so that leaving the loop
early (break, an exception, a disconnected client) closes the
stream and stops the model call.
Example
# 1. Start streaming
result = ai.generate_stream(prompt='Write a haiku about rain.')
# 2. Print chunks as they arrive; the context manager closes the stream
async with result.stream as stream:
async for chunk in stream:
print(chunk.text, end='')
print()
# => Soft rain on the roof...
# 3. Await the final response
final = await result.response
print(final.finish_reason)
# => stop
Source code in packages/genkit/src/genkit/_ai/_aio.py
2207 2208 2209 2210 2211 2212 2213 2214 2215 2216 2217 2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228 2229 2230 2231 2232 2233 2234 2235 2236 2237 2238 2239 2240 2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 2253 2254 2255 2256 2257 2258 2259 2260 2261 2262 2263 2264 2265 2266 2267 2268 2269 2270 2271 2272 2273 2274 2275 2276 2277 2278 2279 2280 2281 2282 2283 2284 2285 2286 2287 2288 2289 2290 2291 2292 2293 2294 2295 2296 2297 2298 2299 2300 2301 2302 | |
lookup_model(name)
async
Return the model action registered under name, or None.
Plugin models that haven't been used yet are resolved through the
plugin, so 'googleai/gemini-flash-latest' works before any
generate call. Background models aren't included; they run through
generate_operation.
Source code in packages/genkit/src/genkit/_ai/_aio.py
904 905 906 907 908 909 910 911 912 913 | |
lookup_tool(name)
async
Return the tool registered under name, or None.
The same handle @ai.tool returns, so a tool defined in one module
can be called or passed in tools=[...] from another without
importing it.
Example
# 1. Define a tool in one module
@ai.tool
async def menu_price(dish: str) -> float:
return 14.5
# 2. Find it by name somewhere else
price_tool = await ai.lookup_tool('menu_price')
response = await ai.generate(prompt='How much is the ramen?', tools=[price_tool])
print(response.text)
# => The ramen is $14.50.
Source code in packages/genkit/src/genkit/_ai/_aio.py
915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 | |
lookup_value(*, kind, name)
Return the value defined under kind and name, or None.
Source code in packages/genkit/src/genkit/_ai/_aio.py
942 943 944 | |
middleware(cls=None, *, name=None, description=None)
middleware(cls: type[MiddlewareT]) -> type[MiddlewareT]
middleware(
*,
name: str | None = None,
description: str | None = None
) -> Callable[[type[MiddlewareT]], type[MiddlewareT]]
Decorator that registers a custom middleware on this app's registry.
Source code in packages/genkit/src/genkit/_ai/_aio.py
671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 | |
prompt(name, *, variant=None, input_schema=None, output_schema=None)
prompt(
name: str,
*,
variant: str | None = None,
input_schema: None = None,
output_schema: None = None
) -> Prompt[Any, Any]
prompt(
name: str,
*,
variant: str | None = None,
input_schema: type[InputT],
output_schema: None = None
) -> Prompt[InputT, Any]
prompt(
name: str,
*,
variant: str | None = None,
input_schema: None = None,
output_schema: type[OutputT]
) -> Prompt[Any, OutputT]
prompt(
name: str,
*,
variant: str | None = None,
input_schema: type[InputT],
output_schema: type[OutputT]
) -> Prompt[InputT, OutputT]
Look up a prompt by name and optional variant.
Source code in packages/genkit/src/genkit/_ai/_aio.py
1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 | |
run(name, fn, *, metadata=None)
async
Run a function as a discrete traced step within a flow.
fn takes no arguments and returns an awaitable, so async def
functions, lambda: fetch(i), and callables with an async def
__call__ all work. Same shape as JS ai.run('step', fn).
Example
# 1. Each fan-out step gets its own span in the trace
menus = await asyncio.gather(*[ai.run(f'fetch-{i}', lambda i=i: fetch_menu(i)) for i in range(3)])
# 2. Keyword form still works
total = await ai.run(name='sum', fn=lambda: sum_prices(menus))
Source code in packages/genkit/src/genkit/_ai/_aio.py
2601 2602 2603 2604 2605 2606 2607 2608 2609 2610 2611 2612 2613 2614 2615 2616 2617 2618 2619 2620 2621 2622 2623 2624 2625 2626 2627 2628 2629 2630 2631 2632 2633 2634 2635 2636 2637 2638 2639 | |
run_main(coro)
Run your main coroutine and return its result, like asyncio.run.
With reflection on (GENKIT_ENV=dev, which genkit start sets, or
GENKIT_REFLECTION_ENABLED=true), it keeps the process alive after
main returns, so the Dev UI can keep running your flows, until
Ctrl+C, SIGTERM, or the reflection server stops on its own. With
reflection off, it returns as soon as main does.
If main raises while reflection is on, the error is logged and the
Dev UI stays up. The error is raised once the process stops. Ctrl+C
after a successful main raises KeyboardInterrupt, and SIGTERM
returns main's result.
Example
from genkit import Genkit
from genkit_google_genai import GoogleAI
# 1. Initialize Genkit and define a flow
ai = Genkit(plugins=[GoogleAI()], model=GoogleAI.gemini('gemini-flash-latest'))
@ai.flow
async def suggest_dish(cuisine: str) -> str:
response = await ai.generate(prompt=f'Suggest one {cuisine} dish.')
return response.text
# 2. Run a quick check from your script's entry point
async def main() -> None:
print(await suggest_dish('Thai'))
# 3. Start it
ai.run_main(main())
# => Green curry with chicken
# `python main.py` exits here. Under `genkit start`, the process
# stays up for the Dev UI until you press Ctrl+C.
Source code in packages/genkit/src/genkit/_ai/_aio.py
1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 1724 1725 1726 1727 1728 1729 | |
tool(name=None, *, description=None, input_schema=None)
tool(
name: str | None = None,
*,
description: str | None = None,
input_schema: (
type[BaseModel] | dict[str, object] | None
) = None
) -> _ToolDecorator
tool(
func: Callable[
[ToolContext[ChunkT], object], Awaitable[OutputT]
],
) -> Tool[Any, OutputT, ChunkT]
tool(
func: DefaultInputCtxFn[
ToolContext[ChunkT], InputT, OutputT
],
) -> DefaultedTool[InputT, OutputT, ChunkT]
tool(
func: Callable[
[InputT, ToolContext[ChunkT]], Awaitable[OutputT]
],
) -> Tool[InputT, OutputT, ChunkT]
tool(
func: CtxDefaultInputFn[
ToolContext[ChunkT], InputT, OutputT
],
) -> DefaultedTool[InputT, OutputT, ChunkT]
tool(
func: Callable[
[ToolContext[ChunkT], InputT], Awaitable[OutputT]
],
) -> Tool[InputT, OutputT, ChunkT]
tool(
func: Callable[[object], Awaitable[OutputT]],
) -> Tool[Any, OutputT]
tool(
func: Callable[
[ToolContext[ChunkT]], Awaitable[OutputT]
],
) -> Tool[None, OutputT, ChunkT]
tool(
func: DefaultInputFn[InputT, OutputT],
) -> DefaultedTool[InputT, OutputT]
tool(
func: Callable[[InputT], Awaitable[OutputT]],
) -> Tool[InputT, OutputT]
tool(
func: Callable[[], Awaitable[OutputT]],
) -> Tool[None, OutputT]
Decorator to register a function as a tool.
Can be used bare (@ai.tool) or with arguments (@ai.tool(),
@ai.tool('custom_name', description='...')).
The return annotation is what the model binds as outputSchema.
Awaiting the tool directly returns what the function returned.
A tool takes at most one input, and that input's type is the schema the
model fills in. For several fields, use one Pydantic model. To read the
request context or interrupt, add a parameter annotated
ToolContext, in any position. Any other second parameter raises
TypeError when the tool is defined.
Example
@ai.tool async def current_weather(city: str) -> str: return f'Sunny in {city}'
class Forecast(BaseModel): city: str days: int = 3
@ai.tool async def forecast(input: Forecast, ctx: ToolContext) -> str: user = ctx.context.get('user_id') return f'{input.days}-day forecast for {input.city} ({user})'
res = await ai.generate(prompt='Weather in Paris?', tools=['current_weather', 'forecast'])
Source code in packages/genkit/src/genkit/_ai/_aio.py
604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 | |
current_session()
Return the active agent session, or None if not inside a session.
Source code in packages/genkit/src/genkit/exp/_api.py
44 45 46 | |
genkit.exp.agent
Agent types. Experimental — import the instance from genkit.exp.
from genkit.exp import Genkit
from genkit.exp.agent import InMemorySessionStore
from genkit_google_genai import GoogleAI
ai = Genkit(plugins=[GoogleAI()])
agent = ai.define_agent(
name='weatherAgent',
model=GoogleAI.gemini('gemini-flash-latest'),
system='Weather assistant.',
store=InMemorySessionStore(),
)
AgentFn = Callable[[SessionRunner, AgentContext], Awaitable[AgentResult]]
module-attribute
Custom agent entrypoint: takes a SessionRunner and the run context, returns an AgentResult.
ChunkTransform = Callable[[AgentStreamChunk], AgentStreamChunk | None]
module-attribute
Reshapes a stream chunk before a client sees it; returning None drops the chunk.
SaveFn = Callable[[SessionSnapshot | None], SessionSnapshot | None]
module-attribute
Synchronous save mutator: gets a copy of the stored snapshot (or None), returns the new one or None to skip.
StateT = TypeVarExt('StateT', bound=BaseModel, default=Any)
module-attribute
Type of a session's custom state: a Pydantic model, or Any for schemaless sessions.
StateTransform = Callable[[SessionState], SessionState]
module-attribute
Redacts session state before a client sees it; must return a state, never None.
TERMINAL_STATUSES = frozenset({SnapshotStatus.COMPLETED, SnapshotStatus.FAILED, SnapshotStatus.ABORTED})
module-attribute
Final snapshot statuses: a store rejects any later status change, and status streams end after one.
Agent
Bases: genkit._core._action.BidiAction[genkit._core._model.AgentInput, genkit._core._model.AgentOutput, genkit._core._model.AgentStreamChunk, genkit._core._model.AgentInit], genkit._ai._agents._client.AgentClient[genkit._ai._agents._session.StateT], typing.Generic[genkit._ai._agents._session.StateT]
The low-level agent primitive: a BidiAction that lives in the registry.
Created by define_agent / define_custom_agent. As a BidiAction it's
the thing that gets registered and served over HTTP. It's also an
AgentClient, so the ergonomic chat surface (chat/load_chat/
get_snapshot/abort) is inherited rather than reimplemented — it's the
same client used for remote agents, just pointed at an in-process transport.
So talking to a local agent and a remote one go through one client, not two.
The action generics are pinned to the agent turn shape: each turn's input is
an AgentInput, streamed chunks are AgentStreamChunk, the turn result
is an AgentOutput, and init (session identity) is an AgentInit.
Source code in packages/genkit/src/genkit/_ai/_agents/_base.py
105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 | |
config_schema
property
The config class set at definition time, from config_schema= or the fn annotation.
input_class
property
The action's input annotation as a concrete class, when it is one.
params
property
The function's input and context parameters, as Genkit passes them.
__call__(input=NO_INPUT, *, context=None)
async
__call__(
input: None = None,
*,
context: dict[str, Any] | None = None
) -> OutputT
__call__(
input: InputT, *, context: dict[str, Any] | None = None
) -> OutputT
Call the action directly, returning just the response value.
Source code in packages/genkit/src/genkit/_core/_action.py
1350 1351 1352 1353 1354 1355 1356 1357 | |
__init__(*, name, bidi_fn, store=None, state_transform=None, state_schema=None, description=None, metadata=None)
Initialise Agent; transport is inferred from the action + store.
Source code in packages/genkit/src/genkit/_ai/_agents/_base.py
124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 | |
abort(snapshot_id)
async
Aborts a running snapshot on the server.
The return is the snapshot's status from before this call, not after.
pending means the turn was still running and this call cancelled it
(the row is now aborted). A terminal status (completed,
failed, aborted) means the turn had already finished — this
call did not rewrite it. None means nothing was observed (no row,
or the store never ran the abort write).
Source code in packages/genkit/src/genkit/_ai/_agents/_client.py
691 692 693 694 695 696 697 698 699 700 701 | |
abort_snapshot_data(snapshot_id)
async
Abort a running snapshot.
The return is the snapshot's status from before this call, not after.
pending means the turn was still running and this call cancelled it
(the row is now aborted). A terminal status means the turn had
already finished — this call did not rewrite it. None means
nothing was observed (no store, no row, or the store never ran the
abort write). This Python server always returns the previous status.
Source code in packages/genkit/src/genkit/_ai/_agents/_base.py
201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 | |
action_fn(input, ctx)
async
Adapt the bidi fn to the plain Action fn shape.
The bidi fn reads its per-turn inputs from an async stream and emits
chunks through a callback — the same pair a plain action fn gets on its
ctx. A live chat supplies ctx._input_stream; a one-shot run has just
the single input, which we hand over as a one-item stream. init
(session identity) rides the run's init channel, separate from inputs.
Source code in packages/genkit/src/genkit/_core/_action.py
2236 2237 2238 2239 2240 2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 | |
chat(*, snapshot_id=None, session_id=None, messages=None, artifacts=None, state=None)
Starts a new session, or resumes one.
Store-backed agents take snapshot_id and/or session_id.
snapshot_id resumes that exact row (it must be completed).
session_id continues the conversation: skip a failed / aborted /
pending leaf, or start fresh if there isn't a completed turn yet.
Passing both is fine: the snapshot picks the row, the session id
checks it belongs to that session.
messages / artifacts / state are only for client-managed
agents (no store). Mixing those with a resume id, or sending a state
blob to a store-backed agent, raises :class:AgentInitError.
Source code in packages/genkit/src/genkit/_ai/_agents/_client.py
615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 | |
get_snapshot(*, snapshot_id=None, session_id=None)
async
Reads a stored snapshot without starting a session.
Pass exactly one of snapshot_id or session_id. A session
lookup returns the newest row even when that turn failed or was
aborted. chat(session_id=) is how you continue from the last
good turn.
Source code in packages/genkit/src/genkit/_ai/_agents/_client.py
676 677 678 679 680 681 682 683 684 685 686 687 688 689 | |
get_snapshot_data(*, snapshot_id=None, session_id=None)
async
Read a stored snapshot without starting a session.
Pass exactly one of snapshot_id or session_id. A session
lookup returns the newest row even when that turn failed or was
aborted. chat(session_id=) is how you continue from the last
good turn.
Source code in packages/genkit/src/genkit/_ai/_agents/_base.py
178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 | |
load_chat(*, snapshot_id=None, session_id=None)
async
Loads a stored snapshot onto a new chat.
Pass exactly one of snapshot_id or session_id. The chat is
pointed at that row as stored — including a failed, aborted, or
still-pending turn. send() resumes that row, so a non-completed
leaf is rejected. To continue the conversation, use chat(session_id=).
Source code in packages/genkit/src/genkit/_ai/_agents/_client.py
650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 | |
run(input=NO_INPUT, *, on_chunk=None, context=None, telemetry_labels=None, init=None)
async
run(
input: None = None,
*,
on_chunk: Callable[[ChunkT], None] | None = None,
context: dict[str, Any] | None = None,
telemetry_labels: dict[str, object] | None = None,
init: InitT | None = None
) -> ActionResponse[OutputT]
run(
input: InputT,
*,
on_chunk: Callable[[ChunkT], None] | None = None,
context: dict[str, Any] | None = None,
telemetry_labels: dict[str, object] | None = None,
init: InitT | None = None
) -> ActionResponse[OutputT]
Run the action and return its result with trace ids.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
input
|
genkit._core._action.InputT | None
|
The input, validated against the input schema. Required unless the action takes no input or its function gives the input a default; omit it to use that default. On a flow or tool, an explicit None is a value and is validated like any other. |
genkit._core._action.NO_INPUT
|
on_chunk
|
collections.abc.Callable[[genkit._core._action.ChunkT], None] | None
|
Called with each chunk the action streams. |
None
|
context
|
dict[str, typing.Any] | None
|
Request context (auth, headers, ...), read as |
None
|
telemetry_labels
|
dict[str, object] | None
|
Custom labels set as span attributes. |
None
|
init
|
genkit._core._action.InitT | None
|
Session init value for a bidi action (e.g. a served agent's session identity). Validated against the init schema; plain actions ignore it. |
None
|
Returns:
| Type | Description |
|---|---|
genkit._core._action.ActionResponse[genkit._core._action.OutputT]
|
ActionResponse with the result, trace id and span id. |
Raises:
| Type | Description |
|---|---|
genkit._core._error.GenkitError
|
If input validation fails (INVALID_ARGUMENT status). |
Source code in packages/genkit/src/genkit/_core/_action.py
1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 | |
stream(input=NO_INPUT, *, context=None, telemetry_labels=None, init=None)
stream(
input: None = None,
*,
context: dict[str, Any] | None = None,
telemetry_labels: dict[str, object] | None = None,
init: InitT | None = None
) -> StreamResponse[ChunkT, OutputT]
stream(
input: InputT,
*,
context: dict[str, Any] | None = None,
telemetry_labels: dict[str, object] | None = None,
init: InitT | None = None
) -> StreamResponse[ChunkT, OutputT]
Return a StreamResponse: iterate .stream for chunks, await .response for the result.
Takes the same keyword arguments as run, minus on_chunk. If the
action raises, both the loop and .response raise.
Source code in packages/genkit/src/genkit/_core/_action.py
1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 | |
stream_bidi(init=None, *, context=None, telemetry_labels=None)
async
Start a bidirectional streaming session over the single stream() primitive.
Opens an input channel, kicks off _stream(input_stream=channel), and
returns a BidiConnection whose send/close push per-turn inputs into that
channel while receive/output read the run's chunks and final result.
init is the session identity for the whole connection; per-turn
inputs arrive later via BidiConnection.send. It's sugar — all
execution goes through the same run()/stream() path as any other action.
Source code in packages/genkit/src/genkit/_core/_action.py
2254 2255 2256 2257 2258 2259 2260 2261 2262 2263 2264 2265 2266 2267 2268 2269 2270 2271 2272 2273 2274 2275 2276 2277 2278 2279 2280 2281 2282 | |
AgentChat
Bases: typing.Generic[genkit._ai._agents._client.StateT]
A stateful conversation session with an agent.
Public surface: read snapshot_id, session_id, state, messages,
and artifacts; call send / send_stream, resume /
resume_stream, detach, and abort. Everything prefixed with _
is internal wiring.
state is the agent's custom session state; messages and artifacts
are the running conversation and files. The chat tracks all three directly,
so a client-managed resume just hands them back (chat(messages=chat.messages,
state=chat.state, artifacts=chat.artifacts)). snapshot_id and
session_id are the server store's resume handles.
The chat keeps these fields in sync with each turn's output and rebuilds the
per-turn resume payload from them via _wire_init.
Source code in packages/genkit/src/genkit/_ai/_agents/_client.py
1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 | |
messages
property
Running view of the conversation, built the same way in both modes.
A turn's messages are stitched from its chunk stream — the only channel
that carries the in-between tool-request/tool-response steps — with the
final reply taken from the turn's output when it's there (it carries the
metadata a resume needs). The chunks are identical whether state is server-
or client-managed, so there's one path here. For server-managed sessions
the durable snapshot in the store stays the source of truth — use
get_snapshot() / load_chat to read it (e.g. after a detached
turn, whose chunks this session never saw).
snapshot_id
property
Store resume handle, kept in sync with the latest turn (store-backed only).
state
property
The agent's custom session state, kept live as the turn streams.
abort()
async
Stops the session's server-side work by aborting its current snapshot.
The return is the snapshot's status from before this call, not after.
pending means the turn was still running and this call cancelled it
(the row is now aborted). A terminal status means the turn had
already finished. None means nothing was observed.
Against this SDK the return is always the previous status. After abort
the chat still points at the aborted leaf — chat(session_id=…)
before the next send(). The local transcript keeps the prompt;
it was still asked.
Raises:
| Type | Description |
|---|---|
ValueError
|
if there's no snapshot to abort — the agent is
client-managed (no store) or no turn has produced a snapshot yet.
For a client-side stop that just detaches the caller, use
|
Source code in packages/genkit/src/genkit/_ai/_agents/_client.py
1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 | |
detach(input)
async
Runs a turn in the background on the server and returns a poll handle.
Source code in packages/genkit/src/genkit/_ai/_agents/_client.py
1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 | |
get_snapshot()
async
Reads the current snapshot from the server store, if store-backed.
Source code in packages/genkit/src/genkit/_ai/_agents/_client.py
1243 1244 1245 1246 1247 | |
resume(*, respond=None, restart=None, metadata=None)
async
Continues a conversation from an interrupt and returns the response.
Sugar for :meth:send with a resume payload. For streaming, use
:meth:resume_stream.
Source code in packages/genkit/src/genkit/_ai/_agents/_client.py
1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 | |
resume_stream(*, respond=None, restart=None, metadata=None)
Continues a conversation from an interrupt and returns an in-flight turn.
respond answers paused tool requests; restart re-runs them with
new metadata. Build each part with the interrupt's respond / restart
helpers. metadata rides along on the resumed tool message (the same
resume_metadata a plain generate call takes). Sugar for
:meth:send_stream with a resume payload.
Source code in packages/genkit/src/genkit/_ai/_agents/_client.py
1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 | |
send(input)
async
Runs a turn and returns the completed response.
A turn that finishes failed still returns: finish_reason is
failed and .error says why. It raises GenkitError only
when the call never got a turn result (dropped connection, init
misuse), and puts the chat back where it was before the send. A
dropped connection may still have run the turn server-side, so
reload with chat(session_id=...) before resending a turn whose
tools have side effects. Cancelling or timing out the call is a
client-side detach instead: it raises CancelledError /
TimeoutError and keeps the prompt. Pumps the transport so custom-state
patches and message stitching still apply; does not expose a chunk
stream. For incremental chunks use :meth:send_stream.
Source code in packages/genkit/src/genkit/_ai/_agents/_client.py
1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 | |
send_stream(input)
Runs a turn and returns a handle with .stream / .response.
Streaming form of :meth:send (like Action.stream / ai.generate_stream):
same underlying turn, with chunks delivered to the caller. To stop a turn,
call turn.abort() for a client-side detach (the server finishes in the
background), or wrap the stream/response in asyncio.timeout(...) /
cancel the surrounding task — both detach the same way and then surface the
deadline. Use chat.abort() to halt server-side work on a store-backed
agent.
Source code in packages/genkit/src/genkit/_ai/_agents/_client.py
1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 | |
AgentChunk
dataclass
Bases: typing.Generic[genkit._ai._agents._client.StateT]
Represents a structured stream chunk yielded during a turn.
Source code in packages/genkit/src/genkit/_ai/_agents/_client.py
175 176 177 178 179 180 181 182 183 184 185 186 187 188 | |
AgentClient
Bases: typing.Generic[genkit._ai._agents._client.StateT]
Transport-backed agent client — wraps any AgentTransport and implements AgentAPI.
This is the one ergonomic surface (chat/load_chat/get_snapshot/
abort) for talking to an agent, whether it's remote (HTTP transport) or
in-process (the local agent action). get_snapshot and load_chat
read the store; chat() is how you start or continue a conversation.
state_schema types the custom session state so the resulting chat hands
back a validated model instead of a bare dict; leave it None for untyped state.
Source code in packages/genkit/src/genkit/_ai/_agents/_client.py
594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 | |
abort(snapshot_id)
async
Aborts a running snapshot on the server.
The return is the snapshot's status from before this call, not after.
pending means the turn was still running and this call cancelled it
(the row is now aborted). A terminal status (completed,
failed, aborted) means the turn had already finished — this
call did not rewrite it. None means nothing was observed (no row,
or the store never ran the abort write).
Source code in packages/genkit/src/genkit/_ai/_agents/_client.py
691 692 693 694 695 696 697 698 699 700 701 | |
chat(*, snapshot_id=None, session_id=None, messages=None, artifacts=None, state=None)
Starts a new session, or resumes one.
Store-backed agents take snapshot_id and/or session_id.
snapshot_id resumes that exact row (it must be completed).
session_id continues the conversation: skip a failed / aborted /
pending leaf, or start fresh if there isn't a completed turn yet.
Passing both is fine: the snapshot picks the row, the session id
checks it belongs to that session.
messages / artifacts / state are only for client-managed
agents (no store). Mixing those with a resume id, or sending a state
blob to a store-backed agent, raises :class:AgentInitError.
Source code in packages/genkit/src/genkit/_ai/_agents/_client.py
615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 | |
get_snapshot(*, snapshot_id=None, session_id=None)
async
Reads a stored snapshot without starting a session.
Pass exactly one of snapshot_id or session_id. A session
lookup returns the newest row even when that turn failed or was
aborted. chat(session_id=) is how you continue from the last
good turn.
Source code in packages/genkit/src/genkit/_ai/_agents/_client.py
676 677 678 679 680 681 682 683 684 685 686 687 688 689 | |
load_chat(*, snapshot_id=None, session_id=None)
async
Loads a stored snapshot onto a new chat.
Pass exactly one of snapshot_id or session_id. The chat is
pointed at that row as stored — including a failed, aborted, or
still-pending turn. send() resumes that row, so a non-completed
leaf is rejected. To continue the conversation, use chat(session_id=).
Source code in packages/genkit/src/genkit/_ai/_agents/_client.py
650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 | |
AgentContext
Bases: genkit._core._action.ActionContext[genkit._core._model.AgentStreamChunk]
Execution context passed to a custom agent function.
Provides ctx.send_chunk(chunk) for streaming :class:AgentStreamChunk
items and ctx.context for caller context data.
Source code in packages/genkit/src/genkit/_ai/_agents/_runtime.py
280 281 282 283 284 285 | |
context
property
The request context (auth, headers, ...) the caller attached.
is_streaming
property
True if a streaming callback is registered.
streaming_callback
property
The streaming callback, if any.
Use this when you need to pass the callback to another action. For sending chunks directly, use send_chunk() instead.
send_chunk(chunk)
Send a streaming chunk to the client.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
chunk
|
genkit._core._action.ChunkT
|
The chunk data to stream. |
required |
Source code in packages/genkit/src/genkit/_core/_action.py
1053 1054 1055 1056 1057 1058 1059 1060 | |
AgentFinishReason
Bases: genkit._core._compat.StrEnum
AgentFinishReason data type class.
Source code in packages/genkit/src/genkit/_core/_typing.py
37 38 39 40 41 42 43 44 45 46 47 48 | |
AgentInit
Bases: genkit._core._base.GenkitModel
Init payload whose state uses the public SessionState type.
Source code in packages/genkit/src/genkit/_core/_model.py
1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
AgentInitError
Bases: genkit._core._error.PublicError
API misuse on agent init that must surface as a thrown/HTTP error.
Covers calling an agent with an init that does not match its state-management
mode (e.g. sending state to a server-managed agent). Recoverable pre-turn
problems (missing or non-resumable snapshot, invalid custom state) are not
subclasses of AgentInitError, so define_custom_agent absorbs them
into finish_reason='failed'.
It is a PublicError: the message names only the init fields and ids the
caller sent, so a served agent returns its status, message, and
details.reason and a remote client reads the same reason it would
in-process.
Source code in packages/genkit/src/genkit/_ai/_agents/_runtime.py
328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 | |
__init__(*, status, message, reason=None)
Initialize an AgentInitError.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
status
|
genkit._core._error.StatusName
|
The status name for this error. |
required |
message
|
str
|
Caller-facing sentence; must not include server-side data. |
required |
reason
|
genkit._core._error.ErrorReason | None
|
Stable reason, sent on the wire as |
None
|
Source code in packages/genkit/src/genkit/_ai/_agents/_runtime.py
343 344 345 346 347 348 349 350 351 | |
AgentInput
Bases: genkit._core._base.GenkitModel
Turn input whose message is the public Message type.
Source code in packages/genkit/src/genkit/_core/_model.py
1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
AgentInterrupt
Bases: typing.Generic[genkit._ai._agents._client.InputT, genkit._ai._agents._client.OutputT]
Represents a tool request interrupt that paused the turn.
Source code in packages/genkit/src/genkit/_ai/_agents/_client.py
191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 | |
respond(output)
Tool response for chat.resume(respond=[...]).
Carries the output only. Part.respond is the call that attaches
interruptResponse metadata.
Source code in packages/genkit/src/genkit/_ai/_agents/_client.py
207 208 209 210 211 212 213 214 215 216 217 | |
restart(*, resumed_metadata=None)
Tool request for chat.resume(restart=[...]).
The tool sees a resume (ctx.is_resumed() is true). Pass
resumed_metadata when it needs the approval payload.
The request input stays the one the model asked for — a client cannot re-run a tool with arguments the model never sent.
Source code in packages/genkit/src/genkit/_ai/_agents/_client.py
219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 | |
AgentOutput
Bases: genkit._core._base.GenkitModel
Turn output whose message and artifacts are the public types.
Source code in packages/genkit/src/genkit/_core/_model.py
1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
AgentResponse
dataclass
Bases: typing.Generic[genkit._ai._agents._client.StateT]
Completed turn result — client-side wrapper around AgentOutput with rich accessors.
Source code in packages/genkit/src/genkit/_ai/_agents/_client.py
240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 | |
artifacts
property
Artifacts emitted during this turn.
data
property
Structured-output value of the response message, if the model returned one.
error
property
Why this turn failed, or None when it didn't.
Same fields generate puts on response.error (status, message,
details, and reason), so one check reads both. A failed turn that
carried no error still gets one, so if res.error: never reads a
failure as success.
finish_message
property
Human-readable detail when a turn ends abnormally (e.g. blocked or failed).
finish_reason
property
Why the turn ended.
interrupts
property
Tool requests that paused this turn.
media
property
First media part of the response message, if any.
message
property
The response message.
A failed or aborted turn's wire message is the previous reply. Showing it would look like this turn answered, so this is None and every accessor that reads the reply (text, data, interrupts, ...) comes back empty.
reasoning
property
Concatenated reasoning the model exposed for this turn.
session_id
property
Session id this turn belongs to (store- or client-managed).
snapshot_id
property
Server snapshot id after this turn, if store-backed.
text
property
Full text content of the response message.
tool_requests
property
Tool requests in the response message.
AgentResult
Bases: genkit._core._base.GenkitModel
Agent result whose message and artifacts are the public types.
Source code in packages/genkit/src/genkit/_core/_model.py
1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
AgentStreamChunk
Bases: genkit._core._base.GenkitModel
One streamed piece of an agent turn.
Source code in packages/genkit/src/genkit/_core/_model.py
2190 2191 2192 2193 2194 2195 2196 2197 2198 2199 2200 2201 2202 2203 2204 2205 2206 2207 2208 2209 2210 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
AgentTransport
Bases: typing.Protocol, typing.Generic[genkit._ai._agents._client.StateT_co]
Interface implemented by the transport layer (local or websocket).
Source code in packages/genkit/src/genkit/_ai/_agents/_client.py
127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 | |
abort_snapshot(snapshot_id)
async
Aborts the specified snapshot on the server.
Source code in packages/genkit/src/genkit/_ai/_agents/_client.py
165 166 167 | |
get_snapshot(*, snapshot_id=None, session_id=None)
async
Retrieves a session snapshot from the server store.
Source code in packages/genkit/src/genkit/_ai/_agents/_client.py
156 157 158 159 160 161 162 163 | |
run_turn(*, agent_input, init)
async
Run a single turn, returning a (stream, output) pair.
The transport must drive the turn to completion on its own — the returned
output awaitable has to resolve whether or not the caller consumes the
stream. The stream is an optional live view of the same turn, never the
thing that advances it, so await output works without draining chunks.
Concretely: read your own transport (socket, queue, SSE) to the end in a
background task; don't rely on the client to pull the stream for you.
init describes how to resume the conversation for this turn. A stateful
in-process transport reads it only when it opens its connection; a
stateless transport (HTTP) replays it on every request. The session keeps
it current via _wire_init so the transport never has to know how state
is managed.
Source code in packages/genkit/src/genkit/_ai/_agents/_client.py
133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 | |
AgentTurn
Bases: typing.Generic[genkit._ai._agents._client.StateT]
A single in-flight turn — read .stream for chunks, .response for the result.
Same handle shape as action.stream(): the turn runs whether or not you read
.stream, and .response resolves to the final result either way.
Source code in packages/genkit/src/genkit/_ai/_agents/_client.py
435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 | |
response
property
The turn's final result. The turn runs whether or not you read .stream.
stream
property
The turn's chunk stream.
Cancelling the consumer (asyncio.timeout(...) or a task cancel around the
loop) detaches the turn like turn.abort() — the client stops listening and
the server finishes in the background — then the cancellation propagates so the
deadline still surfaces.
abort()
async
Detaches the client from this turn: stops streaming and settles the result now.
This is a client-side abort. The server turn keeps running to completion
in the background so its work still lands; we just stop listening and
resolve the awaited result as cancelled. The prompt you sent stays in
history — it was still asked — so the session reads like a turn that
simply got no reply. To actually halt server-side work on a store-backed
agent, use chat.abort().
Source code in packages/genkit/src/genkit/_ai/_agents/_client.py
496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 | |
Artifact
Bases: genkit._core._base.GenkitModel
Named session file whose parts are the public Part type.
Source code in packages/genkit/src/genkit/_core/_model.py
1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
DetachedTask
Bases: typing.Generic[genkit._ai._agents._client.StateT]
A handle to a background (detached) turn running on the server.
Source code in packages/genkit/src/genkit/_ai/_agents/_client.py
1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 | |
abort()
async
Aborts the detached task on the server.
The return is the snapshot's status from before this call, not after.
pending means this call cancelled a still-running turn (the row is
now aborted). A terminal status means the turn had already
finished. None means nothing was observed.
This does not edit the originating chat's transcript. The prompt was
still asked; reload via chat(session_id=...) before the next send
so you are not still pointed at the aborted leaf.
Source code in packages/genkit/src/genkit/_ai/_agents/_client.py
1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 | |
poll(interval=1.0)
async
Yields the task's snapshot every interval seconds until it settles.
Re-reads the server snapshot on a fixed cadence and stops once it reaches
a terminal status (completed, failed, aborted, or expired), so a caller
can drive a live status UI with a plain async for. For just the final
result, await wait instead.
Source code in packages/genkit/src/genkit/_ai/_agents/_client.py
1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 | |
wait(interval=1.0)
async
Polls until the task settles and returns its final snapshot.
Source code in packages/genkit/src/genkit/_ai/_agents/_client.py
1602 1603 1604 1605 1606 1607 1608 1609 | |
FileSessionStore
Bases: genkit._ai._agents._session.SessionStoreLock, genkit._ai._agents._session.SessionStore[genkit._ai._agents._session.StateT], genkit._ai._agents._session.SnapshotSubscriber, typing.Generic[genkit._ai._agents._session.StateT]
File-backed snapshot store: one <snapshot_id>.json per snapshot.
Source code in packages/genkit/src/genkit/_ai/_agents/_session_stores/_file_store.py
49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 | |
lock
property
Return a loop-local asyncio.Lock for this store instance.
__init__(directory, *, reject_ambiguous_session=False, max_persisted_chain_length=None)
Create the store, ensuring directory exists.
See :class:InMemorySessionStore for reject_ambiguous_session.
max_persisted_chain_length caps how many snapshots of a chat's
history stay on disk: once a chain grows past it, the oldest turns are
deleted on each save so a long-lived conversation doesn't accumulate
files forever. Resuming and continuing still work, but you lose the
ability to rewind or branch past the retained window. Leave it unset to
keep the full history.
Source code in packages/genkit/src/genkit/_ai/_agents/_session_stores/_file_store.py
52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 | |
delete_sync(snapshot_id)
Delete a snapshot file, tolerating one that's already gone.
Source code in packages/genkit/src/genkit/_ai/_agents/_session_stores/_file_store.py
97 98 99 100 101 102 | |
get_snapshot(*, snapshot_id=None, session_id=None, context=None)
async
Return a snapshot by id, or the session's latest leaf, read from disk.
Source code in packages/genkit/src/genkit/_ai/_agents/_session_stores/_file_store.py
144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 | |
on_snapshot_status_change(snapshot_id, *, context=None)
async
Return an iterator that yields this snapshot's status changes.
Call aclose() (or contextlib.aclosing) to unsubscribe early;
natural end after a terminal status also unsubscribes.
Source code in packages/genkit/src/genkit/_ai/_agents/_session_stores/_file_store.py
187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 | |
path(snapshot_id)
Return the file path for a snapshot ID.
Source code in packages/genkit/src/genkit/_ai/_agents/_session_stores/_file_store.py
76 77 78 79 | |
prune_chain_sync(leaf)
Trim a chat's ancestry to the newest max_persisted_chain_length turns.
Walks parent_id back from the just-written snapshot and deletes the
oldest links past the cap. The retained oldest snapshot keeps pointing at
its (now-deleted) parent, so history reconstruction simply stops at the
window's edge.
Source code in packages/genkit/src/genkit/_ai/_agents/_session_stores/_file_store.py
104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 | |
read_session_sync(session_id)
Read all snapshots for a session from disk synchronously.
Source code in packages/genkit/src/genkit/_ai/_agents/_session_stores/_file_store.py
127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 | |
read_sync(snapshot_id)
Read and validate a snapshot from disk synchronously.
Source code in packages/genkit/src/genkit/_ai/_agents/_session_stores/_file_store.py
81 82 83 84 85 86 87 | |
save_snapshot(snapshot_id, fn, *, context=None)
async
Read-modify-write a snapshot on disk, prune the chain, and notify subscribers.
Source code in packages/genkit/src/genkit/_ai/_agents/_session_stores/_file_store.py
161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 | |
write_sync(snapshot)
Atomically write a snapshot to disk synchronously.
Source code in packages/genkit/src/genkit/_ai/_agents/_session_stores/_file_store.py
89 90 91 92 93 94 95 | |
HttpAgentTransport
Bases: genkit._ai._agents._client.AgentTransport[genkit._ai._agents._transports._http.StateT]
Client-side agent transport that talks to a remote agent over HTTP.
Source code in packages/genkit/src/genkit/_ai/_agents/_transports/_http.py
117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 | |
__init__(url, *, get_snapshot_url=None, abort_url=None, headers=None, state_management, timeout=DEFAULT_AGENT_TIMEOUT_SECONDS)
Initializes the HTTP transport.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
url
|
str
|
Agent turn endpoint (e.g. |
required |
get_snapshot_url
|
str | None
|
|
None
|
abort_url
|
str | None
|
|
None
|
headers
|
genkit._ai._agents._transports._http.HeadersProvider | None
|
Static headers, or a function called per request (sync or async). |
None
|
state_management
|
genkit._ai._agents._types.StateManagement
|
Declares server- vs client-managed state. |
required |
timeout
|
float | None
|
Seconds to wait for the next bytes from the server. On a
streamed turn this is the longest allowed gap between chunks,
not a cap on the whole turn. |
genkit._ai._agents._transports._http.DEFAULT_AGENT_TIMEOUT_SECONDS
|
Source code in packages/genkit/src/genkit/_ai/_agents/_transports/_http.py
120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 | |
abort_snapshot(snapshot_id)
async
Aborts the specified snapshot on the server.
Source code in packages/genkit/src/genkit/_ai/_agents/_transports/_http.py
310 311 312 313 314 315 | |
get_snapshot(*, snapshot_id=None, session_id=None)
async
Retrieves a session snapshot from the server.
Source code in packages/genkit/src/genkit/_ai/_agents/_transports/_http.py
295 296 297 298 299 300 301 302 303 304 305 306 307 308 | |
run_turn(*, agent_input, init)
async
Runs a single turn over HTTP using a streaming POST request.
Source code in packages/genkit/src/genkit/_ai/_agents/_transports/_http.py
206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 | |
InMemorySessionStore
Bases: genkit._ai._agents._session.SessionStoreLock, genkit._ai._agents._session.SessionStore[genkit._ai._agents._session.StateT], genkit._ai._agents._session.SnapshotSubscriber, typing.Generic[genkit._ai._agents._session.StateT]
In-memory snapshot store. State is lost when the process exits.
Source code in packages/genkit/src/genkit/_ai/_agents/_session_stores/_inmemory_store.py
62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 | |
lock
property
Return a loop-local asyncio.Lock for this store instance.
__init__(*, reject_ambiguous_session=False)
Create the store.
When reject_ambiguous_session is set, a session_id lookup on a
history that has forked (more than one leaf) raises instead of picking
the most recent branch — useful when accidental branching should surface
loudly.
Source code in packages/genkit/src/genkit/_ai/_agents/_session_stores/_inmemory_store.py
65 66 67 68 69 70 71 72 73 74 75 | |
get_snapshot(*, snapshot_id=None, session_id=None, context=None)
async
Return a snapshot by id, or the session's latest leaf, as a deep copy.
Source code in packages/genkit/src/genkit/_ai/_agents/_session_stores/_inmemory_store.py
77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 | |
on_snapshot_status_change(snapshot_id, *, context=None)
async
Return an iterator that yields this snapshot's status changes.
Call aclose() (or contextlib.aclosing) to unsubscribe early;
natural end after a terminal status also unsubscribes.
Source code in packages/genkit/src/genkit/_ai/_agents/_session_stores/_inmemory_store.py
118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 | |
save_snapshot(snapshot_id, fn, *, context=None)
async
Read-modify-write a snapshot in memory and notify status subscribers.
Source code in packages/genkit/src/genkit/_ai/_agents/_session_stores/_inmemory_store.py
96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 | |
JsonPatchOp
Bases: genkit._core._compat.StrEnum
JsonPatchOp data type class.
Source code in packages/genkit/src/genkit/_core/_typing.py
58 59 60 61 62 63 64 65 66 | |
JsonPatchOperation
Bases: genkit._core._base.GenkitModel
Model for jsonpatchoperation data.
Source code in packages/genkit/src/genkit/_core/_typing.py
242 243 244 245 246 247 248 249 250 251 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
Session
Bases: typing.Generic[genkit._ai._agents._session.StateT]
Holds conversation state with asyncio-safe read/write access.
Parameterize with a custom-state type when agents carry typed custom
blobs (Session[MyState]). Wire storage stays SessionState.custom.
version bumps on every mutation so the runtime can skip redundant
snapshot writes without deep-comparing state.
Source code in packages/genkit/src/genkit/_ai/_agents/_session.py
223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 | |
add_artifacts(artifacts)
async
Append artifacts; replace by name if artifact.name already exists.
Source code in packages/genkit/src/genkit/_ai/_agents/_session.py
298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 | |
on_artifact_changed(listener)
Register a callback invoked after add_artifacts mutates state.
Source code in packages/genkit/src/genkit/_ai/_agents/_session.py
251 252 253 | |
on_custom_changed(listener)
Register a callback invoked after update_custom mutates state.
Source code in packages/genkit/src/genkit/_ai/_agents/_session.py
247 248 249 | |
state()
async
Deep copy of current state.
Source code in packages/genkit/src/genkit/_ai/_agents/_session.py
263 264 265 266 | |
SessionRunner
Bases: typing.Generic[genkit._ai._agents._session.StateT]
Per-turn input loop for one agent invocation.
AgentFn calls session_runner.run(handle_turn); after each turn the runtime
emits snapshots and turnEnd chunks via on_end_turn.
Source code in packages/genkit/src/genkit/_ai/_agents/_runtime.py
102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 | |
result()
async
Last message, artifacts, and finish reason from the current session.
Source code in packages/genkit/src/genkit/_ai/_agents/_runtime.py
245 246 247 248 249 250 251 252 253 254 | |
run(fn)
async
Consume inputs from the intake queue, calling fn for each turn.
Each turn is wrapped in a trace span runTurn-N (1-based). Inbound
messages are automatically added to the session before fn is called.
When a store is configured the turn's snapshot id is reserved up front
and handed to fn via TurnContext so handlers can name external
resources before the turn runs; the snapshot at turn end reuses that id.
After fn returns, on_end_turn is called (snapshot + chunk emission)
and turn_index is incremented.
Turn failures resolve gracefully: the invocation finishes with
finish_reason=failed and error on AgentOutput, and the
turn loop stops.
Source code in packages/genkit/src/genkit/_ai/_agents/_runtime.py
144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 | |
seed_last_good_state()
async
Capture initial session state as the fallback for first-turn failures.
Source code in packages/genkit/src/genkit/_ai/_agents/_runtime.py
139 140 141 142 | |
SessionSnapshot
Bases: genkit._core._base.GenkitModel
Snapshot whose state uses the public SessionState type.
Source code in packages/genkit/src/genkit/_core/_model.py
1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
SessionState
Bases: genkit._core._base.GenkitModel
Session state whose conversation uses Message and Artifact.
Source code in packages/genkit/src/genkit/_core/_model.py
1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
SessionStore
Bases: typing.Protocol, typing.Generic[genkit._ai._agents._session.StateT_co]
Structural interface for snapshot persistence backends.
Minimum: get_snapshot + save_snapshot.
Optional detach/abort support: implement SnapshotSubscriber as well.
The StateT parameter names the custom-state shape a store round-trips,
so a typed store agrees with its agent's state_schema. It's a phantom
over the snapshot wire format (which stays plain JSON), so leaving it off
just defaults to Any.
Source code in packages/genkit/src/genkit/_ai/_agents/_session.py
87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 | |
get_snapshot(*, snapshot_id=None, session_id=None, context=None)
async
Retrieve a snapshot by id or the latest leaf for a session.
context is optional request side-channel data (e.g. auth) so
backends can isolate tenants; in-memory/file stores ignore it.
Source code in packages/genkit/src/genkit/_ai/_agents/_session.py
99 100 101 102 103 104 105 106 107 108 109 110 111 | |
save_snapshot(snapshot_id, fn, *, context=None)
async
Atomically read-modify-write a snapshot under snapshot_id.
fn receives the existing snapshot (or None for new) and returns the snapshot to persist, or None to skip. Stores may call fn more than once under contention, so it must not mutate external state; recording what it observed is fine as long as a retried call may overwrite the record (last call wins).
Callers reserve the id up front (reserve_snapshot_id) so it can be
known before the write — e.g. handed to a turn handler via
TurnContext. When no row exists yet, this creates under that id.
The store also fills created_at and defaults status when left empty.
context is optional request side-channel data (e.g. auth) so
backends can isolate tenants; in-memory/file stores ignore it.
Source code in packages/genkit/src/genkit/_ai/_agents/_session.py
113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 | |
SnapshotStatus
Bases: genkit._core._compat.StrEnum
SnapshotStatus data type class.
Source code in packages/genkit/src/genkit/_core/_typing.py
69 70 71 72 73 74 75 76 77 | |
SnapshotStatusStream
Bases: typing.Protocol
Async iterator of snapshot status changes.
Consume with async for. Call aclose() (or contextlib.aclosing)
to unsubscribe — Python does not tear subscriptions down on break alone.
Natural end after a terminal status also unsubscribes.
Source code in packages/genkit/src/genkit/_ai/_agents/_session.py
142 143 144 145 146 147 148 149 150 151 152 153 154 155 | |
SnapshotSubscriber
Bases: typing.Protocol
Optional capability that makes a store's snapshots abortable/detachable.
Aborting itself is just a save_snapshot that flips a pending snapshot to
aborted — there's no separate abort method. This is the other half: a way to
notice that flip (e.g. when a different request aborts a detached turn
that's still running) so the runtime can cancel the background work. A store
that can't signal status changes can't support detach.
Source code in packages/genkit/src/genkit/_ai/_agents/_session.py
158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 | |
on_snapshot_status_change(snapshot_id, *, context=None)
async
Async stream of the snapshot's persisted status changes.
Yields committed statuses such as pending, completed, failed, or aborted.
expired is computed on read when a pending heartbeat goes stale — it
is not written to the store and does not appear on this stream. Use
get_snapshot / resolve paths for that liveness check.
Iteration ends when the run resolves (the last yielded status is a
persisted terminal) or when the store is closed locally (ends without
one). Call aclose() (or wrap with contextlib.aclosing) to
unsubscribe early — break alone is not enough. context scopes
the subscription for multi-tenant stores; stores without tenancy accept
and ignore it.
Source code in packages/genkit/src/genkit/_ai/_agents/_session.py
169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 | |
TurnContext
dataclass
Per-turn context handed to a custom-agent handler before the turn runs.
snapshot_id is reserved at turn start (when a store is configured) and is
the id the snapshot persisted at turn end will reuse. That lets a handler
name external, snapshot-correlated resources — e.g. a worktree or scratch
directory — up front, then commit them under that id, so a later rollback to
the snapshot can restore the external state too. None when the agent has
no store (client-managed).
Source code in packages/genkit/src/genkit/_ai/_agents/_types.py
53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 | |
TurnEnd
Bases: genkit._core._base.GenkitModel
Model for turnend data.
Source code in packages/genkit/src/genkit/_core/_typing.py
285 286 287 288 289 290 291 292 293 | |
model_dump(**kwargs)
Dump model with Genkit defaults (by_alias=True, exclude_none=True).
Source code in packages/genkit/src/genkit/_core/_base.py
130 131 132 133 134 135 | |
model_dump_json(**kwargs)
Dump model to JSON with Genkit defaults.
Source code in packages/genkit/src/genkit/_core/_base.py
137 138 139 140 141 142 | |
TurnResult
dataclass
What an agent turn function returns to tell the loop how the turn ended.
Source code in packages/genkit/src/genkit/_ai/_agents/_types.py
70 71 72 73 74 75 | |
apply_json_patch(*, doc, patch)
Apply RFC 6902 JSON Patch operations to a document and return the transformed copy.
Lenient by design so a stream of deltas stays robust: an add/replace whose
parent container is missing initializes it, and a remove/replace of a missing
member is a no-op. test is honored (raises on mismatch); an unrecognized
op raises rather than silently letting client state drift.
Source code in packages/genkit/src/genkit/_ai/_json_patch.py
246 247 248 249 250 251 252 253 254 255 256 257 | |
apply_save(*, existing, snapshot_id, fn)
Run a save mutator and stamp the result under snapshot_id, or None to skip.
When existing is None this creates under the reserved id. Mutators that
only update (abort, heartbeat) return None when existing is missing.
Terminal statuses are absorbing: once a snapshot is completed, failed, or
aborted, a mutation may still rewrite its state or metadata, but changing
its status raises FAILED_PRECONDITION — an aborted run must never be
rewritten into a completed one.
Source code in packages/genkit/src/genkit/_ai/_agents/_session_stores/_util.py
140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 | |
diff_json(*, from_value, to_value)
Return add/remove/replace ops that transform from_value into to_value.
Source code in packages/genkit/src/genkit/_ai/_json_patch.py
59 60 61 62 63 | |
iterate_statuses(q, *, on_close=None)
Expose a status queue as a closable async stream.
Source code in packages/genkit/src/genkit/_ai/_agents/_session_stores/_util.py
250 251 252 253 254 255 256 | |
remote_agent(url, *, get_snapshot_url=None, abort_url=None, headers=None, state_management, state_schema=None, timeout=DEFAULT_AGENT_TIMEOUT_SECONDS)
Create a remote agent client over HTTP. timeout is as on :class:HttpAgentTransport.
Source code in packages/genkit/src/genkit/_ai/_agents/_transports/_http.py
331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 | |
require_one_selector(*, snapshot_id, session_id)
Enforce that a get_snapshot call names exactly one of snapshot_id / session_id.
Source code in packages/genkit/src/genkit/_ai/_agents/_session_stores/_util.py
87 88 89 | |
session_id_of(snapshot)
Session a snapshot belongs to, preferring the top-level id over state's.
Source code in packages/genkit/src/genkit/_ai/_agents/_session_stores/_util.py
80 81 82 83 84 | |