Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
a745df8
Moved Core to the Direct Stream reads and appends.
moetemp Nov 15, 2026
436ad1a
Moved Core to the Stream WIT with the Workflow `iter` read.
moetemp Nov 15, 2026
d3788de
Pinned the Stream generator to nexgen with sync helpers and regenerated.
moetemp Nov 15, 2026
f16ae14
Gave Core the Stream runtime of each Store on the bridge.
moetemp Nov 15, 2026
3ae1427
Added the Workflow `iter` read, which Core runs in the Workflow Task.
moetemp Nov 15, 2026
efbe301
Made Core append a Workflow's Records inside the Workflow Task.
moetemp Nov 15, 2026
c2655d1
Let a replay with no Stream config start a read that History holds.
moetemp Nov 15, 2026
8198e0d
Tested the Direct reads and appends of a Stream in Workflow code.
moetemp Nov 15, 2026
5e2ef66
Documented the Direct reads and appends and their costs.
moetemp Nov 15, 2026
bca1c8c
Moved Core to the fixes of the Direct Stream calls.
moetemp Nov 15, 2026
52d4d85
Documented that a cap on the last Record of a closed Stream gives Done.
moetemp Nov 15, 2026
4c4dda3
Tested failed replays, the capped Done and a run's end with a read open.
moetemp Nov 15, 2026
82caea0
Replayed Direct calls whose ref names no Store with no Stream config.
moetemp Nov 15, 2026
89d4cdb
Woke every task that waits on a read, and kept a bad body readable.
moetemp Nov 15, 2026
02ad6a3
Ran the codec on the Stream bodies of Direct calls with no context.
moetemp Nov 15, 2026
2068b11
Gave Core the Store runtimes only from a Worker with Workflows.
moetemp Nov 15, 2026
4722987
Tested the fixes of the review, and wrapped three long lines.
moetemp Nov 15, 2026
82487dd
Moved Core to its fix of the transition coverage report.
moetemp Nov 15, 2026
2b4cf54
Fitted the Redis tests to the Direct append and tested iter on Redis.
moetemp Nov 15, 2026
8b48c58
Raised a failed read's error at every step of its loop.
moetemp Nov 15, 2026
bdc5fb2
Said that a Store isn't meant to be written in Python.
moetemp Nov 15, 2026
6c76edc
Moved Core to the Direct close of a Stream.
moetemp Nov 15, 2026
8849be4
Made Core close a Workflow's Stream after the task that asked for it.
moetemp Nov 15, 2026
b4fed08
Documented the Direct close in two Workflow Tasks.
moetemp Nov 15, 2026
9dab5da
Tested the Direct close of a Stream in Workflow code.
moetemp Nov 15, 2026
d922e85
Wrapped two lines of the Streams README.
moetemp Nov 15, 2026
95c7cf3
Moved Core to its Stream tests that wait for events.
moetemp Nov 15, 2026
1738876
Gave a read of Workflow code no idle bound when `idle_timeout` is None.
moetemp Nov 16, 2026
20d27ae
Documented that a read of Workflow code waits for the Stream's close.
moetemp Nov 16, 2026
2f4d5ab
Tested a read with no idle bound, and a zero idle timeout.
moetemp Nov 16, 2026
48196e0
Moved Core to a read that looks at the Store before its timeout.
moetemp Nov 16, 2026
edab085
Moved Core to its Stream metrics and the shutdown wait for a close.
moetemp Nov 16, 2026
c2720e0
Documented the Stream metrics and the shutdown wait for a close.
moetemp Nov 16, 2026
4ba8729
Tested the Stream metrics and a shutdown with a close in flight.
moetemp Nov 16, 2026
1cd1d9c
Pinned the nexgen fork commit that has the helper method in one PR.
moetemp Nov 16, 2026
6922639
Moved Core to the read-ahead option and the smaller Stream marker.
moetemp Nov 16, 2026
947a8fe
Added `StreamConfig.workflow_read_ahead_bytes` for Workflow reads.
moetemp Nov 16, 2026
9902b3b
Showed a decision in Workflow code as the example of `iter`.
moetemp Nov 16, 2026
0f9f085
Tested the tool-call example of `iter` and the read-ahead bound.
moetemp Nov 16, 2026
ded2c29
Moved Core to the API's `WorkflowProducer` comment.
moetemp Nov 16, 2026
f1a3638
Moved Core to its review fixes of the Direct Stream calls.
moetemp Nov 16, 2026
a71f077
Added a read batch window and a Workflow failure for lost Records.
moetemp Nov 16, 2026
0594415
Documented the idle default of a read and Core's review fixes.
moetemp Nov 16, 2026
d0bfe02
Tested the batch window and a Workflow failure for lost Records.
moetemp Nov 16, 2026
7325df1
Moved Core to the Stream WIT with one read for both targets.
moetemp Nov 17, 2026
e29e33c
Said that a close on a Python Store runs in a Local Activity.
moetemp Nov 17, 2026
f66743f
Fitted the close tests to the Local Activity close of the base branch.
moetemp Nov 17, 2026
3129f1f
Pinned the nexgen fork commit with the Workflow poll iterators.
moetemp Nov 17, 2026
ec7303b
Served each Workflow read call from Core's long-lived read.
moetemp Nov 17, 2026
eba8804
Documented the Workflow read with `iter_read_items`.
moetemp Nov 17, 2026
5fe21e2
Fixed the type checks of the Workflow read and its tests.
moetemp Nov 17, 2026
5bdbd90
Tested a decision that depends on the activations of a read.
moetemp Nov 17, 2026
b693a58
Wrote the review's example with a counter task for each Record.
moetemp Nov 17, 2026
7e9dec3
Added tests for taking one Stream page in several activations.
moetemp Nov 17, 2026
844a42d
Moved Core to the retry of a Store outage for Direct calls.
moetemp Nov 17, 2026
ac0e580
Tested Direct calls and a replay through a short Store outage.
moetemp Nov 17, 2026
7125279
Documented the retry of a Store outage for Direct calls.
moetemp Nov 17, 2026
dd5af8a
Moved Core to Direct reads with no batch window.
moetemp Nov 17, 2026
b6df792
Tested the 256 Record page cap and the refused long offset.
moetemp Nov 17, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -228,8 +228,9 @@ jobs:
- run: uv tool install poethepoet
- run: uv sync --all-extras
- run: cargo install --locked nexgen --version 0.2.8 --features advanced --force
# The Stream package needs the `main` of nexgen, which no release has yet.
- run: cargo install --locked --git https://github.com/temporalio/nexgen --rev 9653a0f5a845df0c5059554f1b5813db6a74cd1a --features advanced --root "$RUNNER_TEMP/nexgen-stream" nexgen
# The Stream package needs `@nexus.helper` with `sync=true`, which only a nexgen fork has.
# This commit of the fork also has the poll iterators of the Workflow target.
- run: cargo install --locked --git https://github.com/moetemp/nexgen --rev 2785462ecdaba5da7a27051fbaaccbaee26ae9e0 --features advanced --root "$RUNNER_TEMP/nexgen-stream" nexgen
- run: echo "NEX_GEN_STREAM_BIN=$RUNNER_TEMP/nexgen-stream/bin/nexgen" >> "$GITHUB_ENV"
- run: poe build-develop
- run: poe gen-protos
Expand Down
7 changes: 7 additions & 0 deletions changelog/added/patient-river-heron.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
Add `read`, `iter_read` and `iter_read_items` to the Workflow `Stream`, so Workflow code reads a Stream with the same calls as a client. Core reads the Store inside the Workflow Task. A page that Workflow code gets in one activation has at most 256 Records, and the rest come in the next activation with no wait. A loop that asks for its next page in the same activation keeps one read open, and a read that no code waits on closes before the activation completes, so it never holds a Workflow Task open. With a `wait`, a quiet Stream gives empty pages, so a loop on a Stream that can be quiet for long uses `iter_read` and leaves after an empty page. A call that waits on a quiet Stream holds its Workflow Task open, with one Workflow Task about every 8 s. The calls of one Workflow Task write one marker in History. `StreamConfig.workflow_read_replay_from` decides whether the marker holds the Records (`StreamReplaySource.HISTORY`) or only their offsets and a digest, which a replay reads from the Store again (`StreamReplaySource.STORE`, the default). With `HISTORY`, one Workflow Task's marker holds at most 1 MiB of Records, and the read goes on in the next task.
A Workflow's append on the Redis Store or `MemoryStreamStore` now runs in Core inside the Workflow Task, with no Local Activity. It writes as the Workflow, with its Workflow Task as the epoch.
A Workflow's close on the Redis Store or `MemoryStreamStore` now runs in Core, with no Activity, and spans two Workflow Tasks. The task that calls `close` records the intent, Core closes the Stream after the server took that task, and the next task gives the result, so a failed task closes nothing. Await the close: a graceful shutdown of the Worker waits at most 10 s for a close that the code doesn't await, and a Worker that dies when the Workflow ends loses it. An append and a close of Workflow code always carry the Workflow Task as their epoch. An append is at-least-once: after a Worker crash or an eviction, the rerun of its task can write a second copy with the epoch of the new task.
Core records metrics for the Direct calls of a Stream, such as `temporal_stream_read_total`, `temporal_stream_read_records` and `temporal_stream_close_latency`, each with the attribute `stream_store`, and `temporal_stream_close_abandoned` for a close that a graceful shutdown gave up.
Add `StreamConfig.workflow_read_ahead_bytes`, the most bytes of one read's Records that wait in the Worker for Workflow code, 4 MiB by default.
Add `temporalio.exceptions.StreamReplayError`. With it in `workflow_failure_exception_types`, a replay that can't read the Records of a read from the Store again fails the Workflow, and not only the Workflow Task.
Core tries a read, an append or a close of Workflow code again while its Store is unavailable, from 100 ms up to 2 s apart, for at most 30 s. Only then does Workflow code get the retryable `StreamError`. A replay tries its Store for 5 s. If the Store stays down, only the Workflow Task fails, with `StreamReplayUnavailable`, also with `StreamReplayError` in `workflow_failure_exception_types`.
9 changes: 6 additions & 3 deletions scripts/gen_nexus_system_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,10 @@
workflow_init_path = base_dir / "temporalio" / "workflow" / "__init__.py"
NEX_GEN_VERSION = "0.2.8"
# The Stream package needs a newer nexgen, which rejects the other WITs. One pin is enough
# again once the api WITs and a nexgen release agree. No release has nexgen commit 9653a0f5,
# the `main` of nexgen, yet, so NEX_GEN_STREAM_BIN points at a build of it.
# again once the api WITs and a nexgen release agree. The Stream WIT uses `@nexus.helper` with
# `sync=true` and `@nexus.support-type`, which only the fork https://github.com/moetemp/nexgen
# has. Its commit 2785462e also has the poll iterators of the Workflow target. So
# NEX_GEN_STREAM_BIN points at a build of that commit.
NEX_GEN_STREAM_VERSION = "0.2.12"
proto_files = [
gen_protos.api_proto_dir
Expand Down Expand Up @@ -87,7 +89,8 @@ def nex_gen_stream_command() -> list[str]:
bin_path = os.environ.get("NEX_GEN_STREAM_BIN")
if not bin_path:
raise RuntimeError(
"NEX_GEN_STREAM_BIN must point at a nexgen build of commit 9653a0f5"
"NEX_GEN_STREAM_BIN must point at a nexgen build of commit 2785462e "
"of https://github.com/moetemp/nexgen"
)
version = subprocess.check_output([bin_path, "--version"], text=True).strip()
if version != f"nexgen {NEX_GEN_STREAM_VERSION}":
Expand Down
8 changes: 5 additions & 3 deletions temporalio/api/stream/v1/message_pb2.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -167,8 +167,8 @@ global___ActivityEpoch = ActivityEpoch
@typing.final
class WorkflowProducer(google.protobuf.message.Message):
"""Experimental. Might change in the future.
A Workflow run Producer. On the server's Streams, a Workflow writes as this Producer in its
Workflow Task. On a Store, Workflow code writes through a Local Activity, so its writes carry a
A Workflow run Producer. Workflow code that writes in its Workflow Task writes as this
Producer, with that task as the epoch. A write that it makes through a Local Activity has a
Local Activity Producer.
"""

Expand All @@ -180,7 +180,9 @@ class WorkflowProducer(google.protobuf.message.Message):
"""Required. The run id of the Workflow run, not of its chain. Each run is its own Producer."""
@property
def epoch(self) -> global___WorkflowTaskEpoch:
"""The Workflow Task of the write."""
"""The Workflow Task of the write. Workflow code leaves it empty, and the SDK sets it before
the write goes to the Store.
"""

def __init__(
self,
Expand Down
1 change: 1 addition & 0 deletions temporalio/bridge/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

59 changes: 53 additions & 6 deletions temporalio/bridge/_visitor.py
Original file line number Diff line number Diff line change
Expand Up @@ -308,6 +308,38 @@ async def _visit_coresdk_workflow_activation_ResolveNexusOperation(
if o.HasField("result"):
await self._visit_coresdk_nexus_NexusOperationResult(fs, o.result)

async def _visit_temporal_api_stream_v1_StreamRecord(
self, fs: VisitorFunctions, o: Any
):
if o.HasField("body"):
await self._visit_temporal_api_common_v1_Payload(fs, o.body)

async def _visit_coresdk_workflow_activation_StreamReadEnd(
self, fs: VisitorFunctions, o: Any
):
if o.HasField("failure"):
await self._visit_temporal_api_failure_v1_Failure(fs, o.failure)

async def _visit_coresdk_workflow_activation_ResolveStreamRead(
self, fs: VisitorFunctions, o: Any
):
for v in o.records:
await self._visit_temporal_api_stream_v1_StreamRecord(fs, v)
if o.HasField("end"):
await self._visit_coresdk_workflow_activation_StreamReadEnd(fs, o.end)

async def _visit_coresdk_workflow_activation_ResolveStreamAppend(
self, fs: VisitorFunctions, o: Any
):
if o.HasField("failed"):
await self._visit_temporal_api_failure_v1_Failure(fs, o.failed)

async def _visit_coresdk_workflow_activation_ResolveStreamClose(
self, fs: VisitorFunctions, o: Any
):
if o.HasField("failed"):
await self._visit_temporal_api_failure_v1_Failure(fs, o.failed)

async def _visit_coresdk_workflow_activation_WorkflowActivationJob(
self, fs: VisitorFunctions, o: Any
):
Expand Down Expand Up @@ -353,6 +385,18 @@ async def _visit_coresdk_workflow_activation_WorkflowActivationJob(
await self._visit_coresdk_workflow_activation_ResolveNexusOperation(
fs, o.resolve_nexus_operation
)
elif o.HasField("resolve_stream_read"):
await self._visit_coresdk_workflow_activation_ResolveStreamRead(
fs, o.resolve_stream_read
)
elif o.HasField("resolve_stream_append"):
await self._visit_coresdk_workflow_activation_ResolveStreamAppend(
fs, o.resolve_stream_append
)
elif o.HasField("resolve_stream_close"):
await self._visit_coresdk_workflow_activation_ResolveStreamClose(
fs, o.resolve_stream_close
)

async def _visit_coresdk_workflow_activation_WorkflowActivation(
self, fs: VisitorFunctions, o: Any
Expand Down Expand Up @@ -492,6 +536,11 @@ async def _visit_coresdk_workflow_commands_ScheduleNexusOperation(
if o.HasField("input"):
await self._visit_nexus_operation_input_payload(fs, o.input)

async def _visit_coresdk_workflow_commands_AppendToStream(
self, fs: VisitorFunctions, o: Any
):
await self._visit_payload_container(fs, o.bodies)

async def _visit_coresdk_workflow_commands_WorkflowCommand(
self, fs: VisitorFunctions, o: Any
):
Expand Down Expand Up @@ -551,6 +600,10 @@ async def _visit_coresdk_workflow_commands_WorkflowCommand(
await self._visit_coresdk_workflow_commands_ScheduleNexusOperation(
fs, o.schedule_nexus_operation
)
elif o.HasField("append_to_stream"):
await self._visit_coresdk_workflow_commands_AppendToStream(
fs, o.append_to_stream
)

async def _visit_coresdk_workflow_completion_Success(
self, fs: VisitorFunctions, o: Any
Expand Down Expand Up @@ -681,12 +734,6 @@ async def _visit_temporal_api_streamservice_v1_ReadRecordsRequest(
) -> None:
pass

async def _visit_temporal_api_stream_v1_StreamRecord(
self, fs: VisitorFunctions, o: Any
):
if o.HasField("body"):
await self._visit_temporal_api_common_v1_Payload(fs, o.body)

async def _visit_temporal_api_streamservice_v1_ReadRecordsResponse(
self, fs: VisitorFunctions, o: Any
):
Expand Down
2 changes: 2 additions & 0 deletions temporalio/bridge/proto/common/__init__.py
Original file line number Diff line number Diff line change
@@ -1,13 +1,15 @@
from .common_pb2 import (
ExternalStorageMetrics,
NamespacedWorkflowExecution,
StreamReadStop,
VersioningIntent,
WorkerDeploymentVersion,
)

__all__ = [
"ExternalStorageMetrics",
"NamespacedWorkflowExecution",
"StreamReadStop",
"VersioningIntent",
"WorkerDeploymentVersion",
]
8 changes: 5 additions & 3 deletions temporalio/bridge/proto/common/common_pb2.py

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

39 changes: 39 additions & 0 deletions temporalio/bridge/proto/common/common_pb2.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,45 @@ else:

DESCRIPTOR: google.protobuf.descriptor.FileDescriptor

class _StreamReadStop:
ValueType = typing.NewType("ValueType", builtins.int)
V: typing_extensions.TypeAlias = ValueType

class _StreamReadStopEnumTypeWrapper(
google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper[
_StreamReadStop.ValueType
],
builtins.type,
):
DESCRIPTOR: google.protobuf.descriptor.EnumDescriptor
STREAM_READ_STOP_UNSPECIFIED: _StreamReadStop.ValueType # 0
STREAM_READ_STOP_DONE: _StreamReadStop.ValueType # 1
"""The Stream is closed, and the read gave its last Record."""
STREAM_READ_STOP_MAX_RECORDS: _StreamReadStop.ValueType # 2
"""The read gave `max_records` Records."""
STREAM_READ_STOP_MAX_BYTES: _StreamReadStop.ValueType # 3
"""The next Record would pass `max_bytes`."""
STREAM_READ_STOP_IDLE: _StreamReadStop.ValueType # 4
"""No Record came for `idle_timeout`."""
STREAM_READ_STOP_TIMEOUT: _StreamReadStop.ValueType # 5
"""`timeout` passed since the read started."""

class StreamReadStop(_StreamReadStop, metaclass=_StreamReadStopEnumTypeWrapper):
"""Why a read of a Stream ended. A read that failed has no stop."""

STREAM_READ_STOP_UNSPECIFIED: StreamReadStop.ValueType # 0
STREAM_READ_STOP_DONE: StreamReadStop.ValueType # 1
"""The Stream is closed, and the read gave its last Record."""
STREAM_READ_STOP_MAX_RECORDS: StreamReadStop.ValueType # 2
"""The read gave `max_records` Records."""
STREAM_READ_STOP_MAX_BYTES: StreamReadStop.ValueType # 3
"""The next Record would pass `max_bytes`."""
STREAM_READ_STOP_IDLE: StreamReadStop.ValueType # 4
"""No Record came for `idle_timeout`."""
STREAM_READ_STOP_TIMEOUT: StreamReadStop.ValueType # 5
"""`timeout` passed since the read started."""
global___StreamReadStop = StreamReadStop

class _VersioningIntent:
ValueType = typing.NewType("ValueType", builtins.int)
V: typing_extensions.TypeAlias = ValueType
Expand Down
15 changes: 14 additions & 1 deletion temporalio/bridge/proto/external_data/__init__.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,19 @@
from .external_data_pb2 import LocalActivityMarkerData, PatchedMarkerData
from .external_data_pb2 import (
LocalActivityMarkerData,
PatchedMarkerData,
StreamAppendEntry,
StreamCloseEntry,
StreamMarkerData,
StreamReadEnded,
StreamReadEntry,
)

__all__ = [
"LocalActivityMarkerData",
"PatchedMarkerData",
"StreamAppendEntry",
"StreamCloseEntry",
"StreamMarkerData",
"StreamReadEnded",
"StreamReadEntry",
]
Loading
Loading