Skip to content

Readers

An IndexTransform defines which source value belongs at every result position. A Reader defines how a particular backend obtains those values. Readers do not define indexing semantics, partitioning, scheduling, or result ownership.

Reader.read_into(source, context, out) receives a ReadContext whose transform maps zero-origin output-buffer coordinates to global coordinates in source, with context.transform.domain.shape == out.shape. Its optional projection is the existing plan for a partitioned read. The projection's chunk_transform remains chunk-local, its cell_transform describes result placement, and its chunk_domain describes the grid cell. The global read transform and the projection's chunk transform deliberately use different coordinate frames.

An implementation must fill every cell of out in place, preserve the global transform's exact values, order, and dtype, and return None. It must neither replace nor retain out, which may be a strided writable view. Backend exceptions propagate unchanged. Derived part views share their reader and may be resolved concurrently, so a stateful reader owns its own synchronization.

Reader wrappers compose by intercepting this one operation and forwarding the same source, context, and output buffer to an inner reader:

class RecordingReader:
    def __init__(self, inner):
        self.inner = inner
        self.calls = []

    def read_into(self, source, context, out, /):
        self.calls.append((source, context, out))
        self.inner.read_into(source, context, out)


inner = RecordingReader(numpy_reader)
outer = RecordingReader(inner)
view = LazyArray.from_numpy(array).with_reader(outer)
values = view.result()

Both wrappers observe the same three objects, in outer-to-inner order. This delegation pattern supports policies such as logging and caching without library-defined wrapper primitives.

zarr_indexing.reader

Backend reader protocol and built-in system-memory implementations.

__all__ module-attribute 

__all__ = [
    "BasicReader",
    "NumPyReader",
    "ReadContext",
    "Reader",
    "UnitStepReader",
    "basic_reader",
    "numpy_reader",
    "unit_step_reader",
]

basic_reader module-attribute 

basic_reader: Final = BasicReader()

numpy_reader module-attribute 

numpy_reader: Final = NumPyReader()

unit_step_reader module-attribute 

unit_step_reader: Final = UnitStepReader()

BasicReader

Reader for system-memory sources exposing basic integer/slice indexing.

Each transform is decomposed into the smallest enclosing positive-slice slab and a residual transform. The slab is read once with basic indexing, so fancy or negative-step selections may over-read, and the residual is then lowered through NumPy system-memory operations into the supplied buffer.

Slice results must permit conversion to NumPy system memory. Device arrays that reject implicit conversion require a custom reader responsible for transferring values into the supplied system-memory output buffer.

Examples:

>>> transform = IndexTransform.from_shape((6,))[1:5:2]
>>> source = np.arange(6)
>>> out = np.empty(transform.domain.shape, dtype=source.dtype)
>>> BasicReader().read_into(source, ReadContext(transform), out)
>>> out.tolist() == source[1:5:2].tolist()
True
Source code in src/zarr_indexing/reader.py 
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
class BasicReader:
    """Reader for system-memory sources exposing basic integer/slice indexing.

    Each transform is decomposed into the smallest enclosing positive-slice
    slab and a residual transform. The slab is read once with basic indexing,
    so fancy or negative-step selections may over-read, and the residual is
    then lowered through NumPy system-memory operations into the supplied
    buffer.

    Slice results must permit conversion to NumPy system memory. Device arrays
    that reject implicit conversion require a custom reader responsible for
    transferring values into the supplied system-memory output buffer.

    Examples
    --------
    >>> transform = IndexTransform.from_shape((6,))[1:5:2]
    >>> source = np.arange(6)
    >>> out = np.empty(transform.domain.shape, dtype=source.dtype)
    >>> BasicReader().read_into(source, ReadContext(transform), out)
    >>> out.tolist() == source[1:5:2].tolist()
    True
    """

    __slots__ = ()

    def read_into(self, source: Any, context: ReadContext, out: Any, /) -> None:
        """Read one transform through a positive-slice slab and residual lowering."""
        transform = context.transform
        key, residual = _decompose_basic(transform)
        block = np.asanyarray(source[key])
        out[...] = _lower(block, residual)

__slots__ class-attribute instance-attribute 

__slots__ = ()

read_into 

read_into(
    source: Any, context: ReadContext, out: Any
) -> None

Read one transform through a positive-slice slab and residual lowering.

Source code in src/zarr_indexing/reader.py 
126
127
128
129
130
131
def read_into(self, source: Any, context: ReadContext, out: Any, /) -> None:
    """Read one transform through a positive-slice slab and residual lowering."""
    transform = context.transform
    key, residual = _decompose_basic(transform)
    block = np.asanyarray(source[key])
    out[...] = _lower(block, residual)

NumPyReader

Reader optimized for NumPy system-memory arrays.

This is the reader selected by LazyArray.from_numpy. It applies the complete transform with NumPy operations and is applicable to numpy.ndarray sources, including numpy.ma.MaskedArray.

Examples:

>>> transform = IndexTransform.from_shape((3, 4))[::2, 1:3]
>>> source = np.arange(12).reshape(3, 4)
>>> out = np.empty(transform.domain.shape, dtype=source.dtype)
>>> NumPyReader().read_into(source, ReadContext(transform), out)
>>> out.tolist()
[[1, 2], [9, 10]]
Source code in src/zarr_indexing/reader.py 
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
class NumPyReader:
    """Reader optimized for NumPy system-memory arrays.

    This is the reader selected by
    [`LazyArray.from_numpy`][zarr_indexing.lazy_array.LazyArray.from_numpy]. It
    applies the complete transform with NumPy operations and is applicable to
    `numpy.ndarray` sources, including `numpy.ma.MaskedArray`.

    Examples
    --------
    >>> transform = IndexTransform.from_shape((3, 4))[::2, 1:3]
    >>> source = np.arange(12).reshape(3, 4)
    >>> out = np.empty(transform.domain.shape, dtype=source.dtype)
    >>> NumPyReader().read_into(source, ReadContext(transform), out)
    >>> out.tolist()
    [[1, 2], [9, 10]]
    """

    __slots__ = ()

    def read_into(self, source: Any, context: ReadContext, out: Any, /) -> None:
        """Read one transform through a narrowed slab into `out`."""
        transform = context.transform
        key, residual = _decompose_basic(transform)
        block = np.asanyarray(source[key])
        out[...] = _lower(block, residual)

__slots__ class-attribute instance-attribute 

__slots__ = ()

read_into 

read_into(
    source: Any, context: ReadContext, out: Any
) -> None

Read one transform through a narrowed slab into out.

Source code in src/zarr_indexing/reader.py 
154
155
156
157
158
159
def read_into(self, source: Any, context: ReadContext, out: Any, /) -> None:
    """Read one transform through a narrowed slab into `out`."""
    transform = context.transform
    key, residual = _decompose_basic(transform)
    block = np.asanyarray(source[key])
    out[...] = _lower(block, residual)

ReadContext dataclass

A source-global transform and optional projection for a partitioned read.

Examples:

>>> transform = IndexTransform.from_shape((6,))[1:5:2]
>>> context = ReadContext(transform)
>>> context.transform.domain.shape
(2,)
>>> context.projection is None
True
Source code in src/zarr_indexing/reader.py 
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
@dataclass(frozen=True, slots=True)
class ReadContext:
    """A source-global transform and optional projection for a partitioned read.

    Examples
    --------
    >>> transform = IndexTransform.from_shape((6,))[1:5:2]
    >>> context = ReadContext(transform)
    >>> context.transform.domain.shape
    (2,)
    >>> context.projection is None
    True
    """

    transform: IndexTransform
    """Maps zero-origin output-buffer coordinates to global coordinates in the source."""

    projection: ChunkProjection | None = None
    """The partition plan when this read is one part of a partitioned view, else `None`."""

projection class-attribute instance-attribute 

projection: ChunkProjection | None = None

The partition plan when this read is one part of a partitioned view, else None.

transform instance-attribute 

transform: IndexTransform

Maps zero-origin output-buffer coordinates to global coordinates in the source.

__init__ 

__init__(
    transform: IndexTransform,
    projection: ChunkProjection | None = None,
) -> None

Reader

Bases: Protocol

Backend adapter that fills supplied system-memory result buffers.

A reader may be shared by every view and part derived from one LazyArray. Part reads may run concurrently, so a stateful implementation must synchronize its own mutable state. LazyArray deliberately adds no serialization.

Examples:

The protocol is not runtime_checkable; an object satisfies it by exposing a conforming read_into, as basic_reader does:

>>> transform = IndexTransform.from_shape((6,))[1:5:2]
>>> source = np.arange(6)
>>> out = np.empty(transform.domain.shape, dtype=source.dtype)
>>> basic_reader.read_into(source, ReadContext(transform), out)
>>> out.tolist()
[1, 3]
Source code in src/zarr_indexing/reader.py 
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
class Reader(Protocol):
    """Backend adapter that fills supplied system-memory result buffers.

    A reader may be shared by every view and part derived from one
    [`LazyArray`][zarr_indexing.lazy_array.LazyArray]. Part reads may run
    concurrently, so a stateful implementation must synchronize its own
    mutable state. `LazyArray` deliberately adds no serialization.

    Examples
    --------
    The protocol is not `runtime_checkable`; an object satisfies it by
    exposing a conforming `read_into`, as `basic_reader` does:

    >>> transform = IndexTransform.from_shape((6,))[1:5:2]
    >>> source = np.arange(6)
    >>> out = np.empty(transform.domain.shape, dtype=source.dtype)
    >>> basic_reader.read_into(source, ReadContext(transform), out)
    >>> out.tolist()
    [1, 3]
    """

    def read_into(
        self,
        source: Any,
        context: ReadContext,
        out: np.ndarray[Any, Any],
        /,
    ) -> None:
        """Fill `out` with the exact source values selected by `context`.

        `context.transform` maps zero-origin coordinates in the output buffer
        to global coordinates in `source`, and its domain shape equals
        `out.shape`. `context.projection`, when present, is the corresponding
        partition plan: its `chunk_transform` is chunk-local, its
        `cell_transform` describes result placement, and its `chunk_domain`
        describes the grid cell. Fill every cell in place, preserving the
        transform's exact values, order, and dtype, then return `None`. Do not
        replace or retain `out`; it may be a strided writable view rather than
        an owning array.

        Backend exceptions propagate unchanged. Because callers may resolve
        parts concurrently through the same reader object, stateful readers
        are responsible for synchronizing their own state.
        """
        ...

read_into 

read_into(
    source: Any,
    context: ReadContext,
    out: ndarray[Any, Any],
) -> None

Fill out with the exact source values selected by context.

context.transform maps zero-origin coordinates in the output buffer to global coordinates in source, and its domain shape equals out.shape. context.projection, when present, is the corresponding partition plan: its chunk_transform is chunk-local, its cell_transform describes result placement, and its chunk_domain describes the grid cell. Fill every cell in place, preserving the transform's exact values, order, and dtype, then return None. Do not replace or retain out; it may be a strided writable view rather than an owning array.

Backend exceptions propagate unchanged. Because callers may resolve parts concurrently through the same reader object, stateful readers are responsible for synchronizing their own state.

Source code in src/zarr_indexing/reader.py 
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
def read_into(
    self,
    source: Any,
    context: ReadContext,
    out: np.ndarray[Any, Any],
    /,
) -> None:
    """Fill `out` with the exact source values selected by `context`.

    `context.transform` maps zero-origin coordinates in the output buffer
    to global coordinates in `source`, and its domain shape equals
    `out.shape`. `context.projection`, when present, is the corresponding
    partition plan: its `chunk_transform` is chunk-local, its
    `cell_transform` describes result placement, and its `chunk_domain`
    describes the grid cell. Fill every cell in place, preserving the
    transform's exact values, order, and dtype, then return `None`. Do not
    replace or retain `out`; it may be a strided writable view rather than
    an owning array.

    Backend exceptions propagate unchanged. Because callers may resolve
    parts concurrently through the same reader object, stateful readers
    are responsible for synchronizing their own state.
    """
    ...

UnitStepReader

Reader for sources whose basic indexing accepts only step-1 slices.

Each transform is decomposed into the smallest enclosing ascending unit-step slab and a residual transform, so the source only ever receives slice(start, stop, 1) on every axis — the one form an API without general strided reads (an FFI binding, an HTTP range endpoint) supports. BasicReader instead pushes strided and reversed slices down, which reads less but asks more of the source.

The residual lowering applies strides, reversals, and gathers to the in-memory block, so a strided selection over-reads its cover by the stride factor. Partitioning the wrapping LazyArray (with_parts) bounds each cover by a part.

Slice results must permit conversion to NumPy system memory, exactly as for BasicReader.

Examples:

The source below is only ever asked for step-1 slices — here the cover slice(1, 4, 1) — and the stride is replayed against the block:

>>> transform = IndexTransform.from_shape((6,))[1:5:2]
>>> source = np.arange(6)
>>> out = np.empty(transform.domain.shape, dtype=source.dtype)
>>> UnitStepReader().read_into(source, ReadContext(transform), out)
>>> out.tolist()
[1, 3]
Source code in src/zarr_indexing/reader.py 
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
class UnitStepReader:
    """Reader for sources whose basic indexing accepts only step-1 slices.

    Each transform is decomposed into the smallest enclosing ascending
    unit-step slab and a residual transform, so the source only ever receives
    `slice(start, stop, 1)` on every axis — the one form an API without
    general strided reads (an FFI binding, an HTTP range endpoint) supports.
    `BasicReader` instead pushes strided and reversed slices down, which
    reads less but asks more of the source.

    The residual lowering applies strides, reversals, and gathers to the
    in-memory block, so a strided selection over-reads its cover by the
    stride factor. Partitioning the wrapping
    [`LazyArray`][zarr_indexing.lazy_array.LazyArray] (`with_parts`) bounds
    each cover by a part.

    Slice results must permit conversion to NumPy system memory, exactly as
    for `BasicReader`.

    Examples
    --------
    The source below is only ever asked for step-1 slices — here the cover
    `slice(1, 4, 1)` — and the stride is replayed against the block:

    >>> transform = IndexTransform.from_shape((6,))[1:5:2]
    >>> source = np.arange(6)
    >>> out = np.empty(transform.domain.shape, dtype=source.dtype)
    >>> UnitStepReader().read_into(source, ReadContext(transform), out)
    >>> out.tolist()
    [1, 3]
    """

    __slots__ = ()

    def read_into(self, source: Any, context: ReadContext, out: Any, /) -> None:
        """Read one transform through an ascending unit-step slab into `out`."""
        transform = context.transform
        key, residual = _decompose_unit_step(transform)
        block = np.asanyarray(source[key])
        out[...] = _lower(block, residual)

__slots__ class-attribute instance-attribute 

__slots__ = ()

read_into 

read_into(
    source: Any, context: ReadContext, out: Any
) -> None

Read one transform through an ascending unit-step slab into out.

Source code in src/zarr_indexing/reader.py 
196
197
198
199
200
201
def read_into(self, source: Any, context: ReadContext, out: Any, /) -> None:
    """Read one transform through an ascending unit-step slab into `out`."""
    transform = context.transform
    key, residual = _decompose_unit_step(transform)
    block = np.asanyarray(source[key])
    out[...] = _lower(block, residual)