◼ Apache Ignite Product Properties

Apache Ignite 2

Binary marshalling, binary builders and storage structures

Public product properties reconstructed from test evidence.

Binary marshalling, binary builders and storage structures

This page is a public product-spec reconstruction from Apache Ignite 2 test sources at revision 486a6367610c4c891b729e89373695ef83165345.

Binary-mode and storage-structure tests describe how Ignite preserves object shape, metadata, builder mutations, marshalling identity, generated message serializers, and ordered index internals visible through higher-level cache/query behavior.

User-facing guarantees

Traceable requirements

Only negative or ambiguous test status is annotated after a test name, for example status=failed, status=flaky, status=ignored, or status=mixed.

BIN-MARSHAL-SCALAR-ROUNDTRIP — binary marshalling preserves null

Requirement: Ignite binary marshalling preserves null, Java primitive wrapper values, UUID/IgniteUuid identifiers, SQL temporal values, and representative UTF-8 strings across marshal/unmarshal.

Why: Users rely on cache values, messages, and metadata-bearing objects to survive binary transport without changing identity-like values or basic data.

Boundary: The scalar tests use representative values, not full domains, locale-sensitive formatting, timezone conversion, or cross-version cluster compatibility. The string tests distinguish legacy and newer UTF-8 behavior but do not prove every malformed Unicode sequence.

Evidence tests:

BIN-MARSHAL-ARRAY-ROUNDTRIP — binary marshalling preserves arrays of primitive values

Requirement: Ignite binary marshalling preserves arrays of primitive values, common scalar object values, Object arrays, binary-object arrays, and repeated array references inside arrays or object fields.

Why: Distributed cache entries and internal messages frequently carry collections of values; array content and repeated references must survive binary transport for correct object graphs.

Boundary: The tests cover small arrays and selected element types. They do not establish large-array limits, null elements in every array kind, cyclic arrays beyond repeated references, or interoperability between binary-array compatibility modes.

Evidence tests:

BIN-MARSHAL-FIELD-ACCESS — Binary objects expose marshalled fields with their declared value and type

Requirement: Binary objects expose marshalled fields with their declared value and type, including nested objects, SQL temporal fields, decimal fields, and Class fields, while deserialization reconstructs equivalent user objects.

Why: Ignite clients and server internals inspect binary fields without full deserialization; field APIs must match the logical user object.

Boundary: The coverage is reflective and marshal-aware decimal classes plus one object with class fields. It does not prove arbitrary generic class metadata or schema evolution of field types.

Evidence tests:

BIN-MARSHAL-DETACHED-READS — Detached binary reads preserve map entries

Requirement: Detached binary reads preserve map entries, collection members, typed arrays, arrays of collections, and binary collection elements when binaries are unwrapped for platform interop.

Why: Platform integrations and binary APIs must detach nested structures without losing element identity or changing container shape.

Boundary: The tests use local marshalling helpers and fixed payload shapes. They do not prove cross-language serialization formats, null-heavy collections, ordering for unordered containers, or concurrent access.

Evidence tests:

BIN-MARSHAL-METADATA-EQUALITY — Binary object equality

Requirement: Binary object equality, duplicate field naming, and field-order metadata behave deterministically across marshaller configurations, heap/offheap forms, builders, Binarylizable objects, and metadata serialization.

Why: Binary objects are used as comparable keys, metadata carriers, and schema descriptors; equivalent logical objects must compare correctly and metadata order/name resolution must stay predictable.

Boundary: testFieldOrder and testFieldOrderByBuilder return early when sorted-field mode is enabled, so they prove insertion/declaration order only for unsorted-field configuration. Duplicate-field checks partly use Java assert, so runtime assertion settings matter for part of the proof.

Evidence tests:

BTREE-SIZE-AND-LOCK-INVARIANTS — B+Tree reports size and filtered size consistently with visible contents during sequential put/remove operations and lea

Requirement: Ignite B+Tree reports size and filtered size consistently with visible contents during sequential put/remove operations and leaves no page locks after operations.

Why: Storage indexes need reliable cardinality and lock cleanup so cache operations, scans, and maintenance code do not leak resources or miscount rows.

Boundary: Single local test tree only; no persistence restart, real cache index integration, or multi-node topology.

Evidence tests:

BTREE-CONCURRENT-SIZE-BOUNDS — During concurrent put/remove activity

Requirement: During concurrent put/remove activity, B+Tree size remains either exact at synchronization points or inside a bounded range when modification races are intentionally allowed; the tree validates and releases locks afterward.

Why: Operators and internal components need storage metrics and scans that remain sane under write load without livelock or leaked page locks.

Boundary: These are stress-style in-memory tests with timing and CPU-dependent interleavings. They prove bounded behavior for configured page capacities and loop counts, not linearizable size under all races.

Evidence tests:

BTREE-CONCURRENT-STRUCTURE-INVARIANTS — B+Tree structural maintenance remains valid while puts

Requirement: B+Tree structural maintenance remains valid while puts, removes, range removes, and iteration-like lookups interleave around split, merge, and empty-leaf edge cases.

Why: Storage indexes must survive high-churn workloads without leaving unreachable rows, invalid tree shape, or exceptions during page split/merge races.

Boundary: Some checks are exception/validation based rather than comparing a complete model of contents. Randomized and concurrent scheduling means they are not exhaustive proofs for every tree shape.

Evidence tests:

MSG-CODEGEN-SERIALIZER-SHAPE — Ignite’s message annotation processor generates expected serializer and

Requirement: Ignite’s message annotation processor generates expected serializer and, when needed, marshaller companions for supported Message shapes: simple messages, collections, maps, inherited messages, multiple messages, enum sets, marshallable messages, and JDK-marshalled messages.

Why: Communication message classes depend on generated companions for stable wire IO and object marshalling without hand-written boilerplate.

Boundary: Golden-source equivalence proves generated Java text for fixture classes, not runtime wire compatibility or all possible field type combinations.

Evidence tests:

MSG-CODEGEN-VALIDATION — message processor rejects unsupported or ambiguous message definitions at compile time

Requirement: The message processor rejects unsupported or ambiguous message definitions at compile time, including wrong @Order usage, matrices, POJO fields, Throwable fields, unsupported raw/generic shapes, explicit compressed-message internals, invalid compression annotations, and invalid @NioField/@Marshalled combinations.

Why: Failing invalid message schemas during compilation prevents broken wire protocols and runtime marshalling failures.

Boundary: Several negative tests assert only failure, and some assert diagnostic substrings. They do not prove complete diagnostic stability or every unsupported Java type.

Evidence tests:

MSG-CODEGEN-ENUM-MAPPERS — Enum mapping annotations are accepted only for enum fields or nested enum collections/maps

Requirement: Enum mapping annotations are accepted only for enum fields or nested enum collections/maps, one enum field cannot combine incompatible enum types, and a given enum type must use a consistent mapper across messages while reuse of the same mapper is allowed.

Why: Stable enum wire encoding requires one unambiguous mapper per enum type and clear rejection of mapper annotations that cannot apply.

Boundary: The tests cover TransactionIsolation and selected fixture structures only. They do not prove mapper behavior for every enum container nesting pattern.

Evidence tests:

MSG-CODEGEN-MARSHALLED-DEPLOYABLE — processor generates marshaller companions for @Marshalled object

Requirement: The processor generates marshaller companions for @Marshalled object, collection, map, and array-backed-map fields, and deployer companions for cache-aware deployable messages including nested cache messages and custom deployment hooks.

Why: Cache communication messages must correctly marshal user payloads and deploy binary/cache-object fields using available cache context hooks.

Boundary: Golden-source comparison checks generated source, not runtime deployment against live caches. The cache context failure is one fixture diagnostic.

Evidence tests:

MSG-QUERYINDEX-COPY — QueryIndexMessage preserves all non-static QueryIndex fields needed to reconstruct an equivalent QueryIndex

Requirement: QueryIndexMessage preserves all non-static QueryIndex fields needed to reconstruct an equivalent QueryIndex.

Why: Query index definitions must travel through internal cache/query messages without losing index name, indexed fields, type, or inline size.

Boundary: This is a shape guard plus one reconstruction case. It does not prove backward compatibility if QueryIndex gains new non-static fields.

Evidence tests:

A4-BIN-001 — Binary marshalling preserves exception diagnostics

Requirement: When Ignite binary-marshals a Throwable, the deserialized exception must retain a usable stack trace produced through the Java serialization path.

Why: Users and operators need exceptions stored or transported through Ignite to remain diagnosable after deserialization.

Boundary: Covers a simple runtime exception only, not causes, suppressed exceptions, custom exception fields, or cross-version transport.

Evidence tests:

A4-BIN-002 — Binary marshalling round-trips common collection and map implementations

Requirement: Ignite binary marshalling must preserve equality of standard Java collections/maps and selected custom collection implementations with their contents.

Why: Cache values and compute/job payloads commonly embed Java collection types and users expect logical contents to survive round-trip marshalling.

Boundary: Covers selected in-memory collection shapes only; no distributed cache, concurrent mutation, comparator compatibility, or very large collections. testMixedRawCollections only proves no exception for two marshaller configurations.

Evidence tests:

A4-BIN-003 — Binary marshalling preserves map entries and externalizable/nested values

Requirement: Standalone Map.Entry values and externalizable objects nested in enclosing objects must survive binary marshalling with their key/value or externalized state intact.

Why: User objects may contain map-entry views or externalizable payloads and should not lose meaning when cached or transferred.

Boundary: Does not prove mutable entry aliasing, custom Externalizable failure handling, or non-void class metadata generally.

Evidence tests:

A4-BIN-004 — Binary objects expose stable field access and deserialize to equivalent reflective/Binarylizable objects

Requirement: Binary-marshalled user objects must expose primitive, scalar, array, enum, collection, map, and nested object fields through BinaryObject.field while deserializing back to an equivalent object.

Why: Ignite binary mode lets users inspect selected fields without full deserialization and later recover the original object.

Boundary: Covers local marshalling with configured type metadata and selected classes; no schema evolution across releases, cross-language clients, or persistence restart.

Evidence tests:

A4-BIN-005 — Binary object builder can read, set, remove, and rebuild fields without losing object structure

Requirement: BinaryObjectBuilder must preserve existing data while applying field mutations, including null assignments, removals, empty-object rebuilds, repeated rebuilds, and cyclic references.

Why: Users rely on binary builders for partial updates without deserializing entire objects or corrupting object graphs.

Boundary: Covers local builder usage only; handle-to-collection equality is asserted rather than identity because the builder does not support preserving the same collection handle.

Evidence tests:

A4-BIN-006 — Binary builder default mappers preserve type IDs, hash codes, field values, and metadata for supported field types

Requirement: With full-name basic name/ID mapping and compact footer, binary objects built through the default builder API must compute configured type IDs, use BinaryArrayIdentityResolver hash semantics, retain supported scalar/array/object/container field values, and publish field metadata.

Why: Users need predictable type identity, equality/hash behavior, and field metadata when building binary objects dynamically.

Boundary: Type coverage is broad but local; it does not prove schema conflict resolution, cross-node metadata propagation, or non-default custom mapper combinations in this builder class.

Evidence tests:

A4-BIN-007 — Binary object copy applies selective field replacements across primitive and object fields

Requirement: Copying a BinaryObject through the builder must preserve unspecified fields and make supplied replacement fields visible both through binary field access and after deserialization.

Why: Users need reliable partial updates of binary values in cache operations and compute pipelines.

Boundary: Does not prove rejection of wrong replacement field types; the negative check for invalid type is commented out in source.

Evidence tests:

A4-BIN-008 — Binary marshalling honors custom serializers, custom ID/name mappers, and duplicate identity protections

Requirement: Binary marshalling must apply per-type serializers/mappers ahead of global defaults where configured, use default mapping where custom mappers are absent, detect duplicate type IDs or duplicate simple mapped names, and expose registered custom type descriptors consistently.

Why: Users need deterministic binary identity and serializer behavior to interoperate across nodes and avoid corrupting metadata.

Boundary: Several checks inspect internal BinaryContext maps and exact IDs; this is stronger for implementation compatibility than for public API wording. Duplicate-name/type negative tests cover selected collision shapes only.

Evidence tests:

A4-BIN-009 — Binary graph marshalling preserves object handles, detached objects, read/write replacement, proxy serialization, and offheap equality

Requirement: Binary marshalling must maintain object-graph semantics for cycles, repeated handles, detached binary objects, writeReplace/readResolve, dynamic proxies, cached deserialized values, offheap binary object equality/hash/field access, and thread-local buffer release.

Why: Ignite stores complex object graphs in heap and offheap forms; users need identity-sensitive graphs and resource cleanup not to regress.

Boundary: Mostly local heap/offheap unit fixtures; no persistence restart, cross-classloader, or remote node transport. testObjectContainingProxy does not actually assert a container object field despite the method name.

Evidence tests:

A4-BTREE-001 — B+Tree reports retry exhaustion as a checked failure

Requirement: The page-memory B+Tree must surface an IgniteCheckedException instead of silently succeeding when configured retry count is exhausted during inserts.

Why: Storage operations must not hide structural access failures; callers need a checked failure they can handle.

Boundary: Does not assert the exception message or exact retry counter behavior.

Evidence tests:

A4-BTREE-002 — B+Tree maintains search, removal, size, root-level, and page-lock invariants across insert/remove order matrices

Requirement: The page-memory B+Tree must support ordered put/remove sequences under small page capacities and varied put/remove directions, including range removals, while keeping lookup results, empty-tree size/root-level, structural validation, and page-lock cleanup correct.

Why: Cache indexes depend on B+Tree structural invariants regardless of insertion/removal order or small-page split/merge pressure.

Boundary: Internal test tree only; does not prove real cache/index integration, crash recovery, or persistence restart.

Evidence tests:

A4-BTREE-003 — B+Tree invoke/put/remove randomized operations preserve map-equivalent contents

Requirement: The B+Tree invoke, put, and remove operations must remain equivalent to a reference map under randomized local operation streams while preserving structural validity and releasing page locks.

Why: Index operations combine conditional updates and removes; data loss or duplicate keys would corrupt query/index behavior.

Boundary: Uses generated randomness and periodic content checks; no deterministic coverage of all operation orders.

Evidence tests:

A4-BTREE-004 — B+Tree handles massive concurrent put/remove batches without structural corruption

Requirement: Under multithreaded insert and removal pressure, B+Tree must end with expected size and ordered contents, validate structure, and not leak page locks.

Why: Concurrent index maintenance must remain consistent during high write/remove pressure.

Boundary: Stress-style local fixture with fixed key counts and page capacities; no transactional or persistence guarantees.

Evidence tests:

A4-BTREE-005 — B+Tree supports concurrent randomized put/remove/invoke plus range scans

Requirement: During concurrent put/remove/invoke operations, B+Tree range cursors must return keys within bounds in strictly increasing order with no duplicates, and final tree contents/size must match the reference map.

Why: Query/index scans must remain ordered and consistent while writes are happening.

Boundary: Uses per-key striped test locks for writes and stress timing; does not prove all production concurrency interleavings.

Evidence tests:

Important boundaries