Kongming HV

kongming is a library implementing operations on sparse binary hypervectors for cognitive computing applications.

While ergonomic APIs are accessible via Python module of kongming for better usability, the core engine is implemented in Rust for maximum efficiency.

See Hypervectors for an introduction to hyperdimensional computing and, more relevant to this package, the sparse binary representation and computation.

License

The Python source code, examples, and documentation in this repository are licensed under the MIT License.

The core engine distributed via PyPI (kongming-rs-hv) is proprietary.

Install

pip install kongming-rs-hv

See Installation for supported platforms and verification steps.

Published notebooks

See Notebook Platforms for all available notebooks and platform details.

Guides

Language Support

This documentation covers code snippets in multiple languages (if available) side by side.

• Python: bindings to the underlying Rust implementation (public kongming-rs-hv on PyPI);
• Go: canonical / reference implementation in proprietary package;
• Rust: parallel implementation, carefully maintained in feature parity;

Docs versioning

The documentation on yangzh.github.io/hv stays in lockstep with the latest kongming-rs-hv release on PyPI. Whatever you read there matches what pip install kongming-rs-hv gives you.

The main branch of this repository is the working head — it may describe APIs or examples that haven’t been released yet: if you browse the raw markdown on GitHub, expect it to occasionally be ahead of the published site.

Reference

The work was initially outlined in this arxiv paper, built on top of the work from many others, and here is the citation:

Yang, Zhonghao (2023). Cognitive modeling and learning with sparse binary hypervectors. arXiv:2310.18316v1 [cs.AI]

Feedback

Found a bug, have a question, or want to suggest an improvement? Open an issue on GitHub.

Concepts

The ideas behind this library, independent of any particular language binding. Start here for the mental model; jump to the API Reference for the concrete surface and practical usage.

Hypervectors

What is Hyperdimensional Computing?

Hyperdimensional computing (HDC) represents concepts as high-dimensional vectors (also called hypervectors) and manipulates them with algebraic operations, typically the dimension (of vectors) can be as high as thousands or millions.

The key insight is that random vectors in high-dimensional spaces are nearly orthogonal: giving each concept a unique, robust representation that tolerates potential ambiguity and interference, all without central orchestration.

In that sense, the traditional mantra of curse of dimensionality becomes the blessing of dimensionality.

Motivated readers should perform their own background research on this topic and make judgement of their own. There are quite a few introductory papers covering this topic.

Sparse Binary Representation

kongming specializes in sparse binary hypervectors. Each vector has a fixed, large number of dimensions (e.g., 65,536/64K or 1,048,576/1M), but only a very small fraction of them are “on” (set to 1). The sparsity is controlled by the Model configuration.

Furthermore, we focus on a special sparse binary configuration: SparseSegmented where each vector is divided into equal-sized segments, and exactly one bit is ON per segment. For example, each of the 8bit-model hypervectors will have total dimension of $2^{16}=65536$, divided into $256$ segments, where each segment of $256$ dimension will have one (and only one) ON bit.

Alternatively you can imagine each SparseSegmented hypervector as a list of phasors, where the offset of ON bit (within the host segment) represents the discretized phase.

In general, this unique constraint enables:

• Compact storage: only the offset of ON bit need to be stored, and we only need to store the bare entropy for the presentation;
• Efficient operations: Unlike neural nets, where weights are recorded in float-point numbers, binary operations can be performed very efficiently with modern memory / CPUs, and without the need of GPU for either float-point operations or matrix manipulations.

Similarity and distance measure

Two vectors are compared via overlap — the count of segments with the same ON bit offset. This is conceptually equivalent to a dimension-wise AND operation.

Naturally, a vector’s overlap with itself equals its cardinality $M$.

For a model with dimension $N$ and sparsity $s$, the expected overlap between two random vectors $A$ and $B$ is:

$$\text{E}[O(A,B)]=N{s}^2=1$$

Actually, the overlap (of random vectors) follows a Poisson distribution with $\lambda =1$.

The commonly-used distance measure (or dis-similar measure) for binary vectors is Hamming Distance, equivalent to a bitwise XOR operation. As we discussed (and proved) in the paper, the overlap and Hamming distance for sparse binary hypervectors are two sides of the same coin, with the following equation:

$$2\times O(A,B)+H(A,B)=2M$$

The closer two vectors in Hamming space, the more overlap they have.

Supported Models

A Model determines the total number of dimensions (width), how those dimensions are divided into segments (cardinality and sparsity), and therefore implies critical storage and compute characteristics.

Model properties

All model functions take a Model enum value and return the derived property:

How to Choose a Model

• MODEL_64K_8BIT: Fast prototyping, tiny memory footprint, and high performance (due to SIMD). Good for tests, experiments and production.
• MODEL_1M_10BIT: General-purpose, balances performance and storage.
• MODEL_16M_12BIT: General-purpose, for the adventurous.
• MODEL_256M_14BIT / MODEL_4G_16BIT: Very high capacity, not there yet.

In general, larger models provide more orthogonal space (lower collision probability) at the cost of more memory per vector.

Operators

kongming provides two core algebraic operations on sparse binary hypervectors.

Bind

Binding ($\otimes$) combines two vectors into a result that is dissimilar to both inputs. It is the multiplicative operation in the VSA algebra.

Mathematically

$$A\otimes B=B\otimes A\text{(commutative)}$$

$$(A\otimes B)\otimes C=A\otimes (B\otimes C)\text{(associative)}$$

$$A\otimes I=A\text{(where I is an identity vector)}$$

$$A\otimes A^{-1}=I\text{(inverse)}$$

$$O(A\otimes B,A)\approx O(A\otimes B,B)\approx \text{noise}\text{(dissimilarity)}$$

Implementation: segment-wise offset addition modulo segment size.

Check out original paper for details.

Check out code snippets from the API reference.

Release

Occasionally we use release, as the equivalent of division.

$$A\oslash B=A\otimes B^{-1}$$

Note that release is anti-commutative: $$(A\oslash B{)}^{-1}=B\oslash A$$

Check out code snippets from the API reference.

Bundle

Bundling ($\oplus$) creates a superposition of vectors, which is similar to all inputs. It is the additive operation in the VSA algebra.

$$S=\sum _{i,\oplus }{A}_i$$

Mathematically

$$O(S,A_i)\gg O_{\text{random}}\text{(similarity to each member)}$$

$$O(S,X)\approx O_{\text{random}}\text{for }X\notin \{A_i\}\text{(dissimilarity to non-members)}$$

Check out original paper for details on bundle operator.

Check out code snippets from the API reference.

Composites

Composites combine multiple hypervectors into higher-level conceptual structures. Each composite type uses a different combination strategy, preserving different semantics between its members.

All composites follow the same contract (interface in Go and traits in Rust) and can be hierarchically nested: for example, a Set can contain Sparkles, Knots, or even other Sets.

Sparkle ✨: the primitive

Before any composite, there is the Sparkle ✨ — the atomic, named hypervector everything below is built upon. A raw SparseSegmented 🍡 is just a bit pattern, and a Sparkle ✨ is that pattern with an identity:

$$S=\text{expand}(D,P)$$

where $D$ (the Domain) is a semantic namespace — “animals”, “role”, “country” — and $P$ (the Pod) names the individual within it: a word, a numeric seed, a prewired constant.

The (Domain, Pod) seed deterministically expands into the vector, so the same triple (model, domain, pod) yields the same Sparkle ✨ in every run and every engine — Go, Rust, and Python are bit-identical.

For this reason, only domain and pod are needed, instead of the raw per-segment offsets, which is a significant saving both on-wire and on-storage.

Two properties that carry the most weight:

• Distinct Sparkles ✨ are always quasi-orthogonal, without any central orchestration. They can be safely used as bricks for high-level construction;
• The markers and keys in the formulas below — $S_{marker}$, $S_{step}$, $K_i$ — are themselves Sparkles. The whole composite algebra bootstraps from this one primitive type.

Use when: you need a stable, deterministic identity for an atomic concept — a word, a role, an entity — as a leaf for the composites below.

Check out code snippets from the API reference.

Set 🫧

An unordered collection of concepts.

$$S=S_{marker}\otimes (\sum _{i,\oplus }{M}_i)$$

where $S_{marker}$ is a special marker to distinguish the set itself from its individual members.

Use when: you need to represent “these things together” without order.

Check out code snippets from the API reference.

Sequence 📿

An ordered collection.

$$S=S_{marker}\otimes (\sum _{i,\oplus }{M}_i\otimes S_{step}^i)$$

where $S_{step}$ is a generic hypervector for positional encoding.

$S_{marker}$ is a special marker to distinguish a sequence from its individual members.

Use when: order matters (e.g., words in a sentence, events in time).

Check out code snippets from the API reference.

Octopus 🐙

A key-value structure. Each key (a string) is converted to a Sparkle ✨ and bound with its corresponding value before bundling.

$$S=\sum _{i,\oplus }{K}_i\otimes V_i$$

Use when: you need to represent structured records with named attributes.

Check out code snippets from the API reference.

Knot 🪢

The result of binding (multiplicative composition) of hypervectors.

$$S=\prod _{i,\otimes }{M}_i$$

Unlike direct bind operator, Knot 🪢 keeps tracking of its members for serialization and introspection.

Use when: you need a reversible association between concepts.

Check out code snippets from the API reference.

Parcel 🎁

The result of bundling (additive composition) of hypervectors.

$$S=\sum _{i,\oplus }{M}_i$$

Unlike direct bundle operator, Parcel 🎁 keeps tracking of its members for serialization and introspection.

Use when: you need a superposition of concepts, with optional weights.

Check out code snippets from the API reference.

Dart 🎯

A one-directional reference between two hypervectors.

$$P=T^{-1}\otimes H=H\oslash T$$

A Dart 🎯 is “thrown” from a tail T to a head H — the direction is semantic (from → to), while algebraically the link is recoverable from either end. Dart 🎯 is the structured wrapper for the release.

Given the Dart 🎯 of $P$:

$$H=P\otimes T$$ $$T=P^{-1}\otimes H$$

Use when: you need a directed link — edges, mappings, “from→to” relations — where either endpoint can still be recovered given the other.

Check out code snippets from the API reference.

Summary

Near Neighbor Search

Near Neighbor Search (NNS) retrieves chunks from the storage substrate in increasing order of Hamming distance (from a query).

As we mentioned earlier, this is equivalent to a strictly decreasing order of overlap (between query and candidate). Overlap generally encodes semantic relevance, and this translates to a list of semantically relevant candidates.

This NNS module has linear time complexity with a very low constant — in expectation only a few tally operations per stored entry — so query cost grows gently with the number of entries in the storage substrate.

It leverages an underlying Associative Index for efficient recovery of candidates. The Associative Index is a semantic index that enables fast similarity-based lookup over stored hypervectors. Conceptually it turns a key-value substrate (item memory) into an associative memory — one where retrieval is by content similarity, not by exact content or key match.

Unlike approximate nearest neighbor methods (LSH, HNSW, etc.), the choice of sparse and binary hypervectors makes this practical and exact. The NNS module computes exact overlap counts via the associative index. There is no approximation error and no index-specific parameters to tune.

Jump to the API reference for Near-Neighbor Search.

Learner

A Learner performs online bundling over a stream of observations, in the style of Hebbian learning: each incoming pattern claims a share of a fixed representational budget, weighted by how often it is seen.

Conceptually the Learner itself is a single hypervector whose content is the weighted superposition of everything it has experienced, and recovery is by overlap (e.g. via the near-neighbor-search module, NNS): frequent patterns read back strongly, rare ones faintly, unseen ones at chance level.

The fixed representational budget

The representational budget is what gives a Learner its character and the limitation we will address later in LearnerPool.

A Learner is not a pure container for all experiences but a distribution that sharpens or flattens, which makes it ideal for learning distributions: transition frequencies, co-occurrence statistics — and gives it a natural capacity, beyond which no experiences can be reliably recovered.

To give you a concrete example, an 8bit learner, at its core, is a sparse binary hypervector with $256$ segments, each containing a single ON bit. Given the inherent noise, only a few dozen unique patterns can survive the dilution and maintain recognizable weights to be picked up by the NNS module.

Diversity vs repetition

What fills a Learner is the diversity of its past experiences, the unique count of experienced patterns (and their frequencies), instead of simple repetition.

To make this plain, observing a repeated pattern mostly re-distributes weight already committed, while each genuinely novel pattern makes its mark by diluting all existing residents.

Jump to the API reference for Learner.

LearnerPool

LearnerPool is an aggregate over member Learners for improved scalability. Instead of dedicating one Learner to one particular use case, a pool holds a fixed roster of member Learners that can collectively support many learning tasks in parallel.

Each learning task is identified by a particular address, so the conceptual design goal is to have a Learner-like component where an observation can be written into a given address, and with improved capacity.

The solution LearnerPool takes is to return a small access circle of member Learners for each addressed access, which can expand based on actual need.

Why a pool

A single Learner has a hard capacity ceiling: as more distinct patterns are experienced, each one’s share of ON bits dilutes, until they become indistinguishable from random noise. This can happen past a few dozen unique patterns for an 8bit Learner, for example.

This capacity limitation (of the classic Learner) bites in various scenarios:

• high fan-out addresses saturate and silently forget everything;
• the long tail of low fan-out addresses wastes almost all of its dedicated capacity.

A pool solves both issues exactly as its name suggests, by pooling many member Learners together. Most light addresses still get a nearly-private member, while heavy addresses recruit as many members as their content genuinely needs, up to the whole pool. At the end of the day, an individual member can serve a mixture of low and high fan-out addresses, orchestrated internally by its own scheduling logic.

The internal “orchestration” (or rewiring) is mathematically sound, stable, and requires no manual intervention. The member Learners organize organically rather than piling up naively: if each Learner were a basic “neuron”, a LearnerPool is then an organism that behaves intelligently and coherently toward a common goal.

Fixed resource consumption

Unlike the typical arrangement of one Learner per learning task which can grow unbounded, a LearnerPool uses a fixed amount of resources: entities of varying fan-out intelligently share the same pool.

The roster size is set at pool creation and cannot grow afterwards: there is currently no incremental expansion short of a full retrain. When every member a write could reach is full, the pool refuses the write rather than degrade what it already holds.

So another way to understand LearnerPool: it’s an addressable collection of elastically scalable (up to the fixed pool capacity) Learners.

Diversity vs. repetition

The LearnerPool picks a suitable member Learner by ensuring the incoming pattern can be recalled reliably later. Note that expansion follows diversity (of experiences) rather than crude repetition.

This also implies the scheduler can always find one suitable member, unless the whole pool is out of capacity, in which case the write will fail: all members can be used as reserves as needed.

Using bigger learners

Using bigger Learners (for example, 10bit, 12bit, etc.) is a completely orthogonal direction for expanding capacity. However, if you are comfortable with simpler 8bit learners, adding more members is more straightforward.

Another practical consideration: 8bit offsets are byte-aligned (one byte per segment; 16bit is two), which unlocks low-level SIMD on all supported platforms and boosts performance significantly. 10bit/12bit/14bit offsets are not byte-aligned and do not vectorize this way — so stick with 8bit if you care about performance.

Jump to the API reference for LearnerPool.

HV

The core hypervector API. This module provides the building blocks for hyperdimensional computing: vector types, algebraic operators, and model configuration.

Common Utilities

Functions and types used across all HyperBinary types.

Models

See Concepts: Hypervectors for the full overview.

Model Enum

model0 = hv.MODEL_64K_8BIT

model1 = hv.MODEL_1M_10BIT

Model Functions

hv.width(hv.MODEL_1M_10BIT)           # total dimensions
hv.cardinality(hv.MODEL_1M_10BIT)     # ON bit count
hv.sparsity(hv.MODEL_1M_10BIT)        # sparsity
hv.segment_size(hv.MODEL_1M_10BIT)    # dimensions per segment

See also: SparseOperation — Model + seeded RNG for deterministic vector generation.

Domain & Pod

A Domain models the semantic grouping for hypervectors, providing the high 64-bit half of a Seed128. A Pod is a slot within a Domain, providing the low 64-bit half. The (Domain, Pod) pair uniquely identifies a Sparkle.

Domain Constructors

# From a name string (hashed to a 64-bit id)
d = hv.Domain("animals")

# Same as above
d = hv.Domain.from_name("animals")

# From a raw 64-bit id
d = hv.Domain.from_id(0x1234567890abcdef)

# From a domain prefix enum and a name suffix
# The id is computed as xxhash(prefix_label + "." + name)
d = hv.Domain.from_prefix_and_name(hv.DOMAIN_PREFIX_NLP, "concept")

# Accessors
d.id()              # u64
d.name()            # str (empty if constructed from id)
d.domain_prefix()   # int (0 = UNKNOWN if no prefix was set)
d.is_default()      # True if id == 0

Domain Prefix Constants

Domain prefixes provide namespacing for domains. When a prefix is set, the domain id is derived from the prefix label (and optional name), ensuring consistent hashing across languages.

Pod Constructors

Pods can be seeded by a string word, a raw uint64, or a prewired enum value.

# From a word string (hashed to a 64-bit seed)
p = hv.Pod("cat")

# Same as above
p = hv.Pod.from_word("cat")

# From a raw 64-bit seed
p = hv.Pod.from_seed(42)

# From a prewired enum value
p = hv.Pod.from_prewired(hv.PREWIRED_SET_MARKER)
p = hv.Pod.from_prewired(hv.PREWIRED_STEP)

# Accessors
p.seed()       # u64
p.word()       # str (empty if constructed from seed or prewired)
p.prewired()   # int (0 if not prewired)
p.is_default() # True if seed == 0

Prewired Constants

Prewired pods are infrastructure-level constants with fixed seeds:

Polymorphic arguments (Python-only)

Most Python factories that take a Domain or Pod accept the underlying primitives directly — you rarely need to wrap them explicitly:

# Domain — four equivalent forms in any factory expecting a Domain:
memory.by_item_key("animals", "cat")
memory.by_item_key(hv.Domain.from_name("animals"), "cat")
memory.by_item_key(0x1234, "cat")                           # from numeric id
memory.by_item_key((hv.DOMAIN_PREFIX_NLP, "concept"), "p")  # from (prefix, name)

# Pod — Prewired enum is recognized:
memory.new_terminal("internal", hv.PREWIRED_STEP)           # Pod from Prewired
memory.new_terminal("animals", "cat")                       # Pod from word
memory.new_terminal("animals", 0xCAFE_BABE)                 # Pod from raw seed

For the parallel polymorphism on Seed128, see Seed128 → Polymorphic arguments.

Seed128

A Seed128 is a 128-bit seed to drive a random number generator.

The current random number generator expects 2 64-bit seeds: the same (seed_high, seed_low) pair always produces the same sequence of random numbers, enabling reproducible and deterministic vector generation across runs and languages.

Constructors

# From Domain and Pod arguments (each accepts Domain/Pod, int, or str)
seed = hv.Seed128("animals", "cat")                # domain name + pod word
seed = hv.Seed128(0, 42)                           # default domain + raw pod seed
seed = hv.Seed128("animals", 42)                   # domain name + raw pod seed
seed = hv.Seed128(hv.Domain("animals"), hv.Pod("cat"))  # explicit Domain/Pod objects

# Zero seed
seed_zero = hv.Seed128.zero()                      # (0, 0)

# Random seed from a SparseOperation
seed_rand = hv.Seed128.random(so)                  # consumes two u64 from the RNG

# Accessors
seed.domain()                                      # Domain object
seed.pod()                                         # Pod object
seed.high()                                        # u64 (domain id)
seed.low()                                         # u64 (pod seed)

Usage

All composite constructors take a Seed128, as seed for the bundle operator:

seed = hv.Seed128("fruits", "fruit_set")

s = hv.Set(seed, a, b, c)
seq = hv.Sequence(seed, a, b, c)

Polymorphic arguments (Python-only)

Anywhere a Python factory expects a Seed128 (composite constructors like hv.Set / hv.Sequence / hv.Octopus, the hv.bundle operator, etc.) you can pass either a Seed128 instance or a (domain, pod) tuple — the binding extracts and constructs the seed for you.

The tuple composes with the polymorphic forms accepted by Domain and Pod (see Domain & Pod → Polymorphic arguments), so each side can itself be a string / int / Prewired enum / (prefix, name) tuple — letting you skip the hv.Seed128(...) wrap entirely:

# Equivalent to hv.Sequence(hv.Seed128("words", "hi"), m1, m2):
seq = hv.Sequence(("words", "hi"), m1, m2)

# Tuple form composes with Domain's (DomainPrefix, str) tuple:
seq = hv.Sequence(((hv.DOMAIN_PREFIX_NLP, "concept"), "myseq"), m1, m2)

# And with Pod's Prewired enum:
seq = hv.Sequence(("internal", hv.PREWIRED_STEP), m1, m2)

SparseOperation

A SparseOperation instance wraps a Model, a random number generator, and potentially other information related to the sparse operation in general.

Constructor

so = hv.SparseOperation(hv.MODEL_1M_10BIT, 0, 42)

# Explicit RNG backend (keyword-only; omit for the KONGMING_RNG default, hv.RNG_PHILOX_4X64)
# Constants: hv.RNG_PHILOX_4X64, hv.RNG_XOSHIRO_256PP, hv.RNG_PCG_DXSM, hv.RNG_XOROSHIRO_128PP.
so2 = hv.SparseOperation(hv.MODEL_1M_10BIT, 0, 42, rng_hint=hv.RNG_XOSHIRO_256PP)

Methods

so.model()        # Model enum

so.width()        # width for this model

so.cardinality()  # cardinality for this model

so.sparsity()     # sparsity for this model

so.uint64()       # next random number

so.rng_hint()     # RNG backend (an hv.RNG_* constant)

Usage: Generating Random Vectors

so = hv.SparseOperation(hv.MODEL_1M_10BIT, 0, 42)
sparkle = hv.Sparkle.random(hv.Domain("domain"), so)

Utilities

Similarity

hv.overlap(a, b)    # Overlap

hv.hamming(a, b)    # Hamming distance

hv.equal(a, b)      # Equality check

Identity Check

v=hv.Sparkle.identity(model)

hv.is_identity(v)   # True if v is an identity vector

Hash Utilities

hv.hash64_from_string("hello")   # deterministic u64 hash from string
hv.hash64_from_bytes(b"\x01\x02") # deterministic u64 hash from bytes
hv.curr_time_as_seed()            # current time as a u64 seed
hv.kongming_studio_seed()         # fixed studio seed constant

HyperBinary Types

All vector types conform to a common HyperBinary interface, kept at feature parity across the underlying engines.

v.model()        # Model enum
v.width()
v.cardinality()
v.stable_hash()  # unique hash for this vector
v.seed128()
v.exponent()

Lazy materialization

Every type except SparseSegmented is defined by a recipe — much more compact than raw offsets.

The raw offsets will be computed and cached on first usage by APIs such as core(), stable_hash(), overlap and similarity, serialization of SparseSegmented, etc. Other API calls, such as model(), domain(), pod(), exponent(), never materialize anything.

This is invisible to callers: the vector holds the same semantic content whenever you ask, but it means constructing vectors you never observe is nearly free, which is the typical and common use case for hypervectors.

compact() releases the cached content again, recursively through members. The recipe is retained, so the next observation recomputes exactly the same bits.

The tale of two equalities

Two levels are available, differing only in how hard they work:

• equal_lazy compares recipes (and already-known content hashes), so it never materializes anything. It is conservative: a True answer is always correct, while a False answer may mean “not provable this cheaply” — two vectors of different concrete types, for example, or a coincidence that only the actual offsets would reveal.
• equal starts with the lazy check and falls back to comparing content hashes, materializing if it must. Use it when the answer must be exact.

Concrete Types

SparseSegmented 🍡

The most foundational vector type — a sparse binary hypervector where each segment has exactly one ON bit at the recorded offset location. All other types (Sparkle, Set, Sequence, etc.) ultimately contain a SparseSegmented in memory for processing, whenever necessary.

Structure

The offsets are bit-packed according to the model’s sparsity bits — they do not align to byte boundaries. This trades a small CPU cost for compact, uniform storage that works both in memory and on disk.

Identity vector: when offsets is blank (zero storage), the vector is the identity vector where all offsets are 0.

Constructors

# Identity
ss = hv.SparseSegmented.identity(model)

Key Methods

ss.is_identity()  # True if identity vector

ss2 = ss.power(2)
inv = ss.power(-1)

# Similarity
hv.overlap(a, b)   # Count of matching ON bits
hv.hamming(a, b)   # Count of differing segments

ss.offsets()   # returns all offsets
ss.on(idx)     # True if global bit index is ON
ss.offset(seg) # the ON offset within one segment

Sparkle ✨

Sparkles are the atomic building block for higher-level constructs. Domain is a logical namespace that groups related Sparkle instances. Pod acts as the secondary identifier for a Sparkle instance.

Sparkle is deterministic: the same (domain, pod) pair always produces the same offsets, across all sessions and engines. For this reason, the (model, domain, pod) triple uniquely identifies a Sparkle, and we store the triple rather than the raw offsets for huge space saving.

Sparkle Constructors

# From a word string
s0 = hv.Sparkle.from_word(model, "animals", "cat")

# From a numeric seed
s1 = hv.Sparkle.from_seed(model, "animals", 42)

# From a prewired enum
s2 = hv.Sparkle.from_prewired(model, "animals", hv.PREWIRED_SET_MARKER)

# Identity vector
s3 = hv.Sparkle.identity(model)

# Random (from SparseOperation)
so = hv.SparseOperation(hv.MODEL_1M_10BIT, 0, 42)
s4 = hv.Sparkle.random("animals", so)

# From domain + pod directly — primary constructor
s5 = hv.Sparkle(model, "animals", pod)

Key Methods

s0.model()         # Model enum
s0.stable_hash()   # Deterministic and unique hash
s0.exponent()      # Current exponent (1 for base vector)

s0_square=s0.power(2)     # Returns p-th power (new Sparkle)
hv.equal(s0, s0_square)   # s0_square = s0^2, different from original s0.
       
core0=s0.core()     # Returns underlying SparseSegmented
core0.offsets()    # The raw offsets for each segment.

Pretty-printing

# Pretty-printing, or s.__str__()
print(s0)
# ✨:🔗animals,🌱cat

# More detailed information, or s.__repr__()
s
# hint: SPARKLE
# model: MODEL_1M_10BIT
# stable_hash: 9725717137035622833
# domain:
#   name: animals
# pod:
#   word: cat

During pretty-printing of Sparkle instances, you may notice special emoji for domain / pods.

Learner 💫

Learners are designed to perform online bundling for a stream of observations, in the form of Hebbian learning.

The representational budget is fixed, in the form of segment count $M$ from a single hypervector: what matters is the distribution of weights among observed vectors.

Constructors

learner = hv.Learner(model, hv.Seed128(0, 42))

# age-1 learner that starts having seen `obs`.
learner = hv.Learner(model, hv.Seed128(0, 42), initial=obs)

# optional keyword-only rng_hint pins the RNG backend (default: process-wide).
learner = hv.Learner(model, hv.Seed128(0, 42), rng_hint=hv.RNG_PHILOX_4X64)

# a randomly-initialized learner.
learner = hv.Learner.random(so)

Feeding Observations

learner.bundle(a)                 # single observation

learner.bundle_multiple(b, 3)     # with weight multiplier

Inspection

learner.age()                # total observed weight (int)
learner.blank()              # bool: whether this learner is blank (nothing observed yet)

learner.model()              # identity accessors: model / domain / pod
learner.domain()
learner.pod()

learner.support(a)           # overlap above the chance baseline, saturating at 0
learner.weight(a)            # support, normalized to [0.0, 1.0]

Deferred Observations

A young Learner does not need to build its raw buffer immediately: instead it defers the observations themselves as a small list of (vector, weight) pairs, and only materializes the buffer once keeping the list no longer pays. A deferred observation is a recipe, typically far smaller than a full offsets buffer, so young learners cost a fraction of a materialized buffer, in memory and on the wire.

This also implies:

• Repeats are free. Bundling a pattern the learner already holds bumps that entry’s weight when the recipes match lazily (EqualLazy); a repeat arriving in a different representation may land as a fresh entry. A learner that sees the same pattern a thousand times still holds one entry, and never materializes at all.

learner.has_deferred_data()  # True while observations are still deferred

for entry in learner.deferred_data():
    print(entry)             # the deferred observations, in arrival order

learner.compact()            # drop incidental materializations (content unchanged)

See also

• Learner concepts — the fixed budget, diversity vs. repetition.
• LearnerPool — pooled Learners behind address-keyed access circles.

LearnerPool 🎱

A LearnerPool is an aggregate over member Learners for improved scalability: a fixed roster of members that collectively serves many address-keyed learning tasks in parallel. Writes to an address land on a small access circle of members; heavy addresses recruit more members as needed while light addresses keep a nearly-private one. See the concepts chapter for why and how.

Constructors

# model, member domain, roster size
pool = hv.LearnerPool(hv.MODEL_64K_8BIT, "pool", 65536)
pool.init()          # fill the still-empty roster with fresh Learners

Writing

bundle stores data under addr. Omitting the address stores the pattern auto-associatively.

pool.bundle(data, addr=addr)            # hetero-associative write
pool.bundle(data, addr=addr, multiple=3)  # with weight

When reaching the collective capacity of the pool, it refuses future write.

Reading

We support wwo read scenarios:

• support — evidence that probe was stored under addr (a scalar, noise-subtracted). This is the discriminative read: “how strongly does the pool associate addr → probe?”
• read_members — the content/experiences of the circle’s members, one hypervector each. This is the generative read: use them as attractors for near-neighbor search when you don’t know the probe in advance.

s = pool.support(addr, probe)      # scalar evidence

for attractor in pool.read_members(addr):
    ...                            # feed into NNS / overlap checks

Introspection

pool.total()             # fixed member count
pool.load()              # total write mass W = Σ member ages
pool.unique_estimated()  # ≈ distinct items held in this pool
pool.member_domain()     # the members' domain

Persistence

A pool serializes as a small self-describing sentinel — its metadata as a LearnerPoolProto — plus one ordinary chunk per trained member.

The pool serializes as metadata plus per-member contents. Round-trip via the memory layer: the pool sentinel is self-describing, so loading needs no config.

# load from a substrate view in one call.
pool = memory.load_learner_pool(view, member_domain)

# or manually: metadata first, members via a loader callback
pool = hv.LearnerPool.from_proto_bytes(raw)
pool.hydrate(lambda domain, pod: ...)   # return Learner or None (blank)

See also

• LearnerPool concepts — why a pool, access circles, diversity vs. repetition.
• Learner — the member primitive.

Set 🫧

An unordered collection of hypervectors. See Composites: Set for the conceptual overview.

Constructor

s = hv.Set(hv.Seed128(0, 42), first, second, third)

Notable methods

# All these will be approximately 1/3 of the total cardinality.
hv.overlap(s.unmasked(), first)
hv.overlap(s.unmasked(), second)
hv.overlap(s.unmasked(), third)

Sequence 📿

An ordered collection of hypervectors with positional encoding. See Composites: Sequence for the conceptual overview.

Constructor

# Constructing a sequence, with logical index start at 1 (default to 0).
seq = hv.Sequence(hv.Seed128(0, 42), first, second, third, start=1)

Derived Sequences: Append / Prepend / Reset

Append, Prepend, and Reset all return a new Sequence — a Sequence is an immutable value, so the receiver is never changed.

• Append(more...) — members added at the end. start is unchanged.
• Prepend(more...) — members added at the front; start decrements by len(more) so existing members keep their positional binding.
• Reset(start) — shift the starting index. Returns an equal Sequence when start equals the current start.

The result equals what you’d get by building a fresh NewSequence(seed, new_start, all_members...) — the domain/pod seed is preserved.

seq = hv.Sequence(hv.Seed128(0, 42), a, b, c)

# Append / Prepend are variadic and return new Sequences.
s1 = seq.append(d, e)       # [a, b, c, d, e]; seq unchanged
s2 = seq.prepend(x, y)      # [x, y, a, b, c], start -= 2; seq unchanged
s3 = seq.reset(10)          # starting index 10; seq unchanged

Octopus 🐙

A key-value composite where each value is bound with its key’s Sparkle. See Composites: Octopus for the conceptual overview.

Constructor

Keys are Pods. In Python, strings (and any value polymorphically convertible to Pod) are accepted and auto-converted.

oct = hv.Octopus(hv.Seed128(0, 42), ["color", "shape"], red, circle)

Key Methods

oct.value_by_key("color")  # accepts Pod | str | int | Prewired

Knot 🪢

Knot 🪢 contains the result of binding (multiplicative composition) of hypervectors, while tracking its member parts for serialization and introspection. See Composites: Knot.

Constructor

# More commonly via the bind operator:
k = hv.bind(a, b)

Extending a Knot

An existing Knot can be extended with additional parts via expand. This returns a new Knot (as a Knot is an immutable value) — equivalent to re-binding all parts from scratch but without reconstructing the base.

k = hv.bind(a, b)
k2 = k.expand(c)  # k2 is equivalent to hv.bind(a, b, c); k is unchanged

Parcel 🎁

Parcel 🎁 contains the result of bundling (additive composition) of hypervectors, while tracking its members and bundling seed for serialization and debugging. See Composites: Parcel.

Constructors

# Direct, with optional per-member weights:
p = hv.Parcel(hv.Seed128(10, 1), a, b, c, weights=[0.6, 0.2, 0.2])

Key Methods

p.count()      # member count
p.members()    # the tracked members

Dart 🎯

A one-directional reference between two hypervectors. A Dart is “thrown” from a tail to a head: $$P=H\otimes T^{-1}=H\oslash T$$

See Composites: Dart.

Constructor

# Or via the release operator (note the order: release(head, tail)):
p = hv.release(head, tail)

Endpoints

A Dart retains references to its endpoints.

p.tail()          # thrown from ...
p.head()          # ... to

Recovering endpoints

p = hv.release(h, t)                    # the Dart thrown t → h
recovered_h = hv.bind(p, t)             # ≈ h
recovered_t = hv.bind(p.power(-1), h)   # ≈ t

Anti-commutativity

Dart (and the release operator that constructs it) is anti-commutative:

$$\text{Dart}(T,H)=\text{Dart}(H,T{)}^{-1}$$

Operators

See Concepts: Operators for the full overview.

Bind

bound = hv.bind(a, b)
released = hv.release(bound, b)  # this will recover `a`

hv.equal(a, b)                   # hash equality

Release

Extracts one component from a binding: $A\oslash B=A\otimes B^{-1}$

bound = hv.bind(role, filler)
recovered = hv.release(bound, role)  # Dart; ≈ filler at the bit level

Expand (extend a Knot)

Extends an existing Knot with additional operands without re-binding from scratch. k.expand(c) on k = bind(a, b) returns a new Knot equal to bind(a, b, c) — a Knot is an immutable value, so k itself never changes.

k = hv.bind(a, b)
k2 = k.expand(c)            # k2 is equivalent to hv.bind(a, b, c); k unchanged

Bundle

p = hv.bundle(hv.Seed128(10, 1), a, b, c)

Customizing runtime behavior

Environment Variables

All environment variables are read once on first access and cannot be changed at runtime. Unset variables use the documented default.

KONGMING_RNG

Selects the pseudo-random number generator backend used for hypervector generation.

All four are bit-parity across the Go and Rust engines. Any unrecognized value falls back to philox.

Changing this affects all generated vectors: Sparkle offsets, Learner bundling, Cyclone patterns. Vectors generated with different backends are not compatible.

export KONGMING_RNG=xoshiro++

Querying the Current Environment

Use global_env() to inspect all active settings at runtime. Returns a GlobalEnv protobuf message — new fields added to the proto automatically appear.

>>> hv.global_env()
rng_hint: PHILOX_4X64

Misc

Display

All HyperBinary types have a compact, emoji-prefixed string representation for quick visual inspection. See HyperBinary Types for type symbols.

Python __str__ and __repr__

__str__ (triggered by print()) returns the compact emoji form:

>>> a = hv.Sparkle.from_word(hv.MODEL_64K_8BIT, hv.d0(), "hello")
>>> print(a)
✨:🌐0x..c862,🫛0x..80e4

__repr__ (triggered by evaluating a variable in the shell or notebook) returns a detailed, developer-friendly YAML representation:

>>> a
hint: SPARKLE
model: MODEL_64K_8BIT
stable_hash: 12345678
domain:
  id: ...
pod:
  seed: 12345

Serialization

# HyperBinary → protobuf message
msg = hv.to_message(sparkle)

# protobuf message → HyperBinary
obj = hv.from_message(msg)

# raw proto bytes → HyperBinary
obj = hv.from_proto_bytes(data)

# proto bytes → YAML string (for debugging)
hv.format_to_yaml(data)

Memory

The memory package provides persistent and in-memory storage for hypervectors with semantic indexing and near-neighbor search.

The core abstraction is a Chunk — an immutable identity (Sparkle) paired with a mutable semantic code (any HyperBinary). Chunks are stored in a Substrate (pluggable storage backend), queried via ChunkSelectors, and created via ChunkProducers.

Chunk

The fundamental storage unit in the memory system. A Chunk mostly carries a semantic code (any HyperBinary type) along with various diagnostic information.

Structure

Inspection

Chunks are typically created via producers (see Producers) — or directly from a code (memory.Chunk(code, note="", extra=msg); the id derives from the code’s domain/pod) — and inspected after retrieval (see Selectors).

chunk = memory.first_picked(view, memory.by_item_key("animals", "cat"))

chunk.id               # Sparkle
chunk.code             # HyperBinary
chunk.note             # str
chunk.extra_message()  # deserialized protobuf message, or None

Substrate & Views

A Substrate is a pluggable storage backend. It provides transactional views for reading and writing chunks.

View Pattern

All storage access goes through views:

• SubstrateView — read-only, supports key lookup and prefix scanning
• SubstrateMutableView — extends SubstrateView with write staging and atomic commit (to underlying storage)

# Read-only view (context manager)
with storage.new_view() as view:
    # Check if chunk exists, without actually reading it back.
    exists = view.chunk_exists("animals", "cat")

    cat_chunk = view.read_chunk("animals", "cat")

# Mutable view (auto-commits on clean exit, rollback on exception).
# Stage writes by running producers against the view via
# producer.produce(view) — the recommended path for batched writes.
with storage.new_mutable_view() as view:
    memory.new_terminal("words", "hi").produce(view)
    memory.from_sequence_members("words", "greet", members,
                                  enable_semantic_indexing=True).produce(view)
    # commits automatically

Storage Backends

InMemory

Volatile, in-process storage. All data lost on exit. Best for testing and ephemeral caches.

storage = memory.InMemory(hv.MODEL_64K_8BIT, "my_store")

Embedded

Persistent, single-machine storage backed by an embedded key-value store. Suitable for local development and moderate-scale deployments.

storage = memory.Embedded(hv.MODEL_64K_8BIT, "/path/to/store")

ScyllaDB (Distributed)

Distributed storage via Cassandra-compatible ScyllaDB. For high-scale, multi-node deployments.

# Not exposed yet...

Selectors

ChunkSelectors are composable query builders for reading chunks from the substrate. Each selector defines how to locate and return matching chunks.

NNS (Near-Neighbor Search)

Wraps a single attractor to perform near-neighbor search. For multiple attractors, compose them with joiner(...) first.

result = memory.first_picked(
    view, memory.nns(
        memory.set_members(memory.by_item_key("sets", "my_set"))))

Each attractor conceptually provides “the center of attraction” for candidates: the NNS accepts one or more attractors, to perform the actual near-neighbor search work, by interacting with underlying associative index.

Forward attractors

Roughly forward attractors try to find parts from a given a composite.

memory.set_members(memory.by_item_key("sets", "my_set"))

memory.sequence_member(memory.by_item_key("seqs", "my_seq"), pos=2)

memory.tentacle(memory.by_item_key("records", "person"), "name")

Reverse Attractors

Roughly reverse attractors try to locate composites given a part.

memory.set_attractor(memory.by_item_key("animals", "cat"), "sets")

memory.sequence_attractor(memory.by_item_key("animals", "cat"), 0, "seqs")

memory.octopus_attractor("color", memory.by_item_key("colors", "red"))

Analogical Reasoning

AnalogicalReasoner(dst, src, feature) performs analogical reasoning (“A is to B as C is to ?”): for each chunk c yielded by dst, it computes Bind(c.code, feature, Inverse(src)) and forwards to NNS. Model is implicit in src / feature.

Given the analogy “king is to queen as man is to ?”:

king   = hv.Sparkle(model, "role", "king")
queen  = hv.Sparkle(model, "role", "queen")
man    = hv.Sparkle(model, "role", "man")

# Analogy: "king is to queen as man is to ?"
#   src     = king   (the known source of the relationship)
#   feature = queen  (the known feature/attribute of src)
#   dst     = man    (the target; we want to find its corresponding feature)
#
# src = king (known source), feature = queen (known relation), dst = man.
# Modifier = queen ⊗ inverse(king); applied to man → "woman".
memory.nns(
    memory.analogical_reasoner(memory.with_code(man), king, queen))

Direct WithCodeModifier / WithIDModifier

For ad-hoc patterns that don’t fit a named attractor, use the primitives directly. They take a precomputed HyperBinary modifier and apply Bind(code, modifier) or Bind(id, modifier) to each yielded chunk:

memory.with_code_modifier(inner_selector, modifier_vec)
memory.with_id_modifier(inner_selector, modifier_vec)

Other Selectors

ByItemKey

Exact lookup by domain + pod.

sel = memory.by_item_key("animals", "cat")

ByItemDomain

All chunks in a given domain (prefix scan).

sel = memory.by_item_domain("animals")

WithCode / WithSparkle

Literal selector — returns a hypervector directly, no storage lookup.

sel = memory.with_code(some_hv)

sel = memory.with_sparkle("animals", "cat")

Joiner

Union of multiple selectors — returns results from each of the inner selectors.

sel = memory.joiner(
    memory.by_item_key("animals", "cat"),
    memory.by_item_key("animals", "dog"),
)

Range

Limits results to [start, start+limit). limit=0 (default) implies no limit, and iteration continue until there is no more results.

sel = memory.range_sel(
    memory.by_item_domain("animals"), start=0, limit=10)

OnlyDomain

Filters inner selector results by given domain.

sel = memory.only_domain(
    "animals", inner_selector)

Working with Results

FirstPicked — get the first match

Returns the first chunk matching the selector. Returns an error if nothing is found.

# Returns the first matching Chunk (with .id, .code, .note, .extra_message())
chunk = memory.first_picked(view, selector)
print(chunk.id, chunk.code, chunk.note)

mem_get — eager batch read (Chunks only)

Returns every match as a list[Chunk]. No extras — any per-result SelectorExtra produced by the selector (e.g. NNS scores) is discarded. Use this when you only need the Chunks.

chunks = storage.mem_get(selector)        # list[Chunk]
for chunk in chunks:
    print(chunk.id, chunk.note)

lazy_selector_iter — stream Chunks with extras

Yields (Chunk, Optional[SelectorExtra]) tuples one at a time. This is the only way to access per-result SelectorExtra in Python; mem_get drops it.

# Streaming — useful for large result sets or early termination
for chunk, extra in memory.lazy_selector_iter(view, selector):
    print(chunk.id, extra)
    if done():
        break

# Eager with extras — wrap in list()
results = list(memory.lazy_selector_iter(view, selector))
# results: list[tuple[Chunk, Optional[SelectorExtra]]]

Producers

ChunkProducers are write builders that create and persist chunks in the substrate. Each producer encapsulates the logic for constructing a specific type of chunk.

Producer Options

Producer options are additional information supplied to producer constructor to tweak behavior.

# `note` indiciates additional note for the new terminal chunk.
memory.new_terminal("d", "p", note="annotation")

# `enable_semantic_indexing` indicates we need to index the semantic code 
# (on top of the id vector).
memory.from_set_members("d", "p", members, enable_semantic_indexing=True)

Concrete Producers

NewTerminal

Creates a chunk whose code equals its identity (a bare Sparkle). Useful for registering atoms/symbols.

with storage.new_mutable_view() as view:
    memory.new_terminal("fruits", "apple", note="an apple").produce(view)

NewLearner

Creates a fresh Learner chunk for online learning.

with storage.new_mutable_view() as view:
    memory.new_learner("learners", "my_learner", note="a learner").produce(view)

FromSetMembers

Creates a Set from stored members.

with storage.new_mutable_view() as view:
    memory.from_set_members(
        "sets",
        "fruit_set",
        memory.by_item_domain("fruits"),
    ).produce(view)

FromSequenceMembers

Creates a Sequence from stored members with positional encoding.

with storage.new_mutable_view() as view:
    memory.from_sequence_members(
        "seqs",
        "greeting",
        memory.joiner(
            memory.by_item_key("words", "hello"),
            memory.by_item_key("words", "world"),
        ),
        start=0,
    ).produce(view)

FromKeyValues

Creates an Octopus (key-value composite) from keys and value selectors.

with storage.new_mutable_view() as view:
    memory.from_key_values(
        "records",
        "obj1",
        keys=["color", "shape"],
        values=memory.joiner(
            memory.by_item_key("colors", "red"),
            memory.by_item_key("shapes", "circle"),
        ),
    ).produce(view)

NewDart

Creates a Dart 🎯 chunk — thrown from a tail chunk to a head chunk. Both selectors must resolve to a single chunk; the produced Dart’s bit-level value is Inv(tail.id) ⊗ head.id.

with storage.new_mutable_view() as view:
    memory.new_dart(
        "edges", "earth_to_moon",
        memory.by_item_key("planets", "earth"),
        memory.by_item_key("planets", "moon"),
    ).produce(view)

ClusterUpdater

Feeds an observed chunk into an existing Learner, updating its accumulated code via bundling. The bundle multiplier defaults to 1; pass an explicit override (multiple=N in Python) to fold the same observation in repeatedly.

with storage.new_mutable_view() as view:
    # With explicit multiplier:
    memory.cluster_updater(
        learner=memory.by_item_key("learners", "my_learner"),
        observed=memory.by_item_key("fruits", "apple"),
        multiple=3,
    ).produce(view)

Notebook Quick Start

This guide walks through using Kongming HV in a Jupyter notebook, cell by cell.

Tips

• Reproducibility: Use fixed seeds in SparseOperation for deterministic results across reruns.
• Visualization: Use pandas DataFrames for overlap matrices — they render nicely in Jupyter.
• Performance: The Rust backend is fast. Building 10,000 vectors takes under a second on MODEL_64K_8BIT.
• Model choice: Start with MODEL_64K_8BIT for exploration. Switch to MODEL_1M_10BIT or larger for production workloads.

Notebook Platforms

Setup and behavior differ across Jupyter, Google Colab, and Binder. This page covers the key differences.

Try Online

Notebook	Platform	Link
first.ipynb	Google Colab
first.ipynb	Binder
memory.ipynb	Google Colab
lisp.ipynb	Google Colab

Comparison

Jupyter (Local)

Install once in your terminal, then use in any notebook:

pip install kongming-rs-hv

# Cell 1 — no restart needed
from kongming import hv

For development workflows with frequent code changes, use autoreload:

%load_ext autoreload
%autoreload 2

Google Colab

Colab runs in the cloud with a fresh environment each session. Install in the first cell:

# Cell 1 — install
!pip install kongming-rs-hv

After the first install, Colab requires a runtime restart:

1. Go to Runtime → Restart runtime (or use the button Colab shows after install)
2. Then run the remaining cells

# Cell 2 — after restart
from kongming import hv
model = hv.MODEL_64K_8BIT

Subsequent sessions on the same notebook will need the install cell again — Colab does not persist pip packages across sessions.

Saving work: Use google.colab.drive to mount Google Drive for persistent storage:

from google.colab import drive
drive.mount('/content/drive')
# Then use paths like /content/drive/MyDrive/...

Binder

Binder builds a Docker image from your repo’s requirements.txt and launches a Jupyter server. No account needed.

• First launch: Takes 2–5 minutes to build the environment
• Subsequent launches: Faster if the image is cached
• No install needed: kongming-rs-hv is pre-installed from requirements.txt
• Ephemeral: All work is lost when the session times out (~10 min idle)

# Cell 1 — works immediately, no install
from kongming import hv

Choosing a Platform

Interactive Notebooks

For deeper walkthroughs, open these notebooks directly:

Notebook	Description	Colab
first.ipynb	Introduction to hypervectors, bind/bundle operations, and composites
memory.ipynb	In-memory and persistent storage, near-neighbor search with attractors, and export to disk
lisp.ipynb	VSA-based LISP interpreter where every data structure is a hypervector

See also: LISP Interpreter — a full example built on the core API.

Walkthrough

A step-by-step introduction to Kongming HV in a notebook, cell by cell.

Setup

# Cell 1: Install and import
# !pip install kongming-rs-hv pandas

from kongming import hv
import pandas as pd

model = hv.MODEL_64K_8BIT
so = hv.SparseOperation(model, 0, 1)

Building a Vocabulary

# Cell 2: Create vectors for a set of words
words = ["cat", "dog", "fish", "bird", "tree", "rock"]
vectors = {w: hv.Sparkle.from_word(model, "vocab", w) for w in words}

print(f"Created {len(vectors)} vectors")
print(f"Model: {model}, Cardinality: {hv.cardinality(model)}")

Output:

Created 6 vectors
Model: 1, Cardinality: 256

Similarity Matrix

# Cell 3: Compute pairwise overlap
data = {}
for w1 in words:
    data[w1] = {w2: hv.overlap(vectors[w1], vectors[w2]) for w2 in words}

pd.DataFrame(data, index=words)

Output:

The diagonal is 256 (cardinality = perfect self-overlap). Off-diagonal values are near 0-2 (random noise), confirming the vectors are near-orthogonal.

Learning from Observations

# Cell 4: Create a learner and feed it observations
learner = hv.Learner(model, hv.Seed128(0, so.uint64()))

# "cat" seen 3 times, "dog" once, "bird" once
for _ in range(3):
    learner.bundle(vectors["cat"])
learner.bundle(vectors["dog"])
learner.bundle(vectors["bird"])

print(f"Learner age: {learner.age()}")

Output:

Learner age: 5

Probing the Learner

# Cell 5: Check what the learner remembers
results = []
for w in words:
    ov = hv.overlap(learner, vectors[w])
    results.append({"word": w, "overlap": ov})

df = pd.DataFrame(results).sort_values("overlap", ascending=False)
df

Output:

“cat” has the highest overlap — roughly its weight share of the cardinality (3/5 × 256 ≈ 154). “dog” and “bird” (seen 1x each) sit near 1/5 × 256 ≈ 51. Unseen words stay at noise level.

Binding: Role-Filler Pairs

# Cell 6: Create a structured representation
#   "a cat that is red"
color_role = hv.Sparkle.from_word(model, "role", "color")
animal_role = hv.Sparkle.from_word(model, "role", "animal")

red = hv.Sparkle.from_word(model, "color", "red")
blue = hv.Sparkle.from_word(model, "color", "blue")
cat = vectors["cat"]

# Bind role with filler, then bundle the pairs
learner2 = hv.Learner(model, hv.Seed128(0, so.uint64()))
learner2.bundle(hv.bind(color_role, red))
learner2.bundle(hv.bind(animal_role, cat))

# Probe: "what color?" — release the role from the learned bundle
query = hv.release(learner2, color_role)
print(f"red overlap:  {hv.overlap(query, red)}")    # high (~128, its bundle share)
print(f"blue overlap: {hv.overlap(query, blue)}")   # ~1
print(f"cat overlap:  {hv.overlap(query, cat)}")    # ~1

Python Quick Start

See also: Notebook Quick Start for interactive Jupyter walkthroughs.

Installation

PyPI

pip install kongming-rs-hv

Supported Platforms

Verifying Installation

import kongming
print(kongming.__version__)  # e.g. "5.0.0", as of Aug. 2026. Yours should be newer.

from kongming import hv
print(hv.MODEL_64K_8BIT)  # should print 1

Import Paths

The package exposes two main modules:

from kongming import hv       # hypervector operations
from kongming import memory   # storage and selectors

Model constants are available directly on hv:

hv.MODEL_64K_8BIT      # 1
hv.MODEL_1M_10BIT      # 2
hv.MODEL_16M_12BIT     # 3
hv.MODEL_256M_14BIT    # 4
hv.MODEL_4G_16BIT      # 5

Docker

If you’d rather not install anything on your host, you can run kongming-rs-hv inside a container. This works on any system with Docker — no Python, no virtualenv, no wheel compatibility to worry about.

One-liner: throwaway Python REPL

Drop straight into a Python shell with the package preinstalled:

docker run --rm -it python:3.12-slim sh -c "\
    pip install --quiet --root-user-action=ignore \
        --disable-pip-version-check kongming-rs-hv && python"

--rm removes the container on exit. Nothing is persisted. Re-running reinstalls from PyPI, which takes a few seconds. The --root-user-action=ignore and --disable-pip-version-check flags silence pip’s root-user and upgrade notices, which are harmless inside a throwaway container.

Reusable image

For repeat use, build a small image once:

# Dockerfile
FROM python:3.12-slim
RUN pip install --no-cache-dir --disable-pip-version-check kongming-rs-hv
CMD ["python"]

docker build -t kongming-hv .
docker run --rm -it kongming-hv

To run a script from the host instead of an interactive REPL, mount the current directory:

docker run --rm -v "$PWD":/work -w /work kongming-hv python my_script.py

JupyterLab in a container

For interactive exploration with notebooks:

# Dockerfile.jupyter
FROM python:3.12-slim
RUN pip install --no-cache-dir --disable-pip-version-check \
    kongming-rs-hv jupyterlab
WORKDIR /notebooks
EXPOSE 8888
CMD ["jupyter", "lab", "--ip=0.0.0.0", "--no-browser", \
     "--ServerApp.token=''", "--ServerApp.password=''"]

docker build -f Dockerfile.jupyter -t kongming-hv-jupyter .
docker run --rm -p 8888:8888 -v "$PWD":/notebooks kongming-hv-jupyter

Open http://localhost:8888 in your browser. Notebooks saved under /notebooks are persisted to the mounted host directory.

The disabled token/password above is fine for local use. Do not expose this container on a public network without adding authentication.

Quick Example

A minimal example showing the core operations:

from kongming import hv

# Create hypervectors
a = hv.Sparkle.from_word(hv.MODEL_64K_8BIT, hv.d0(), "hello")
b = hv.Sparkle.from_word(hv.MODEL_64K_8BIT, hv.d0(), "world")
print(f'Overlap: {hv.overlap(a, b)}')  # Near orthogonal (~1)

# Bind: result is dissimilar to both inputs
bound = hv.bind(a, b)
print(f'{hv.overlap(bound, a)=}, {hv.overlap(bound, b)=}')  # ~1, ~1

# Bundle: result is similar to both inputs
bundled = hv.bundle(hv.Seed128(10, 1), a, b)
print(f'{hv.overlap(bundled, a)=}, {hv.overlap(bundled, b)=}')  # high, high

What’s Happening

1. Sparkle.from_word generates a deterministic hypervector from a word. Same word always produces the same vector.
2. Two unrelated vectors have near-zero overlap (~1) — random high-dimensional vectors are nearly orthogonal.
3. hv.bind(a, b) produces a vector dissimilar to both (low overlap). Binding is reversible.
4. hv.bundle(seed, a, b) produces a vector similar to both (high overlap). Different seeds produce different but equally valid results.

Walkthrough

A deeper exploration of the Python API, covering vector creation, similarity, random generation, power/permutation, and online learning.

Creating Vectors

from kongming import hv

model = hv.MODEL_64K_8BIT

# Create sparkles (atomic vectors) from words
cat = hv.Sparkle.from_word(model, "animals", "cat")
dog = hv.Sparkle.from_word(model, "animals", "dog")

# Same inputs always produce the same vector
cat2 = hv.Sparkle.from_word(model, "animals", "cat")
assert cat.stable_hash() == cat2.stable_hash()

Measuring Similarity

# Random vectors have ~1 overlap
print(hv.overlap(cat, dog))   # ≈ 1 (near-orthogonal)

# A vector is maximally similar to itself
print(hv.overlap(cat, cat))   # = 256 (= cardinality)

Using SparseOperation for Random Generation

so = hv.SparseOperation(model, 123, 456)

# Generate random sparkles
a = hv.Sparkle.random("my_domain", so)
b = hv.Sparkle.random("my_domain", so)

# Each call to so produces a new random seed
print(hv.overlap(a, b))  # ≈ 1

Power and Permutation

# Power creates a permuted vector
s = hv.Sparkle.from_word(model, "pos", "step")
s2 = s.power(2)
s3 = s.power(3)

# Different powers are near-orthogonal
print(hv.overlap(s, s2))   # ≈ 1
print(hv.overlap(s, s3))   # ≈ 1

# Inverse: power(-1) undoes power(1)
s_inv = s.power(-1)
# bind(s, s_inv) ≈ identity

Online Learning with Learner

learner = hv.Learner(model, hv.Seed128(0, 42))

# Feed observations one at a time
learner.bundle(cat)
learner.bundle(cat)   # seen twice — stronger signal
learner.bundle(dog)

# The learned vector is more similar to cat (seen 2x)
print(hv.overlap(learner, cat))  # higher
print(hv.overlap(learner, dog))  # lower but above random

Examples

Standalone runnable scripts under examples/ — each demonstrates a different facet of hypervector computing. Click through for the walkthrough.

Operators from Scratch

Standalone script: operators.py

This example implements the bind, release, and bundle operators in pure Python using only the low-level offset API, then verifies correctness against the library’s built-in implementations.

The script does not call hv.bind(), hv.release(), or hv.bundle() for computation — it reimplements them to show how they work at the offset level.

Bind

Per-segment offset addition modulo segment size:

for seg in range(cardinality):
    result[seg] = (core_a.offset(seg) + core_b.offset(seg)) % segment_size

Properties:

• Result is nearly orthogonal to both inputs (overlap ≈ 1)
• Commutative: bind(a, b) == bind(b, a)
• Associative: bind(a, b, c) == bind(bind(a, b), c)

Release (Unbind)

Per-segment offset subtraction modulo segment size:

for seg in range(cardinality):
    result[seg] = (core_c.offset(seg) - core_k.offset(seg)) % segment_size

Properties:

• release(bind(a, b), b) = a (exact recovery)
• Multi-release: release(release(bind(a, b, c), c), b) = a

Bundle

PRNG-based random selection among inputs. For each segment, a seeded PRNG picks which input vector contributes its offset. The selection probability is proportional to each input’s weight.

# Compute cumulative anchors from weights (weights sum to 1.0).
# For equal weights [0.33, 0.33, 0.33]: anchors ≈ [21845, 43690, 65535]
# For weighted [0.6, 0.2, 0.2]:        anchors ≈ [39321, 52428, 65535]
cumulative = 0.0
anchors = []
for w in weights:
    cumulative += w
    anchors.append(int(cumulative * 65535))

for seg in range(0, cardinality, 4):
    r = so.uint64()                          # one PRNG call → 4 × 16-bit values
    for j in range(4):
        dial = (r >> (48 - 16 * j)) & 0xFFFF # extract 16-bit random value
        chosen = first input whose anchor >= dial
        result[seg + j] = cores[chosen].offset(seg + j)

Properties:

• Result is similar to all inputs (overlap ≈ weight × cardinality)
• Not reversible — information is lost

Note: The library’s bundle folds inputs into a Learner sequentially, each replacing a weight-proportional random subset of segments. The per-segment selection frequencies match the classic draw shown above, but the exact segment choices are internal state, not public contract — so the script verifies bind/release bit-for-bit and bundle statistically.

Running

pip install kongming-rs-hv

python operators.py

Mexican Dollar

Standalone scripts: mexican_dollar.py | mexican_dollar_memory.py

The “What’s the Dollar of Mexico?” problem is a classic demonstration of analogical reasoning with hypervectors. It shows how structured knowledge about countries can be encoded, and how algebraic operations can answer analogy questions without explicit programming.

The Problem

Given knowledge about three countries:

We want to answer questions like:

• “What is the Dollar of Mexico?” → Peso
• “What is the Washington DC of Mexico?” → Mexico City
• “What is the Dollar of Sweden?” → Krona

How It Works

Each country is encoded as a bundled set of role-filler bindings:

$$\text{US}=\sum _{\oplus }\left(\text{code}\otimes \text{usa},\text{capital}\otimes \text{dc},\text{currency}\otimes \text{dollar}\right)$$

$$\text{Mexico}=\sum _{\oplus }\left(\text{code}\otimes \text{mex},\text{capital}\otimes \text{mexico\_city},\text{currency}\otimes \text{peso}\right)$$

$$\text{Sweden}=\sum _{\oplus }\left(\text{code}\otimes \text{swe},\text{capital}\otimes \text{stockholm},\text{currency}\otimes \text{krona}\right)$$

To find “the Dollar of Mexico”, we compute a transfer vector from US to Mexico:

$$T_{\text{US}\to \text{Mexico}}=\text{Mexico}\oslash \text{US}$$

Then apply it to Dollar:

$$\text{result}=\text{dollar}\otimes T_{\text{US}\to \text{Mexico}}$$

The result will have high overlap with Peso — the analogical answer.

The same transfer works for Sweden:

$$T_{\text{US}\to \text{Sweden}}=\text{Sweden}\oslash \text{US}$$

$$\text{result}=\text{dollar}\otimes T_{\text{US}\to \text{Sweden}}\approx \text{krona}$$

Code (Manual)

Full script: mexican_dollar.py. The essence — each country is a bundle of role ⊗ filler pairs, and one release + one bind answers the analogy:

us_record = hv.bundle(hv.Seed128.random(so),
    hv.bind(country_code, usa), hv.bind(capital, dc), hv.bind(currency, dollar))
# ... mexico_record, sweden_record likewise ...

transfer_to_mexico = hv.release(mexico_record, us_record)
mexican_dollar = hv.bind(dollar, transfer_to_mexico)

hv.overlap(mexican_dollar, peso)    # 32/32 — the answer
hv.overlap(mexican_dollar, dollar)  #  2    — noise
hv.overlap(mexican_dollar, krona)   #  0    — noise

The same transfer answers “the Washington DC of Mexico?” (→ mexico_city, 29/32) and, via release(sweden_record, us_record), “the Dollar of Sweden?” (→ krona, 26/32).

Code (with AnalogicalReasoner)

Full script: mexican_dollar_memory.py. When the country records live in storage — filler terminals plus one Octopus per country, staged via the producer API — analogical_reasoner does the transfer for you:

result = memory.first_picked(view,
    memory.nns(
        memory.analogical_reasoner(
            memory.with_code(mex_code), src=us_code, feature=fillers["USD"])))
print(result.id)  # → ✨:🌱MXN

analogical_reasoner computes the transfer vector feature ⊗ inverse(src) internally and uses near-neighbor search to find the best match in memory — no manual algebra needed.

Why It Works

The transfer vector $T=\text{Mexico}\oslash \text{US}$ captures the structural mapping between the two records. When applied to any filler from the US record, it maps it to the corresponding filler in the Mexico record — because the role-filler binding structure is preserved by the algebra.

This is a form of analogical reasoning: no explicit rules, no lookup tables — just algebraic operations on high-dimensional vectors.

See Also

• Concepts: Operators — algebraic foundations
• Operators — bind, release, bundle
• Octopus — key-value composite used for country records
• Memory: Selectors — analogical_reasoner, nns, with_code
• Near-neighbor search — how the reasoner finds answers

Bulk Storage Benchmark

Standalone script: bulk_storage.py

This example populates a storage with a large number of random terminal chunks, then queries a few by key to verify correctness. It demonstrates how to batch-create items and measure throughput.

Note associative index is also prepared in the process, and near-neighbor search is available immediately upon successful conclusion of all writing.

Motivated readers can further improve this script to test various producers or selectors.

What it does

Full script: bulk_storage.py. The core loop is three lines — pick a backend, write N terminals through mem_set, then spot-check ids:

storage = memory.InMemory(args.model)   # or memory.Embedded(args.model, path)

for i in range(args.count):
    storage.mem_set(memory.new_terminal(args.domain, str(i)))

# verify: stored id must equal the deterministic Sparkle for the same key
chunk = storage.get(args.domain, str(idx))
assert hv.equal(chunk.id, hv.Sparkle.from_word(args.model, args.domain, str(idx)))

The script wraps this with argparse (count, model, backend, path), timing, and throughput reporting.

Usage

# Default: 10K chunks, in-memory storage substrate.
python bulk_storage.py

# Embedded (disk-backed storage substrate).
python bulk_storage.py --backend embedded

# Embedded with a specific path (tip: use a tmpfs mount for near-in-memory speed)
python bulk_storage.py --backend embedded --path /dev/shm/my_bench

# Custom count
python bulk_storage.py -n 100000

# Different model, 1 implies MODEL_64K_8BIT model, etc.
python bulk_storage.py -n 10000 --model 1

Word Indexer

Standalone script: word_indexer.py

This example encodes ~5,000 English words as Sequences of per-letter Sparkles, then queries them by exact word or by positional suffix (“six-letter words ending in er”, “eleven-letter words ending in tion”) using multi-attractor near-neighbor search.

It demonstrates four ideas together:

• Using Sparkle as a stable per-symbol code (one Sparkle per a–z).
• Using Sequence with a Pod-derived seed so chunks are addressable both by word (exact) and by structure (positional).
• The ChunkProducer API (new_terminal, from_sequence_members, joiner) staged through a batched SubstrateMutableView via producer.produce(view).
• Multi-attractor similar search over SequenceAttractors for positional conjunctive queries.

The general idea

letters domain                       words domain
─────────────                        ────────────
"a" → Sparkle_a                      "the"      → Sequence(t, h, e)
"b" → Sparkle_b                      "language" → Sequence(l, a, n, g, u, a, g, e)
"c" → Sparkle_c                      ...
...                                  Pod   = word         (exact lookup key)
"z" → Sparkle_z                      note  = word         (recoverable in results)
                                     members = letter Sparkles in order

• Letters as Sparkles. Pre-write 26 random-looking Sparkles, one per a–z, into a letters domain via new_terminal(letters, ch). Each letter’s Pod is the letter itself, so you can fetch it by by_item_key("letters", "e").
• Words as Sequences. Each word is a Sequence in a words domain whose ordered members are the letter-Sparkles spelling it, built by from_sequence_members(...) with a joiner(...) of per-letter by_item_key selectors. The Sequence’s Pod is the word, so exact lookup is by_item_key("words", "language").
• note carries the word string. Each word-chunk is written with note=<word>, so chunk.note recovers the word in result loops without decoding the Pod.

Batched writes via the ChunkProducer API

This example uses the producer API end-to-end. Producers compute their chunks at produce() time against a mutable view, mirroring Go’s producer.Produce(ctx, view) and Rust’s producer.produce(view, index). Storage’s new_mutable_view() is a context manager with transactional semantics:

• All writes staged by producer.produce(view) calls between __enter__ and __exit__ go into a single batch.
• Clean exit auto-commits; an exception inside the block discards everything.
• view.commit() mid-block flushes the current batch and lets you continue staging — useful for pacing memory pressure on large ingests.

Letters and words go into two separate views; the second commits every BATCH_SIZE = 1000 words:

with storage.new_mutable_view() as view:
    for ch in "abcdefghijklmnopqrstuvwxyz":
        memory.new_terminal("letters", ch).produce(view)
    # auto-commits on __exit__

with storage.new_mutable_view() as view:
    for i, w in enumerate(words, start=1):
        members = memory.joiner(*[memory.by_item_key("letters", ch) for ch in w])
        # enable_semantic_indexing=True: index the Sequence's code so suffix
        # queries (sequence_attractor) can find words by structure.
        memory.from_sequence_members(
            "words", w, members, note=w, enable_semantic_indexing=True,
        ).produce(view)
        if i % BATCH_SIZE == 0:
            view.commit()
    # trailing writes auto-commit on __exit__

See Substrate & Views for the full view API.

Multi-attractor NNS

A sequence_attractor(member_selector, pos, domain) is a positional constraint: “Sequences in domain whose member at pos overlaps with member_selector”. Position is 0-based.

similar(joiner(...)) evaluates all attractors and ranks Sequences by combined overlap. With multiple attractors, the result is a conjunction — a chunk must satisfy each positional constraint to score well.

For “six-letter words ending in er”:

memory.similar(
    memory.joiner(
        memory.sequence_attractor(memory.by_item_key("letters", "e"), 4, WORDS_DOMAIN),
        memory.sequence_attractor(memory.by_item_key("letters", "r"), 5, WORDS_DOMAIN),
    )
)

This returns Sequences with e at index 4 and r at index 5 — i.e., the last two characters of a six-letter word.

For “eleven-letter words ending in tion”, anchor t/i/o/n at positions 7/8/9/10.

Counting and ranged results

storage.mem_get(selector) returns the full ranked result list as a Python list. Two helpers shape the output:

range_sel(inner, start, limit) consumes its inner selector, so to demonstrate both count and top 10 the example builds the NNS selector twice (cheap; the substrate work dominates).

See Working with Results for more on shaping selector output. When you need per-result SelectorExtra (e.g. NNS scores) or lazy iteration, reach for memory.lazy_selector_iter(view, selector) — mem_get returns Chunks only.

Running

pip install kongming-rs-hv
python examples/word_indexer/word_indexer.py

Expected output shape:

Ingested 4982 words in 40s.

by word 'the': 1 match(es) [0.9 ms]
   1. the

by word 'people': 1 match(es) [0.1 ms]
   1. people

****er  (6 letters): 712 match(es) [~30 ms]
   1. closer
   ...

*******tion (11 letters): 847 match(es) [~25 ms]
   1. information
   ...

Approximate timings on an Apple Silicon laptop with the InMemory backend:

A note on enable_semantic_indexing

For NNS by composite structure (i.e. “find Sequences whose member at position N matches X”), each word’s producer is constructed with enable_semantic_indexing=True. This impresses the Sequence’s code into the associative index alongside the chunk’s id-Sparkle (which is always indexed). Without the flag, only the id is indexed and sequence_attractor queries return zero hits.

The letter terminals are written without the flag because their code is the id-Sparkle, so id-only indexing is sufficient.

Switching to persistent storage

InMemory is fine for a demo. For a persistent store, swap one line:

storage = memory.Embedded(MODEL, "/path/to/db")

Everything else is identical.

Data attribution

Word-frequency data in top5000.txt is sourced from www.wordfrequency.info (top-5000 English words). Please credit the source when reusing this data.

Format (tab-separated, no header):

Rank    Word    POS    Frequency    Dispersion

See Also

• Sparkle ✨ — per-symbol code used for letters
• Sequence 📿 — ordered composite used for words
• Attractors — sequence_attractor and friends
• Near-Neighbor Search — nns over multiple attractors
• Substrate & Views — batched mutable views
• Working with Results — range_sel, mem_get

LISP Interpreter

A LISP interpreter where every data structure — atoms, cons cells, lists, closures — is encoded as a hypervector. No traditional memory allocation, no pointers, no garbage collector. All computation happens through hypervector algebra.

Two Implementations

The LISP interpreter ships in two forms, both feature-identical:

Rust (built-in)

The kongming-rs-hv package includes a Rust-based LISP interpreter built directly on the internal Rust API and primitives. This implementation is compiled into the Python wheel and accessible via from kongming.lisp import LispEnv.

Since it operates on Rust-native hypervector types with zero Python overhead, it delivers the best performance for production use.

Python (open-source)

For research and study, we provide a pure-Python implementation of the same interpreter, built entirely on the public Python API of kongming-rs-hv. It mirrors the Rust implementation’s architecture but uses Python-level operations (hv.bind, hv.bundle, hv.release, etc.), making the underlying hypervector mechanics fully transparent and easy to modify.

This implementation is ideal for:

• Understanding how LISP primitives map to hypervector operations
• Experimenting with alternative encodings or evaluation strategies
• Teaching and prototyping

Quick Start

pip install kongming-rs-hv

# Pure Python
from kongming.pylisp import LispEnv

env = LispEnv()
env.eval("(CAR (QUOTE (A B C)))")       # => "A"
env.eval("(CDR (QUOTE (A B C)))")       # => "(B C)"
env.eval("(CONS (QUOTE A) (QUOTE B))")  # => "(A . B)"

# Rust (same API, same results)
from kongming.lisp import LispEnv

env = LispEnv()
env.eval("(CAR (QUOTE (A B C)))")       # => "A"

Supported Forms

McCarthy’s 7 Primitives (1960)

Extensions

Examples

# Lambda
env.eval("((LAMBDA (X) (CAR X)) (QUOTE (A B C)))")  # => "A"

# Define a reusable function
env.eval("(DEFINE SECOND (LAMBDA (L) (CAR (CDR L))))")
env.eval("(SECOND (QUOTE (X Y Z)))")                 # => "Y"

# Recursion with LABEL
env.eval(
    "(DEFINE LAST (LAMBDA (L) "
    "  ((LABEL REC (LAMBDA (X) "
    "    (COND ((ATOM (CDR X)) (CAR X)) "
    "          (T (REC (CDR X)))))) L)))"
)
env.eval_full("(LAST (QUOTE (A B C)))")              # => "C"

How It Works

Each LISP value is a Sparkle — a sparse binary hypervector seeded by its content. Atoms like A, B, CAR are sparkles in a symbol domain.

A cons cell (a . b) is encoded as:

cell = bundle(bind(a, LHS), bind(b, RHS))

where LHS and RHS are fixed tag sparkles. The cell is stored under a fresh random sparkle id. To extract:

car(id) = cleanup(release(cell, LHS))
cdr(id) = cleanup(release(cell, RHS))

The release operation is noisy — it produces an approximate result. Cleanup uses near-neighbor search (NNS) over the substrate’s associative index to find the exact stored sparkle that best matches the noisy probe.

File Structure

examples/pylisp/
  __init__.py      # Package entry point
  types.py         # HyperBinary type alias
  env.py           # LispEnv: domains, symbols, lexicon, substrate
  cons.py          # Cons cells: cons, car, cdr, cleanup via NNS
  reader.py        # S-expression tokenizer and parser
  evaluator.py     # Single-step and fixed-point evaluator
  lambda_.py       # Beta-reduction with currying and shadowing
  printer.py       # Hypervector → S-expression display
  test_pylisp.py   # 16 tests mirroring the Rust integration suite

Storage Backends

# In-memory (default, volatile)
env = LispEnv()

# Persistent (Embedded disk-backed)
env = LispEnv(path="/tmp/my_lisp_db")

Running Tests

pip install pytest kongming-rs-hv
pytest examples/pylisp/test_pylisp.py -v

Notebook

We provide a Colab notebook that runs both implementations side by side, demonstrating correctness parity and performance comparison:

References

• Hey, Pentti, We Did it!: A fully Vector-Symbolic Lisp — the paper that inspired this implementation
• Peter Norvig’s (How to Write a (Lisp) Interpreter (in Python)) — the original Lispy that the Python implementation builds upon

Dependency Parser

Everything in this book so far — Sparkles, composites, learners, pools, near-neighbor search — was built for a purpose: a demonstration that VSA can offer a unique and novel perspective in cognitive computing / AI.

This chapter serves that purpose: a dependency parser in which the entire language model is a hypervector substrate.

Unlike traditional NLP with heavy reliance on explicit frequency tables, and unlike neural networks where gradients are computed via backpropagation, our training/inference features:

• A transparent representation of the underlying language models that encourages inspection and enables incremental improvements;
• The language models are generic in the sense that new languages can be added without idiosyncrasy or much tweaking;
• An efficient representation far more compact than existing models, see Evaluations;
• An efficient computation with mostly binary operations, no need for floating-point computations or expensive GPUs.

Wernicke’s area in the brain is widely hypothesized to host the generic neural circuitry for language understanding: the solution I hope to present here will be the computational counterpart of it.

For readers new to this topic (or to this project), the best starting point is my paper, Cognitive modeling and learning with sparse binary hypervectors (full citation in the introduction): it lays the foundation — sparse binary hypervectors and their operators — that everything in this chapter builds on.

The project is divided into the following pages:

Introduction

The original idea came from a random bump into the stanza NLP package (from Stanford) and its web-based rendering.

What the website shows is a dependency parse for human languages: the sentence split into tokens, each token tagged with its part of speech, lemma, and named-entity type, and, for the interesting part, every token is attached to a head token by a labeled edge (subject-of, object-of, modifier-of, …). The nodes and edges form a dependency tree representing the sentence’s grammatical structure.

Parsing is one of the fundamental tasks in NLP, and serves as a pre-processor for many downstream language-understanding applications: information extraction, question answering, knowledge-graph construction all start from exactly these parse trees.

A brief history, and where we are

The dominant technique before neural networks was the HMM family: Hidden Markov Models and their hierarchical variants, where decoding recovers a sequence of latent states with learned transition and emission probabilities. Most modern parsers have since switched to specially-trained neural networks, which lifted accuracy substantially at the cost of opaque models, heavy computation, and hundreds of megabytes of storage per language model.

State-space models, generally

HMMs belong to a much larger family: state-space models (SSMs). The shared idea is fundamentally simple: a latent state summarizes everything about the past that matters, a transition rule evolves it step by step, and each observation is emitted from the current state. Choose discrete states and you have the HMM; choose linear-Gaussian ones and you have the Kalman filter; let a network learn the transition and you have an RNN. Under this lens, sequence understanding is always the same job: maintain a compressed state, and keep it honest against the observations.

SSMs are having a renaissance. Modern deep-learning variants (the S4/Mamba line) rediscovered that a recurrent state gives linear-time, constant-memory sequence processing, where attention pays quadratic cost and must hold the whole window, and made it competitive at scale.

The classical virtues of SSMs never went away either: explicit and inspectable states, principled inference (Viterbi decoding, Kalman filtering) instead of learned approximation, and reasonable model performance from a modest amount of data. What killed the classical SSM in practice was the state-space explosion: in NLP domain, rich linguistic states make the transition tables astronomically large and their statistics hopelessly sparse.

The parser in this project is best understood as a revival of the hierarchical HMM rebuilt on hypervectors, attacking that exact weakness head-on. High-dimensionality offers practically unlimited orthogonal vectors to work with, superposition keeps an enormous state space, potentially hierarchical in the same shape, and evidence pooling tames the sparse statistics.

It keeps the SSM virtues the neural approximations have given up: transparency and inference you can trace. The latent states are still there: they are spines of grammar edges, and decoding is still Viterbi over a beam. What changed is the substrate: instead of explicit probability tables, every statistic lives in superposed sparse binary vectors, learned over time and read back efficiently and on-demand by similarity.

The goal

Functionally, the parser sets out to do what the stanza demo does: given a tokenized sentence in English, Chinese, or any desired language in the future, it emits the same artifacts: per-token part of speech, lemma, named-entity spans, and the labeled head edges that form a complete dependency tree. Text corpora of different languages are condensed into their own language models which can generalize, and the same engine runs decoding with no language-specific priors or code paths.

Matching that output contract, however, is the baseline rather than the point. The focus here is not to achieve performance parity with state-of-the-art neural networks, but to demonstrate the feasibility of an alternative model with the following characterizations:

• compaction — the trained model for 2 languages (English and Chinese for now) fits in tens of MBs (see Evaluations for more details);
• efficiency — training is a single pass, and decoding is mostly integer/bitwise computation with no GPU needed;
• transparency — every decoding decision is transparent: you can print, trace, and improve it incrementally.

Bootstrapping

Unsurprisingly, this parser learns from annotated trees. The annotations are produced by an existing parser (stanza, following the Universal Dependencies conventions): a typical teacher–student setup. The teacher supplies a training tree for each incoming sentence, and the student learns to reproduce and generalize them in an entirely different representation. The quality ceiling is therefore the teacher’s; the demonstration is about the representation, not about outrunning the state of the art.

Further reading

The rest of the chapter walks the pipeline:

• Training: how trees become substrate writes;
• Decoding: how token streams become trees again;
• Evaluations: measured quality;
• Discussions: why it works, and what’s next.

References

State-space models, classical and modern:

• L. R. Rabiner — A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. Proc. IEEE 77(2), 1989. DOI
• S. Fine, Y. Singer, N. Tishby — The Hierarchical Hidden Markov Model: Analysis and Applications. Machine Learning 32(1), 1998. DOI
• A. Gu, K. Goel, C. Ré — Efficiently Modeling Long Sequences with Structured State Spaces (S4). ICLR 2022. arXiv:2111.00396
• A. Gu, T. Dao — Mamba: Linear-Time Sequence Modeling with Selective State Spaces. 2023. arXiv:2312.00752
• T. Dao, A. Gu — Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality (Mamba-2). ICML 2024. arXiv:2405.21060

Parsing and annotations:

• P. Qi, Y. Zhang, Y. Zhang, J. Bolton, C. D. Manning — Stanza: A Python Natural Language Processing Toolkit for Many Human Languages. ACL 2020 (System Demonstrations). arXiv:2003.07082
• M.-C. de Marneffe, C. D. Manning, J. Nivre, D. Zeman — Universal Dependencies. Computational Linguistics 47(2), 2021. ACL Anthology

Training

Training turns a corpus of parsed sentences into a substrate, in a single pass.

This page walks the training pipeline conceptually: from raw text to what, exactly, ends up as a fully-functional language model.

Preparing the corpus

For reference, we used 4231 English and 4231 Chinese sentences from the public Wikipedia corpus for training. The training itself can be done on a single MacBook Air in ~12 minutes.

Preparing the text

The corpus starts as plain text, one batch per language: curated sentences covering the vocabulary and constructions the model should absorb and later generalize in typical decoding scenarios.

The text is split into individual sentences up front, and each sentence is handed to the annotation step on its own, one sentence per request.

Bootstrapping the parsed trees

A teacher parser (i.e., stanza) annotates each incoming sentence: tokens, part of speech, lemma, morphological features, named-entity spans, and the head arc that makes the dependency tree. The annotated sentences are stored as compact per-language archives: training doesn’t need the live parser or the raw text, for efficiency reasons.

Beyond various annotations, the training pipeline further constructs a dependency tree per sentence. The tree has a single ROOT, with each leaf node mapping to a surface token and each arc annotated with its dependency relation, following the Universal Dependencies conventions.

From trees to spines

The local grammar and syntactic dependency are captured by the nodes and edges of the dependency tree.

A feats captures a unique combination of rich linguistic features, such as part-of-speech, lemma, incoming dependency label, and structural markers such as is_head (indicating whether this token is the head among its siblings, under their common parent). An edge, on the other hand, is the parent and child feats pair, as a deterministic Sparkle instance.

Each surface token is ultimately emitted by its spine, the hierarchy of interlocked edges from the root down. Siblings under a subtree share edges up to the subtree root, before diverging into their unique edges.

Within the state-space setup, the latent state at token t is that token’s spine, namely its full hierarchical path in the tree, from the root edge down to its leaf. Concretely, a state is a Sequence whose members are the spine’s edges.

Three properties make this state space feasible where classical HMMs can choke:

• Representation of states: the space of possible spines is combinatorially vast, which hypervectors have no problem handling. A state, at its core, is conceptually just another hypervector, sharing the same shape as any other state, with a potentially different length. Practically, thanks to lazy materialization, composing a state on the fly reduces to simple bookkeeping, with very little extra cost;
• No tabulation: instead of highly sparse frequency tables, a learner is a far more natural representation for encoding probabilities, only at cells where transitions do happen. Furthermore, LearnerPool offers an elegant solution for the high dynamic range of fan-outs;
• Content-addressable by construction: two surface tokens emitted by the same grammatical path share the same state, by construction: that identity is what lets transition statistics pool.

The downstream inference task, naturally, is to recover the most likely sequence of states/spines, before reconstructing the dependency tree.

Looking back, hierarchical inference over such a state space is a generic problem, applicable well beyond linguistic parsing: anywhere the potential state space is too immense to tabulate. A robust training/inference pipeline that accommodates a vast and rich state space is a valuable by-product of this project in its own right.

What is learned

Two kinds of learning/writes happen, side by side.

The inventory — ordinary chunks, written once (create-if-missing, idempotent across repeats):

• every distinct feats: the collection of all known combinations of linguistic features in a language;
• every distinct edge: its parent-and-child pair of feats;
• every distinct token — an Octopus containing feats, lemma, and surface text (this is where the lemma can be recovered from the surface token at decode time);
• every named entity (see below).

The statistics — writes into one per-language LearnerPool, three conceptual families:

Every occurrence adds an extra piece of information, and the frequency is the statistic: a transition seen a thousand times reads back a thousand times stronger than one seen once, at least in principle.

Nothing is normalized at training time, as a self-normalization process is at play: observations compete for a Learner’s fixed representational budget, so read-back strengths are effectively relative probabilities.

The sentinels: BEGIN and END

Every sibling chain is conceptually bracketed: BEGIN → e₁ → e₂ → … → END. The two brackets are treated very differently, and the asymmetry is deliberate:

END is trained. The last child’s out-transition to END is recorded like any other, so every edge carries a measured probability that its chain closes after it, which is needed by the decoder.

BEGIN is not trained as a transition source. “How do chains open?” has enormous fan-out at fine granularity (thousands of distinct edges): one address accumulating a transition to every chain-opening edge in the corpus would saturate its pool members and the statistic would drown. Instead, chains open through the down family — the first child’s edge is the parent’s climb anchor.

So BEGIN’s information is actually trained and recorded in the down family instead of the out family. Every down entry is implicitly a BEGIN target: an edge that begins a subtree under its parent (the sentence root is merely the outermost instance). The same relation is addressed from the opposite — and favorable — low-fan-out side: sharded per parent across thousands of addresses instead of concentrated at one — and the decoder recovers the chain-opening probability hop-wise through observation confidence.

Named entities

Entities get special treatment because they behave like single tokens with internal structure. Entity members carry the entity’s signature (a content hash), so entity-internal edges are distinct identities — “York” under “New York” is not confused with a generic proper-noun attachment.

An entity’s internal membership is trained exactly once, as the members are fixed for each named entity, by definition.

See also

• Composites: encoding sequences and hierarchical states
• Learner: encoding transition probabilities
• LearnerPool: accommodating the high dynamic range of fan-outs

Decoding

Decoding inverts training: tokens arrive one at a time, and the linguistic parser must recover the most likely sequence of states that could have emitted them, then eventually convert that sequence back into a dependency tree.

The engine is a Viterbi beam over the hierarchical state space, and this page walks one token’s journey through each step.

What the decoder carries

Between tokens, the decoder maintains a beam: the $K$ best hypotheses so far. Each surviving state itself is a spine, outlining the hierarchical path for the previous token, plus its accumulated Viterbi score $\alpha$. True to form, the beam slice itself is stored as a superposition of all surviving state vectors, weighted by their confidence, plus a parallel superposition of backpointers recording where each state came from.

At the start of a sentence the beam holds a single seed state anchored at BEGIN, and each primed language contributes its own seed — the beam arbitrates between languages on evidence alone, with no explicit language switch anywhere in the engine: we have a natural way to handle multi-lingual tokens.

Decoding cycle

1. Start as candidate leaves

The obs family answers: which leaf edges may have emitted this word? Each answer is a candidate leaf, already weighted by observation frequency.

2. Climb

From each candidate leaf, the decoder climbs the down family: each hop reads “whose child is this edge?” and yields the plausible parents, again frequency-weighted. The result is a set of extended candidate spines, each carrying a joint observation confidence.

3. Graft and score

Each candidate spine may attach to each of the beam’s previous states at any layer of that state’s spine. For every (prev_state, layer, candidate) triple, admission is a single fitness test to the out family: does the candidate’s top edge fit the prev_state’s edge at this layer? The answer becomes the transition confidence. Two structural rules apply:

• The root gate. Layer zero hosts only root edges; a mid-tree edge proposing itself as a new root is structurally illegal and is dropped without a read. Genuine root openings get a uniform prior instead — this is the flip side of BEGIN being untrained (see Training).
• The conclusion discount. Grafting at layer L implicitly closes every existing layer below L for the previous state: each close-pending level pays its measured END cost. An expected closing costs almost nothing, a surprising one costs considerably more. This is how the decoder balances “attach deep, continue the phrase” against “attach high, close the clause” with statistics instead of heuristics.

The candidate’s score is then the classic Viterbi:

$$\alpha ={\alpha }_{prev}\times P_{transition}\times P_{observation}$$

4. Ghosts

If no real candidate survived the placement for the current beam, that might be the case for an out-of-vocabulary word, or a simple typo. The decoder asks each beam state’s leaf edge for its predictions without observations. The best few become ghost candidates, priced low but non-zero, while participating in continuing the beam.

One guard applies: a ghost is purely prediction, not an observation, so any named-entity claim it carries is checked against the actual input text and stripped on mismatch — no phantom entities.

5. Select

All admitted hypotheses, real and ghost, are ranked together by confidence $\alpha$ and only the top $K$ survive. The new states are superposed, along with their backpointers, and the cycle begins.

Backtrace

When the sentence ends, the best-scoring endpoint is unwound through the backpointers, recovering the winning state sequence: one spine per token, in order. This is the Viterbi answer: the most likely path through the state space, given everything observed so far.

Reassembly

The spine sequence is folded back into a dependency tree: tokens are bucketed by their parent edge, the is_head marker identifies exactly the head member of each subtree, to which all other members attach.

Along the way each token’s lemma is recovered from its trained token record, and entity spans are re-emitted from the carried annotations. The output is the same artifact the teacher produced: a full dependency parse.

Properties worth noticing

• Streaming: one token in, one beam update; the carried state (of the decoder) is the superposition of surviving hypotheses, fixed size regardless of sentence length and hypothesis count.
• Inspectable: every admission, discount, and ranking above is an integer you can print: there is no layer of the decision you cannot open.

See also

• LearnerPool — the access-circle reads under every step.

Evaluations

Setup

Note: the numbers below were measured on the Rust engine over the full held-out validation tier (the Go and Rust engines are kept at bit-parity and decode identically).

Combined corpus of 4231 English + 4231 Chinese sentences for training; the statistics live in a unified LearnerPool (MODEL_64K_8BIT) per language.

Validation tier: 950 held-out sentences — 484 English (5.8K tokens) and 466 Chinese (5.1K tokens), never seen in training.

Substrate stats

The entire trained model — every transition statistic and open-class inventory for English and Chinese — is a 17 MB substrate on disk.

The trained substrate: 336,564 chunks, with Payload types:

• 192,731 SPARKLE;
• 58,129 OCTOPUS;
• 85,704 LEARNER, that made up 2 LearnerPools;

Pool health, per 65,536-member language pool:

Held-out parse quality

Production configuration (coarse tier on), per language and combined (full-tier run on the pre-redesign substrate; the sampled tables below are current):

Entity matching is a multiset match on (type, text) between the gold entities and the surfaced ones, over parsed sentences.

Training corpus vs held-out

The same sweep per language over sampled tiers (English at 10%, Chinese at 5%), on the consolidated substrate: the training corpus bounds what the substrate retained; the held-out gap is the generalization cost.

The coarse tier’s contribution

Ablating the coarse-to-fine backoff on the same substrate and the same full validation tier (tier off → on), head attachment:

The gains come from trained coarse-class evidence rescuing transitions the fine-grained statistics never saw — not from structural guessing.

Decode throughput

Single decoder, sequential decode, on the same MacBook Air that trains the model in ~12 minutes (sampled tiers, consolidated substrate):

The width-addressed dense posting index (no hashing on the decode-side Collect/Add) shaved ~10 ms/token off both languages on top of the pool consolidation: +23% for English; Chinese remains dominated by pool-read volume from beam churn, not per-probe cost.

Discussions

State-space models, the generic learning system

The state-space framing suggests this parser is a special case of something more general. Where a state space is too immense to tabulate, possibly due to the hierarchical nature of the real world, the classical approach becomes fundamentally intractable. Hypervectors sidestep this because a hierarchical state keeps the same fixed width as a flat one, with simple and intuitive bookkeeping from this package.