Native: find_nearest_approximate sends the query by address and reads the hits from one buffer, not as JSON (refs #388, #315)

docs/modules/ROOT/pages/how-to/vectors.adoc

@@ -459,6 +459,8 @@ These rules hold for the quantization arguments of the method:
 * `encoding="INT8"` is for a `BINARY` property that already holds INT8 bytes. Set `quantization` to `"NONE"` with it. The call raises `ValueError` when both are `INT8`, because the vector would be quantized twice.
 * A bucket that holds fewer vectors than `pq_clusters` is too small to train a codebook. The index still builds: the engine skips PQ for that bucket and searches it exactly, so `find_nearest_approximate` returns the exact neighbours with exact scores from it. A bucket with `pq_clusters` vectors or more is trained and answers approximately.
 
+On the `arcadedb` (native) package, `find_nearest_approximate` hands the engine its query as a float32 array by address and reads the hits from one buffer that Python provides, with no JSON text on either side. The call returns the same list as before: the same records in the same order, the same RID texts, and scores of the same type with the same bits. A query crosses this way when it is a one-dimensional `float16`, `float32` or `float64` NumPy array, or a list or tuple of `float` and `int` values (an `int` beyond 2**53 is not one), and has no NaN or infinity in it; an `allowed_rids` filter crosses this way when every entry is a plain `#bucket:position` text or a record. Anything else keeps the JSON text route, which returns the same hits and only takes longer: a generator, a NumPy scalar or `None` in a list, an integer array, an empty query, a `float64` value that float32 cannot hold. `ARCADEDB_NATIVE_VINDEX_BIN=0` in the environment turns the address route off. A native library that is older than this change falls back to the text route by itself. `find_nearest` and the other searches are statements and are not changed by this: they already send a NumPy query as float32 and read their rows as a result set.
+
 == Rebuild the graph
 
 Inserts, updates and deletes collect as pending changes. When enough of them have collected, the engine rebuilds the graph in the background, and searches keep running. How many is enough comes from the database setting `arcadedb.vectorIndex.rebuildGraphRatio` and from `mutations_before_rebuild`; see xref:how-to/databases-and-queries.adoc#database-settings[Database settings] for how to change the ratio.

native/src/main/java/org/humemai/nativelib/ArcadeNative.java

@@ -681,6 +681,11 @@ public static CCharPointer insertColp(final IsolateThread thread, final long h,
 
   private static final ThreadLocal<String> LAST_ERROR = new ThreadLocal<>();
 
+  /** Records the error of an entry point that answers with a long (-1), for arcadedb_n_last_error on the same thread. */
+  static void recordError(final Throwable t) {
+    LAST_ERROR.set(errorJson(t));
+  }
+
   /**
    * A command (flags bit 0 selects command, else query) whose first rows are pulled but not encoded (as flags bit 1 of arcadedb_n_exec).
    * Returns the cursor id (> 0, read with arcadedb_n_next, closed with arcadedb_n_rs_close or arcadedb_n_rs_close_many), 0 when the

native/src/main/java/org/humemai/nativelib/NativeExtras.java

@@ -359,6 +359,7 @@ private static String q(final String text) {
    * ops: metadata {type, prop} | approximate {type, prop, vector, k, allowed} | build {type, prop}. The index is the one the type has on the
    * property; its bucket indexes are searched in turn (the caller merges). metadata describes the first bucket index, as the JPype wrapper does.
    * build rebuilds the graph of every bucket index now (buildVectorGraphNow), as the JPype wrapper's build_graph_now() does (#162).
+   * approximate has a binary twin, arcadedb_n_vindex_bin (below); this one stays for the queries that entry cannot carry.
    */
   @CEntryPoint(name = "arcadedb_n_vindex")
   @SuppressWarnings("unchecked")
@@ -367,15 +368,7 @@ public static CCharPointer vindex(final IsolateThread thread, final long dbh, fi
       final Database db = ArcadeNative.db(dbh);
       final Map<String, Object> a = args(argsJson);
       final String type = (String) a.get("type"), prop = (String) a.get("prop");
-      final TypeIndex typeIndex = db.getSchema().getType(type).getPolymorphicIndexByProperties(prop);
-      if (typeIndex == null)
-        throw new IllegalArgumentException("No vector index on " + type + "[" + prop + "]");
-      final List<LSMVectorIndex> parts = new ArrayList<>();
-      for (final IndexInternal index : typeIndex.getIndexesOnBuckets())
-        if (index instanceof LSMVectorIndex lsm)
-          parts.add(lsm);
-      if (parts.isEmpty())
-        throw new IllegalArgumentException("Underlying index is not an LSM vector index");
+      final List<LSMVectorIndex> parts = vectorParts(db, type, prop);
       final StringBuilder sb = new StringBuilder("{\"ok\":true");
       switch (ArcadeNative.str(op)) {
       case "metadata" -> {
@@ -413,15 +406,11 @@ public static CCharPointer vindex(final IsolateThread thread, final long dbh, fi
         }
         sb.append(",\"hits\":[");
         boolean firstHit = true;
-        for (final LSMVectorIndex lsm : parts) {
-          final List<Pair<RID, Float>> hits = allowed != null ? lsm.findNeighborsFromVectorApproximate(query, k, allowed)
-              : lsm.findNeighborsFromVectorApproximate(query, k);
-          for (final Pair<RID, Float> hit : hits) {
-            if (!firstHit)
-              sb.append(',');
-            firstHit = false;
-            sb.append("[\"").append(hit.getFirst()).append("\",").append(hit.getSecond()).append(']');
-          }
+        for (final Pair<RID, Float> hit : approximateHits(parts, query, k, allowed)) {
+          if (!firstHit)
+            sb.append(',');
+          firstHit = false;
+          sb.append("[\"").append(hit.getFirst()).append("\",").append(hit.getSecond()).append(']');
         }
         sb.append(']');
       }
@@ -437,6 +426,85 @@ public static CCharPointer vindex(final IsolateThread thread, final long dbh, fi
     }
   }
 
+  /** The bucket indexes of the vector index that `type` has on `prop`. */
+  private static List<LSMVectorIndex> vectorParts(final Database db, final String type, final String prop) {
+    final TypeIndex typeIndex = db.getSchema().getType(type).getPolymorphicIndexByProperties(prop);
+    if (typeIndex == null)
+      throw new IllegalArgumentException("No vector index on " + type + "[" + prop + "]");
+    final List<LSMVectorIndex> parts = new ArrayList<>();
+    for (final IndexInternal index : typeIndex.getIndexesOnBuckets())
+      if (index instanceof LSMVectorIndex lsm)
+        parts.add(lsm);
+    if (parts.isEmpty())
+      throw new IllegalArgumentException("Underlying index is not an LSM vector index");
+    return parts;
+  }
+
+  /** The PQ-scored hits of every bucket index in turn, one list (up to k hits per bucket index; the caller merges). */
+  private static List<Pair<RID, Float>> approximateHits(final List<LSMVectorIndex> parts, final float[] query, final int k,
+      final Set<RID> allowed) {
+    final List<Pair<RID, Float>> all = new ArrayList<>();
+    for (final LSMVectorIndex lsm : parts)
+      all.addAll(allowed != null ? lsm.findNeighborsFromVectorApproximate(query, k, allowed)
+          : lsm.findNeighborsFromVectorApproximate(query, k));
+    return all;
+  }
+
+  /**
+   * The "approximate" op of arcadedb_n_vindex with no JSON on either side: no number text for the query, no hit text for the answer.
+   * type and prop are C strings. The query is `dims` float32 values at `query` (the engine narrows the JSON route's numbers to float32
+   * as well, so the search sees the same floats). `allowed` holds `allowedPairs` (bucket, position) pairs of int64, or is NULL with 0
+   * for no filter. `out` has room for `capHits` hits of three int64 words each: the RID's bucket, its position, and the score as the bits of
+   * a double. The score is the double that the JSON route's number text of the same float parses to (Float.toString, then
+   * Double.parseDouble: the shortest decimal of the float), so a caller reads the same bits from either route.
+   *
+   * Returns the number of hits n >= 0. When n <= capHits the n hits are in `out`; when n is larger nothing is written, and the caller calls
+   * again with room for n. -1 is an error, its text is read with arcadedb_n_last_error on the same thread (no reply text: no malloc, no
+   * arcadedb_free and no JSON decode in the caller).
+   */
+  @CEntryPoint(name = "arcadedb_n_vindex_bin")
+  public static long vindexBin(final IsolateThread thread, final long dbh, final CCharPointer type, final CCharPointer prop, final int k,
+      final CFloatPointer query, final long dims, final CCharPointer allowed, final long allowedPairs, final CCharPointer out,
+      final long capHits) {
+    try {
+      if (dims < 0 || dims > Integer.MAX_VALUE || allowedPairs < 0 || capHits < 0 || capHits > Integer.MAX_VALUE / 24)
+        throw new IllegalArgumentException(
+            "vector search arguments out of range (dims " + dims + ", allowed " + allowedPairs + ", capacity " + capHits + ")");
+      if (dims > 0 && query.isNull())
+        throw new IllegalArgumentException("no query vector");
+      final Database db = ArcadeNative.db(dbh);
+      final List<LSMVectorIndex> parts = vectorParts(db, ArcadeNative.str(type), ArcadeNative.str(prop));
+      final float[] q = new float[(int) dims];
+      for (int i = 0; i < q.length; i++)
+        q[i] = query.read(i);
+      Set<RID> allowedSet = null;
+      if (allowedPairs > 0) {
+        if (allowed.isNull())
+          throw new IllegalArgumentException("no allowed RIDs");
+        final Pointer pairs = (Pointer) allowed;
+        allowedSet = new HashSet<>();
+        for (long i = 0; i < allowedPairs; i++)
+          allowedSet.add(
+              new RID((int) pairs.readLong(WordFactory.signed(16L * i)), pairs.readLong(WordFactory.signed(16L * i + 8))));
+      }
+      final List<Pair<RID, Float>> hits = approximateHits(parts, q, k, allowedSet);
+      final int n = hits.size();
+      if (n > capHits)
+        return n;
+      final Pointer o = (Pointer) out;
+      for (int i = 0; i < n; i++) {
+        final Pair<RID, Float> hit = hits.get(i);
+        o.writeLong(24 * i, hit.getFirst().getBucketId());
+        o.writeLong(24 * i + 8, hit.getFirst().getPosition());
+        o.writeLong(24 * i + 16, Double.doubleToRawLongBits(Double.parseDouble(Float.toString(hit.getSecond()))));
+      }
+      return n;
+    } catch (final Throwable t) {
+      ArcadeNative.recordError(t);
+      return -1;
+    }
+  }
+
   // ---- GraphBatch
 
   private record BatchEntry(GraphBatch batch, Database db) {

src/arcadedb/_native/lib.py

@@ -37,6 +37,9 @@
 # ARCADEDB_NATIVE_GB_BIN: the float32 vectors of a GraphBatch op (create_vertices rows, new_edges props) go through arcadedb_n_gb_bin in a float32 side buffer, not as number text
 GB_BIN = os.environ.get("ARCADEDB_NATIVE_GB_BIN", "1") != "0"
 
+# ARCADEDB_NATIVE_VINDEX_BIN: the query vector of find_nearest_approximate crosses as a float32 array by address and the hits come back in one packed buffer (arcadedb_n_vindex_bin), not as a JSON list of numbers and a JSON list of [rid, score]
+VINDEX_BIN = os.environ.get("ARCADEDB_NATIVE_VINDEX_BIN", "1") != "0"
+
 # the directory the engine log, logging.properties and the crash files go to (None until the library is loaded)
 LOG_DIR_USED = None
 
@@ -255,16 +258,18 @@ class _Params(ctypes.Structure):
             "arcadedb_n_async": [P, L, S, S],
             "arcadedb_n_async_bin": [P, L, S, S, P, N],
             "arcadedb_n_vindex": [P, L, S, S],
+            "arcadedb_n_vindex_bin": [P, L, S, S, N, P, L, P, L, P, L],
             "arcadedb_n_import": [P, L, S],
             "arcadedb_n_export": [P, L, S, S, N, S],
             "arcadedb_n_cb": [P, S, S],
             "arcadedb_n_alloc": [P, L],
             "arcadedb_n_set_upcall": [P, P],
             "arcadedb_free": [P, P],
         }
-        # the entry points that answer with a number, not with the address of a reply text: a cursor id (arcadedb_n_cmd) and the
-        # transaction state (arcadedb_n_tx); -1 is an error for both, its text is read with arcadedb_n_last_error
-        returns_long = ("arcadedb_n_cmd", "arcadedb_n_tx")
+        # the entry points that answer with a number, not with the address of a reply text: a cursor id (arcadedb_n_cmd), the
+        # transaction state (arcadedb_n_tx) and the hit count of arcadedb_n_vindex_bin; -1 is an error for all three, its text is
+        # read with arcadedb_n_last_error
+        returns_long = ("arcadedb_n_cmd", "arcadedb_n_tx", "arcadedb_n_vindex_bin")
         for name, argtypes in sigs.items():
             fn = getattr(lib, name, None)
             if fn is None:  # a library built before this entry point existed
@@ -280,7 +285,7 @@ class _Params(ctypes.Structure):
                 )
                 else ctypes.c_longlong if name in returns_long else ctypes.c_void_p
             )
-        global BINARY_PARAMS, CMD_PATH, CLOSE_BATCH, INSERT_ROWS, EXEC_MANY, NEXT_BIN, GB_BIN, TS_BIN
+        global BINARY_PARAMS, CMD_PATH, CLOSE_BATCH, INSERT_ROWS, EXEC_MANY, NEXT_BIN, GB_BIN, TS_BIN, VINDEX_BIN
         if not hasattr(
             lib, "arcadedb_n_exec_bin"
         ):  # a library built before the binary path
@@ -305,6 +310,10 @@ class _Params(ctypes.Structure):
             lib, "arcadedb_n_async_bin"
         ):  # a library built before the address table of append_samples
             TS_BIN = False
+        if not hasattr(
+            lib, "arcadedb_n_vindex_bin"
+        ):  # a library built before the binary approximate search
+            VINDEX_BIN = False
         _isolate = isolate
         _creator = threading.get_ident()
         _local.thread = _Thread(thread, owned=False)
@@ -505,8 +514,27 @@ def tx(handle, op) -> int:
     return lib.arcadedb_n_tx(th, handle, op)
 
 
+def vindex_bin(
+    handle, type_name, prop, k, query, dims, allowed, pairs, out, cap
+) -> int:
+    """``arcadedb_n_vindex_bin``: the approximate search of a vector index with no JSON on either side.
+
+    ``type_name`` and ``prop`` are bytes; ``query`` is the address of ``dims`` float32 values; ``allowed`` is the address of ``pairs``
+    (bucket, position) int64 pairs, or 0 with 0 pairs for no filter; ``out`` is the address of room for ``cap`` hits of three int64
+    words (bucket, position, the score as the bits of a double). Returns the hit count n: the n hits are in ``out`` when ``n <= cap``,
+    and nothing is written when it is larger (call again with room for n). -1 is an error, read with :func:`last_error`.
+    The caller keeps every buffer alive until the call returns.
+    """
+    lib = _lib or _load()
+    t = _local.__dict__.get("thread")
+    th = t.handle if t is not None else _thread_handle()
+    return lib.arcadedb_n_vindex_bin(
+        th, handle, type_name, prop, k, query, dims, allowed, pairs, out, cap
+    )
+
+
 def last_error() -> str:
-    """The error of the last failed :func:`cmd` or :func:`tx` on this thread, as the JSON text of an error reply."""
+    """The error of the last failed :func:`cmd`, :func:`tx` or :func:`vindex_bin` on this thread, as the JSON text of an error reply."""
     return call("arcadedb_n_last_error")