Repository navigation
Conversation
…ject_hook and without reading its side array (refs #315) to_list() and iter_dicts() of the JVM wheel build a json object_hook for every batch and read the side array of every batch, whether or not the batch holds a tagged value. These tests count the hook factory, the object_hook argument of json.loads and the batch elements that are read, for a result of plain values (no hook, no read of the side array) and for every kind of value the bridge tags (the hook stays). They also compare to_list() and iter_dicts() with the decoding that always builds the hook, over single shapes, a mixed result and batch boundaries. They fail on the current decoder (every batch gets a hook and a read of the side array) and pass once the decoder looks for the tag escape in the batch text first. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…s text without an object_hook and without reading its side array (refs #315) Every batch was decoded with json.loads(text, object_hook=_typed_hook(side)) and its side array was read first, also when the batch held only plain values. The hook is a Python call per JSON object and the read of the side array is a JPype operation per batch, which for iter_dicts (one row per crossing) is one per row. TypedRows writes every tag as {"\u0001": [kind, payload]} with the escape spelled out, writes a control character of a string as the same escape, and never writes the raw character, and every side array entry is referred to by a tag in the text. A text without the escape therefore holds no tag and no side index, the hook would return each object unchanged, and plain json.loads gives the same value. The check is a substring scan of a str that already exists (_loads_typed_row did it for one row). A string that holds U+0001 or the text of the escape only keeps the hook, which then changes nothing. No Java change, and the native wheel is untouched. The speed effect is not measured: it waits for a timed A/B of a candidate wheel against 26.11.1.post2. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…taframe on the JVM wheel (refs #315) Round trips of every Arrow temporal type through a real load on both wheels (every timestamp unit with and without a zone, to the nanosecond; date32 and date64 before 1970; nulls; batches; vertex and parallel loads; pandas and Polars frames), and JVM-only tests that count what one load does: no java.time call and no convert_python_to_java per cell, and each timestamp or date column reaching the bridge as one TemporalColumn per batch. The counting tests fail on the code before, which builds one LocalDateTime or LocalDate per cell in Python and converts every element again. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…te column crosses as one long[] of epoch ticks, not one JPype call per cell (refs #315) _temporal_to_java made one LocalDateTime.ofEpochSecond or LocalDate.ofEpochDay call per cell, and insert_columns converted every element of that list again with convert_python_to_java: a million-row timestamp column was two million JPype calls. The column is now the integer epoch values in one long[] (plus one boolean[] when a row is null) wrapped in the new bridge class TemporalColumn (dateTimes(long[] ticks, long ticksPerSecond, boolean[] nulls) and dates(long[] ticks, long ticksPerDay, boolean[] nulls)), and DocumentBatcher builds each LocalDateTime or LocalDate as it fills the document of that row, as StringColumn does for text. The reading rules are the ones of the code before: UTC wall clock, a zoned timestamp is already UTC epoch time, a naive one is the clock it shows, the unit's precision is kept (a nanosecond tick is not cut to microseconds), a tick before the epoch floors, date64 is milliseconds divided by 86,400,000 with a floor. The factory checks the smallest and the largest tick, so a value past the years of java.time still raises before a row is written. The per-cell path is retired: the wheel carries its own bridge jar, so no fallback for an older one. Not measured: the speed claim waits for a timed A/B against 26.11.1.post2. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ly text, no free and no JSON decode (refs #315) arcadedb_n_tx returns the transaction state as one integer (bit 0 active, bits 1 to 31 depth, bits 32 to 62 commits, -1 for an error whose text lib.last_error holds), as arcadedb_n_cmd returns a cursor id. These tests pin that and that an error still reaches Python as the same typed exception with the same message: - with db.transaction() is three foreign calls (mark, begin, commit), all arcadedb_n_tx: no strlen, no copy into a str, no arcadedb_free (was twelve); - begin, commit, rollback, is_transaction_active and the mark each make exactly one foreign call; - no JSON decode in a transaction, not even for the mark that reads active and depth; - the layout of the integer, -1 for an unknown handle with the error in last_error, commit without begin, rollback with nothing to roll back, a failed statement in a block, nested with blocks, an injected failure on each entry point, a database closed with a transaction open. They fail on the old code: lib.tx does not exist there, and the foreign-call and decode counts are higher. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…not a JSON text (refs #315) A begin, a commit, a rollback and a mark each crossed into the library and got back a malloc'd JSON text of about 60 bytes that Python read with strlen, a copy into a str and arcadedb_free, and mostly never looked at. with db.transaction() is three crossings (the mark, begin, commit), so twelve foreign calls and three strings built, copied and freed. arcadedb_n_tx now returns a long: bit 0 active, bits 1 to 31 depth, bits 32 to 62 the low 31 bits of the commit count, -1 for an error. It is the way arcadedb_n_cmd returns a cursor id: the error text is kept per thread and read with arcadedb_n_last_error, built by the same errorJson as before, so the typed exception and its message do not change. Each crossing is one foreign call, and with db.transaction() is three. The Python side reads the fields of the integer inline, so the mark, the begin and rollback states the unwind and run_in_transaction read, and is_transaction_active no longer decode anything. Not measured: the speed claim waits for a timed A/B of a candidate wheel against 26.11.1.post2. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ossings, not three (refs #317) The transaction check of a saved record's save() cannot refuse while auto-transaction is on, yet it still crossed into the library (arcadedb_n_tx) before the SELECT and the UPDATE. These tests pin two crossings there, bare and inside a transaction, with and without the read-back of the row, and pin what must not move: the check is back the moment auto-transaction is off (first, before any read), a stale record is refused under auto-transaction too, and the stored values and the next save's baseline are those of an update inside a transaction for NULL, float, text, date-time, date, Decimal, nested values, a float32 array and a value set unchanged. The two-crossing tests fail on the code before the change (three crossings); the rest pass on both. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…te when auto-transaction is on (refs #317) _save_update asked is_transaction_active() (an arcadedb_n_tx crossing) first and only then looked at the auto-transaction flag, so the answer was thrown away whenever the flag was on. The operands are swapped: the flag is a Python attribute, the state is a crossing. With auto-transaction on, an update of a saved record is two crossings (the SELECT of the stale-record check and the UPDATE), not three. With it off nothing changes: the check still comes first, outside a transaction and inside one. The check is not replaced by the engine's refusal for the default mode: the engine runs the before-update callbacks and the property validation before it looks for a transaction (read in LocalDatabase.updateRecord of the 26.12.1-SNAPSHOT source), and an UPDATE of a record that is gone raises nothing, so a save outside a transaction would run callbacks, or pass silently, instead of raising "Transaction not begun" before it reads. A Python-side record of the transaction state is not exact either: a SQL COMMIT or ROLLBACK statement ends the transaction without passing the wheel's transaction calls. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… no failed assert leaves a transaction open (refs #315) CI of the first push (run 43478436168) failed two tests on a wrong assumption of mine: I expected the outer begin to raise the depth by one. LocalDatabase.getNestedTransactions() is the size of the thread's stack of transaction contexts (DatabaseContext.DatabaseContextTL.transactions). The stack always holds the thread's base context once the thread has used the database, so the depth is 1 while idle. The outer begin reuses that base context (depth 1, active). Only a begin while a transaction is active pushes a context (2), the commit or rollback of that level pops it (1, still active), and the outermost end leaves the base context on the stack (1, idle). popIfNotLastTransaction never removes the last one. The native tx entry point returns exactly that number, as the old JSON reply did, so the change is right and the two expectations were not. The fixture now rolls back whatever a test left open before it drops the database, so a failed assert is one failure and not also a "Cannot drop the database in transaction" teardown error. The close-and-reopen test does the same in a finally. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…s the one the seven-call path writes (refs #388) The first test counts the calls on proxies of the source and target vertex, the database and GraphCalls: the parent makes seven calls on the Java objects and none on GraphCalls.newEdge. The others compare the edge with one written by the former code (reproduced in the test) for the three forms of target, edges with and without properties, a source read before other commits, and failures. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…rossing, through GraphCalls.newEdge (refs #388) The lookup of the source by record id inside the active transaction and the target's identity were seven JPype calls (getIdentity, isValid, lookupByRID, asVertex, getIdentity, isValid, newEdge). GraphCalls.newEdge does the same steps in Java, with the same rules: a source with a valid identity and a database is looked up again, the target goes in as its identity when that is valid. The wheel carries its own bridge jar, so the seven-call route is gone. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
tae898
added
python
7 tasks
…py passes an importer setting that the latest engine refuses; refs #388) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…in and commit), and edge lists are read through a fresh vertex (refs #388) The tagged 26.11.1 JVM run failed three tests in every leg, all in tests/test_vertex_new_edge_crossings.py, and all three are the tests, not the new code: - test_new_edge_is_one_call_into_the_bridge put a counting proxy in place of db._java_db before opening the transaction, and the transaction's own begin goes through DbCalls.stackMark(db), which JPype refuses for a proxy ("No matching overloads found"). The proxies are now on only around the new_edge call. - test_the_edge_is_the_one_the_seven_call_path_writes and test_an_edge_without_properties_and_several_edges_of_one_vertex listed the edges of vertex objects read before the commit. A vertex object is the state it had when it was read: GraphEngine.getMostUpdatedVertex (26.11.1) hands it the newest record only inside the transaction that wrote it, so a stale object lists nothing (the same for the seven-call path, and the new_edge docstring says the object is not refreshed). The lists are read through a fresh lookup, and one assertion pins that the stale object is left as it was on both routes. The code is unchanged: the properties, the edge class and the failure parity assertions of the same file passed in all legs. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
On the JVM wheel,
Vertex.new_edge()resolved both ends of the edge by record id in Python:getIdentity,isValid,lookupByRIDandasVertexfor the source,getIdentityandisValidfor the target, thennewEdge. That is seven JPype calls for one edge, where the native wheel makes one (#317).Now it is one call to a new method of
GraphCalls(src/java/com/arcadedb/python/GraphCalls.java):The rules are the ones the Python code had, moved into Java. A source with a valid identity is looked up again by it when a database is given, inside the active transaction, so a vertex object read before another edge was committed does not fail with
ConcurrentModificationException(the case oftests/test_stale_vertex_new_edge.py); otherwise it is used as it is. The target goes in as its identity when that is valid, and the engine loads it. Whether the edge is bidirectional is still the edge type's. The property values are converted in Python as before and cross as the varargs. The wheel carries its own bridge jar, so the seven-call route is gone.Related issues
Refs #388 and #315. This is not one of the ten ranked rows of #388: I found it reading the same code for what the ten leave out.
Performance changes
Not measured yet, so no figure here. The change removes crossings, which the test counts, and I have not timed it. The speed claim waits for a timed A/B of a candidate wheel against 26.11.1.post3, as CONTRIBUTING asks.
The figure for that A/B:
Vertex.new_edge(label, target)with no properties and with three properties (a string, an int and a float), for 1, 100 and 10,000 edges between vertices read once, each batch in one transaction, on a vertex type that has one edge type. The unit of one operation is onenew_edgecall, reported per edge. Wheel:arcadedb-jvm. The parity digest covers, for every edge, the property map, the Java class of the edge and the record ids at both ends, read back after the commit. The no-change control isgraph_batch().new_edgesfor the same edges (a path this change does not touch), and an A/A pair.Memory: the change allocates the property list that Python built before (two items per property) and nothing else. No batch or second copy of an input is held.
How I checked it
The tests come first in the first commit and the change in the second, so a run of the first commit shows them failing on the old code.
tests/test_vertex_new_edge_crossings.py(JVM wheel only, listed inNATIVE_NOT_APPLICABLE): one test counts the calls on proxies of the source and target vertex, the database andGraphCallsand expects exactlyGraphCalls.newEdge(the old code makes seven calls on the Java objects and none onGraphCalls). The others compare the edge with one written by the former code, reproduced in the test: properties of six kinds, the Java class of the edge, both ends read back throughget_out_edgesandget_in_edges, the three forms of target (Python vertex, Java vertex, Java RID), edges without properties, a source read before two commits, and the class of the failure for a missing edge type, a value Java cannot take and an unsaved source.tests/test_stale_vertex_new_edge.py(both wheels) already covers the vertex that was read before other commits.uvx pre-commit run --fileson the changed files, Bandit as CI runs it, pyflakes.CI: to be filled in once the legs have run.
Docs
No page or the README describes how
new_edgecrosses, so none needs an update.Checklist
uvx pre-commit run --files <changed files>passes🤖 Generated with Claude Code