Repository navigation
Vollständiger ReqIF-Import und -Export (Phasen 1–5) - #6
Merged
Merged
Conversation
Documents the current coverage of the ReqIF 1.2 schema, the findings from XSD validation and round-trip experiments, and a phased plan to reach schema-valid, lossless import and export. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JHowUptcqVmPtthduSbdvN
reqif.xsd imports xml.xsd, driver.xsd and the XHTML modules by http URL, which the hardened schema factory refuses to fetch, so validating against the official schema always failed. Remote imports are now resolved by file name from the directory of reqif.xsd. A missing import is answered with an empty input instead of being handed back to the parser, and it is reported by name together with its URL. Warnings raised while loading the schema are no longer swallowed. validateAgainstSchema now also accepts a file on disk, read through a hardened SAX reader so findings carry line numbers. The tests download the official schema set on first use and cache it (REQIF_SCHEMA_DIR, cached on CI). Without network they are skipped locally and fail on CI. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JHowUptcqVmPtthduSbdvN
All nine identifiable ReqIF elements (datatypes, enum values, spec types, attribute definitions, spec objects, spec relations, specifications, spec hierarchy nodes, relation groups) now share an Identifiable base class that mirrors the ReqIF metamodel. LAST-CHANGE, DESC, LONG-NAME and ALTERNATIVE-ID are read for every element and written back. Properties a document does not declare stay absent instead of being written empty. ALTERNATIVE-ID is written in the nested form the schema prescribes; the flat form previous versions wrote is still read. An element without the optional LONG-NAME no longer crashes the parser, and one without the required IDENTIFIER is rejected with a ReqIFParseException. Reading attribute values was duplicated between SpecObject and Specification. It now lives in AttributeValueReader. Values of definitions without a LONG-NAME are keyed by the definition id, so they no longer overwrite each other. The schema round-trip test surfaced two writer defects that are fixed here: SPEC-RELATION-GROUPS was written before SPECIFICATIONS, although REQ-IF-CONTENT is a sequence, and XHTML attributes without content were written as an empty THE-VALUE. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JHowUptcqVmPtthduSbdvN
After a fatal error the parser throws a new exception for the same problem, so telling reported findings apart by object identity listed every fatal error twice. Findings are now told apart by position and message. XmlUtils goes back to string literals for the ALTERNATIVE-ID lookup, so the util package does not depend on the model packages that depend on it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JHowUptcqVmPtthduSbdvN
The writer fills in what the schema requires but a model may lack, typically one built with ReqIFBuilder. The model itself is not changed. - LAST-CHANGE of every element: the header CREATION-TIME, or the time of writing (injectable clock) if the header has none. - Header CREATION-TIME, REQ-IF-TOOL-ID, REQ-IF-VERSION, SOURCE-TOOL-ID and TITLE are always written; empty ones get defaults. - MULTI-VALUED is written for every enumeration attribute definition, and OTHER-CONTENT for every enum value, also when false or empty. - REAL datatypes keep ACCURACY, MIN and MAX. Missing values default to 15 digits and the full double range. Infinity is written and read in xsd notation (INF, -INF) for bounds and values alike. The builder gives the header a free default identifier and refuses inconsistent REAL bounds. A missing header is now a validation error, because the schema requires it and no identifier can be invented for it. The builder example from the README and a round trip of a document with REAL bounds, MULTI-VALUED="false" and an empty OTHER-CONTENT are now schema-valid. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JHowUptcqVmPtthduSbdvN
ReqIFValidator now reports as errors everything the writer cannot repair and the schema would reject: - identifiers that are not valid xsd:IDs; - LAST-CHANGE, CREATION-TIME and DATE values that are not xsd:dateTime; - a REQ-IF-VERSION other than 1.0; - enum values without an integer KEY; - default values that are not in schema notation (they are written verbatim). Enum values now take part in the identifier uniqueness check. Values outside their datatype's MAX-LENGTH or MIN/MAX are warnings, because the schema cannot express those limits. String length is counted in characters. Findings come in a stable order. The xsd lexical rules live in util.XsdTypes (which replaces XsdDouble): NCName, dateTime with the leap year rule and without year 0000, integer, double and boolean. XHTML default values are written as markup in a THE-VALUE element instead of as an attribute the schema does not allow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JHowUptcqVmPtthduSbdvN
A review of the claim "a document without validation errors is written schema-valid" found cases where it did not hold. They are fixed: - Identifiers with non-ASCII characters are checked by the JDK's own schema validator, whose name tables are older than XML 1.0 fifth edition, so validator and schema always agree. - Text with characters XML cannot hold (control characters, unpaired surrogates) is an error; writing it used to fail with an IOException. - A malformed XHTML default, an enumeration default given as text (now also not written), and a relation group without type, source or target specification are errors. - The parser keeps only the TOOL-EXTENSIONS of REQ-IF itself, not elements of that name nested in a vendor extension. Schema findings on a DOM source carry no position, so merging findings with equal messages hid distinct problems. Only the fatal error a parser throws a second time is now skipped. The remaining exception is documented: which XHTML elements a value may use is only checked by validateAgainstSchema. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JHowUptcqVmPtthduSbdvN
The writer emits XHTML default values as markup since the previous commits, but the parser only read defaults from a THE-VALUE attribute, so they were lost on the next read. XHTML defaults are now read from the THE-VALUE element and kept as rendered XHTML, like XHTML values. xsd:boolean allows "1" for true, which the validator accepts, but the parser read it as false. It now reads "1" as true as well. Writer and validator share one ordering of attribute values (AttributeValue.BY_DEFINITION_ID). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JHowUptcqVmPtthduSbdvN
Attribute values were kept in a HashMap keyed by LONG-NAME and padded with the defaults of all unset attributes. Values of definitions that share a LONG-NAME overwrote each other, document order was lost, and the writer turned every default (and neutral values such as false or 0) into an explicit value. Spec objects, spec relations and specifications now extend SpecElementWithAttributes, as in the ReqIF metamodel. It keeps the values the element carries itself by attribute definition IDENTIFIER, in document order: - getValues() / getValue(definitionID): what the document contains; this is what the writer writes and the validator checks. - getAttributes() / getAttribute(name): the effective values by LONG-NAME, with definition defaults filled in, derived on each call so it never goes stale; an explicit value always wins over a default of the same name. The view is read-only now, because changes through it would not be written. - setValue / removeValue to change values; the content category is derived again afterwards. An explicit empty enumeration selection is written as such; left out, the definition's default would apply on the next read. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JHowUptcqVmPtthduSbdvN
The parser dropped several parts of the schema, so they were lost on a round trip. They are now modelled, read and written: - integer values of any size (xsd:integer); getValue() stays an Integer while the value fits, getBigInteger() is always exact; - xml:lang of the document and REPOSITORY-ID of the header; - IS-EDITABLE of attribute definitions; IS-EDITABLE, IS-TABLE-INTERNAL and EDITABLE-ATTS of spec hierarchy nodes; - THE-ORIGINAL-VALUE and IS-SIMPLIFIED of XHTML values and defaults (e.g. an embedded OLE object of which THE-VALUE only holds a preview), and XHTML content made of a single p instead of a div; - the OBJECT reference of a hierarchy node, also when it cannot be resolved, so it is written back and the validator names it. An xsd:boolean flag that is not true/false/1/0 is read as unknown instead of being flipped to false. LosslessRoundTripTest reads a schema-valid document that uses every element and attribute of the schema, writes it and compares both with CanonicalXml, which ignores only what the schema leaves open. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JHowUptcqVmPtthduSbdvN
ReqIFz only kept png and jpg files, so embedded OLE objects, PDFs, GIFs
and other attachments were lost on reading and when re-packing. Every
entry that is not a ReqIF document is now kept in getAttachmentFiles();
getPictureFiles() lists the common image types among them.
ReqIFzWriter.write(ReqIFz, ...) re-packs all attachments, copied straight
from the extracted files instead of being loaded into memory.
The archive is extracted into a temporary directory that close() deletes,
no longer into a "<name>_unzipped" directory next to the source file,
which failed on read-only locations and was never cleaned up. Archives can
be read from an InputStream.
Reading is more robust: document entries are recognized case-insensitively
(".REQIF"), backslash entry names are normalized, a duplicate entry
replaces the earlier one instead of failing, and no stream or file handle
is left open when reading fails.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JHowUptcqVmPtthduSbdvN
A single broken element - a value referencing an unknown attribute definition, an unresolvable type reference, an element without IDENTIFIER, THE-VALUE="abc" for an integer - used to make the whole document unreadable with a NullPointerException or NumberFormatException. The parser now passes a ParseIssues collector through the model constructors. What cannot be used is skipped, everything else is kept, and each problem is recorded in ReqIFDocument.getParseIssues(): - elements without IDENTIFIER are skipped; - elements with an unresolvable type reference are kept with that reference, which is written back; only their values are skipped; - values of unknown attribute definitions are skipped, the others kept; - values and defaults that do not fit their datatype are kept as written and written back unchanged; the validator reports them; - identifiers used twice within one kind are reported. ReqIFValidator.validate reports parse issues as warnings: what is still wrong with the model is reported as an error by its own checks, and stops being reported once fixed. The model constructors without ParseIssues read as tolerantly as before and discard the issues. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JHowUptcqVmPtthduSbdvN
- XHTML rendering keeps the xml: prefix of xml:lang and xml:space. As "lang" they clashed with a lang attribute on the same element, and the validator rejected the rendered default as not well-formed. - An XHTML default read from a document is not checked again by the validator; it is well-formed by construction. - SpecHierarchy.getXHTMLContent / getXHTMLDivContent take the object's own XHTML values first, so the default of another attribute no longer stands in for the object's text. - The README states that typed values are written in canonical notation. - The gap analysis records the Phase 3 results. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JHowUptcqVmPtthduSbdvN
- Values: set/setXhtml/setEnum/removeValue on spec objects, relations and specifications, through a shared AttributeValueFactory that also backs the builder. - Hierarchy: addChild/removeChild on specifications and hierarchy nodes; nodes know their parent, so a node is in one place only and never below itself; levels and sections follow every change; flattened lists are cached until the structure changes. - Content: add methods refuse identifiers already in use anywhere in the document; addSpecHierarchy, moveSpecHierarchy and removeSpecHierarchy; removeSpecObject lifts the children of nodes that show the object and drops its relations, checked up front so a refusal changes nothing; removeSpecRelation, removeSpecification and removeRelationGroup keep relation groups consistent; spec types and datatypes in use are not removed. The maps returned by the getters are read-only. - Relations and groups can be rewired. - LAST-CHANGE: new and changed elements are modified; the writer gives them the time of writing and, once the output succeeded, takes it into the model, so writing again reports no new change. Validating and buildDocument leave the model alone; the builder's documents start unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JHowUptcqVmPtthduSbdvN
- Relation group types and relation groups, with references checked. - with(id, ...) sets LONG-NAME, DESC, LAST-CHANGE (checked as xsd:dateTime) and ALTERNATIVE-ID on any element added before. - lang(...) for xml:lang; toolExtension(...) for vendor XML, refused unless it consists of elements of a namespace of their own, as the schema requires; all extensions go into one TOOL-EXTENSIONS. - IS-EDITABLE of attribute definitions; IS-TABLE-INTERNAL, IS-EDITABLE and EDITABLE-ATTS of hierarchy nodes; simplified XHTML values with THE-ORIGINAL-VALUE, also for specifications. - Builder calls are atomic: a refused call leaves its identifiers free. Spec types must be of the kind the element needs. - EDITABLE-ATTS: the model only takes attributes of the node's object, the writer names each reference after its definition's kind, and the validator reports references to attributes the object does not have. - The element name of attribute definitions is derived in one place. - README and analysis updated for Phase 4. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JHowUptcqVmPtthduSbdvN
- A hierarchy node shows its object's own XHTML value in the order of the spec type's definitions; an own value without content is not replaced by another attribute's default, defaults are only computed when the object has no XHTML value; an unresolved object gives null, not an NPE. - getBlocks() on XHTML values holds the "single p instead of div" rule in one place. - Rendering keeps attributes of other vocabularies with their namespace (well-formed, nothing lost) and drops xhtml:-prefixed duplicates. - The validator again checks defaults that are not of an XHTML datatype. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JHowUptcqVmPtthduSbdvN
Exports of IBM DOORS, Siemens Polarion, ReqIF Studio, Enterprise Architect, Capella and StrictDoc, and the reference files of the ReqIF Implementor Forum, taken from the test suite of the Apache-2.0 licensed reqif project (src/test/resources/corpus, with origin and license). CorpusTest checks for every document that it is written back as read - or differs for a stated reason - that a second write changes nothing, that the validator raises no error for a schema-valid file and that a document it finds no error in is written schema-valid; archives are re-packed with all entries. What the corpus turned up: - Values that cannot be interpreted - of an element whose spec type is missing, of an undeclared attribute definition, or of a kind that does not match the definition (Polarion writes XHTML ReqIF.Text as ATTRIBUTE-VALUE-STRING) - were dropped or converted. They are now kept as UninterpretedAttributeValue and written back as read, ReqIF elements moved into the namespace written. - An attribute definition whose datatype is missing was dropped; it is kept as read, with the properties the model changes. - xsi:schemaLocation of REQ-IF was lost; it is kept, with the namespace of a ReqIF draft replaced by ReqIF 1.2. - Relations and relation groups may reference objects and specifications of another document of the archive (GLOBAL-REF, ReqIF Implementor Forum TC1700): ReqIFValidator.validate(ReqIFz) and validate(Map) resolve them across documents. - The validator checks values kept as written against their own kind and warns when it does not match the definition. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JHowUptcqVmPtthduSbdvN
ImplementationGuideCheck reports what tools exchanging ReqIF rely on beyond the schema: - standard attributes with the datatype tools expect: dates DATE, ReqIF.ForeignDeleted BOOLEAN, ReqIF.Text XHTML, names and descriptions STRING or XHTML - derived from what real exports use; - embedded objects with a MIME type and a relative reference; an object that is not a PNG image, such as an OLE object, with a nested PNG preview; also in XHTML defaults; - IS-SIMPLIFIED and THE-ORIGINAL-VALUE together; - for an archive, every referenced file present, resolved against the document's folder and percent-decoded; inline data: URIs are left alone. Findings are warnings, files missing from an archive errors. The reference files of the ReqIF Implementor Forum raise none. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JHowUptcqVmPtthduSbdvN
Measured with the 19 MB reference file of the ReqIF Implementor Forum (TC4000, 10,000 objects): reading took 2.5 s, now 0.5 s; writing 0.3 s; about 100 MB of heap. Quadratic steps found and removed: - XmlUtils walked children with NodeList.item(i), which costs up to the list length when lists are walked nested; it now follows siblings, and firstDescendantByLocalName stops at the first match. - Removing a spec object rebuilt and renumbered the whole sibling list per lifted node, and walked every node of every specification; now only the parents of the affected nodes change, each in one step (HierarchyParent.setChildren). - Appending a top-level node renumbered all sections; moves copied the sibling list (getChildCount/getChild instead). - Looking up hierarchy nodes walked the tree on every add, move and removal; specifications keep an index by IDENTIFIER that follows each change. Children are kept in a list with a map beside it. - The builder copied all identifiers per call to undo a failed one; it now records only the call's own. - Every XHTML value created a new DocumentBuilderFactory; one configured factory is kept. Plain-value constructors no longer take a node placed elsewhere half way. LargeDocumentTest builds, writes, reads and changes 20,000 objects and moves 20,000 nodes within a generous limit. A StAX reader/writer is deferred: at about 10 KB of heap per object it pays off only well above 100,000 objects. README and gap analysis record Phase 5. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JHowUptcqVmPtthduSbdvN
- Values and definitions kept as written from a document without namespace join the ReqIF namespace when written, instead of being written with xmlns="" (ReqIF element names are upper case). - ImplementationGuideCheck spells archive entries like references (./ and .. collapsed, decoded), reports a reference that climbs above the archive root instead of matching a root entry, and tells objects apart by data and type without rendering them. - A value of an undeclared attribute definition is reported as "A value" instead of "Attribute value 'null'". - ReqIFConst.kindOf is used where the kind of an element name was taken apart by hand. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JHowUptcqVmPtthduSbdvN
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Setzt den Plan aus
IST-SOLL-ANALYSE.mdum: Import und Export von ReqIF 1.2 sind vollständig, schemagültig und verlustfrei, Dokumente lassen sich bearbeiten, und die Bibliothek ist gegen echte Tool-Exporte getestet. Jede Phase wurde mit mehreren Review-Runden umgesetzt; der aktuelle Stand steht in Abschnitt 0 der Analyse.Phase 1 – Schemaprüfung und gemeinsame Basis
validateAgainstSchemaprüft gegen das offizielle OMG-XSD, ohne online zu gehen (lokaler Resolver für die http-Importe); die Tests laden das Schema einmal herunter (in der CI gecacht).Identifiable:LAST-CHANGE,DESC,LONG-NAME,ALTERNATIVE-IDwerden für alle Elementarten gelesen und geschrieben.ALTERNATIVE-ID, falsche Reihenfolge inREQ-IF-CONTENT, leeresTHE-VALUE.Phase 2 – Schemagültiger Export
LAST-CHANGE, Header-Felder,MULTI-VALUED,OTHER-CONTENT, REAL-Grenzen), ohne das Modell zu ändern.Phase 3 – Verlustfreier Round-Trip
xml:lang,REPOSITORY-ID,IS-EDITABLE,EDITABLE-ATTS,THE-ORIGINAL-VALUE, …)..reqifz: alle Anhänge bleiben erhalten, Entpacken ins Temp-Verzeichnis,InputStream-Konstruktor.getParseIssues()statt in Exceptions.LosslessRoundTripTestmit kanonischem XML-Vergleich.Phase 4 – Bearbeitung und Builder
LAST-CHANGEwird für neue und geänderte Elemente automatisch gepflegt.with(...),lang, Tool-Extensions, Editierbarkeit, vereinfachte XHTML-Werte).Phase 5 – Interoperabilität und Skalierung
src/test/resources/corpus/README.md.CorpusTestprüft Verlustfreiheit, Stabilität und die Übereinstimmung von Validator und Schema.UninterpretedAttributeValueerhalten;xsi:schemaLocationbleibt erhalten; Verweise zwischen Dokumenten eines Archivs (GLOBAL-REF) perReqIFValidator.validate(ReqIFz).ImplementationGuideCheckfür die Konventionen des ReqIF Implementation Guide (Standardattribute, eingebettete Objekte und OLE-Vorschau,IS-SIMPLIFIED, Anhänge im Archiv).LargeDocumentTestsichert 20.000 Objekte ab. StAX ist bewusst zurückgestellt (Begründung in der Analyse).API-Änderungen
getAttributes()ist schreibgeschützt; Werte ändern überset…/setValue/removeValue.ReqIFCoreContentsind schreibgeschützt; geändert wird überadd…/remove…, undadd…lehnt vergebene IDs ab.getName()liefert für Elemente ohneLONG-NAMEnull.ReqIFzentpackt nicht mehr neben die Quelldatei.ReqIFBuildExceptionist eineIllegalArgumentException.Tests
🤖 Generated with Claude Code
https://claude.ai/code/session_01JHowUptcqVmPtthduSbdvN
Generated by Claude Code