HDT archive ingest
HDT (Header-Dictionary-Triples) archive ingest — decode + id-translation throughput when loading .hdt archives into a sparq Graph.
Latest commit ef8d69fa · every metric smaller-is-better · each group header shows a competitive summary computed live from real same-box competitor numbers (or an honest placeholder where none exist yet).
Canonical same-box comparison on ip-172-31-15-13 at snikmeta.hdt (328 triples, /root/sparq/crates/sparq-hdt/tests/fixtures/snikmeta.hdt) (git e9e72412). Across 1 query whose solution count cross-checked across engines, sparq returned the correct count on every one and was the fastest engine on 1. The fastest competitor ranged 20.1×–20.1× sparq’s time (median 20.1×).
Measurement modes differ — sparq run in-process via the CLI, while answer over an HTTP SPARQL adapter — so on small result sets these absolute cross-engine ratios reflect per-query harness/transport overhead as much as engine speed; read them for cross-engine ordering, not as an audited head-to-head.
| Benchmark | sparq | Unit |
|---|---|---|
HDT direct_eq_upstream — load-and-decode gatesnikmeta_direct_eq_upstream | 1 | count |
HDT distinct_predicates — load-and-decode gatesnikmeta_distinct_predicates | 23 | count |
HDT rdf_type_triples — load-and-decode gatesnikmeta_rdf_type_triples | 49 | count |
HDT terms — load-and-decode gatesnikmeta_terms | 205 | count |
HDT triples — load-and-decode gatesnikmeta_triples | 328 | count |
HDT — gzipped-N-Triples / HDT load-time ratio (advisory)hdt_vs_ntgz_load_s | 3.6 | ratio |
HDT — load+decode .hdt to a native Graph (advisory)hdt_load_s | 0.039 | s |
Same-box cross-engine comparison
- sparq· CLI in-process (sparq-cli bench, COUNT mode)
- hdt-cpp (rdfhdt)· in-container decode (Docker rdfhdt/hdt-cpp, hdt2rdf: decode .hdt → N-Triples; container spawn excluded, hdt2rdf process start included)
| Query | Rows | sparqe9e72412 | hdt-cpp (rdfhdt)rdfhdt/hdt-cpp@sha256:36682e1d0202371ba06f50c187a5244f5cf8276c2be59286ec2ab07b9b63b073 | counts |
|---|---|---|---|---|
| decode | 328 | 527 µs | 10,579 µs | ✓ |
Canonical same-box hdt gather · snikmeta.hdt (328 triples, /root/sparq/crates/sparq-hdt/tests/fixtures/snikmeta.hdt) · min-of-1 · ip-172-31-15-13 · git e9e72412 · counts cross-checked engine-vs-engine and vs expected rows. Absolute cross-engine ratios still include measurement-mode overhead (in-process CLI vs HTTP adapter) — read for cross-engine ordering. single gather; counts cross-checked engine-vs-engine.