Vendor codec 2.65.0 (from vaptvupt-codec tag v2.65.0), re-applying the two
in-tree audit patches on top: the ANS decode safe-zone 2*SAFEZONE_MAX_RUN
reserve (heap-overflow guard, not yet upstream) and the AVX2 offset-read
bound in vv_decoder.c.
Two settings in the integration layer were leaving most of the codec's ratio
on the table:
* vaptvupt_api.c forced opts.format_v2=1 for balanced+extreme. Since codec
v2.61.0 that routes text through the binary/greedy path and HALVES the
extreme-mode text ratio (codec-level 7.6x -> 3.7x). The codec auto-enables
format_v2 for binary-detected input on its own, so stop forcing it: text
keeps the optimal parser, binary still gets v2.
* auto_block_size() capped the extreme block at 512 KiB. The block IS the
codec's LZ window, so the 'large-window extreme' parser could never match
past 512 KiB. Scale block size with level (128 KiB fast -> 8 MiB extreme).
Because block size also sets --dedup granularity (a large block rarely
finds a byte-exact duplicate), --dedup now overrides to a small 256 KiB
block so block-level dedup still works.
Measured, level 9 extreme: text 3.77x->5.98x (+58%), logs 7.21x->9.07x (+26%),
json 8.25x->9.38x, source 4.93x->5.63x. Wire format unchanged (v1.6); 5.0.0
and 5.1.0 archives interoperate both directions (verified, all modes). Bump
ZUPT_CODEC_RELEASE to 2.65.0 and ZUPT_VERSION_STRING to 5.1.0.
The app aborted ('QThread: Destroyed while thread is still running')
whenever an async job completed — reported as 'the software closes
automatically after selecting files and adding my key'. Reproduced
deterministically with a headless flow driver: run_async's finish()
dropped the (QThread, Worker) refs from parent._jobs right after
t.quit(), while the OS thread was still winding down; the next cyclic-GC
pass collected the live QThread wrapper and Qt aborted the process.
Fix: finish() (queued from Worker.done) now only re-enables the UI and
calls t.quit(); a release() slot connected to QThread.finished (queued)
does t.wait() and only then drops the refs — the thread is provably dead
before its wrapper can be collected.
Hardening from adversarial review of the fix:
- Worker.run: catch-all except -> done.emit(-1, ..., str(exc)) so no
exception can strand a job with the button disabled forever (fatal
under PyQt6); errors='replace' on the pipes so non-UTF-8 CLI output
cannot raise mid-read.
- Worker.cancel() + _cancelled flag: kills the child CLI on window close
and closes the cancel-before-Popen startup race.
- ZuptWindow.closeEvent: confirm 'Quit and abort it?' when jobs are
running (a killed disk restore is destructive — never silent), then
cancel + quit + wait(3000) each thread; if one cannot be joined,
os._exit(0) instead of letting teardown abort.
- Drop dead _thread/_worker single-slot attrs from the pre-_jobs scheme.
Verified: full GUI function matrix (keygen hybrid/pq-only/export,
compress PQ-hybrid x3 / password / pq-only, extract all modes with
byte-identical round-trips, verify, info, two concurrent jobs, confirm-
close mid-job, instant close after start) — 16/16 PASS on offscreen and
xcb, no aborts; --selftest OK.
On Sway 1.12 + Qt 6.9 (Guix, NVIDIA) Qt-Wayland deadlocks before mapping:
WAYLAND_DEBUG shows the client completes xdg_toplevel setup but never sends
the initial wl_surface.commit, so the compositor never sends configure and
the surface never maps — the event loop runs, the app prints its startup
notice, and no window ever appears. This reproduces with a bare PySide6
QLabel, so it is a toolkit/compositor bug, not ours; no Qt env knob
(fractional-scale disable, scale pinning) unblocks it, while the same
window maps instantly on XWayland.
Fix: a map watchdog on Wayland platforms. An event filter LATCHES the first
Expose on the toplevel QWindow (sampling isExposed() at a deadline would
misfire: a healthy hidden window — other workspace, scratchpad, locker —
reads unexposed ~100 ms after frame callbacks stop). If no expose ever
arrived after 4 s, re-exec the same process with QT_QPA_PLATFORM=xcb.
Safety rails: a sentinel env var (VAPTVUPT_XCB_FALLBACK_DONE) makes a
second fallback impossible even if '-platform wayland' argv (which outranks
the env override) brings the child up on Wayland again; execve failure is
caught and degrades to the no-fallback message; frozen bundles reuse argv
as-is (PyInstaller sets argv[0] to the exe); DISPLAY-unset systems just get
an honest notice; VAPTVUPT_NO_XCB_FALLBACK=1 opts out.
Verified live: wayland launch relaunches at 4 s and the window appears in
the sway tree (title 'VaptVupt 5.0.0', visible, tiled) — first time the GUI
is actually on screen on this machine; latch flips true where expose events
exist (offscreen); sentinel path stays wayland with no exec; selftest OK.
e15329a's window-activation code crashed the GUI at startup on Wayland:
raise_()/activateWindow()/move() go through Qt-Wayland's xdg restack/
xdg-activation paths, which SEGSEGV in PySide6 6.9 as shipped on Guix
(faulthandler pinpointed win.raise_(); QT_QPA_PLATFORM=wayland reproduces,
--selftest — plain show() — never crashed). Gate the whole center/raise/
activate block behind platformName() == "xcb": on X11 it is the canonical
show-then-activate sequence and verified working under XWayland; on Wayland
the compositor maps and focuses the new toplevel itself (raise/activate were
no-ops there even before they crashed). Strict equality keeps wayland-egl
and any unknown platform on the safe plain-show path. Centering is computed
before show() again, as pre-e15329a, so X11 windows map already-centered.
Also make every sys.stderr access None-safe: PyInstaller --windowed (>=5.8,
i.e. the shipped Windows .exe) sets sys.stderr = None, so the unguarded
startup notice would have raised AttributeError right after show() and
killed the window; a dead pipe similarly raises on flush. Guard the notice
(+ try/except OSError), the import-error message, and the debug discovery
echo. Verified: launch stays alive on wayland, xcb, and offscreen-with-
fd2-closed; --selftest passes offscreen/wayland incl. VAPTVUPT_DEBUG=1
with stderr closed.
Center the main window on the active screen and raise()/activateWindow()
it on launch, so it cannot open off-screen or behind the focused frame on
a tiling compositor (Sway/i3/Hyprland) — the usual cause of reports that
'the GUI won't start / is stuck': it launched, but was not visible.
Add lightweight non-GUI flags handled before the QApplication is built:
--version print GUI + pinned-CLI version and Qt binding
--help usage
--selftest build the full UI, spin the event loop once, exit 0
so a successful launch can be verified with no display (headless/CI/remote).
Also emit a one-line 'window open' notice to stderr so a terminal launch
is not mistaken for a hung shell.
Adds packaging/guix/vaptvupt.scm so VaptVupt is reproducibly installable on
GNU Guix (guix package -f). The GUI launcher sets LD_LIBRARY_PATH to the Qt6
leaf libraries PySide6 needs but does not carry in its RUNPATH — libGL (mesa),
libxkbcommon, the X11/xcb family, libzstd (from zstd's "lib" output), harfbuzz,
icu, wayland, glib, dbus, ... Without them, `import PySide6.QtWidgets` fails with
"libGL.so.1 / libzstd.so.1: cannot open shared object file" and the GUI prints
"requires PySide6 or PyQt6" on Guix (which has no global /usr/lib to satisfy
them). Qt's own libraries are deliberately excluded from LD_LIBRARY_PATH so they
resolve via PySide6's RUNPATH (a second copy causes private-API symbol clashes).
Also puts python-shiboken-6's site-packages on GUIX_PYTHONPATH.
Verified: full PySide6 QtWidgets/QtGui/QtCore import + QApplication, and the
windowed GUI launches, on this Guix system.
Version bumped to 5.0.0 across include/zupt.h, all packaging recipes, man
page, and docs.
Audit fixes (pre-5.0.0 review):
- src/zupt_format.c: overflow-safe bound in the solid-mode `test` path
(off+sz could wrap and drive an OOB read in zupt_xxh64 on a crafted archive;
the extract path was already hardened, the test path was not).
- gui: run_async now marshals the completion callback onto the GUI thread with
QueuedConnection (a bare functor connected DirectConnection and touched
widgets off the worker thread); Extract auto-detect note survives the log
clear via a new `info` param.
- .github/workflows/ci.yml: trigger on `master` (was main/develop, so CI never
ran); `make dist` tarball is vaptvupt-*.tar.gz not zupt-*; the ASAN PQ
round-trip uses native --pq (was --pq-sdk, which fails on the source-only
build and blocked the release job).
Documentation:
- New AUDIT.md (methodology, FIPS 203 conformance validation, findings, repro).
- CHANGELOG 5.0.0 entry covers the FIPS 203 conformance fix + BREAKING note and
the GUI/CLI/security/packaging work.
- README "What's new in 5.0.0", download tables (incl. Windows/macOS/BSD +
portable GUI), version-history row.
- SECURITY.md + THREAT_MODEL.md: ML-KEM-768 documented as FIPS 203, validated
byte-for-byte against OpenSSL 3.5.
- Accuracy fixes: man page (--kdf default is PBKDF2 on source-only; codec
2.60.4), rpm %description, debian control/copyright, homebrew header
(no vendored library on source-only builds).
make check 16/16 (FIPS 203 conformance 3/3, all distro-safe checks).
The in-tree ML-KEM-768 was round-3 CRYSTALS-Kyber mislabelled "FIPS 203" and
was NOT interoperable with a compliant ML-KEM. Discovered and fixed by
validating against OpenSSL 3.5's FIPS 203 ML-KEM-768 as an oracle.
Three deviations, all fixed in src/zupt_mlkem.c:
1. Matrix  transpose convention. FIPS 203 K-PKE.KeyGen samples
Â[i][j] = SampleNTT(XOF(ρ, j, i)) and K-PKE.Encrypt uses (ρ, i, j); the
code had both index orders swapped. It was self-consistent (encaps/decaps
round-tripped) but transposed vs the standard — which is exactly why a
self-consistency-only round-trip test never caught it. With the same seed,
keygen now produces a byte-identical ek to OpenSSL.
2. Shared-secret KDF. FIPS 203 returns K = G(m‖H(ek))[0:32] directly; removed
the round-3 final K = KDF(K̄‖H(c)) step (encaps + decaps success key).
3. Implicit rejection. Now K̄ = J(z‖c) = SHAKE256(z ‖ full-ciphertext) instead
of the round-3 KDF(z‖H(c)).
Validation (tests/test_mlkem_fips203.sh + mlkem_fips203_harness.c, wired into
make check): against OpenSSL 3.5 ML-KEM-768 —
- deterministic keygen (same d‖z seed) -> byte-identical ek
- our encaps -> OpenSSL decap: shared secret matches
- OpenSSL encap -> our decaps: shared secret matches
The harness feeds a fixed random stream (MLKEM_RAND) so the FIPS 203 seed is
reproducible; the test skips gracefully without an ML-KEM-capable openssl.
BREAKING: --pq / --pq-only keys and archives from <= 4.2.1 no longer decrypt
(the KEM math changed). Regenerate keys and re-encrypt. Password mode and plain
compression are unaffected; wire format stays v1.6. make check 16/16.
Fixes the "GUI functions don't work on GNU Guix" report and a batch of
adversarial-audit findings across the CLI, decoder, crypto and packaging.
GUI (gui/src/zupt_gui.py)
- Root cause of the broken GUI: every encryption path defaulted to the
libzuptsdk "SDK v2" modes, which are absent from the source-only build and
exit 1. Reworked Keys/Compress/Extract around the native modes with a
build-aware PQ-mode selector: Hybrid (--pq, default), Full PQ (--pq-only),
and SDK v2 only when the binary reports WITH_SDK support. Capability is
detected from `version` ("Build:"/"KDF:" lines) with a `help` fallback.
- Extract/Verify gain a PQ private-key input with auto-detect (reads the
archive via `info` to pick --pq vs --pq-only). Verify could not verify any
PQ archive before (password field only).
- run_async now holds a LIST of in-flight (thread, worker) refs; DiskTab's two
buttons previously shared one slot, so a second op GC'd the first QThread
mid-run. About tab corrected (codec 2.60.4, PBKDF2 default, --pq-only, URL).
CLI (src/zupt_main.c)
- compress -p <archive> <files> swallowed the archive name as the password and
truncated the first input file (silent data loss, exit 0). Added a
data-loss guard: refuse to overwrite an existing non-.zupt file as the output
archive unless -y/--force; plus a self-overwrite guard.
- compress <archive> <src> -p <pw> wrote an UNENCRYPTED archive (exit 0)
because options after the first positional were treated as files. Now errors
on a misplaced option (with a `--` escape for real dashed filenames).
- Removed duplicated dead --pq-box/--pq-sdk branches. version/banner/usage now
state the build's real KDF (PBKDF2 on source-only) and repo URL; examples
lead with native --pq / --pq-only.
Security (audit findings)
- vv_decoder.c: the two AVX2 fast-path token decoders read a 2-/3-byte match
offset after only checking 1 byte remained -> heap over-read on a crafted
archive. Added the `ip + off_bytes > ip_end` guard the tail path already had.
- zupt_crypto.c: wipe ML-KEM/X25519 secret-key buffers when hybrid decrypt
init fails on key read (matches the pq-only path).
- zupt_format.c: bound attacker-controlled encryption_header_off before the
(off_t)+7 arithmetic in the info enc_type reader (avoid signed-overflow UB).
Packaging (would fail source-only)
- debian/rules (staged into debian/zupt, installed vendored .so), aur/PKGBUILD,
nix/flake.nix, homebrew (vendored .so + nonexistent AUDIT.md, stale /zupt
URLs) now build source-only. opensuse spec %files ships the shell completions
make install writes (rpmbuild no longer fails on unpackaged files);
_service + debian/control point at the vaptvupt repo; KDF claim corrected.
Cross-platform GUI packaging (new)
- packaging/portable/: OS-agnostic GUI package (zupt_gui.py + .bat/.command/.sh
launchers + README) that runs on Windows/macOS/Linux/BSD with Python+PySide6.
- .github/workflows/cross-platform.yml: builds real native binaries on
windows-latest + macos-latest runners (CLI, PyInstaller GUI, Inno Setup
installer, .dmg) and the portable zip, attaching them to the release on tag.
- packaging/windows/vaptvupt-gui.iss: Inno Setup installer script.
Validation: make check 16/16 (all distro-safe checks), GUI imports + command
contracts verified against the fixed CLI. test_help_consistency updated to
assert the truthful (build-aware) default KDF.
Version fields, changelogs (rpm/deb/openSUSE), and download references
across README/INSTALL/DISTRIBUTION/gui-README bumped to 4.2.1 for the
info-label point release. GUI stays 1.3.0. No code change here.
`vaptvupt info` mislabelled full post-quantum (--pq-only, enc_type 0x06)
archives as "PQ Hybrid: YES (ML-KEM-768 + X25519)". Full-PQ archives set
the generic ZUPT_FLAG_PQ_HYBRID header flag (the enc_type byte is what
distinguishes hybrid 0x02 from pure 0x06), but info only checked the flag.
info now seeks to hdr.encryption_header_off, reads the real enc_type from
the encryption-header block, and reports the actual mode: "ML-KEM-768
only, no classical layer" for --pq-only, and hybrid / SDK-v2 / sealed-box
for the others. Reader-side only — no wire-format change; existing 4.2.0
archives are relabelled correctly with no re-encryption.
The CLI AppImage no longer bundles libzuptsdk/libpqvaptvupt (removed in
the 4.1.0 source-only switch); the binary links only libc/libm/pthread
from the host. Drop the vendored-library install and the LD_LIBRARY_PATH
in AppRun, and default VERSION to 4.2.0.
- 2026-07-09 is a Thursday; fix the weekday in the rpm, debian, and
openSUSE 4.2.0 changelog entries (was "Wed").
- packaging/rpm/vaptvupt.spec: drop the stale vendored-library install
and %files entries (the libraries were removed in 4.1.0's source-only
switch), build with WITH_SDK=0, and list the actual vaptvupt binary
and shell completions. Mirrors the already-source-only openSUSE spec.
Add a native full post-quantum encryption mode and fix a critical
keystream-reuse bug in deduplicated encrypted archives.
Full post-quantum mode (--pq-only)
- New envelope type 0x06 (ZUPT_ENC_PQ_ONLY): ML-KEM-768 (FIPS 203) as
the sole key-establishment mechanism, with no classical X25519
component. Archive key = SHA3-512(ml_ss || ml_ct || "ZUPT-PQ-ONLY-v1").
- For compliance postures that require a single NIST-standardised PQ
primitive with no classical KEM in the envelope (CNSA 2.0-style
"PQ-only"). Hybrid --pq stays the recommended default; --pq-only has
no classical fallback, so a break of ML-KEM-768 alone breaks it.
- keygen --pq-only / keygen --pub --pq-only (ZPQK magic, 1200B pub /
3600B priv; not interchangeable with hybrid --pq keys). Wrong or
tampered ciphertext is rejected via ML-KEM FO implicit rejection plus
the HMAC-SHA256 Encrypt-then-MAC envelope. In-tree, default build.
Security (critical): AES-256-CTR keystream reuse under --dedup
- Dedup assigns block sequence 0 to every data block (the sentinel that
keeps cross-file dedup references authenticating consistently). The
per-block nonce was base_nonce XOR block_seq, so under --dedup every
block collapsed to the same nonce, reusing the CTR keystream across
distinct plaintexts (a many-time-pad). Each block now uses a fresh
random 128-bit nonce stored in the block prefix and bound into the
block MAC; block_seq is still bound as MAC AAD. Regression test:
tests/test_dedup_nonce.sh. Re-encrypt any --dedup encrypted archives
written by <= 4.1.0.
Other
- keygen --sdk / --box on a source-only build now fails with a clear
message pointing to native --pq / --pq-only (or a WITH_SDK=1 build).
- Documentation: README, SECURITY, THREAT_MODEL, man page, CHANGELOG,
and all packaging recipes updated for the new mode and the security
fix; version bumped to 4.2.0. Wire format v1.6 unchanged (0x06 is
additive).
Validation: make check 16/16, quick suite 11/11 (incl. PQ-only),
dedup-nonce regression (all block nonces distinct), cppcheck clean.
The SDK-mode regression tests (audit, F-08/F-10/F-11/F-12, pq-box, KDF
transparency) now skip cleanly when built without libzuptsdk, and the
constant-time test links libzuptsdk only when present (its primitives are
native). Also drop a dead 'seqs_decoded' variable in vv_ans.c that failed
the exact-size test's -Werror=unused-but-set-variable. 'make check' is
green source-only, so the openSUSE %check no longer blocks the build.
Add an Acknowledgements section crediting the openSUSE maintainer for the
packaging work under packaging/opensuse/. Drop the AI-assistant line from
.gitignore (that ignore now lives in .git/info/exclude). Core code unchanged.
Build from source with no vendored binaries:
- Remove prebuilt libzuptsdk.so / libpqvaptvupt.so (and a stray .pyc). The
default build needs only a C compiler + make; it links no external library
and installs no .so. The libzuptsdk-backed modes (Argon2id KDF, --pq-sdk,
--pq-box) are gated behind an opt-in `make WITH_SDK=1`. The default password
KDF is PBKDF2-SHA256 and --pq (native ML-KEM-768 + X25519) is the built-in PQ
mode. openSUSE/RPM/deb/AUR/Homebrew/Nix recipes bumped to 4.1.0; the openSUSE
spec now builds source-only (%files ships no .so, %build/%install WITH_SDK=0).
Fix: multithreaded encrypted archives were unextractable on the native AEAD
path. The parallel compress/decompress workers skipped the F-09 frame-preface
AAD that the serial path and the archive's AAD_PREFACE flag bind into every
block MAC, so each multithreaded block failed authentication. The workers now
bind the preface via a shared serializer; output is byte-identical across
thread counts and interoperates with single-threaded archives (also fixes
`--kdf pbkdf2 -t N` in any build).
Security hardening (crafted-archive memory safety + crypto):
- LZH raw code-length stack overflow + huff_lut OOB write
- overflow-safe bounds in parse_index and solid-mode extract (heap OOB read)
- SEQ decoder safe-zone heap overflow (litlen+matchlen reserve)
- require the per-block ENCRYPTED flag on encrypted archives (plaintext forgery)
- cap archive-supplied PBKDF2 iteration count (KDF-amplification DoS)
- non-elidable secret wipe in the SDK path; restored disk images created 0600
Docs: remove AUDIT.md / BENCHMARKS.md / ROADMAP.md; trim marketing/AI-styled
text and correct KDF/PQ facts across README, SECURITY, INSTALL, DISTRIBUTION,
THREAT_MODEL, THIRD-PARTY-NOTICES, the man page, and packaging READMEs. Wire
format v1.6 unchanged.
Major release. Highlights:
- Codec: vendored VaptVupt codec moves to canonical 2.60.4 security
release. Fixes a high-severity OOB heap write in the AVX2 decode fast
path (reachable on a valid stream sized to exactly content_size, both
tail variants). Brings CBMC-formally-verified BCJ filters with
automatic ELF/PE/Mach-O detection. Compressed output stays
byte-identical (ratio gate Δ 0.00%); wire format unchanged at v1.6.
- New --pq-box sealed-box recipient mode (vendored libpqvaptvupt 0.6.0):
ML-KEM-768 + X25519 combined via HKDF-SHA256 with domain separation,
AES-256-CTR + HMAC-SHA256 EtM. Legacy --pq and --pq-sdk stay readable.
- F-16: discloses and fixes a pre-existing data-loss defect in the
<= 3.8.0 in-tree BCJ encoder. Full back-compat matrix decodes
byte-exact under 4.0.0; every readable pre-4.0 archive remains readable.
Repository hygiene:
- Sync full 4.0.0 source tree (codec, crypto, SDK, GUI, packaging, tests).
- Remove internal scratch files (PROMPT.md, FORMAL_AUDIT_PROMPT.md)
and superseded version-specific docs (INTEGRATION_PROTOCOL_2.60.4.md,
docs/FINDINGS-2.x.md) and a stray test binary.
- Refresh README download/install section to real 4.0.0 release assets;
bump version badge to 4.0.0.
- Add .gitignore for build outputs (keeps vendored prebuilt libraries).
- Corrected LZHP prediction encoding in disk backups to prevent data corruption
- Disabled spurious SOLID flag for per-block disk archives
- Shared write_enc_header() across all encryption paths to eliminate format mismatches
- Enabled solid compression with PQ encryption support
- Updated block device restore to use O_SYNC + fsync/sync
- Improved Termux/Android host detection for safer builds
- Made zupt_w8(), zupt_w16le(), zupt_w64le() non-static for shared use
- Removed all shipped .o files from tarball (fixes aarch64/Termux linker errors with x86_64 objects)
- Added arch-safety guard in Makefile to auto-detect and remove incompatible .o files
- Switched default compiler from gcc to cc (Termux uses clang)
- Skipped -lpthread on Android (bionic provides pthreads)
- Added Android detection via uname -o
- Fixed Keccak UB: ROL64(x,0) no longer expands to undefined x >> 64
- Achieved zero UBSan/ASAN issues across all PQ crypto paths
- Moved sys/syscall.h include to file scope with proper __linux__ guard
Release stats:
- 73 files, 159KB, zero .o artifacts
- 70/70 tests passing
- Fully clean under ASAN + UBSan
Note: full-disk encryption (--disk) deferred to v2.2.0 (requires raw device I/O, sparse detection, and privilege handling)
- Integrated `zupt_mac_verify_ct` in `zupt_decrypt_buffer()` to replace C XOR loop for HMAC-SHA256
- Integrated `zupt_ct_select_32` in `zupt_mlkem768_decaps()` to replace C `cmov()` for FO transformation
- Added `include/zupt_jasmin.h` with extern declarations and ABI docs
- Added `#ifdef ZUPT_USE_JASMIN` guards with clean C fallbacks in `zupt_crypto.c` and `zupt_mlkem.c`
- Makefile now auto-detects `jasmin/*.s`, assembles and links with `-DZUPT_USE_JASMIN`
Closes#3
