fix: eliminate security vulnerabilities and strengthen test coverage#13
Merged
fix: eliminate security vulnerabilities and strengthen test coverage#13
Conversation
- Remove numpy.load(allow_pickle=True) in compress_octree.py and training_pipeline.py to prevent arbitrary code execution via crafted .npy/.npz files. Metadata now saved as JSON sidecar; optimizer variables saved as individual .npy files with numeric dtypes. - Add path validation in training_pipeline.py (traversal guard) and evaluation_pipeline.py (existence check) for checkpoint loading. - Replace model.save() with model.save_weights() in cli_train.py to avoid full SavedModel format vulnerable to Keras deserialization RCE. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…age analysis Bug fixes in source: - compress_octree: _save_debug_info no longer pickles dicts (only saves ndarrays) - compress_octree: save_compressed converts NaN/Inf scalars to None for valid JSON - training_pipeline: path validation uses Path.relative_to() to prevent prefix collision bypass (e.g. checkpoints_evil matching checkpoints prefix) New tests (26 total, 213 → 239): - test_compress_octree (13): NaN/Inf metadata, empty grid, no-normals roundtrip, missing sidecar/grid files, debug pickle prevention, metadata value fidelity, numpy scalar types, dtype change documentation, E2E quality check - test_training_pipeline (9): path traversal/absolute/prefix-collision rejection, NaN in optimizer vars, save before training, missing weights, partial optimizer files, old pickle format ignored, optimizer state value fidelity - test_evaluation_pipeline (3): no checkpoint configured, empty string checkpoint, missing checkpoint raises FileNotFoundError - test_integration (1): checkpoint resume preserves eval loss Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ixes Remove 37 tracked .pyc/egg-info build artifacts from git index. Convert all intra-package imports in src/ to relative imports and add a meta-path import hook in conftest.py so test files work unchanged. Fix setup.py dependencies (add tensorflow, tf-probability, etc; remove pytest from install_requires), set version 2.0.0, require Python >=3.10. Update pyproject.toml target-version to py310. Replace hardcoded CI test file list with pytest discovery. Fix evaluation_pipeline --checkpoint CLI arg not being applied. Fix Popen.__exit__ unconditionally terminating finished processes. Fix mp_report compression_ratio direction (higher is better). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ation - Add src/file_io.py with read_off(), read_ply(), read_point_cloud() dispatcher - Fix data_loader.py: use file_io for both .off/.ply, guard divide-by-zero in normalization - Fix ds_mesh_to_pc.py: triangulate n-gon faces, replace centroid with barycentric sampling - Fix ds_pc_octree_blocks.py: replace broken TF PLY parser, remove dual file write - Fix octree_coding.py and compress_octree.py: half-open intervals prevent boundary duplication - Fix cli_train.py: use read_point_cloud instead of read_off for .ply files Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…t optimization Replace continuous log-PDF with discretized CDF-difference probability mass function for correct entropy coding. Fix quantization to use standard grid (round(y - mean), not round((y - mean) / scale)). Add missing z_bits (hyper-latent rate) to total bitrate across all entropy models. Switch training pipeline to joint rate-distortion optimization with gradient clipping and configurable lambda_rd. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…de consistency Phase 4 of adversarial code review fixes: - Replace CENICGDN with standard GDN (Balle 2016) supporting inverse (IGDN) for synthesis - Fix SynthesisTransform to always use Conv3DTranspose (was incorrectly downsampling with SpatialSeparableConv) - Fix DeepCompressModel.call() to pass quantized y_hat to synthesis, apply sigmoid to output - Fix DeepCompressModelV2 compress/decompress for all 6 entropy model types - Fix HybridAttentionEntropyModel: proper attention-to-params projection (replaces concat hack) - Fix ChannelContextEntropyModel inference path for first channel group - Fix quick_benchmark: proper decode timing, Shannon entropy estimate, V1 unpack - Fix training_pipeline: empty validation guard, V1 unpack - Fix evaluation_pipeline: V1 unpack - 238 tests pass, ruff clean Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…roundtrip, numerical stability Add 6 new test files with 141 tests validating mathematical correctness: - test_entropy_correctness: PMF validity, rate computation, quantization behavior - test_causality: MaskedConv3D causal masks, raster-scan ordering, AutoregressiveContext - test_roundtrip: V1/V2 compress/decompress shape/bounds/determinism, gradient flow - test_numerical: GDN/IGDN stability, entropy model extreme values, constants correctness - test_data_pipeline: OFF/PLY I/O, mesh sampling, point cloud partitioning - test_benchmarks: Benchmark utilities, timing methodology, comparison functions Full suite: 379 passed, 0 failures. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
.pyc/egg-info artifacts, fixed packaging (setup.pydeps, Python version), converted to relative imports, fixedparallel_process.pyunconditional terminate andmp_report.pycompression ratio directionResult: 379 tests passing, 0 failures. Addresses all 66 findings from adversarial code review.
Test plan
ruff check src/ tests/— passes cleanpytest tests/ -v -m "not gpu and not slow"— 379 passed, 0 failurestest_entropy_correctness.py,test_causality.py,test_roundtrip.py,test_numerical.py,test_data_pipeline.py,test_benchmarks.py🤖 Generated with Claude Code