Compact small files with measured outcomes #18

New issue

Closed

opened 2026-10-11 19:10:47 +00:00 by phg · 1 comment

phg commented

2026-10-11 19:10:47 +00:00

Owner

Parent

IODD Tool specification

What to build

Preview and perform small-file compaction on fixture volumes, showing the agreed objective, moved files, limits, and actual free-run result.

Acceptance criteria

The native Compact workflow uses the resolved objective, threshold, work limits, preview, and confirmation contract.
It can relocate eligible already-contiguous files as well as fragmented files, without assuming single-file defrag alone covers compaction.
Execution uses verified FAT32/exFAT transaction contracts and preserves moved/unmoved content and relevant metadata.
Before/after free-run metrics and per-file outcomes demonstrate achieved benefit or an explicit no-benefit/limited-work result.
No arbitrary packing heuristic, VHD-sized hole, or unlimited optimization is claimed unless the approved policy requires and verifies it.
Stop and injected errors preserve recoverability and produce a truthful fresh snapshot.
Independent fixtures include early holes, unsuitable files, no-benefit layouts, and the selected threshold/work-cap boundaries.

Blocked by

Defragment a file on an exFAT image

Validation contract

Completion requires the applicable independently checked software evidence in Set validation gates and the implementation handoff. A closed policy decision does not establish that implementation tests have passed. Use disposable images/virtual volumes and controlled faults; no hardware-trial or performance gate applies.

Scope

This ticket belongs to the later implementation effort. All planning decisions are resolved; current work covers tracker and documentation updates. The parent specification indexes the authoritative product and software-validation contracts. Forgejo governs live updates; retained local files record the handoff.

## Parent [IODD Tool specification](https://git.s1q.dev/phg/iodd-tool/issues/7) ## What to build Preview and perform small-file compaction on fixture volumes, showing the agreed objective, moved files, limits, and actual free-run result. ## Acceptance criteria - [ ] The native Compact workflow uses the resolved objective, threshold, work limits, preview, and confirmation contract. - [ ] It can relocate eligible already-contiguous files as well as fragmented files, without assuming single-file defrag alone covers compaction. - [ ] Execution uses verified FAT32/exFAT transaction contracts and preserves moved/unmoved content and relevant metadata. - [ ] Before/after free-run metrics and per-file outcomes demonstrate achieved benefit or an explicit no-benefit/limited-work result. - [ ] No arbitrary packing heuristic, VHD-sized hole, or unlimited optimization is claimed unless the approved policy requires and verifies it. - [ ] Stop and injected errors preserve recoverability and produce a truthful fresh snapshot. - [ ] Independent fixtures include early holes, unsuitable files, no-benefit layouts, and the selected threshold/work-cap boundaries. ## Blocked by - [Defragment a file on an exFAT image](https://git.s1q.dev/phg/iodd-tool/issues/16) ## Validation contract Completion requires the applicable independently checked software evidence in [Set validation gates and the implementation handoff](https://git.s1q.dev/phg/iodd-tool/issues/6). A closed policy decision does not establish that implementation tests have passed. Use disposable images/virtual volumes and controlled faults; no hardware-trial or performance gate applies. ## Scope This ticket belongs to the later implementation effort. All planning decisions are resolved; current work covers tracker and documentation updates. The parent specification indexes the authoritative product and software-validation contracts. Forgejo governs live updates; retained local files record the handoff.

phg added the

ready-for-agent

label

2026-10-11 19:10:47 +00:00

phg referenced this issue

2026-10-11 19:10:47 +00:00

Recover interrupted operations in the app #19

phg self-assigned this

2026-10-13 18:03:05 +00:00

phg referenced this issue from a commit

2026-10-13 19:27:24 +00:00

Merge ticket #18: compact small files with reviewed bounds and measured outcomes

phg commented

2026-10-13 19:27:31 +00:00

Author

Owner

Resolution — bounded small-file Compact implemented

Implemented on implement/iodd-tool-spec-7, completed branch 391b463, integration merge 6f6a9b3.

The public operation API and native image workflow preview a bounded beneficial compaction plan, including already-contiguous and fragmented eligible files. Defaults are an inclusive 1 MiB logical threshold and 100 committed moves; reviewed allocation-byte caps and cluster-rounded targets are supported. The declared search considers at most 2048 eligible files, 4096 placement evaluations, the first 4096 free runs for placement proposals and depth 128. Every committed prefix preserves the initial largest free run, and the complete plan must increase it. No-benefit and work-limit results state the bounds without claiming global impossibility. A staging move can commit with zero measured growth; moved files and achieved benefit remain distinct.

Execution retains one exclusive image descriptor, regenerates the full reviewed plan and revalidates remaining identities after each commit. It uses the shared FAT32/exFAT recipe, journal and replay owner. Actual I/O, identity, verification or recovery failures halt subsequent work. Stop completes a safe rollback/forward result before stopping. Per-file results survive final rescan failures; unavailable final metrics are reported as unverified. Create requires its own fresh preview and explicit retry.

The exact final combined source f9b42d4 passed all 182 tests in 820.770 seconds, exit zero, on pinned Xcode27.0/27A266a with ordinary release optimization. Complete Sources and Tests match the evidence-only completed branch. Boundary, mixed-file/metadata, physical write-unit, stale-review, Stop, second-file fault/repeated recovery, legacy nil-witness compatibility and malformed/mismatched full-allocation-witness cases pass. Shared copied-slack and postcommit Stop regressions are reproduced and corrected. Fresh merged checks pass all 26 focused tests/79 cases in 21.498 seconds; release app/worker builds, arm64, plist and both binary macOS15.0 minimum checks pass.

Same-source native FAT32/exFAT completion moved two files/1024 allocated bytes and increased the largest run from 10 to 12 clusters. Material review changes required renewed preview/backup acknowledgement. Actual Copying/Stop on a coherent approximately 1.1 GiB FAT32 source, with an explicitly raised reviewed threshold, committed zero moves and preserved the largest run. Independent complete logical/allocation hashes, identities, metadata and free intervals match; nonrepairing checkers exit zero without changing their input image hashes. All owned apps exited and the GUI lock was released.

Reproducible qualification, exact source/artifact record, full logs and native evidence. This qualifies disposable image workflows. External writes, the complete recovery UI and final signed/notarized distribution remain their later tickets. The requested pause begins after this resolution; no new frontier ticket has been started.

## Resolution — bounded small-file Compact implemented Implemented on [implement/iodd-tool-spec-7](https://git.s1q.dev/phg/iodd-tool/src/branch/implement/iodd-tool-spec-7), completed branch [391b463](https://git.s1q.dev/phg/iodd-tool/commit/391b463e62ef4cf73c3de1c33eeca7088a3f2ae6), integration merge [6f6a9b3](https://git.s1q.dev/phg/iodd-tool/commit/6f6a9b3c1db865751ad7dd164f18b2687a7403f0). The public operation API and native image workflow preview a bounded beneficial compaction plan, including already-contiguous and fragmented eligible files. Defaults are an inclusive 1 MiB logical threshold and 100 committed moves; reviewed allocation-byte caps and cluster-rounded targets are supported. The declared search considers at most 2048 eligible files, 4096 placement evaluations, the first 4096 free runs for placement proposals and depth 128. Every committed prefix preserves the initial largest free run, and the complete plan must increase it. No-benefit and work-limit results state the bounds without claiming global impossibility. A staging move can commit with zero measured growth; moved files and achieved benefit remain distinct. Execution retains one exclusive image descriptor, regenerates the full reviewed plan and revalidates remaining identities after each commit. It uses the shared FAT32/exFAT recipe, journal and replay owner. Actual I/O, identity, verification or recovery failures halt subsequent work. Stop completes a safe rollback/forward result before stopping. Per-file results survive final rescan failures; unavailable final metrics are reported as unverified. Create requires its own fresh preview and explicit retry. The exact final combined source f9b42d4 passed all 182 tests in 820.770 seconds, exit zero, on pinned Xcode27.0/27A266a with ordinary release optimization. Complete Sources and Tests match the evidence-only completed branch. Boundary, mixed-file/metadata, physical write-unit, stale-review, Stop, second-file fault/repeated recovery, legacy nil-witness compatibility and malformed/mismatched full-allocation-witness cases pass. Shared copied-slack and postcommit Stop regressions are reproduced and corrected. Fresh merged checks pass all 26 focused tests/79 cases in 21.498 seconds; release app/worker builds, arm64, plist and both binary macOS15.0 minimum checks pass. Same-source native FAT32/exFAT completion moved two files/1024 allocated bytes and increased the largest run from 10 to 12 clusters. Material review changes required renewed preview/backup acknowledgement. Actual Copying/Stop on a coherent approximately 1.1 GiB FAT32 source, with an explicitly raised reviewed threshold, committed zero moves and preserved the largest run. Independent complete logical/allocation hashes, identities, metadata and free intervals match; nonrepairing checkers exit zero without changing their input image hashes. All owned apps exited and the GUI lock was released. [Reproducible qualification, exact source/artifact record, full logs and native evidence](https://git.s1q.dev/phg/iodd-tool/src/commit/6f6a9b3c1db865751ad7dd164f18b2687a7403f0/docs/validation/compaction.md). This qualifies disposable image workflows. External writes, the complete recovery UI and final signed/notarized distribution remain their later tickets. The requested pause begins after this resolution; no new frontier ticket has been started.

phg closed this issue

2026-10-13 19:27:31 +00:00

phg referenced this issue

2026-10-13 19:28:12 +00:00

IODD Tool specification #7