refactor: consolidate platform engines into senbei-engine

This commit is contained in:
bfloat16
2026-09-06 19:31:19 +08:00
parent cbfacbc31f
commit d436a200ba
66 changed files with 1148 additions and 1012 deletions
+20 -78
View File
@@ -1,98 +1,40 @@
# AGENTS.md
Guidance for AI coding agents (and human contributors) working in this repo.
Guidance for contributors working in this repository.
## Project
Senbei is a static unpacker for Crackproof-protected PE files and protected
Android (AArch64) shared libraries: a Cargo workspace with a pure, panic-free,
no-I/O PE unpacker core (`senbei-pe/`, built on `senbei-crypto/`), il2cpp
metadata de-obfuscators (`senbei-metadata/` for the Windows structural
variant, `senbei-android-metadata/` for the Android seeded-permutation and
embedded-blob variants), the native-only Android pipeline
(`senbei-android-crypto/`, `senbei-android-engine/`, `senbei-android-elf/`),
filesystem/CLI orchestration (`senbei-io/`, including the Android
single-library/package glue in `senbei-io/src/android.rs`), the `senbei`
binary (`senbei-cli/`), WebAssembly bindings (`senbei-wasm/`, outside the
workspace; builds into `web/pkg/`), and the static browser frontend assets
(`web/`). Read `docs/design.md` first.
Senbei is a static unpacker for protected PE files and Android AArch64 shared libraries. The workspace contains `senbei-cli`, `senbei-crypto`, `senbei-elf`, `senbei-engine`, `senbei-io`, `senbei-metadata`, and `senbei-pe`; `senbei-wasm` is a separate crate for the browser frontend.
Read `docs/design.md` before changing architecture or pipeline boundaries.
## Commands
```cmd
cargo build --release :: CLI (default member: senbei-cli)
cargo test --release --workspace :: full suite (golden corpus: samples/, git-ignored)
cargo build --release
cargo test --release --workspace
cargo clippy --workspace --all-targets -- -D warnings
cargo fmt --all
cd senbei-wasm && wasm-pack build --target web --release --out-dir ../web/pkg :: browser build
cd senbei-wasm && wasm-pack build --target web --release --out-dir ../web/pkg
```
The `samples/` corpus is user-managed and absent on CI; without it the
samples test is a no-op pass. `SENBEI_REQUIRE_SAMPLES=1` makes an absent
corpus fail (use this on a private CI that *does* have the corpus). The
Android corpus lives in `samples/android/` (one extracted app tree per
subdirectory) and is covered by `tests/android_samples.rs`;
`SENBEI_ANDROID_SAMPLES` overrides that location. Do not
delete `samples/` with `rm -rf` — it may be a junction; use git
worktree-aware cleanup.
The optional `samples/` corpus is user-managed and ignored by Git. The Android corpus is under `samples/android/` when present. Do not delete sample directories as part of routine cleanup.
## Hard rules
## Crate Boundaries
- **The PE unpacker core stays pure**: `senbei-pe` and `senbei-crypto` have no
file I/O, no `unsafe`, no panics across the public boundary, no
platform-specific code. Everything `senbei-wasm` compiles must keep building
for `wasm32-unknown-unknown` (`cargo check --target wasm32-unknown-unknown`
at the workspace root covers it — the Android crates do compile to wasm, but
nothing on the wasm path calls them).
- **The Android crates are native-only orchestration-style crates**:
`senbei-android-engine`/`senbei-android-elf` memory-map inputs and write a
module workspace to disk (the restore is a two-phase design consuming that
workspace). Keep them off the web app's code paths; `senbei-io`'s
`android.rs` is the only caller the CLI uses.
- **`catch_unwind` does not work on wasm** (the prebuilt std can't unwind; a
caught panic becomes a fatal `unreachable` trap). Native code may rely on
`catch_unpack`, but any routing decision must also work without a catchable
panic: spliced companion inputs route straight to the EXE pipeline, and the
web app isolates every unpack in a disposable Web Worker, retrying trapped
DLLs with `job::unpack_bytes_force_exe`. Never make correctness on wasm
depend on catching a panic.
- **Byte-identical output is the contract.** Any pipeline change must re-run
the full golden corpus; a byte mismatch on any golden is a regression.
- **Trial-and-validate, never trust a heuristic.** A silently wrong offset
produces a silently broken binary — worse than an error. Every layout
candidate must be validated (checksum / structural oracle) with fall-through
to the next candidate.
- **Determinism under parallelism.** Block fan-out must stay byte-identical
regardless of thread count (`SENBEI_THREADS=1` is the sequential reference).
- **Folder scanning: deny-list, never allow-list.** Targets are recognised by
content, not extension, and can carry arbitrary names — there is no closed
set of target extensions an allow-list could enumerate. Only known
bulk-asset formats are excluded.
- **No binaries in the repo** — not as fixtures, not in commits. The only
corpus is the local git-ignored `samples/`. (Issue attachments of
protected inputs are fine when the user is authorized to share them, but
never commit them.)
`senbei-pe` and `senbei-elf` contain basic format parsing and address mapping only. `senbei-engine/src/windows/` contains the PE unpacking pipeline; `senbei-engine/src/android/` contains Android extraction and ELF restoration. `senbei-crypto/src/android/` and `senbei-metadata/src/android/` contain Android-specific primitives; Windows metadata code is under `senbei-metadata/src/windows/`. Shared source stays directly under `src/`.
## Public-repo hygiene (important)
The format crates and PE engine remain free of filesystem I/O. Native Android extraction and restoration may memory-map inputs and write temporary workspaces. The browser binding must continue to compile for `wasm32-unknown-unknown`.
This is a public research repository. In code comments, docs, tests, and
commit messages:
## Hard Rules
- **Never name specific games, publishers, or product codenames.** Refer to
build families generically ("older EXE-64 builds", "the marker-less
layout", "external-companion builds"). Keep offsets/numbers — drop names.
- **Never name specific protected filenames** from real distributions. Test
fixtures use generic names (`app.exe`, `managed.dll`, `daemon.exe`).
Exceptions (platform-standard technology names, allowed): `il2cpp`,
`Unity`, `global-metadata.dat`, the Crackproof magic `KONN`.
- **Never reference other tools, projects, implementations, or paths outside
this repo.** Describe behavior and layout directly; do not mention prior
art, porting, or where any algorithm came from.
- Outputs must be byte-identical to the available golden corpus.
- Layout heuristics must trial and validate every candidate before accepting it.
- Deterministic parallel and sequential paths must produce identical bytes.
- Folder scanning must not open bulk assets. Windows candidates are `.exe`, `.dll`, and `global-metadata.dat`; Android candidates are `.so` and `global-metadata.dat`. Matching `.exe._` and `.dll._` files are auxiliary payloads and are not counted as skipped targets.
- APK, APKS, and XAPK processing must inspect manifests first and extract only `.so` and `global-metadata.dat` entries.
- Do not commit protected or restored binaries. Use generic fixture names and do not add product-specific names or external tool references to public code, docs, tests, or commit messages.
## Conventions
## Documentation
- Comments explain *why* (layout rationale, observed variants, failure modes),
not *what*.
- Rust 2024 edition; clippy-clean at `-D warnings`; rustfmt default style.
- CLI behavior (flags, exit codes, output naming) is documented in
`docs/usage.md` — update the doc when changing behavior.
Use one line for each normal Markdown paragraph. Keep code blocks, table rows, and list items structurally separate. Update `docs/usage.md` when CLI behavior changes.