diff --git a/AGENTS.md b/AGENTS.md index ec7e70c..ef52e28 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,98 +1,40 @@ # AGENTS.md -Guidance for AI coding agents (and human contributors) working in this repo. +Guidance for contributors working in this repository. ## Project -Senbei is a static unpacker for Crackproof-protected PE files and protected -Android (AArch64) shared libraries: a Cargo workspace with a pure, panic-free, -no-I/O PE unpacker core (`senbei-pe/`, built on `senbei-crypto/`), il2cpp -metadata de-obfuscators (`senbei-metadata/` for the Windows structural -variant, `senbei-android-metadata/` for the Android seeded-permutation and -embedded-blob variants), the native-only Android pipeline -(`senbei-android-crypto/`, `senbei-android-engine/`, `senbei-android-elf/`), -filesystem/CLI orchestration (`senbei-io/`, including the Android -single-library/package glue in `senbei-io/src/android.rs`), the `senbei` -binary (`senbei-cli/`), WebAssembly bindings (`senbei-wasm/`, outside the -workspace; builds into `web/pkg/`), and the static browser frontend assets -(`web/`). Read `docs/design.md` first. +Senbei is a static unpacker for protected PE files and Android AArch64 shared libraries. The workspace contains `senbei-cli`, `senbei-crypto`, `senbei-elf`, `senbei-engine`, `senbei-io`, `senbei-metadata`, and `senbei-pe`; `senbei-wasm` is a separate crate for the browser frontend. + +Read `docs/design.md` before changing architecture or pipeline boundaries. ## Commands ```cmd -cargo build --release :: CLI (default member: senbei-cli) -cargo test --release --workspace :: full suite (golden corpus: samples/, git-ignored) +cargo build --release +cargo test --release --workspace cargo clippy --workspace --all-targets -- -D warnings cargo fmt --all -cd senbei-wasm && wasm-pack build --target web --release --out-dir ../web/pkg :: browser build +cd senbei-wasm && wasm-pack build --target web --release --out-dir ../web/pkg ``` -The `samples/` corpus is user-managed and absent on CI; without it the -samples test is a no-op pass. `SENBEI_REQUIRE_SAMPLES=1` makes an absent -corpus fail (use this on a private CI that *does* have the corpus). The -Android corpus lives in `samples/android/` (one extracted app tree per -subdirectory) and is covered by `tests/android_samples.rs`; -`SENBEI_ANDROID_SAMPLES` overrides that location. Do not -delete `samples/` with `rm -rf` — it may be a junction; use git -worktree-aware cleanup. +The optional `samples/` corpus is user-managed and ignored by Git. The Android corpus is under `samples/android/` when present. Do not delete sample directories as part of routine cleanup. -## Hard rules +## Crate Boundaries -- **The PE unpacker core stays pure**: `senbei-pe` and `senbei-crypto` have no - file I/O, no `unsafe`, no panics across the public boundary, no - platform-specific code. Everything `senbei-wasm` compiles must keep building - for `wasm32-unknown-unknown` (`cargo check --target wasm32-unknown-unknown` - at the workspace root covers it — the Android crates do compile to wasm, but - nothing on the wasm path calls them). -- **The Android crates are native-only orchestration-style crates**: - `senbei-android-engine`/`senbei-android-elf` memory-map inputs and write a - module workspace to disk (the restore is a two-phase design consuming that - workspace). Keep them off the web app's code paths; `senbei-io`'s - `android.rs` is the only caller the CLI uses. -- **`catch_unwind` does not work on wasm** (the prebuilt std can't unwind; a - caught panic becomes a fatal `unreachable` trap). Native code may rely on - `catch_unpack`, but any routing decision must also work without a catchable - panic: spliced companion inputs route straight to the EXE pipeline, and the - web app isolates every unpack in a disposable Web Worker, retrying trapped - DLLs with `job::unpack_bytes_force_exe`. Never make correctness on wasm - depend on catching a panic. -- **Byte-identical output is the contract.** Any pipeline change must re-run - the full golden corpus; a byte mismatch on any golden is a regression. -- **Trial-and-validate, never trust a heuristic.** A silently wrong offset - produces a silently broken binary — worse than an error. Every layout - candidate must be validated (checksum / structural oracle) with fall-through - to the next candidate. -- **Determinism under parallelism.** Block fan-out must stay byte-identical - regardless of thread count (`SENBEI_THREADS=1` is the sequential reference). -- **Folder scanning: deny-list, never allow-list.** Targets are recognised by - content, not extension, and can carry arbitrary names — there is no closed - set of target extensions an allow-list could enumerate. Only known - bulk-asset formats are excluded. -- **No binaries in the repo** — not as fixtures, not in commits. The only - corpus is the local git-ignored `samples/`. (Issue attachments of - protected inputs are fine when the user is authorized to share them, but - never commit them.) +`senbei-pe` and `senbei-elf` contain basic format parsing and address mapping only. `senbei-engine/src/windows/` contains the PE unpacking pipeline; `senbei-engine/src/android/` contains Android extraction and ELF restoration. `senbei-crypto/src/android/` and `senbei-metadata/src/android/` contain Android-specific primitives; Windows metadata code is under `senbei-metadata/src/windows/`. Shared source stays directly under `src/`. -## Public-repo hygiene (important) +The format crates and PE engine remain free of filesystem I/O. Native Android extraction and restoration may memory-map inputs and write temporary workspaces. The browser binding must continue to compile for `wasm32-unknown-unknown`. -This is a public research repository. In code comments, docs, tests, and -commit messages: +## Hard Rules -- **Never name specific games, publishers, or product codenames.** Refer to - build families generically ("older EXE-64 builds", "the marker-less - layout", "external-companion builds"). Keep offsets/numbers — drop names. -- **Never name specific protected filenames** from real distributions. Test - fixtures use generic names (`app.exe`, `managed.dll`, `daemon.exe`). - Exceptions (platform-standard technology names, allowed): `il2cpp`, - `Unity`, `global-metadata.dat`, the Crackproof magic `KONN`. -- **Never reference other tools, projects, implementations, or paths outside - this repo.** Describe behavior and layout directly; do not mention prior - art, porting, or where any algorithm came from. +- Outputs must be byte-identical to the available golden corpus. +- Layout heuristics must trial and validate every candidate before accepting it. +- Deterministic parallel and sequential paths must produce identical bytes. +- Folder scanning must not open bulk assets. Windows candidates are `.exe`, `.dll`, and `global-metadata.dat`; Android candidates are `.so` and `global-metadata.dat`. Matching `.exe._` and `.dll._` files are auxiliary payloads and are not counted as skipped targets. +- APK, APKS, and XAPK processing must inspect manifests first and extract only `.so` and `global-metadata.dat` entries. +- Do not commit protected or restored binaries. Use generic fixture names and do not add product-specific names or external tool references to public code, docs, tests, or commit messages. -## Conventions +## Documentation -- Comments explain *why* (layout rationale, observed variants, failure modes), - not *what*. -- Rust 2024 edition; clippy-clean at `-D warnings`; rustfmt default style. -- CLI behavior (flags, exit codes, output naming) is documented in - `docs/usage.md` — update the doc when changing behavior. +Use one line for each normal Markdown paragraph. Keep code blocks, table rows, and list items structurally separate. Update `docs/usage.md` when CLI behavior changes. diff --git a/Cargo.lock b/Cargo.lock index 634342f..7e48c0e 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -418,53 +418,11 @@ dependencies = [ "syn 3.0.4", ] -[[package]] -name = "senbei-android-crypto" -version = "1.2.0" -dependencies = [ - "aes", - "thiserror", -] - -[[package]] -name = "senbei-android-elf" -version = "1.2.0" -dependencies = [ - "memmap2", - "senbei-android-crypto", - "serde", - "serde_json", - "sha2", - "tempfile", - "thiserror", -] - -[[package]] -name = "senbei-android-engine" -version = "1.2.0" -dependencies = [ - "goblin", - "memmap2", - "senbei-android-crypto", - "serde", - "serde_json", - "sha2", - "tempfile", - "thiserror", -] - -[[package]] -name = "senbei-android-metadata" -version = "1.2.0" -dependencies = [ - "serde", - "thiserror", -] - [[package]] name = "senbei-cli" version = "1.2.0" dependencies = [ + "senbei-engine", "senbei-io", "senbei-metadata", "sha2", @@ -475,6 +433,29 @@ dependencies = [ name = "senbei-crypto" version = "1.2.0" dependencies = [ + "aes", + "thiserror", +] + +[[package]] +name = "senbei-elf" +version = "1.2.0" +dependencies = [ + "goblin", + "thiserror", +] + +[[package]] +name = "senbei-engine" +version = "1.2.0" +dependencies = [ + "goblin", + "memmap2", + "senbei-crypto", + "serde", + "serde_json", + "sha2", + "tempfile", "thiserror", ] @@ -487,11 +468,8 @@ dependencies = [ "indicatif", "libc", "owo-colors", - "senbei-android-elf", - "senbei-android-engine", - "senbei-android-metadata", + "senbei-engine", "senbei-metadata", - "senbei-pe", "sha2", "tempfile", "walkdir", @@ -502,12 +480,15 @@ dependencies = [ [[package]] name = "senbei-metadata" version = "1.2.0" +dependencies = [ + "serde", + "thiserror", +] [[package]] name = "senbei-pe" version = "1.2.0" dependencies = [ - "senbei-crypto", "thiserror", ] diff --git a/Cargo.toml b/Cargo.toml index bc5afe4..ea860f8 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,11 +1,9 @@ [workspace] members = [ - "senbei-android-crypto", - "senbei-android-elf", - "senbei-android-engine", - "senbei-android-metadata", "senbei-cli", "senbei-crypto", + "senbei-elf", + "senbei-engine", "senbei-io", "senbei-metadata", "senbei-pe", @@ -43,11 +41,9 @@ windows = { version = "0.62", features = [ "Win32_System_SystemInformation", ] } zip = { version = "8", default-features = false, features = ["deflate"] } -senbei-android-crypto = { path = "senbei-android-crypto" } -senbei-android-elf = { path = "senbei-android-elf" } -senbei-android-engine = { path = "senbei-android-engine" } -senbei-android-metadata = { path = "senbei-android-metadata" } senbei-crypto = { path = "senbei-crypto" } +senbei-elf = { path = "senbei-elf" } +senbei-engine = { path = "senbei-engine" } senbei-io = { path = "senbei-io" } senbei-metadata = { path = "senbei-metadata" } senbei-pe = { path = "senbei-pe" } diff --git a/README.md b/README.md index 30fc6f9..75087b4 100644 --- a/README.md +++ b/README.md @@ -1,89 +1,56 @@ # Senbei -A static unpacker for Crackproof-protected 64-bit and 32-bit PE files and -protected Android (AArch64) shared libraries. Point it at a file, an app -package, or a folder and it writes decrypted copies — no launch of the -protected program, no kernel driver, no code runs out of the protected binary. +A static unpacker for Crackproof-protected 64-bit and 32-bit PE files and protected Android AArch64 shared libraries. Point it at a file, an app package, or a folder and it writes decrypted copies without launching the protected program. -> _"Crackproof"? It's senbei (煎餅 — rice cracker). Cracks itself._ +Senbei reads protected input bytes and replays the unpacking algorithm statically. The command-line tool adds filesystem scanning, progress reporting, and logs; `senbei-wasm` provides the browser binding. -Senbei reads a protected `.exe` or `.dll`, replays the unpacking algorithm -entirely in memory, and writes the recovered image to a new file. The core is a -pure, panic-free library with no file I/O; the CLI wraps it with scanning, a -progress bar, and a run log. A browser version (WebAssembly, fully client-side) -lives in [`web/`](web/). +## Crates -## Legal notice and intended use +The workspace contains eight crates: `senbei-cli`, `senbei-crypto`, `senbei-io`, `senbei-metadata`, `senbei-pe`, `senbei-elf`, `senbei-engine`, and `senbei-wasm`. -**Read this before using Senbei.** +`senbei-pe` and `senbei-elf` contain only basic format parsing and address mapping. Protection-specific code is in `senbei-engine/src/windows/` and `senbei-engine/src/android/`. Platform-specific crypto and metadata code is grouped under `senbei-crypto/src/android/`, `senbei-metadata/src/windows/`, and `senbei-metadata/src/android/`. -- Senbei is a research and interoperability tool. It exists to enable lawful - reverse engineering, security research, preservation, and interoperability - with software you already legitimately possess. -- **Only process binaries you own or are explicitly authorized to analyze.** - Depending on your jurisdiction and license agreements, circumventing - technological protection measures may be restricted (for example under - DMCA §1201 in the United States, which contains exemptions for security - research and interoperability). It is your responsibility to ensure your use - is lawful. -- Senbei does not bypass any access control for you: it performs a purely - static transformation of a file already on your disk. It derives everything - it needs from the input file itself, contains no vendor code, and - distributes no cracks or copyrighted content. (One Android packaging - variant's embedded metadata layer is unwrapped with an XOR keystream - recovered from a ciphertext/plaintext pair during analysis of a single - build; that keystream is research output shipped with the unpacker, not a - vendor-distributed key, and builds it doesn't match are left alone.) -- Senbei does not enable online play, license fraud, or cheating, and must not - be used to redistribute decrypted binaries. Do not upload outputs anywhere. -- The authors provide this software "as is", without warranty of any kind, and - accept no liability for misuse. See [LICENSE](LICENSE) (AGPL-3.0). -- "Crackproof" is a trademark of its respective owner; this project is not - affiliated with or endorsed by the protection vendor or any software - publisher. Names are used for identification only. +## Supported Inputs -## What it handles +- Protected Windows `.exe` and `.dll` files, including external `.exe._` and `.dll._` payloads. +- `global-metadata.dat` files with supported method-token layouts. +- Protected Android `.so` files and Android `.apk`, `.apks`, and `.xapk` packages. -| Kind | Description | -| --- | --- | -| `NativeExe` | Crackproof-protected native executable (PE32+ and PE32). | -| `ManagedExe` | Protected .NET executable (has a CLR data directory). | -| `NativeDll` | Protected native (unmanaged) DLL. | -| `ManagedDll` | Protected .NET assembly (has a CLR data directory). | -| `._` companion | Stub + external encrypted payload layout, spliced automatically. | -| `global-metadata.dat` | il2cpp metadata with obfuscated method tokens, de-obfuscated in place. | -| Android `.so` | Protected AArch64 shared library, statically restored (hollowed sections + stripped dynamic tables rebuilt). | -| `.apk` / `.apks` / `.xapk` | App packages; protected entries inside are restored, preserving the package's internal layout. | +Windows scanning probes only `.exe`, `.dll`, and `global-metadata.dat`; companion payloads are consumed through their matching stub and are not counted as skipped files. Android scanning probes only `.so` and `global-metadata.dat`. Android packages are inspected from their ZIP manifests and only matching `.so` and metadata entries are extracted. -Detection is content-based (header key-table at offset 4096, magic `KONN`), -not extension-based — app packages are the one exception, recognised by -extension plus the zip magic because they are containers. Anything -unrecognized is left untouched. - -## Quick start +## Quick Start ```cmd cargo build --release - senbei protected.exe -:: -> unpack\protected.unpack.exe - senbei game.apk -:: -> unpack\game.apk\lib\arm64-v8a\libil2cpp.unpack.so - senbei "C:\Games\MyGame" -:: -> C:\Games\MyGame\unpack\... (recursive, skips non-targets) ``` -Every output is sanity-checked statically; structurally broken results are -flagged as suspect rather than silently trusted. +Outputs are written below an `unpack` directory unless `--out` is supplied. Every restored PE or ELF image passes a structural validation step before it is reported as successful. -## Documentation +## Tests -- [Usage reference](docs/usage.md) — CLI flags, exit codes, integrity check -- [Design](docs/design.md) — architecture, routing, and error model -- [Development](docs/development.md) — building, testing, environment variables -- [Web version](web/README.md) — run Senbei in a browser +```cmd +cargo test --release --workspace +cargo clippy --workspace --all-targets -- -D warnings +cargo fmt --all -- --check +``` + +The local `test/` corpus can be passed to the CLI for real sample verification. The tracked `samples/` corpus is optional and remains user-managed. + +## Web Build + +```cmd +cd senbei-wasm +wasm-pack build --target web --release --out-dir ../web/pkg +``` + +The generated package is written to the ignored `web/pkg/` directory and can be served with any static HTTP server. + +## Legal Notice + +Use Senbei only for software you own or are authorized to analyze. The project is intended for lawful reverse engineering, security research, preservation, and interoperability. ## License diff --git a/docs/design.md b/docs/design.md index 4cb88b3..2780e71 100644 --- a/docs/design.md +++ b/docs/design.md @@ -1,238 +1,54 @@ # Design -Senbei is a fully static unpacker: it replays the unpacking algorithm on the -file bytes in memory and writes the recovered PE image. No code from the -protected binary is ever executed, no process is launched or attached to, and -no driver or proxy DLL is involved. +Senbei is a fully static unpacker. It reads protected bytes, replays the protection algorithm, validates the result, and writes a recovered image without launching or attaching to the protected program. -## Crate layout +## Crate Layout -Senbei is a Cargo workspace split into a pure core and thin shells around it: +The workspace is organized into eight crates. `senbei-cli` is the command-line entry point, `senbei-io` owns filesystem orchestration, `senbei-wasm` provides browser bindings, `senbei-pe` and `senbei-elf` provide basic format parsing, `senbei-crypto` provides shared primitives, `senbei-metadata` restores metadata, and `senbei-engine` owns protection-specific pipelines. -- **`senbei-pe/`** — the core. Pure functions over byte slices: no file I/O, - no environment access (beyond a few debugging overrides, see - [development.md](development.md)), panic-free at the public boundary (all - internal panics are trapped and converted to `UnpackError::Corrupt`). This - is what the WebAssembly build embeds. -- **`senbei-crypto/`** — cryptographic, checksum, compression, and bytecode - primitives the core is built from. Same purity rules as `senbei-pe`. -- **`senbei-metadata/`** — il2cpp `global-metadata.dat` method-token - de-obfuscation (format version 31; other versions are left untouched). -- **`senbei-android-crypto/`** — container primitives of the Android - (AArch64) protection scheme: the word/record ciphers, the GF(2³²) - transform, the AES-augmented segment transform, and the Huffman/LZ decoder. -- **`senbei-android-engine/`** — stage-1/stage-2 extraction: finds the - appended payload section, decrypts the stage-1 header and stage-2 payload, - and walks the recursive record streams to decode every module. Native-only - (memory-maps the input, writes the module set to a workspace directory). -- **`senbei-android-elf/`** — the restore: replays the decoded target-image - and fixup containers onto a hollowed ELF and rebuilds the dynamic-linker - tables (hash tables, symbols, relocations) the protector stripped. - Native-only. -- **`senbei-android-metadata/`** — the Android metadata variants: the seeded - five-round MethodDef-RID permutation restore (v31), seed discovery, and the - embedded-metadata XOR unwrap (`keystream.rs`). -- **`senbei-io/`** — filesystem and orchestration: recursive folder scanning, - per-run log file, progress bar, Explorer-friendly exit pause, the - single-file/folder orchestration in `job.rs` (incl. the wasm-safe in-memory - byte API used by the web frontend), and `android.rs` — the Android - single-library / folder / app-package orchestration. -- **`senbei-cli/`** — the `senbei` binary: argument parsing + dispatch. The - integration test suite (incl. the golden corpus test) lives in - `senbei-cli/tests/`. +Single-platform source stays directly under `src/`. Multi-platform crates keep platform code below `src/windows/` and `src/android/`, with shared code directly below `src/`. -``` -senbei-cli/ -└── src/main.rs argument parsing + dispatch -senbei-io/src/ -├── job.rs single-file + folder orchestration, out-naming, -│ companion splice, stub overlay/TLS restore, -│ pipeline routing (incl. the wasm-safe byte API) -├── android.rs Android single-library / folder / package -│ orchestration, cross-source dedup -├── scan.rs recursive target discovery (PE + metadata + Android) -├── logfile.rs per-run timestamped log -├── ui.rs progress bar + status lines -└── pause.rs Explorer-friendly exit pause -senbei-metadata/src/ -└── metadata.rs il2cpp global-metadata.dat de-obfuscation +```text +senbei-cli/src/main.rs senbei-crypto/src/ -├── primitives.rs decrypt_data* steps, key derivation -├── bytecode.rs bytecode VM -├── tables.rs constant tables -└── crc32.rs checksum -senbei-pe/src/engine/ pure, panic-free, no-I/O core -├── mod.rs detection + unpack_auto dispatch -├── error.rs structured error taxonomy -├── integrity.rs static post-unpack sanity check -├── parallel.rs deterministic block-parallel fan-out -├── layout/ layout discovery + validation -│ ├── dd8.rs .text dd8 key-formula + shift selection -│ ├── discovery.rs layout candidate discovery (trial-and-validate) -│ └── image.rs PE image reconstruction helpers -├── exe/ -│ ├── pipeline.rs EXE pipeline (PE32+ and PE32 orchestration) -│ └── pipeline/pe32.rs PE32-specific EXE restore -└── dll/ - └── pipeline.rs native + managed DLL pipeline -senbei-android-crypto/src/ -└── protector.rs container ciphers, GF(2^32), Huffman/LZ decoder -senbei-android-engine/src/ -├── stage1.rs payload-section discovery + stage-1 header/payload -├── stream.rs record-stream parsing -├── extract.rs recursive module extraction (writes the workspace) -├── probe.rs protected-library content probe -└── report.rs machine-readable extraction report -senbei-android-elf/src/ -├── restore.rs image restore + dynamic-table rebuild -├── layout.rs ELF layout parsing -├── artifact.rs module-workspace index loading -└── hash.rs SysV/GNU hash table rebuild -senbei-android-metadata/src/ -├── method_tokens.rs seeded RID permutation restore + seed discovery -├── embedded.rs embedded-metadata blob locate + XOR unwrap -└── keystream.rs recovered keystream table (one observed build) +senbei-crypto/src/android/ +senbei-elf/src/ +senbei-engine/src/windows/ +senbei-engine/src/android/ +senbei-io/src/ +senbei-io/src/android/ +senbei-metadata/src/windows/ +senbei-metadata/src/android/ +senbei-pe/src/ +senbei-wasm/src/ ``` -## Detection and routing +`senbei-pe` and `senbei-elf` are format crates only. They do not depend on the unpacking engines, filesystem code, or platform protection logic. -Detection is content-based (`unpacker::detect`), never extension-based: the -key table is derived from the file header and checked against the format -magic, then the PE characteristics classify the input as EXE or DLL and the -CLR data directory splits each into native vs managed (`NativeExe` / -`ManagedExe` / `NativeDll` / `ManagedDll`). The folder scan additionally -classifies Android targets: an ELF64/AArch64 prefix promotes the file to a -full protection probe (`senbei_android_engine::is_protected_libil2cpp`), and a -package extension plus zip magic marks an app package for container -extraction. +## Windows Engine -`unpack_auto` then dispatches: +`senbei-engine/src/windows/` contains PE detection, layout discovery, EXE and DLL restoration, deterministic block parallelism, and structural integrity checks. Candidate layouts are trial-decrypted and validated before an output is accepted. -- `NativeExe` / `ManagedExe` → the EXE pipeline (handles both PE32+ and - PE32). Managed EXEs take the same path: their import-string table is null - (imports are the CLR bootstrap stub), the entry point comes from the - protected header (the config block stores 0 for managed images), and the - COR20 header, BSJB metadata stream, and CLR resources are restored verbatim - from the protected file, mirroring the managed-DLL restore. -- `NativeDll` / `ManagedDll` → the DLL pipeline first; on failure, the EXE - pipeline as a fallback. Two DLL layouts exist in the wild: an older layout - the DLL pipeline parses, and a newer one that protects DLLs with the - EXE-style shell layout instead. The DLL-first order keeps old-layout outputs - byte-identical (the EXE pipeline also "succeeds" on old-layout DLLs but - produces different bytes); the fallback handles the new layout (including - the managed-DLL .NET metadata restore). +External companion inputs are reconstructed as `stub[..4096]` followed by the matching `._` payload. The stub's export and TLS data is overlaid after unpacking because those regions are not present in the encrypted companion. -One routing shortcut bypasses `unpack_auto`: inputs spliced from an external -companion (`job.rs`, both the CLI and the wasm byte API) go **straight to the -EXE pipeline**. The companion layout is definitionally the EXE-style shell, -so the DLL probe can never be right for it — and the probe's rejection of -EXE-shell DLLs relies on a caught panic, which is a fatal trap on targets -without unwinding (WebAssembly). Output bytes are identical to the -probe-then-fallback route. +## Android Engine -## External-companion inputs +`senbei-engine/src/android/extract/` decrypts the stage-1 header and stage-2 record streams and writes a temporary module workspace. `senbei-engine/src/android/restore/` applies decoded image and fixup containers to the hollowed ELF and rebuilds dynamic-linker tables. Both phases validate bounds and table placement before writing output. -Some builds split a protected module into an on-disk loader stub plus an -encrypted `._` companion. When a `._` sibling matches the stub's header -region, `job.rs` splices the two before unpacking and afterwards overlays the -export table and TLS directory from the stub — pieces the encrypted companion -does not carry. All overlay steps are best-effort no-ops when their inputs -can't be mapped, so a malformed stub can never corrupt an otherwise-good -unpack. +Android protection primitives are in `senbei-crypto/src/android/`. Android metadata restoration is in `senbei-metadata/src/android/` and only rewrites MethodDef token fields. The Windows structural metadata transform is in `senbei-metadata/src/windows/`. -## Pipelines +## Scanning and Packages -Both pipelines are **heuristic with trial-and-validate**: where a layout -leaves ambiguity (e.g. which block is the real file decryptor, or a page-XOR -shift), the pipeline tries candidates and validates the result structurally -(an entry-stub oracle, checksum stamps, cluster stamps) instead of trusting -the first match. A validation failure falls through to the next candidate -rather than producing silently wrong output. +Folder scanning uses platform target names to avoid opening bulk assets: Windows candidates are `.exe`, `.dll`, and `global-metadata.dat`; Android candidates are `.so` and `global-metadata.dat`. A Windows `.exe._` or `.dll._` companion is auxiliary input for its sibling stub and is excluded from the skipped count. -Several protected stages are themselves little bytecode programs. The core -includes a small VM (`bytecode.rs`) that generates and interprets those -programs rather than hardcoding each variant's constants. +APK, APKS, and XAPK files are containers. Senbei reads their ZIP manifests first, follows nested APK entries when necessary, and extracts only `.so` and exact `global-metadata.dat` entries. Extraction streams directly to temporary files, so compressed and decompressed copies are not held in memory together. -## The Android pipeline +## Validation -The Android scheme hollows an ELF64/AArch64 shared object: section bodies are -zeroed in the file and the original bytes move into an encrypted payload -appended as a `SHT_LOUSER` section (invisible to the dynamic loader). Restore -is two-phase: +Every heuristic layout uses trial-and-validate. A candidate that fails structural checks, checksums, or table bounds is rejected and the next candidate is tried. A failed restore is reported as an error rather than emitting a silently damaged binary. -1. **Extract** (`senbei-android-engine`): decrypt the stage-1 parameter block - and stage-2 payload from the payload section, then walk the recursive - record streams — each decoded module may interpret a further nested stream - — into a temporary module workspace with a JSON index. -2. **Restore** (`senbei-android-elf`): decode the target-image container onto - a copy of the hollowed file, apply the compact fixup database (the - relocations stripped from `.rela.dyn`), and rebuild the dynamic-linker - tables the loader needs (SysV/GNU hash, symbol and string tables, - `.rela.dyn`/`.rela.plt`). Validation is structural and total: mismatched - container sizes, descriptor bounds, or a rebuilt table overhanging its - section fail the restore rather than emit a broken image. +The PE integrity check verifies headers, section ranges, entry-point mapping, import names, relocation requirements, and managed metadata signatures. Android restoration validates ELF ranges, decoded container sizes, fixup bounds, and rebuilt dynamic tables. -il2cpp metadata comes in three shapes, all routed through -`job::deobfuscate_metadata_to` / `android::restore_metadata_bytes`: +## WebAssembly -- **structural (Windows `-GMD`)**: sparse method tokens remapped to the - contiguous per-module range, keyless, idempotent (`senbei-metadata`). -- **seeded permutation (Android v31)**: MethodDef RIDs permuted by a keyed - five-round transform; the seed is recovered by intersecting per-image key - residues, and the restore validates every RID — a wrong seed errors and the - structural remap takes over (`senbei-android-metadata`). -- **embedded blob**: no metadata file in the app at all; a slim blob sits in - the library's data section under a per-word XOR layer. After a restore the - blob is located by content (two known plaintext header words against the - embedded keystream) and unwrapped to a standalone `global-metadata.dat`. - Key derivation is untraced — the shipped keystream covers the one observed - build, and other builds simply never match the probe. - -Packages (`.apk`/`.apks`/`.xapk`) are containers, not targets: entries are -extracted to a temporary workspace and content-probed like loose files. -Cross-source duplicates (a library loose in the tree *and* inside its -package) are restored once, preferring the loose file, then the `.apk`, then -bundle splits. - -## Integrity check - -Every produced image passes through `integrity::check` — a static, execution- -free sanity check that only flags defects impossible in a correctly unpacked -image (malformed headers, unmapped/non-executable/all-zero/all-int3 entry -point, a native DLL with no base-relocation directory, any import descriptor -whose DLL name is still ciphertext, a managed image whose COR20 header or BSJB -metadata did not survive). See [usage.md](usage.md#integrity-check). -A clean report is not a proof of correctness; a non-clean report is a reliable -"broken" signal. - -## Parallelism - -Section decrypt/decompress blocks write disjoint output spans and read only -immutable input plus snapshotted key tables, so `parallel.rs` fans them out -across worker threads with **byte-identical** output regardless of thread -count. There is no `unsafe`: the buffer is carved with safe `split_at_mut` -chains so the borrow checker proves spans never alias. Overlapping spans (only -possible on corrupt input) degrade to the sequential whole-buffer pass, -preserving the deterministic last-writer-wins behavior of the serial -pipeline. `SENBEI_THREADS=1` forces the sequential path; on targets without -threads (WebAssembly) the sequential path is used automatically. - -## Error model - -The public API never panics: every pipeline runs under a `catch_unwind` -wrapper (`catch_unpack`) that converts a trapped panic to -`UnpackError::Corrupt`, with the default panic hook transiently suppressed. -Size requests are bounds-checked against a 1 GiB `MAX_IMAGE_SIZE` before -allocation so a crafted header cannot abort the process with a huge -allocation. In folder mode each file is isolated: one file's failure is logged -and counted, never fatal to the run. - -**WebAssembly caveat:** the prebuilt wasm std cannot unwind, so a caught -panic becomes a fatal `unreachable` trap there. The DLL-routing probe relies -on this mechanism to reject EXE-shell-layout DLLs, so the web build routes -around it instead of through it: spliced companion inputs skip the probe -entirely (see "Detection and routing"), and the web app isolates every unpack -in a disposable Web Worker — a trapped DLL is retried once in a fresh worker -with the forced-EXE pipeline (`job::unpack_bytes_force_exe`), reproducing the -probe-then-fallback outcome without a catchable panic. A trap on any other -input is reported as a clean error rather than freezing the page. +The browser binding depends on `senbei-engine` through the I/O byte API. Native filesystem and Android package orchestration remain outside the browser workflow. Each browser unpack runs in a disposable worker because WebAssembly cannot recover from a caught panic in the same way as native code. diff --git a/docs/development.md b/docs/development.md index 0206cec..d9de721 100644 --- a/docs/development.md +++ b/docs/development.md @@ -2,125 +2,60 @@ ## Building -Requires a Rust toolchain (MSVC backend is the default on Windows; -`rustup-init.exe` from installs it). The pinned toolchain -and targets are in `rust-toolchain.toml`. +The pinned Rust toolchain is defined in `rust-toolchain.toml`. Build the CLI with `cargo build --release`; the binary is written to `target/release/senbei.exe` on Windows. -```cmd -cargo build --release -``` - -Output: `target\release\senbei.exe`. The binary is self-contained — no driver, -no proxy DLL, no external assets. - -The library and CLI also build for Linux/macOS (`cfg`-gated platform code -only) and for `wasm32-unknown-unknown` (see the [web version](../web/README.md)). +The workspace crates are portable where their APIs are pure. The browser binding is outside the workspace and is checked with `cargo check --manifest-path senbei-wasm/Cargo.toml` or built with `wasm-pack`. ## Testing ```cmd -cargo test --release +cargo test --release --workspace +cargo clippy --workspace --all-targets -- -D warnings +cargo fmt --all -- --check ``` -The suite covers CLI behavior, detection, the folder driver, the run log, and -byte-exact golden tests over `samples/` — a user-managed corpus (git-ignored, -see `samples/README.md`) of real Crackproof inputs plus `.golden.` -reference outputs. Every input goes through `job::unpack_bytes` — the same -routing the CLI uses, so an `._` companion in the corpus is spliced and -the stub export/TLS overlays run — and is gated on **two** checks: the static -integrity check (catches runtime-broken outputs even when a stale golden would -still byte-match) and, when a golden exists, a bit-for-bit comparison. il2cpp -`*.dat` inputs are routed through `metadata::deobfuscate` instead. An empty or -absent corpus is a no-op pass; set `SENBEI_REQUIRE_SAMPLES` to make it fail -instead (useful on a private CI that has the corpus — public CI never does, -since binaries are not committed). +The tracked test suite is safe without protected samples. The optional local `samples/` corpus is user-managed and the ignored `test/` folder can be used for real Windows and Android runs. -> **Note:** goldens encode expected *bytes*, not runtime behavior. A golden -> produced before a pipeline fix may byte-match while still being wrong — the -> integrity check is the second gate for exactly this reason. Re-verify -> goldens against real runs when touching the affected pipeline stages. -> -> **The corpus only protects what it contains.** Wire the test to the routing -> the CLI actually takes (it is), and keep a sample for every layout family — -> marker-based, marker-less, external-companion, PE32, PE32+, native, managed, -> metadata. An unrepresented family has no regression gate at all, which is -> how a "re-run the golden corpus" rule can pass while silently covering -> nothing. +For an Android package, use one command at a time because a protected `.so` can be hundreds of megabytes. APK, APKS, and XAPK tests read the ZIP manifest first and extract only `.so` and `global-metadata.dat` entries. -## Debugging levers (environment variables) +## Environment Variables -- `DD8_SHIFT` — override the `decrypt_data8` page-XOR shift (`99` skips dd8 - entirely). -- `SEL_DIAG` — print the dd8 selector's scores: the per-shift `0xCC` counts and - the plaintext baseline they are compared against (PE32+), and the per-formula - counts, baseline and net gain (PE32). -- `SENBEI_THREADS` — cap the block-parallel fan-out (`1` forces the fully - sequential path). -- `SENBEI_SCAN_ALL` — same as `--scan-all` (probe every file in a folder). -- `SENBEI_ANDROID_SAMPLES` — override the Android corpus location (default - `samples/android/`; see `samples/README.md`). The Android corpus test pins - restored outputs with SHA-256 sidecar files next to each protected input - and documents known restore gaps with empty `.restore-fails` markers. +- `DD8_SHIFT` overrides the PE page-XOR shift; `99` skips that stage. +- `SEL_DIAG` prints PE layout-selector diagnostics. +- `SENBEI_THREADS` caps deterministic block fan-out; `1` forces the sequential reference path. +- `SENBEI_SCAN_ALL` enables the explicit scan-all mode for selected target names. +- `SENBEI_ANDROID_SAMPLES` overrides the Android sample corpus location. ## Conventions -- The `senbei-pe/` core (and its `senbei-crypto/` base) is pure: no file I/O, - no panics across the public boundary, no `unsafe`. Keep it that way — it is - what the WebAssembly build embeds. -- Layout heuristics must **trial-and-validate**: never pick a candidate offset - on shape alone and trust it; validate by decryption/checksum and fall - through to the next candidate on failure. A silent wrong offset produces a - silently broken output, which is worse than an error. -- Output must remain byte-identical against the golden corpus for every - supported layout. When fixing one build family, re-run the full golden - corpus to prove no other family regressed. -- Folder scanning uses a size floor plus an extension **deny**-list, never an - allow-list: targets are recognised by content, not extension, and can carry - arbitrary names, so only known bulk-asset extensions are excluded. The - pre-filter exists because folder-scan cost is per-file I/O latency, not the - walk — probe fewer files, don't parallelize the probe loop. -- `cargo fmt` and `cargo clippy` must stay clean (CI enforces both). +Format crates stay free of filesystem I/O and protection-specific logic. Windows engine code lives below `senbei-engine/src/windows/`, Android engine code below `senbei-engine/src/android/`, and shared code stays directly under each crate's `src/`. -## Repository layout +Layout heuristics must trial and validate every candidate. A failed validation is an error or a fall-through, never a silently accepted offset. -``` -senbei/ -├── Cargo.toml workspace root (members: the senbei-* crates) -├── rust-toolchain.toml pinned toolchain + targets -├── senbei-cli/ senbei binary (default member) -│ └── tests/ CLI, detection, golden, and folder tests -├── senbei-pe/ pure unpacker core (see docs/design.md) -├── senbei-crypto/ crypto/compression primitives -├── senbei-metadata/ il2cpp metadata de-obfuscation -├── senbei-io/ filesystem, scanning, CLI orchestration -├── senbei-wasm/ WebAssembly bindings crate (own Cargo.lock, -│ outside the workspace; builds into web/pkg/) -├── samples/ local-only test corpus (git-ignored) -├── web/ static browser frontend assets (+ built pkg/) -├── docs/ usage, design, and development documentation -└── .github/ CI workflows and issue templates +Outputs must remain byte-identical against the available golden corpus. Run the full workspace tests after changing a pipeline or a metadata layout. + +Folder scanning uses explicit target names to avoid opening bulk assets. External `.exe._` and `.dll._` files are auxiliary data for their sibling stubs and are not independent scan targets. + +## Repository Layout + +```text +senbei-cli/ command-line binary and integration tests +senbei-crypto/ shared crypto and Android crypto primitives +senbei-elf/ basic ELF parsing +senbei-engine/ Windows and Android unpacking engines +senbei-io/ filesystem, package, scanning, and CLI orchestration +senbei-metadata/ Windows and Android metadata restoration +senbei-pe/ basic PE parsing +senbei-wasm/ browser bindings and its own lockfile +web/ static browser frontend +samples/ optional local corpus ``` -## Web build - -See [web/README.md](../web/README.md). In short: +## Web Build ```cmd cd senbei-wasm wasm-pack build --target web --release --out-dir ../web/pkg ``` -then serve `web/` statically and open `index.html`. Everything runs -client-side; no file leaves the browser. - -## Contributing - -Issues and pull requests are welcome. A few ground rules: - -- **Never commit binaries** (protected or decrypted) to the repository — - the only corpus is the local git-ignored `samples/`. Attaching a protected - input file to an issue is welcome if it helps diagnose the problem; only - attach files you are authorized to share. -- Run `cargo test --release`, `cargo clippy`, and `cargo fmt` before - submitting. -- Keep the unpacker core free of I/O, `unsafe`, and platform-specific code. +Serve `web/` with a static HTTP server after the build. The browser never uploads input files. diff --git a/docs/usage.md b/docs/usage.md index 32824e6..658e84d 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -1,163 +1,55 @@ # Usage -``` -senbei [--out DIR] [-v|--verbose] [-q|--quiet]... [--scan-all] - [--no-log] [--no-pause] [-V|--version] [-h|--help] +```text +senbei [--out DIR] [-v|--verbose] [-q|--quiet]... [--scan-all] [--no-log] [--no-pause] [-V|--version] [-h|--help] ``` -Real runs print `Senbei ` once at start. Use `-V` / `--version` to -print the version and exit. +## Single File -## Single file - -The decrypted image is written under `/unpack/` with `.unpack` inserted -before the extension. A `senbei-.log` is written in the same -directory. With `--out DIR`, both the output and the log go into `DIR` instead: +The output is written below `/unpack/` with `.unpack` inserted before the extension. `--out DIR` changes both the output and log directory. ```cmd senbei app.exe -:: -> unpack\app.unpack.exe -:: -> unpack\senbei-YYYYMMDD-HHMMSS.log - senbei app.exe --out C:\out -:: -> C:\out\app.unpack.exe -:: -> C:\out\senbei-YYYYMMDD-HHMMSS.log ``` -Pointing senbei directly at an il2cpp `global-metadata.dat` rewrites its -obfuscated method tokens back to the contiguous per-module range il2cpp -expects; the output is `global-metadata.unpack.dat`, written only when tokens -actually changed. Only metadata format version 31 is rewritten; other versions -are reported and left untouched. +For `global-metadata.dat`, Senbei writes `global-metadata.unpack.dat` only when method tokens change. Unsupported metadata versions remain untouched and are reported as skipped. -## Android targets +## Android Targets -Senbei also restores Android (AArch64) protected shared libraries and app -packages: +Protected `.so` files are restored from their encrypted payload sections and written as `libil2cpp.unpack.so` or the corresponding input name. APK, APKS, and XAPK files are treated as containers: their manifests are read first, nested APKs are followed when necessary, and only `.so` and exact `global-metadata.dat` entries are extracted. -- **`.so`** — a protected library is hollowed out on disk: its original - sections live in an encrypted payload appended to the file, and senbei - rebuilds the static image from it. Output: `libil2cpp.unpack.so`. -- **`.apk`** — entries are extracted to a temporary workspace and - content-probed like loose files; protected libraries and metadata blobs - inside are restored to `//`. -- **`.apks` / `.xapk`** — split-package bundles; each nested `.apk` is opened - and searched the same way, under `///...`. +If a restored library contains embedded metadata, the unwrapped blob is written beside it as `global-metadata.unpack.dat`. Identical loose and package entries are restored once, preferring the loose file. -When a restored il2cpp library carries its metadata embedded in its data -section (no standalone `global-metadata.dat` in the app at all), senbei -unwraps the blob and writes it next to the library as -`global-metadata.unpack.dat`. One observed packaging variant wraps the blob in -a per-word XOR layer whose keys are generated at runtime and stored nowhere; -senbei ships the keystream recovered from the one build known to use it and -content-probes for it — builds with a different keystream are silently -skipped (the library itself is still fully restored). +## Folder Mode -The same content may appear loose in a folder, in its `.apk`, and in a bundle -side by side: identical content is restored once, at the loose file's -destination. A restored library is validated structurally by the restore -itself (the rebuild refuses inconsistent layouts); a protected library that -fails validation counts as an error, not a suspect. +Folder mode walks recursively, skips directories named `unpack`, and mirrors recognized outputs below `/unpack/` or `--out DIR`. Windows candidates are `.exe`, `.dll`, and `global-metadata.dat`; Android candidates are `.so` and `global-metadata.dat`. A matching `.exe._` or `.dll._` payload is consumed by its stub and is excluded from the skipped count. -## Folder mode +The summary has the form `12 unpacked · 3 skipped · 0 errors · 1 suspect · 2 metadata`; the package count is appended when packages were opened. Each file is isolated so one failed target does not stop the folder run. -Senbei walks the directory recursively, skips any subdirectory literally named -`unpack`, and unpacks every file it recognises as protected (by content, not -extension — renamed files and `.bak` backups are still found; packages are the -one exception, recognised by extension plus the zip magic because they are -containers). Results land under `/unpack/` (or `--out DIR`), mirroring -the input tree's relative paths. The run log is written **in that same out -directory**: +## Integrity Check -```cmd -senbei "C:\Games\MyGame" -:: -> C:\Games\MyGame\unpack\... -:: -> C:\Games\MyGame\unpack\senbei-YYYYMMDD-HHMMSS.log -``` +PE outputs are checked for valid headers, section ranges, entry-point mapping, readable import names, relocation requirements, and managed metadata signatures. Android outputs are validated during ELF restoration, including decoded container sizes, fixup bounds, and rebuilt dynamic tables. -Folder mode also picks up `global-metadata.dat` files and external-companion -`._` payloads: a module whose `._` sibling matches its header region is -spliced with the companion automatically (no flag needed) and unpacked as one -image, with the output named for the stub. - -Each file is processed in isolation: an error or panic on one file is caught, -counted, and logged, and the run continues. Folder mode finishes with a summary -line, then duration: - -``` -12 unpacked · 3 skipped · 0 errors · 1 suspect · 2 metadata -done in 1234 ms -``` - -The `packages` count appears (as `· N packages`) only when Android app -packages were processed. - -## Integrity check - -(PE outputs only — Android restores carry their own structural validation; see -[Android targets](#android-targets).) - -A successful unpack is not always a runnable one: a layout heuristic can pick -the wrong offset and leave the entry-point stub or import strings encrypted, so -the pipeline reports success but the OS loader faults at runtime (typically -`0xC0000005`, STATUS_ACCESS_VIOLATION). To catch this, senbei runs a static -sanity check over every output it produces — inspecting the bytes alone, with -no reference image and no execution. - -It flags only defects that cannot occur in a correctly unpacked image: - -- malformed DOS/PE headers, bad optional-header magic, implausible section - count, zero `SizeOfImage`, or section raw-data ranges that run past EOF; -- an entry point that doesn't map into a section, isn't in an executable - section, or whose stub is all zeros or all `0xCC` int3 padding (the classic - left-encrypted symptom); -- a native (unmanaged) DLL with no base-relocation directory — it cannot - survive being mapped at a non-preferred base; -- **any** import descriptor whose DLL name doesn't resolve or isn't readable - ASCII (imports left encrypted) — the whole table is walked, not just the - first entry; -- for a managed assembly, a COR20 header whose `cb` isn't `0x48` or a - MetaData stream missing its `BSJB` signature (the CLR would reject the - image outright). - -The entry-point and import checks are skipped for managed assemblies, whose -native EP and import stub are legitimately not what the native loader expects. - -The check is deliberately conservative: a clean report is **not** a proof of -correctness, but a non-clean report is a reliable "this is broken" signal. A -suspect file is still written (the bytes are the best available) and flagged — -single-file mode prints a warning to stderr, folder mode prints a yellow `!` -line, adds a `SUSPECT` entry to the run log, and counts it in the summary's -`suspect` total (which is additive to `unpacked`). +A clean report is not a proof of correctness, but a non-clean report is a reliable broken-output signal. Suspect PE files are still written and counted separately. ## Flags | Flag | Behavior | | --- | --- | -| `--out DIR` | Write outputs (and the log, unless `--no-log`) under `DIR`. | -| `-v`, `--verbose` | Print detailed per-stage progress (and the destination path) for each file — `[N/9]` stages for PE targets, container/segment lines for Android libraries. In folder mode this replaces the progress bar. | -| `-q`, `--quiet` | Once: hide progress bar and per-file lines; keep banner, summary, and duration. Twice (`-q -q`): suppress all stdio (exit code only). | -| `--no-log` | Do not write `senbei-*.log`. Console output is unchanged by this flag alone. | -| `--scan-all` | Probe every file in a folder, including ones the scan pre-filter skips (under 4128 bytes, or a bulk-asset extension like `.ab`/`.xml`/`.acb`). Much slower on large game trees; finds the same targets in practice. | -| `--no-pause` | Skip the "Press Enter to exit" prompt (for scripted runs). | -| `-V`, `--version` | Print `Senbei ` and exit. | +| `--out DIR` | Write outputs and logs below `DIR`. | +| `-v`, `--verbose` | Print per-stage progress. | +| `-q`, `--quiet` | Hide progress and per-file lines; repeat to suppress all standard output. | +| `--no-log` | Do not write a run log. | +| `--scan-all` | Probe every selected target-name candidate, including files below the size floor. | +| `--no-pause` | Disable the Explorer-friendly Windows exit prompt. | +| `-V`, `--version` | Print the version and exit. | | `-h`, `--help` | Show usage. | -On Windows, when launched from Explorer (the process owns its console) senbei -pauses for Enter before exiting so the window doesn't vanish. `--no-pause` -disables this; it has no effect when stdout is piped or run from another -process. - -## Exit codes +## Exit Codes | Code | Meaning | | --- | --- | -| `0` | Success (single file restored, or folder run with no errors). | -| `1` | At least one file failed, a scan probe was unreadable, or a single-file unpack errored. | -| `2` | Usage error: no path given, unknown option, missing `--out` value, or multiple input paths (help printed). | - -A folder run also fails with `1` when parts of the tree could not be scanned -(unreadable directory entries or files that failed the content probe) — those -are potential missed targets, not clean skips. An il2cpp metadata blob whose -format version senbei does not handle is *not* an error: it is reported, left -untouched, and counted as skipped. +| `0` | The requested restore completed without errors. | +| `1` | A target failed, a scan probe was unreadable, or a single-file restore errored. | +| `2` | The command line was invalid. | diff --git a/samples/README.md b/samples/README.md index 07fed79..5be4af8 100644 --- a/samples/README.md +++ b/samples/README.md @@ -1,105 +1,23 @@ -# senbei/samples +# Samples -Drop-in corpus for the `samples` integration test (`tests/samples.rs`). +This ignored directory is the optional local corpus used by the samples integration test. Protected binaries and restored outputs must never be committed. -This folder is **git-ignored** (only this `README.md` is tracked), so it holds -whatever Crackproof binaries happen to be on your machine. Nothing here is -committed. +Place protected `.exe` and `.dll` files, exact `global-metadata.dat` files, and matching `.exe._` or `.dll._` companion payloads here. A golden output may sit beside an input as `.golden.`. -## What to put here - -Place protected inputs directly in this folder: - -- `*.exe` — Crackproof-protected executables (PE32 or PE32+) -- `*.dll` — Crackproof-protected DLLs (native or managed) -- `*.dat` — il2cpp `global-metadata.dat` blobs (method-token de-obfuscation) - -For an **external-companion** module, copy the `._` payload in as well, -keeping the exact `._` suffix on the full file name. The test splices it the -same way the CLI does; without it the loader stub alone is meaningless and the -splice / export-overlay / TLS-restore code is never exercised. - -Optionally, place a **golden** next to each input — the known-good unpacked -output, named `.golden.`: - -``` +```text samples/ - app.exe <- input - app.golden.exe <- golden (optional) - managed.dll <- input - managed.golden.dll <- golden (optional) - stub.dll <- input (external-companion layout) - stub.dll._ <- its encrypted payload (NOT an input itself) - stub.golden.dll <- golden - global-metadata.dat <- input - global-metadata.golden.dat<- golden - mystery.exe <- input, no golden + app.exe + app.golden.exe + managed.dll + stub.dll + stub.dll._ + stub.golden.dll + global-metadata.dat + global-metadata.golden.dat ``` -The type (EXE vs native/managed DLL vs metadata) is auto-detected from the file -contents, not the extension, so you don't need to classify anything by hand. +Run `cargo test --release --test samples -- --nocapture`. A missing golden prints a warning; a mismatched golden or failed restore fails the test. An empty corpus is a no-op pass. -Since the corpus is the only regression gate on byte-identical output, keep it -broad: each build family, each layout (marker-based and marker-less), and at -least one external-companion pair. A family with no sample here is a family no -test protects. +## Android Corpus -## How the test treats each input - -Run with: - -``` -cargo test --release --test samples -``` - -For every input file, the test runs the same routing the CLI uses -(`job::unpack_bytes`, so companions splice and the stub overlays run) — or -`metadata::deobfuscate` for an il2cpp blob — and then: - -| Situation | Result | -| ------------------------------------------- | ------------------------------- | -| Golden present, bytes **identical** | **pass** | -| Golden present, bytes **differ** | **fail** (test fails) | -| **No golden** found | **warning** (needs manual check)| -| Unpack errored / file unreadable | **fail** | - -Warnings are printed but do not fail the test — they flag outputs you should -eyeball or promote to a golden once verified. Failures fail the test. An empty -or absent folder is a no-op pass. - -To see the per-file warning/pass/fail summary, run with output shown: - -``` -cargo test --release --test samples -- --nocapture -``` - -## Naming rules - -- An **input** is any `*.exe` / `*.dll` / `*.dat` whose name does **not** - contain the `.golden.` segment. -- A **golden** is `.golden.` sitting next to its input. Files with - `.golden.` in the name are never treated as inputs. -- A **companion** is `._` (e.g. `stub.dll._` for `stub.dll`). - Its extension is `_`, so it is never picked up as an input of its own; it is - read only when its base module is processed. - -## Android corpus (`samples/android/`) - -The `android/` subfolder holds Android samples, one **extracted app tree** per -subdirectory (the layout an APK unpacks to: `lib//*.so`, -`assets/.../global-metadata.dat`, ...). The test -(`tests/android_samples.rs`) finds protected AArch64 libraries by content and -restores them through the real pipeline. `SENBEI_ANDROID_SAMPLES` overrides -the corpus location. - -Sidecar conventions (all next to the protected `.so` input): - -| File | Meaning | -| ---- | ------- | -| `.golden.so.sha256` | Expected SHA-256 of the restored library | -| `.golden.metadata.sha256` | Expected SHA-256 of the unwrapped embedded metadata blob (when the library carries one) | -| `.restore-fails` | Empty marker: this input's restore is a known gap and *must* fail (a future fix fails the test, prompting marker removal) | - -A missing sidecar is a warning (with the computed digest printed, ready to -promote), never a failure. App packages (`.apk`/`.apks`/`.xapk`) dropped into -a tree are exercised by folder mode as containers. +`samples/android/` may contain one extracted app tree per subdirectory. Protected libraries are restored through the real pipeline and can carry SHA-256 sidecars named `.golden.so.sha256` and `.golden.metadata.sha256`. An empty `.restore-fails` marker documents a known restore gap. diff --git a/senbei-android-elf/src/lib.rs b/senbei-android-elf/src/lib.rs deleted file mode 100644 index 4c57c0a..0000000 --- a/senbei-android-elf/src/lib.rs +++ /dev/null @@ -1,10 +0,0 @@ -//! Static restoration of the current protected AArch64 `libil2cpp.so`. - -mod artifact; -mod error; -mod hash; -mod layout; -mod restore; - -pub use error::Error; -pub use restore::{RestoreOptions, RestoreReport, restore_libil2cpp}; diff --git a/senbei-android-engine/Cargo.toml b/senbei-android-engine/Cargo.toml deleted file mode 100644 index c164822..0000000 --- a/senbei-android-engine/Cargo.toml +++ /dev/null @@ -1,20 +0,0 @@ -[package] -name = "senbei-android-engine" -version.workspace = true -edition.workspace = true -rust-version.workspace = true -license.workspace = true -description = "Static Stage 1 and Stage 2 extraction for Senbei Android" - -[dependencies] -goblin.workspace = true -memmap2.workspace = true -serde.workspace = true -serde_json.workspace = true -sha2.workspace = true -tempfile.workspace = true -thiserror.workspace = true -senbei-android-crypto.workspace = true - -[lints] -workspace = true diff --git a/senbei-android-metadata/Cargo.toml b/senbei-android-metadata/Cargo.toml deleted file mode 100644 index 2dbaeb7..0000000 --- a/senbei-android-metadata/Cargo.toml +++ /dev/null @@ -1,14 +0,0 @@ -[package] -name = "senbei-android-metadata" -version.workspace = true -edition.workspace = true -rust-version.workspace = true -license.workspace = true -description = "IL2CPP metadata restoration for Senbei Android" - -[dependencies] -serde.workspace = true -thiserror.workspace = true - -[lints] -workspace = true diff --git a/senbei-cli/Cargo.toml b/senbei-cli/Cargo.toml index 8ee1b2a..b2941a6 100644 --- a/senbei-cli/Cargo.toml +++ b/senbei-cli/Cargo.toml @@ -13,6 +13,7 @@ path = "src/main.rs" [dependencies] senbei-io.workspace = true +senbei-engine.workspace = true [dev-dependencies] senbei-io.workspace = true diff --git a/senbei-crypto/Cargo.toml b/senbei-crypto/Cargo.toml index 3ee7e4b..07c8e97 100644 --- a/senbei-crypto/Cargo.toml +++ b/senbei-crypto/Cargo.toml @@ -6,4 +6,5 @@ license.workspace = true description = "Cryptographic and compression primitives for Senbei" [dependencies] +aes.workspace = true thiserror.workspace = true diff --git a/senbei-android-crypto/src/lib.rs b/senbei-crypto/src/android/mod.rs similarity index 74% rename from senbei-android-crypto/src/lib.rs rename to senbei-crypto/src/android/mod.rs index 36c6876..a8eef22 100644 --- a/senbei-android-crypto/src/lib.rs +++ b/senbei-crypto/src/android/mod.rs @@ -1,4 +1,4 @@ -//! Cryptographic and container primitives used by Senbei Android. +//! Android container cryptography and decoding primitives. mod protector; diff --git a/senbei-android-crypto/src/protector.rs b/senbei-crypto/src/android/protector.rs similarity index 99% rename from senbei-android-crypto/src/protector.rs rename to senbei-crypto/src/android/protector.rs index 7954ecc..f86b24f 100644 --- a/senbei-android-crypto/src/protector.rs +++ b/senbei-crypto/src/android/protector.rs @@ -359,7 +359,7 @@ pub struct HuffmanLzDecoder { impl HuffmanLzDecoder { /// Build the full 16-bit prefix lookup used by the static decoder. pub fn new(tree: &[u8]) -> Result { - if tree.len() < 256 * 3 || tree.len() % 3 != 0 { + if tree.len() < 256 * 3 || !tree.len().is_multiple_of(3) { return invalid(format!("invalid Huffman tree size 0x{:x}", tree.len())); } let mut result = Self { @@ -542,6 +542,7 @@ impl HuffmanLzDecoder { } /// Apply the native word transform and optional AES-256-CBC decryption. +#[allow(clippy::chunks_exact_to_as_chunks)] pub fn transform_segment( data: &[u8], seed: u32, diff --git a/senbei-crypto/src/lib.rs b/senbei-crypto/src/lib.rs index 2f99e7f..4401188 100644 --- a/senbei-crypto/src/lib.rs +++ b/senbei-crypto/src/lib.rs @@ -1,5 +1,6 @@ //! Cryptographic, checksum, compression, and bytecode primitives. +pub mod android; pub mod bytecode; pub mod crc32; pub mod primitives; diff --git a/senbei-android-crypto/Cargo.toml b/senbei-elf/Cargo.toml similarity index 60% rename from senbei-android-crypto/Cargo.toml rename to senbei-elf/Cargo.toml index 269e2aa..59602ca 100644 --- a/senbei-android-crypto/Cargo.toml +++ b/senbei-elf/Cargo.toml @@ -1,13 +1,13 @@ [package] -name = "senbei-android-crypto" +name = "senbei-elf" version.workspace = true edition.workspace = true rust-version.workspace = true license.workspace = true -description = "Protector container primitives for Senbei Android" +description = "ELF format parsing and structural utilities for Senbei" [dependencies] -aes.workspace = true +goblin.workspace = true thiserror.workspace = true [lints] diff --git a/senbei-elf/src/lib.rs b/senbei-elf/src/lib.rs new file mode 100644 index 0000000..6994cb3 --- /dev/null +++ b/senbei-elf/src/lib.rs @@ -0,0 +1,54 @@ +//! Basic ELF format parsing shared by the unpacking engine. + +use goblin::elf::{Elf, header::EM_AARCH64, program_header::PT_LOAD}; +use thiserror::Error; + +#[derive(Debug, Error)] +pub enum Error { + #[error("ELF parse failed: {0}")] + Parse(#[from] goblin::error::Error), + #[error("input is not an ELF64 little-endian image")] + NotElf64, + #[error("input is not an AArch64 image")] + NotAarch64, +} + +pub type Result = std::result::Result; + +/// Parse an ELF64 little-endian image. +pub fn parse(data: &[u8]) -> Result> { + let elf = Elf::parse(data)?; + if elf.header.e_ident[4] != 2 || elf.header.e_ident[5] != 1 { + return Err(Error::NotElf64); + } + Ok(elf) +} + +/// Return true when `data` starts with a valid AArch64 ELF64 image. +pub fn is_aarch64(data: &[u8]) -> bool { + parse(data) + .map(|elf| elf.header.e_machine == EM_AARCH64) + .unwrap_or(false) +} + +/// Return the maximum file end among PT_LOAD segments. +pub fn load_file_end(data: &[u8]) -> Result { + let elf = parse(data)?; + Ok(elf + .program_headers + .iter() + .filter(|ph| ph.p_type == PT_LOAD) + .map(|ph| ph.p_offset.saturating_add(ph.p_filesz)) + .max() + .unwrap_or(0)) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn rejects_non_elf() { + assert!(matches!(parse(b"not elf"), Err(Error::Parse(_)))); + } +} diff --git a/senbei-android-elf/Cargo.toml b/senbei-engine/Cargo.toml similarity index 70% rename from senbei-android-elf/Cargo.toml rename to senbei-engine/Cargo.toml index 6d84996..889c58e 100644 --- a/senbei-android-elf/Cargo.toml +++ b/senbei-engine/Cargo.toml @@ -1,19 +1,20 @@ [package] -name = "senbei-android-elf" +name = "senbei-engine" version.workspace = true edition.workspace = true rust-version.workspace = true license.workspace = true -description = "AArch64 ELF restoration for Senbei Android" +description = "Platform unpacking engines for Senbei" [dependencies] +goblin.workspace = true memmap2.workspace = true serde.workspace = true serde_json.workspace = true sha2.workspace = true tempfile.workspace = true thiserror.workspace = true -senbei-android-crypto.workspace = true +senbei-crypto.workspace = true [lints] workspace = true diff --git a/senbei-android-engine/src/error.rs b/senbei-engine/src/android/extract/error.rs similarity index 91% rename from senbei-android-engine/src/error.rs rename to senbei-engine/src/android/extract/error.rs index 2711d68..cd794a7 100644 --- a/senbei-android-engine/src/error.rs +++ b/senbei-engine/src/android/extract/error.rs @@ -19,7 +19,7 @@ pub enum Error { #[error("serialize extraction index: {0}")] Json(#[from] serde_json::Error), #[error("embedded Stage 2 decoder configuration: {0}")] - EmbeddedConfig(#[source] senbei_android_crypto::Error), + EmbeddedConfig(#[source] senbei_crypto::android::Error), #[error( "depth {depth} stream 0x{stream_id:02X} interpreter 0x{interpreter_id:02X} configuration: {source}" )] @@ -28,7 +28,7 @@ pub enum Error { stream_id: u32, interpreter_id: u32, #[source] - source: senbei_android_crypto::Error, + source: senbei_crypto::android::Error, }, #[error( "depth {depth} stream 0x{stream_id:02X} record {record_index} command 0x{command_id:02X} {part}: {source}" @@ -40,7 +40,7 @@ pub enum Error { command_id: u32, part: &'static str, #[source] - source: senbei_android_crypto::Error, + source: senbei_crypto::android::Error, }, #[error("{0}")] Invalid(String), diff --git a/senbei-android-engine/src/lib.rs b/senbei-engine/src/android/extract/mod.rs similarity index 60% rename from senbei-android-engine/src/lib.rs rename to senbei-engine/src/android/extract/mod.rs index 386eb55..3fee210 100644 --- a/senbei-android-engine/src/lib.rs +++ b/senbei-engine/src/android/extract/mod.rs @@ -1,14 +1,12 @@ -//! Pure-static Stage 1 decryption and recursive Stage 2 module extraction. - mod error; -mod extract; +mod pipeline; mod probe; mod report; mod stage1; mod stream; pub use error::Error; -pub use extract::{ExtractOptions, extract_stage2}; +pub use pipeline::{ExtractOptions, extract_stage2}; pub use probe::is_protected_libil2cpp; pub use report::ExtractionReport; pub use stage1::{DEFAULT_CIPHER_CONSTANT, DEFAULT_OUTER_SIZE}; diff --git a/senbei-android-engine/src/extract.rs b/senbei-engine/src/android/extract/pipeline.rs similarity index 98% rename from senbei-android-engine/src/extract.rs rename to senbei-engine/src/android/extract/pipeline.rs index 2663ebd..2b4b476 100644 --- a/senbei-android-engine/src/extract.rs +++ b/senbei-engine/src/android/extract/pipeline.rs @@ -4,20 +4,20 @@ use std::io::Write; use std::path::{Path, PathBuf}; use memmap2::MmapOptions; -use senbei_android_crypto::{Module9bConfig, decode_container}; +use senbei_crypto::android::{Module9bConfig, decode_container}; use serde_json::to_vec_pretty; use sha2::{Digest, Sha256}; use tempfile::NamedTempFile; -use crate::error::{Error, Result, invalid}; -use crate::report::{ +use super::error::{Error, Result, invalid}; +use super::report::{ ArtifactReport, DecoderReport, ExtractionReport, ModuleRegistryEntry, RecordReport, Stage1Report, StreamParent, StreamReport, }; -use crate::stage1::{ +use super::stage1::{ DEFAULT_CIPHER_CONSTANT, DEFAULT_OUTER_SIZE, SHT_LOUSER, Stage1Result, inspect, }; -use crate::stream::{DIRECT_FLAG, Record, parse_record_stream}; +use super::stream::{DIRECT_FLAG, Record, parse_record_stream}; /// Inputs and output locations for one complete static Stage 2 extraction. #[derive(Debug, Clone)] diff --git a/senbei-android-engine/src/probe.rs b/senbei-engine/src/android/extract/probe.rs similarity index 81% rename from senbei-android-engine/src/probe.rs rename to senbei-engine/src/android/extract/probe.rs index 73c250d..c563874 100644 --- a/senbei-android-engine/src/probe.rs +++ b/senbei-engine/src/android/extract/probe.rs @@ -1,8 +1,8 @@ use std::path::Path; -use senbei_android_crypto::Module9bConfig; +use senbei_crypto::android::Module9bConfig; -use crate::stage1::{self, DEFAULT_CIPHER_CONSTANT, DEFAULT_OUTER_SIZE}; +use super::stage1::{self, DEFAULT_CIPHER_CONSTANT, DEFAULT_OUTER_SIZE}; /// Return whether `data` has a supported protected AArch64 IL2CPP layout. #[must_use] diff --git a/senbei-android-engine/src/report.rs b/senbei-engine/src/android/extract/report.rs similarity index 100% rename from senbei-android-engine/src/report.rs rename to senbei-engine/src/android/extract/report.rs diff --git a/senbei-android-engine/src/stage1.rs b/senbei-engine/src/android/extract/stage1.rs similarity index 99% rename from senbei-android-engine/src/stage1.rs rename to senbei-engine/src/android/extract/stage1.rs index 8b52c68..3c86d6c 100644 --- a/senbei-android-engine/src/stage1.rs +++ b/senbei-engine/src/android/extract/stage1.rs @@ -2,7 +2,7 @@ use std::path::Path; use goblin::elf::{Elf, header::EM_AARCH64}; -use crate::error::{Error, Result, invalid}; +use super::error::{Error, Result, invalid}; pub(crate) const SHT_LOUSER: u32 = 0x8000_0000; pub const DEFAULT_CIPHER_CONSTANT: u32 = 0xbf20_165d; diff --git a/senbei-android-engine/src/stream.rs b/senbei-engine/src/android/extract/stream.rs similarity index 98% rename from senbei-android-engine/src/stream.rs rename to senbei-engine/src/android/extract/stream.rs index b1d4e19..f693442 100644 --- a/senbei-android-engine/src/stream.rs +++ b/senbei-engine/src/android/extract/stream.rs @@ -1,6 +1,6 @@ -use senbei_android_crypto::gf32_mul_fixed; +use senbei_crypto::android::gf32_mul_fixed; -use crate::error::{Error, Result, invalid}; +use super::error::{Error, Result, invalid}; pub(crate) const RECORD_SIZE: usize = 0x5c; pub(crate) const DIRECT_FLAG: u32 = 2; diff --git a/senbei-engine/src/android/mod.rs b/senbei-engine/src/android/mod.rs new file mode 100644 index 0000000..40638af --- /dev/null +++ b/senbei-engine/src/android/mod.rs @@ -0,0 +1,10 @@ +//! Android AArch64 extraction and ELF restoration. + +mod extract; +mod restore; + +pub use extract::{ + DEFAULT_CIPHER_CONSTANT, DEFAULT_OUTER_SIZE, Error as ExtractionError, ExtractOptions, + ExtractionReport, extract_stage2, is_protected_libil2cpp, +}; +pub use restore::{Error as RestoreError, RestoreOptions, RestoreReport, restore_libil2cpp}; diff --git a/senbei-android-elf/src/artifact.rs b/senbei-engine/src/android/restore/artifact.rs similarity index 98% rename from senbei-android-elf/src/artifact.rs rename to senbei-engine/src/android/restore/artifact.rs index 464e51d..ceee104 100644 --- a/senbei-android-elf/src/artifact.rs +++ b/senbei-engine/src/android/restore/artifact.rs @@ -3,7 +3,7 @@ use std::path::{Path, PathBuf}; use serde_json::Value; -use crate::error::{Error, Result, invalid}; +use super::error::{Error, Result, invalid}; const REQUIRED_IDS: [u32; 3] = [0x9b, 0x9d, 0x9e]; diff --git a/senbei-android-elf/src/error.rs b/senbei-engine/src/android/restore/error.rs similarity index 94% rename from senbei-android-elf/src/error.rs rename to senbei-engine/src/android/restore/error.rs index 058db5e..13b7d09 100644 --- a/senbei-android-elf/src/error.rs +++ b/senbei-engine/src/android/restore/error.rs @@ -13,7 +13,7 @@ pub enum Error { #[error("cannot parse module index: {0}")] Json(#[from] serde_json::Error), #[error(transparent)] - Crypto(#[from] senbei_android_crypto::Error), + Crypto(#[from] senbei_crypto::android::Error), #[error("{0}")] Invalid(String), } diff --git a/senbei-android-elf/src/hash.rs b/senbei-engine/src/android/restore/hash.rs similarity index 98% rename from senbei-android-elf/src/hash.rs rename to senbei-engine/src/android/restore/hash.rs index 550e012..360b6b8 100644 --- a/senbei-android-elf/src/hash.rs +++ b/senbei-engine/src/android/restore/hash.rs @@ -1,4 +1,4 @@ -use crate::error::{Error, Result, invalid}; +use super::error::{Error, Result, invalid}; #[must_use] pub(crate) fn elf_hash(name: &[u8]) -> u32 { diff --git a/senbei-android-elf/src/layout.rs b/senbei-engine/src/android/restore/layout.rs similarity index 99% rename from senbei-android-elf/src/layout.rs rename to senbei-engine/src/android/restore/layout.rs index 5ffc7e9..853730b 100644 --- a/senbei-android-elf/src/layout.rs +++ b/senbei-engine/src/android/restore/layout.rs @@ -1,4 +1,4 @@ -use crate::error::{Error, Result, invalid}; +use super::error::{Error, Result, invalid}; pub(crate) const SHT_NOBITS: u32 = 8; pub(crate) const SHT_LOUSER: u32 = 0x8000_0000; diff --git a/senbei-engine/src/android/restore/mod.rs b/senbei-engine/src/android/restore/mod.rs new file mode 100644 index 0000000..cce81b3 --- /dev/null +++ b/senbei-engine/src/android/restore/mod.rs @@ -0,0 +1,8 @@ +mod artifact; +mod error; +mod hash; +mod layout; +mod pipeline; + +pub use error::Error; +pub use pipeline::{RestoreOptions, RestoreReport, restore_libil2cpp}; diff --git a/senbei-android-elf/src/restore.rs b/senbei-engine/src/android/restore/pipeline.rs similarity index 99% rename from senbei-android-elf/src/restore.rs rename to senbei-engine/src/android/restore/pipeline.rs index b8b54be..a0243b8 100644 --- a/senbei-android-elf/src/restore.rs +++ b/senbei-engine/src/android/restore/pipeline.rs @@ -5,17 +5,17 @@ use std::path::{Path, PathBuf}; use std::time::Instant; use memmap2::{Mmap, MmapMut, MmapOptions}; -use senbei_android_crypto::{ +use senbei_crypto::android::{ ContainerHeader, HuffmanLzDecoder, Module9bConfig, ProtectedDescriptor, transform_segment, }; use serde::Serialize; use sha2::{Digest, Sha256}; use tempfile::NamedTempFile; -use crate::artifact::load_artifacts; -use crate::error::{Error, Result, invalid}; -use crate::hash::{build_gnu_hash, build_sysv_hash}; -use crate::layout::{ +use super::artifact::load_artifacts; +use super::error::{Error, Result, invalid}; +use super::hash::{build_gnu_hash, build_sysv_hash}; +use super::layout::{ ElfLayout, SHF_ALLOC, SHT_LOUSER, SHT_NOBITS, SectionHeader, align_up, read_i64, read_u32, read_u64, slice, slice_u64, usize_from_u64, }; diff --git a/senbei-engine/src/lib.rs b/senbei-engine/src/lib.rs new file mode 100644 index 0000000..f7db693 --- /dev/null +++ b/senbei-engine/src/lib.rs @@ -0,0 +1,14 @@ +//! Platform-specific unpacking engines. + +pub mod android; +pub mod windows; + +pub use windows::{ + Detected, IntegrityReport, Kind, UnpackError, check_integrity, detect, unpack_auto, + unpack_auto_v, unpack_dll, unpack_dll_v, unpack_exe, unpack_exe_v, +}; + +/// Deterministic worker-thread cap shared by filesystem scanning and engines. +pub fn thread_cap() -> usize { + windows::thread_cap() +} diff --git a/senbei-pe/src/engine/dll/mod.rs b/senbei-engine/src/windows/dll/mod.rs similarity index 100% rename from senbei-pe/src/engine/dll/mod.rs rename to senbei-engine/src/windows/dll/mod.rs diff --git a/senbei-pe/src/engine/dll/pipeline.rs b/senbei-engine/src/windows/dll/pipeline.rs similarity index 100% rename from senbei-pe/src/engine/dll/pipeline.rs rename to senbei-engine/src/windows/dll/pipeline.rs diff --git a/senbei-pe/src/engine/error.rs b/senbei-engine/src/windows/error.rs similarity index 100% rename from senbei-pe/src/engine/error.rs rename to senbei-engine/src/windows/error.rs diff --git a/senbei-pe/src/engine/exe/mod.rs b/senbei-engine/src/windows/exe/mod.rs similarity index 100% rename from senbei-pe/src/engine/exe/mod.rs rename to senbei-engine/src/windows/exe/mod.rs diff --git a/senbei-pe/src/engine/exe/pipeline.rs b/senbei-engine/src/windows/exe/pipeline.rs similarity index 99% rename from senbei-pe/src/engine/exe/pipeline.rs rename to senbei-engine/src/windows/exe/pipeline.rs index a7ec023..0a99717 100644 --- a/senbei-pe/src/engine/exe/pipeline.rs +++ b/senbei-engine/src/windows/exe/pipeline.rs @@ -407,7 +407,7 @@ impl<'a> Unpacker<'a> { // non-critical for false-positive rejection. if v8 < info6 { let delta = info6.wrapping_sub(v8); - if delta <= 0x1000 && delta.is_multiple_of(0x200) { + if delta <= 0x1000 && delta % 0x200 == 0 { anchor = Some(probe); break; } diff --git a/senbei-pe/src/engine/exe/pipeline/pe32.rs b/senbei-engine/src/windows/exe/pipeline/pe32.rs similarity index 100% rename from senbei-pe/src/engine/exe/pipeline/pe32.rs rename to senbei-engine/src/windows/exe/pipeline/pe32.rs diff --git a/senbei-pe/src/engine/integrity.rs b/senbei-engine/src/windows/integrity.rs similarity index 100% rename from senbei-pe/src/engine/integrity.rs rename to senbei-engine/src/windows/integrity.rs diff --git a/senbei-pe/src/engine/layout.rs b/senbei-engine/src/windows/layout.rs similarity index 100% rename from senbei-pe/src/engine/layout.rs rename to senbei-engine/src/windows/layout.rs diff --git a/senbei-pe/src/engine/layout/dd8.rs b/senbei-engine/src/windows/layout/dd8.rs similarity index 100% rename from senbei-pe/src/engine/layout/dd8.rs rename to senbei-engine/src/windows/layout/dd8.rs diff --git a/senbei-pe/src/engine/layout/discovery.rs b/senbei-engine/src/windows/layout/discovery.rs similarity index 100% rename from senbei-pe/src/engine/layout/discovery.rs rename to senbei-engine/src/windows/layout/discovery.rs diff --git a/senbei-pe/src/engine/layout/image.rs b/senbei-engine/src/windows/layout/image.rs similarity index 100% rename from senbei-pe/src/engine/layout/image.rs rename to senbei-engine/src/windows/layout/image.rs diff --git a/senbei-pe/src/engine/mod.rs b/senbei-engine/src/windows/mod.rs similarity index 98% rename from senbei-pe/src/engine/mod.rs rename to senbei-engine/src/windows/mod.rs index 3eb4bcb..84fd291 100644 --- a/senbei-pe/src/engine/mod.rs +++ b/senbei-engine/src/windows/mod.rs @@ -350,8 +350,8 @@ mod tests { }; assert_eq!(message, "test panic"); assert!( - file.ends_with("senbei-pe/src/engine/mod.rs") - || file.ends_with("senbei-pe\\src\\engine\\mod.rs") + file.ends_with("senbei-engine/src/windows/mod.rs") + || file.ends_with("senbei-engine\\src\\windows\\mod.rs") ); assert!(line > 0); assert!(column > 0); @@ -382,8 +382,8 @@ mod tests { }; assert_eq!(message, "worker panic"); assert!( - file.ends_with("senbei-pe/src/engine/mod.rs") - || file.ends_with("senbei-pe\\src\\engine\\mod.rs") + file.ends_with("senbei-engine/src/windows/mod.rs") + || file.ends_with("senbei-engine\\src\\windows\\mod.rs") ); assert!(line > 0); assert!(column > 0); diff --git a/senbei-pe/src/engine/parallel.rs b/senbei-engine/src/windows/parallel.rs similarity index 100% rename from senbei-pe/src/engine/parallel.rs rename to senbei-engine/src/windows/parallel.rs diff --git a/senbei-io/Cargo.toml b/senbei-io/Cargo.toml index d092b0b..b4a85d4 100644 --- a/senbei-io/Cargo.toml +++ b/senbei-io/Cargo.toml @@ -10,11 +10,8 @@ anyhow.workspace = true flate2.workspace = true indicatif.workspace = true owo-colors.workspace = true -senbei-android-elf.workspace = true -senbei-android-engine.workspace = true -senbei-android-metadata.workspace = true +senbei-engine.workspace = true senbei-metadata.workspace = true -senbei-pe.workspace = true sha2.workspace = true tempfile.workspace = true walkdir.workspace = true diff --git a/senbei-io/src/android.rs b/senbei-io/src/android/mod.rs similarity index 91% rename from senbei-io/src/android.rs rename to senbei-io/src/android/mod.rs index d23c1d0..38d677e 100644 --- a/senbei-io/src/android.rs +++ b/senbei-io/src/android/mod.rs @@ -5,24 +5,24 @@ //! The protection scheme hollows out an ELF64/AArch64 shared object and moves //! the original bytes into an encrypted payload appended as a `SHT_LOUSER` //! section; restoration extracts the stage-2 module set -//! ([`senbei_android_engine`]) and rebuilds the static image -//! ([`senbei_android_elf`]). Some il2cpp builds additionally embed their +//! ([`senbei_engine::android`]) and rebuilds the static image +//! ([`senbei_engine::android`]). Some il2cpp builds additionally embed their //! metadata blob — XOR-wrapped, with no standalone `global-metadata.dat` in //! the assets — inside the library's data section; after a successful restore //! the blob is located by content and unwrapped -//! ([`senbei_android_metadata::extract_embedded_metadata`]). +//! ([`senbei_metadata::android::extract_embedded_metadata`]). //! //! All functions in this module are native filesystem orchestration; the web //! app (wasm) never touches them. use std::collections::HashSet; -use std::io::Read; +use std::io::{BufWriter, Write}; use std::path::{Path, PathBuf}; use anyhow::{Context, Result, bail}; use flate2::read::DeflateDecoder; -use senbei_android_elf::{RestoreOptions, restore_libil2cpp}; -use senbei_android_engine::{ExtractOptions, extract_stage2, is_protected_libil2cpp}; +use senbei_engine::android::{ExtractOptions, extract_stage2, is_protected_libil2cpp}; +use senbei_engine::android::{RestoreOptions, restore_libil2cpp}; use sha2::{Digest, Sha256}; use zip::ZipArchive; @@ -97,7 +97,7 @@ pub fn restore_so_file(input: &Path, dest: &Path, verbose: bool) -> Result String { /// canonical form. Both paths are no-ops (`remapped == 0`) on an /// already-clean blob. pub fn restore_metadata_bytes(data: &[u8]) -> anyhow::Result<(Vec, senbei_metadata::Report)> { - if let Ok(discovery) = senbei_android_metadata::discover_method_token_seeds(data) - && discovery.version == 31 - && discovery.images.iter().any(|image| !image.clean) + if let Ok(discovery) = senbei_metadata::android::discover_method_token_seeds(data) + && matches!(discovery.version, 31 | 39) { let mut seeds = discovery.seed_candidates.clone(); if seeds.is_empty() { - seeds.push(senbei_android_metadata::DEFAULT_METHOD_TOKEN_SEED); + seeds.push(senbei_metadata::android::DEFAULT_METHOD_TOKEN_SEED); } // Trial-and-validate: a wrong seed fails the restore's full-coverage // RID check, so ambiguous candidates cost one extra pass each and a // build with an unseeded permutation falls through to the structural // remap rather than producing a silently wrong file. for seed in seeds { - if let Ok((out, report)) = senbei_android_metadata::restore_method_tokens(data, seed) { + if let Ok((out, report)) = senbei_metadata::android::restore_method_tokens(data, seed) { return Ok(( out, senbei_metadata::Report { @@ -234,7 +233,9 @@ pub fn restore_package( nested.push((index, name)); } } else { - direct.push((index, name)); + if crate::scan::is_android_entry_name(&name) { + direct.push((index, name)); + } } } drop(archive); @@ -262,7 +263,9 @@ pub fn restore_package( let Some(entry_name) = entry_name else { bail!("unsafe entry path in `{}`", nested_label.display()); }; - entries.push((nested_index, entry_name)); + if crate::scan::is_android_entry_name(&entry_name) { + entries.push((nested_index, entry_name)); + } } } drop(nested_archive); @@ -324,6 +327,7 @@ fn restore_package_entry( } if is_so { + drop(data); return Ok(match restore_so_file(&entry_path, dest, verbose) { Ok(embedded) => { let mut outcomes = vec![outcome(EntryKind::So, EntryStatus::Restored)]; @@ -407,27 +411,23 @@ fn extract_entry( // `:` appears in `package::entry` labels and is invalid in Windows file // names; sanitize every path-ish separator. let destination = temporary.path().join(key.replace(['\\', '/', ':'], "_")); - let compressed_size = usize::try_from(entry.compressed_size()) - .map_err(|_| anyhow::anyhow!("entry compressed size exceeds usize"))?; - let output_size = - usize::try_from(entry.size()).map_err(|_| anyhow::anyhow!("entry size exceeds usize"))?; - let mut compressed = vec![0_u8; compressed_size]; - entry.read_exact(&mut compressed)?; - let mut output = Vec::with_capacity(output_size); - match entry.compression() { - zip::CompressionMethod::Stored => output.extend_from_slice(&compressed), + let output_size = entry.size(); + let mut output = BufWriter::new(std::fs::File::create(&destination)?); + let written = match entry.compression() { + zip::CompressionMethod::Stored => std::io::copy(&mut entry, &mut output)?, zip::CompressionMethod::Deflated => { - DeflateDecoder::new(compressed.as_slice()).read_to_end(&mut output)?; + let mut decoder = DeflateDecoder::new(&mut entry); + std::io::copy(&mut decoder, &mut output)? } method => bail!("unsupported compression method {method:?} in entry `{key}`"), - } - if output.len() != output_size { + }; + output.flush()?; + if written != output_size { bail!( "entry `{key}` decompressed to 0x{:x}, expected 0x{output_size:x}", - output.len() + written ); } - std::fs::write(&destination, &output)?; Ok(destination) } /// Lowercase hex of a digest output (sha2 0.11's `Array` no longer formats as diff --git a/senbei-io/src/job.rs b/senbei-io/src/job.rs index 7405e94..be2eb92 100644 --- a/senbei-io/src/job.rs +++ b/senbei-io/src/job.rs @@ -1,4 +1,4 @@ -use senbei_pe as unpacker; +use senbei_engine as unpacker; use std::path::{Path, PathBuf}; /// Crackproof header key table lives at this fixed file offset. For the @@ -818,7 +818,7 @@ pub fn run_file_v( // handled entry-by-entry. Anything else falls through to the PE pipeline. let is_android_so = crate::android::is_elf64_aarch64(&prefix) && std::fs::read(input) - .map(|bytes| senbei_android_engine::is_protected_libil2cpp(&bytes)) + .map(|bytes| senbei_engine::android::is_protected_libil2cpp(&bytes)) .unwrap_or(false); let is_android_package = !is_android_so && crate::android::is_app_package(input, &prefix); diff --git a/senbei-io/src/scan.rs b/senbei-io/src/scan.rs index ebf06f8..a58c607 100644 --- a/senbei-io/src/scan.rs +++ b/senbei-io/src/scan.rs @@ -1,4 +1,4 @@ -use senbei_pe::detect; +use senbei_engine::detect; use std::io::Read; use std::path::{Path, PathBuf}; use walkdir::WalkDir; @@ -16,7 +16,7 @@ const DETECT_PREFIX: u64 = 8 * 1024; /// Smallest file that can possibly be a target, so anything shorter is skipped /// without ever being opened. /// -/// A Crackproof module needs ≥ 4128 bytes for [`senbei_pe::detect`]'s key +/// A Crackproof module needs ≥ 4128 bytes for [`senbei_engine::detect`]'s key /// table (it reads the dword at 4124), so the bound is exact for the unpack /// path. An il2cpp `global-metadata.dat` only needs 4 bytes to match its magic, /// but its header alone runs to offset 0xB0 and the images/types/methods tables @@ -25,14 +25,59 @@ const DETECT_PREFIX: u64 = 8 * 1024; /// processable is lost. const MIN_SIZE: u64 = 4128; -/// File extensions that are bulk data by construction and can never be a PE -/// image or an il2cpp metadata blob. -/// -/// This is deliberately a **deny**-list, not an executable allow-list: unknown -/// extensions are still probed. Extensionless files are handled separately by -/// [`denied_name`] because asset stores commonly contain tens of thousands of -/// extensionless chunks; exhaustive probing remains available through -/// `--scan-all`. +const METADATA_FILE_NAME: &str = "global-metadata.dat"; + +fn is_metadata_name(path: &Path) -> bool { + path.file_name() + .and_then(|name| name.to_str()) + .is_some_and(|name| name.eq_ignore_ascii_case(METADATA_FILE_NAME)) +} + +fn is_target_extension(path: &Path) -> bool { + path.extension() + .and_then(|ext| ext.to_str()) + .is_some_and(|ext| { + ext.eq_ignore_ascii_case("exe") + || ext.eq_ignore_ascii_case("dll") + || ext.eq_ignore_ascii_case("so") + }) +} + +fn is_android_package_name(path: &Path) -> bool { + path.extension() + .and_then(|ext| ext.to_str()) + .is_some_and(|ext| { + ext.eq_ignore_ascii_case("apk") + || ext.eq_ignore_ascii_case("apks") + || ext.eq_ignore_ascii_case("xapk") + }) +} + +/// External Windows payloads are consumed through their sibling `.exe`/`.dll` +/// stub. They are valid input bytes, but are not independent unpack targets. +pub(crate) fn is_windows_companion(path: &Path) -> bool { + let Some(name) = path.file_name().and_then(|name| name.to_str()) else { + return false; + }; + let Some(stub_name) = name.strip_suffix("._") else { + return false; + }; + let stub_path = Path::new(stub_name); + stub_path + .extension() + .and_then(|ext| ext.to_str()) + .is_some_and(|ext| ext.eq_ignore_ascii_case("exe") || ext.eq_ignore_ascii_case("dll")) +} + +pub(crate) fn is_android_entry_name(path: &Path) -> bool { + is_metadata_name(path) + || path + .extension() + .and_then(|ext| ext.to_str()) + .is_some_and(|ext| ext.eq_ignore_ascii_case("so")) +} + +/// File extensions that are bulk data by construction and can never be a target. /// /// Set `SENBEI_SCAN_ALL=1` (or pass `--scan-all`) to probe every file regardless. const DENY_EXT: &[&str] = &[ @@ -89,9 +134,7 @@ const DENY_EXT: &[&str] = &[ "sr", ]; -/// Whether `path` can be skipped from its name alone. Extensionless files and -/// files whose extension is on [`DENY_EXT`] are not opened during a default -/// scan. `--scan-all` remains available when exhaustive probing is required. +/// Whether `path` can be skipped from its name alone. fn denied_name(path: &Path) -> bool { let Some(ext) = path.extension() else { return true; @@ -151,16 +194,14 @@ pub struct ScanResult { /// per-file I/O latency, not bandwidth (that tree lives on a user-mode virtual /// disk that tops out near 1,300 IOPS). Thread count barely moves it either. /// -/// So the only lever is **probing fewer files**, which is what [`MIN_SIZE`] and -/// [`DENY_EXT`] do — both decided from the free directory metadata, before any -/// file is opened. On that tree they cut 46,446 probes to 1,814 and the scan -/// from ~40 s to ~2 s while still finding every target. +/// So the only lever is **probing fewer files**, which is what the target-name +/// filter and [`MIN_SIZE`] do — both decided before any file is opened. /// /// The surviving probes (open + short read + magic test) are fanned out across /// worker threads. Directory traversal itself stays serial (one cheap `readdir` /// pass, no file opens) because it feeds the parallel probe. /// -/// Thread count follows [`crate::unpacker::parallel::thread_cap`] (honoring +/// Thread count follows [`senbei_engine::thread_cap`] (honoring /// `SENBEI_THREADS`, `1` = fully sequential). Output order is independent of /// thread count: each worker owns a disjoint contiguous slice of the path list /// and writes the matching disjoint slice of the class list, so results are @@ -224,6 +265,15 @@ pub fn find_targets_opts(root: &Path, scan_all: bool) -> ScanResult { if !entry.file_type().is_file() { continue; } + if is_windows_companion(entry.path()) { + continue; + } + if !is_metadata_name(entry.path()) + && !is_target_extension(entry.path()) + && !is_android_package_name(entry.path()) + { + continue; + } if !scan_all { // Name checks come first so extensionless asset chunks never // trigger even an explicit metadata query. @@ -247,7 +297,7 @@ pub fn find_targets_opts(root: &Path, scan_all: bool) -> ScanResult { // `Some(Class::None)` means "probed, matched neither detector". let n = paths.len(); let mut class: Vec> = vec![Some(Class::None); n]; - let workers = senbei_pe::thread_cap().clamp(1, n.max(1)); + let workers = senbei_engine::thread_cap().clamp(1, n.max(1)); if workers <= 1 { for (p, c) in paths.iter().zip(class.iter_mut()) { *c = classify(p); @@ -314,9 +364,8 @@ pub fn scan_all_env() -> bool { } } -/// Classify one file by content. Reads a short prefix once and tests the -/// Crackproof detector first, then the il2cpp metadata magic, then the -/// Android probes. Returns `None` when the file could not be classified at +/// Classify one named candidate by content. Reads a short prefix once and tests +/// the detector for that platform. Returns `None` when the file could not be classified at /// all — an I/O error opening it (locked, permissions) or a panic inside a /// detector — so the caller counts it as a probe error rather than a clean /// "not a target" skip. @@ -337,22 +386,31 @@ pub fn scan_all_env() -> bool { fn classify(path: &Path) -> Option { let head = read_prefix(path, DETECT_PREFIX)?; let r = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| { - if detect(&head).is_some() { - return Class::Crackproof; + if is_android_package_name(path) && crate::android::is_app_package(path, &head) { + return Class::AndroidPackage; } - if senbei_metadata::is_metadata(&head) { + if is_metadata_name(path) && senbei_metadata::is_metadata(&head) { return Class::Metadata; } - if crate::android::is_elf64_aarch64(&head) + if path + .extension() + .and_then(|ext| ext.to_str()) + .is_some_and(|ext| ext.eq_ignore_ascii_case("exe") || ext.eq_ignore_ascii_case("dll")) + && detect(&head).is_some() + { + return Class::Crackproof; + } + if path + .extension() + .and_then(|ext| ext.to_str()) + .is_some_and(|ext| ext.eq_ignore_ascii_case("so")) + && crate::android::is_elf64_aarch64(&head) && std::fs::read(path) - .map(|bytes| senbei_android_engine::is_protected_libil2cpp(&bytes)) + .map(|bytes| senbei_engine::android::is_protected_libil2cpp(&bytes)) .unwrap_or(false) { return Class::AndroidSo; } - if crate::android::is_app_package(path, &head) { - return Class::AndroidPackage; - } Class::None })); r.ok() @@ -403,7 +461,7 @@ mod tests { } #[test] - fn extensionless_targets_require_exhaustive_scan() { + fn extensionless_targets_are_not_candidates() { let td = tempfile::tempdir().unwrap(); let root = td.path(); let mut blob = vec![0u8; MIN_SIZE as usize + 1]; @@ -414,7 +472,7 @@ mod tests { assert!(filtered.metadata.is_empty()); let exhaustive = find_targets_opts(root, true); - assert_eq!(exhaustive.metadata.len(), 1); + assert!(exhaustive.metadata.is_empty()); } /// A file below the Crackproof key-table bound is skipped without being @@ -479,4 +537,16 @@ mod tests { ); assert_eq!(scan.stats.walk_errors, 0); } + + #[test] + fn companion_payload_is_not_counted_as_skipped() { + let td = tempfile::tempdir().unwrap(); + let root = td.path(); + std::fs::write(root.join("app.exe"), vec![0u8; MIN_SIZE as usize]).unwrap(); + std::fs::write(root.join("app.exe._"), vec![0u8; MIN_SIZE as usize]).unwrap(); + + let scan = find_targets_opts(root, false); + assert_eq!(scan.stats.skipped, 1, "only the stub was probed"); + assert!(is_windows_companion(&root.join("app.exe._"))); + } } diff --git a/senbei-io/src/ui.rs b/senbei-io/src/ui.rs index 2d04e7b..87f8f19 100644 --- a/senbei-io/src/ui.rs +++ b/senbei-io/src/ui.rs @@ -1,6 +1,6 @@ use indicatif::{ProgressBar, ProgressStyle}; use owo_colors::OwoColorize; -use senbei_pe::{IntegrityReport, Kind}; +use senbei_engine::{IntegrityReport, Kind}; use std::path::Path; /// Create a progress bar for `n` items. Hidden when `quiet` is true. diff --git a/senbei-metadata/Cargo.toml b/senbei-metadata/Cargo.toml index c9d49e2..4e19e5b 100644 --- a/senbei-metadata/Cargo.toml +++ b/senbei-metadata/Cargo.toml @@ -4,3 +4,7 @@ version.workspace = true edition.workspace = true license.workspace = true description = "Unity il2cpp metadata de-obfuscation for Senbei" + +[dependencies] +serde.workspace = true +thiserror.workspace = true diff --git a/senbei-android-metadata/src/embedded.rs b/senbei-metadata/src/android/embedded.rs similarity index 99% rename from senbei-android-metadata/src/embedded.rs rename to senbei-metadata/src/android/embedded.rs index 78138be..8ecdb5e 100644 --- a/senbei-android-metadata/src/embedded.rs +++ b/senbei-metadata/src/android/embedded.rs @@ -20,7 +20,7 @@ //! they are rewritten to the standard il2cpp metadata magic and version so the //! output is a well-formed `global-metadata.dat`. -use crate::keystream::{HEADER_KEYS, SEGMENTS}; +use super::keystream::{HEADER_KEYS, SEGMENTS}; /// Standard il2cpp metadata sanity magic written over the patched header. const STANDARD_MAGIC: u32 = 0xfab1_1baf; diff --git a/senbei-android-metadata/src/keystream.rs b/senbei-metadata/src/android/keystream.rs similarity index 100% rename from senbei-android-metadata/src/keystream.rs rename to senbei-metadata/src/android/keystream.rs diff --git a/senbei-android-metadata/src/method_tokens.rs b/senbei-metadata/src/android/method_tokens.rs similarity index 55% rename from senbei-android-metadata/src/method_tokens.rs rename to senbei-metadata/src/android/method_tokens.rs index 1a13817..0c1ce72 100644 --- a/senbei-android-metadata/src/method_tokens.rs +++ b/senbei-metadata/src/android/method_tokens.rs @@ -171,6 +171,9 @@ pub fn restore_method_tokens(data: &[u8], seed: u32) -> Result<(Vec, Report) return Err(Error::NotMetadata); } let version = read_u32(data, 4)?; + if version == 39 { + return restore_v39(data, seed); + } if version != SUPPORTED_VERSION { return Err(Error::UnsupportedVersion(version)); } @@ -368,6 +371,9 @@ pub fn discover_method_token_seeds(data: &[u8]) -> Result { return Err(Error::NotMetadata); } let version = read_u32(data, 4)?; + if version == 39 { + return discover_v39(data); + } if version != SUPPORTED_VERSION { return Ok(SeedDiscoveryReport { version, @@ -550,6 +556,458 @@ fn decrypt_rid_with_key(rid: u32, low: u32, high: u32, key: u32) -> u32 { value + low } +const V39_METHODS: usize = 5; +const V39_PARAMETERS: usize = 10; +const V39_GENERIC_CONTAINERS: usize = 14; +const V39_INTERFACE_OFFSETS: usize = 18; +const V39_TYPES: usize = 19; +const V39_IMAGES: usize = 20; + +#[derive(Debug, Clone, Copy)] +struct V39Layout { + method_offset: usize, + method_count: usize, + method_stride: usize, + method_token_offset: usize, + type_definition_index_width: usize, + type_offset: usize, + type_count: usize, + type_stride: usize, + type_method_start_offset: usize, + type_method_count_offset: usize, + image_offset: usize, + image_count: usize, + image_stride: usize, +} + +fn v39_section(data: &[u8], index: usize) -> Result<(usize, usize, usize)> { + let header = 8_usize + .checked_add( + index + .checked_mul(12) + .ok_or_else(|| Error::Malformed("v39 section header offset overflow".to_owned()))?, + ) + .ok_or_else(|| Error::Malformed("v39 section header offset overflow".to_owned()))?; + let offset = read_u32(data, header)? as usize; + let size = read_u32(data, header + 4)? as usize; + let count = read_u32(data, header + 8)? as usize; + bytes(data, offset, size)?; + Ok((offset, size, count)) +} + +fn v39_index_width(count: usize) -> usize { + if count <= u8::MAX as usize { + 1 + } else if count <= u16::MAX as usize { + 2 + } else { + 4 + } +} + +fn read_v39_index(data: &[u8], offset: usize, width: usize) -> Result { + match width { + 1 => Ok(bytes(data, offset, 1)?[0] as usize), + 2 => Ok(read_u16(data, offset)? as usize), + 4 => Ok(read_u32(data, offset)? as usize), + _ => malformed("v39 index has an unsupported width"), + } +} + +fn parse_v39(data: &[u8]) -> Result { + let (method_offset, method_size, method_count) = v39_section(data, V39_METHODS)?; + let (_, parameter_size, parameter_count) = v39_section(data, V39_PARAMETERS)?; + let (_, _, generic_count) = v39_section(data, V39_GENERIC_CONTAINERS)?; + let (_interface_offset, interface_size, interface_count) = + v39_section(data, V39_INTERFACE_OFFSETS)?; + let (type_offset, type_size, type_count) = v39_section(data, V39_TYPES)?; + let (image_offset, image_size, image_count) = v39_section(data, V39_IMAGES)?; + let parameter_index_width = v39_index_width(parameter_count); + let generic_container_index_width = v39_index_width(generic_count); + let type_definition_index_width = v39_index_width(type_count); + let type_index_width = if interface_count == 0 { + 4 + } else { + let element_size = interface_size + .checked_div(interface_count) + .ok_or_else(|| Error::Malformed("v39 interface-offset size is invalid".to_owned()))?; + element_size + .checked_sub(4) + .filter(|width| matches!(width, 1 | 2 | 4)) + .ok_or_else(|| Error::Malformed("v39 type-index width is invalid".to_owned()))? + }; + let method_stride = 20_usize + .checked_add(type_definition_index_width) + .and_then(|size| size.checked_add(type_index_width)) + .and_then(|size| size.checked_add(parameter_index_width)) + .and_then(|size| size.checked_add(generic_container_index_width)) + .ok_or_else(|| Error::Malformed("v39 method stride overflow".to_owned()))?; + let type_stride = 68_usize + .checked_add( + type_index_width + .checked_mul(3) + .ok_or_else(|| Error::Malformed("v39 type stride overflow".to_owned()))?, + ) + .and_then(|size| size.checked_add(generic_container_index_width)) + .ok_or_else(|| Error::Malformed("v39 type stride overflow".to_owned()))?; + let image_stride = 32_usize + .checked_add( + type_definition_index_width + .checked_mul(2) + .ok_or_else(|| Error::Malformed("v39 image stride overflow".to_owned()))?, + ) + .ok_or_else(|| Error::Malformed("v39 image stride overflow".to_owned()))?; + if method_count.checked_mul(method_stride) != Some(method_size) + || type_count.checked_mul(type_stride) != Some(type_size) + || image_count.checked_mul(image_stride) != Some(image_size) + || parameter_count == 0 && parameter_size != 0 + { + return malformed("v39 table size does not match its compact entry layout"); + } + let type_method_start_offset = 16_usize + .checked_add( + type_index_width + .checked_mul(3) + .ok_or_else(|| Error::Malformed("v39 type method offset overflow".to_owned()))?, + ) + .and_then(|offset| offset.checked_add(generic_container_index_width)) + .ok_or_else(|| Error::Malformed("v39 type method offset overflow".to_owned()))?; + let type_method_count_offset = type_method_start_offset + .checked_add(7 * 4) + .ok_or_else(|| Error::Malformed("v39 type method count offset overflow".to_owned()))?; + let method_token_offset = 4_usize + .checked_add(type_definition_index_width) + .and_then(|offset| offset.checked_add(type_index_width)) + .and_then(|offset| offset.checked_add(4)) + .and_then(|offset| offset.checked_add(parameter_index_width)) + .and_then(|offset| offset.checked_add(generic_container_index_width)) + .ok_or_else(|| Error::Malformed("v39 method token offset overflow".to_owned()))?; + if method_token_offset + .checked_add(4) + .is_none_or(|end| end > method_stride) + || type_method_count_offset + .checked_add(2) + .is_none_or(|end| end > type_stride) + { + return malformed("v39 compact layout fields exceed their records"); + } + // Touch the section base so malformed headers fail before any output copy. + Ok(V39Layout { + method_offset, + method_count, + method_stride, + method_token_offset, + type_definition_index_width, + type_offset, + type_count, + type_stride, + type_method_start_offset, + type_method_count_offset, + image_offset, + image_count, + image_stride, + }) +} + +fn v39_image_methods(data: &[u8], layout: V39Layout, image: usize) -> Result> { + let image_base = layout + .image_offset + .checked_add( + image + .checked_mul(layout.image_stride) + .ok_or_else(|| Error::Malformed("v39 image offset overflow".to_owned()))?, + ) + .ok_or_else(|| Error::Malformed("v39 image offset overflow".to_owned()))?; + let type_start = read_v39_index(data, image_base + 8, layout.type_definition_index_width)?; + let type_count = read_u32(data, image_base + 8 + layout.type_definition_index_width)? as usize; + let type_end = type_start + .checked_add(type_count) + .ok_or_else(|| Error::Malformed("v39 image type range overflow".to_owned()))?; + if type_end > layout.type_count { + return malformed("v39 image type range exceeds the type table"); + } + let mut methods = Vec::new(); + for type_index in type_start..type_end { + let type_base = layout + .type_offset + .checked_add( + type_index + .checked_mul(layout.type_stride) + .ok_or_else(|| Error::Malformed("v39 type offset overflow".to_owned()))?, + ) + .ok_or_else(|| Error::Malformed("v39 type offset overflow".to_owned()))?; + let method_start = read_u32(data, type_base + layout.type_method_start_offset)?; + let method_count = read_u16(data, type_base + layout.type_method_count_offset)? as usize; + if method_start == u32::MAX || method_count == 0 { + continue; + } + let method_start = method_start as usize; + let method_end = method_start + .checked_add(method_count) + .ok_or_else(|| Error::Malformed("v39 type method range overflow".to_owned()))?; + if method_end > layout.method_count { + return malformed("v39 type method range exceeds the method table"); + } + methods.extend(method_start..method_end); + } + Ok(methods) +} + +#[allow(clippy::too_many_arguments)] +fn v39_report( + layout: V39Layout, + changed_tokens: usize, + images_with_methods: usize, + visited_methods: usize, + already_correct_before: usize, + correct_after: usize, + transformed_images: usize, + seed: u32, +) -> Report { + Report { + version: 39, + seed: format!("0x{seed:08X}"), + encryption_status: if changed_tokens == 0 { + "clean".to_owned() + } else { + "encrypted".to_owned() + }, + images: layout.image_count, + images_with_methods, + types: layout.type_count, + methods: layout.method_count, + visited_methods, + already_correct_before, + correct_after, + changed_tokens, + transformed_images, + } +} + +fn restore_v39(data: &[u8], seed: u32) -> Result<(Vec, Report)> { + let layout = parse_v39(data)?; + let mut owners = vec![u32::MAX; layout.method_count]; + let mut output = data.to_vec(); + let mut images_with_methods = 0; + let mut visited_methods = 0; + let mut already_correct_before = 0; + let mut correct_after = 0; + let mut changed_tokens = 0; + let mut transformed_images = 0; + for image in 0..layout.image_count { + let methods = v39_image_methods(data, layout, image)?; + if methods.is_empty() { + continue; + } + images_with_methods += 1; + visited_methods += methods.len(); + let method_base = *methods.iter().min().ok_or_else(|| { + Error::Malformed("v39 nonempty image lost its method minimum".to_owned()) + })?; + let method_last = *methods.iter().max().ok_or_else(|| { + Error::Malformed("v39 nonempty image lost its method maximum".to_owned()) + })?; + if method_last - method_base + 1 != methods.len() { + return validation(format!("v39 image {image} method block is not contiguous")); + } + for &method in &methods { + if owners[method] != u32::MAX { + return malformed(format!("v39 method {method} belongs to multiple images")); + } + owners[method] = image as u32; + } + let mut tokens = Vec::with_capacity(methods.len()); + let mut clean = true; + for method in methods { + let offset = + layout.method_offset + method * layout.method_stride + layout.method_token_offset; + let token = read_u32(data, offset)?; + if token & 0xff00_0000 != METHOD_TOKEN_TABLE { + return malformed(format!("v39 method {method} has a non-MethodDef token")); + } + let expected = (method - method_base + 1) as u32; + let rid = token & 0x00ff_ffff; + if rid == expected { + already_correct_before += 1; + } else { + clean = false; + } + tokens.push((offset, token, expected)); + } + if clean { + correct_after += tokens.len(); + continue; + } + transformed_images += 1; + let low = tokens + .iter() + .map(|(_, token, _)| token & 0x00ff_ffff) + .min() + .unwrap(); + let high = tokens + .iter() + .map(|(_, token, _)| token & 0x00ff_ffff) + .max() + .unwrap(); + if high - low + 1 != tokens.len() as u32 { + return validation(format!( + "v39 image {image} RID interval is not a permutation" + )); + } + for (offset, token, expected) in tokens { + let restored = decrypt_rid(token & 0x00ff_ffff, low, high, seed)?; + if restored != expected { + return validation(format!( + "v39 restored RID {restored} != expected {expected}" + )); + } + let restored_token = METHOD_TOKEN_TABLE | restored; + if restored_token != token { + output[offset..offset + 4].copy_from_slice(&restored_token.to_le_bytes()); + changed_tokens += 1; + } + correct_after += 1; + } + } + if owners.contains(&u32::MAX) { + return malformed("v39 method definitions are not all owned by an image"); + } + if visited_methods != layout.method_count || correct_after != layout.method_count { + return validation(format!( + "v39 method coverage mismatch: visited={visited_methods}, correct={correct_after}, total={}", + layout.method_count + )); + } + Ok(( + output, + v39_report( + layout, + changed_tokens, + images_with_methods, + visited_methods, + already_correct_before, + correct_after, + transformed_images, + seed, + ), + )) +} + +fn discover_v39(data: &[u8]) -> Result { + let layout = parse_v39(data)?; + let mut reports = Vec::with_capacity(layout.image_count); + for image in 0..layout.image_count { + let methods = v39_image_methods(data, layout, image)?; + if methods.is_empty() { + reports.push(ImageKeyDiscovery { + image, + method_count: 0, + modulus: 0, + clean: true, + seed_residues: Vec::new(), + }); + continue; + } + let base = *methods.iter().min().unwrap(); + let last = *methods.iter().max().unwrap(); + if last - base + 1 != methods.len() { + return validation(format!("v39 image {image} method block is not contiguous")); + } + let values = methods + .iter() + .map(|&method| { + let offset = layout.method_offset + + method * layout.method_stride + + layout.method_token_offset; + let token = read_u32(data, offset)?; + if token & 0xff00_0000 != METHOD_TOKEN_TABLE { + return malformed(format!("v39 method {method} has a non-MethodDef token")); + } + Ok((token & 0x00ff_ffff, (method - base + 1) as u32)) + }) + .collect::>>()?; + let count = u32::try_from(values.len()) + .map_err(|_| Error::Validation("v39 image method count exceeds u32".to_owned()))?; + let clean = values.iter().all(|(rid, expected)| rid == expected); + if clean { + reports.push(ImageKeyDiscovery { + image, + method_count: count, + modulus: count / 2, + clean: true, + seed_residues: Vec::new(), + }); + continue; + } + let low = values.iter().map(|(rid, _)| *rid).min().unwrap(); + let high = values.iter().map(|(rid, _)| *rid).max().unwrap(); + if high - low + 1 != count || count < 2 { + return validation(format!( + "v39 image {image} RID interval is not a permutation" + )); + } + let half = count / 2; + let quarter = count / 4; + let mut residues = Vec::new(); + for residue in 0..half { + let key = quarter + residue; + if values + .iter() + .all(|(rid, expected)| decrypt_rid_with_key(*rid, low, high, key) == *expected) + { + residues.push(residue); + } + } + reports.push(ImageKeyDiscovery { + image, + method_count: count, + modulus: half, + clean: false, + seed_residues: residues, + }); + } + let constraints = reports + .iter() + .filter(|image| !image.clean) + .collect::>(); + let mut candidates = Vec::new(); + if let Some(anchor) = constraints.iter().max_by_key(|image| image.modulus) { + // Returning every 32-bit seed is not representable for tiny synthetic + // images (for example, a two-entry image has billions of candidates). + // The caller still tries the known default seed and validates it fully. + if anchor.modulus < 1024 { + return Ok(SeedDiscoveryReport { + version: 39, + images: reports, + seed_candidates: candidates, + }); + } + for &residue in &anchor.seed_residues { + let modulus = u64::from(anchor.modulus); + let mut candidate = u64::from(residue); + while candidate <= u64::from(u32::MAX) { + if constraints.iter().all(|image| { + image.modulus != 0 + && image + .seed_residues + .iter() + .any(|value| candidate % u64::from(image.modulus) == u64::from(*value)) + }) { + candidates.push(candidate as u32); + } + candidate = candidate.saturating_add(modulus); + } + } + } + candidates.sort_unstable(); + candidates.dedup(); + Ok(SeedDiscoveryReport { + version: 39, + images: reports, + seed_candidates: candidates, + }) +} + #[cfg(test)] mod tests { use super::*; @@ -656,4 +1114,66 @@ mod tests { Err(Error::Validation(_)) )); } + + #[test] + fn restores_compact_v39_method_tokens_without_touching_other_fields() { + let method_stride = 25; + let type_stride = 75; + let image_stride = 34; + let method_offset = 0x400; + let type_offset = 0x600; + let image_offset = 0x700; + let method_count = 7_u32; + let mut data = vec![0_u8; image_offset + image_stride]; + put_u32(&mut data, 0, MAGIC); + put_u32(&mut data, 4, 39); + let section = |data: &mut [u8], index: usize, offset: usize, size: usize, count: usize| { + let header = 8 + index * 12; + put_u32(data, header, offset as u32); + put_u32(data, header + 4, size as u32); + put_u32(data, header + 8, count as u32); + }; + section( + &mut data, + V39_METHODS, + method_offset, + method_stride * method_count as usize, + method_count as usize, + ); + section(&mut data, V39_PARAMETERS, 0x300, 1, 1); + section(&mut data, V39_GENERIC_CONTAINERS, 0x320, 1, 1); + section(&mut data, V39_INTERFACE_OFFSETS, 0x340, 6, 1); + section(&mut data, V39_TYPES, type_offset, type_stride, 1); + section(&mut data, V39_IMAGES, image_offset, image_stride, 1); + data.resize(image_offset + image_stride, 0); + // Compact v39 type definition: firstMethod at offset 23, methodCount at 51. + put_u32(&mut data, type_offset + 23, 0); + put_u16(&mut data, type_offset + 51, method_count as u16); + // Compact v39 image definition: firstTypeIndex (one byte) and typeCount. + data[image_offset + 8] = 0; + put_u32(&mut data, image_offset + 9, 1); + for expected in 1..=method_count { + let token = METHOD_TOKEN_TABLE + | encrypted_rid(expected, method_count, DEFAULT_METHOD_TOKEN_SEED); + put_u32( + &mut data, + method_offset + (expected as usize - 1) * method_stride + 13, + token, + ); + } + let (out, report) = + restore_method_tokens(&data, DEFAULT_METHOD_TOKEN_SEED).expect("v39 restore"); + assert_eq!(report.version, 39); + assert_eq!(report.changed_tokens, method_count as usize); + for expected in 1..=method_count { + let offset = method_offset + (expected as usize - 1) * method_stride + 13; + assert_eq!( + read_u32(&out, offset).unwrap(), + METHOD_TOKEN_TABLE | expected + ); + } + let discovery = discover_method_token_seeds(&data).expect("v39 discovery"); + assert_eq!(discovery.version, 39); + assert!(discovery.seed_candidates.is_empty()); + } } diff --git a/senbei-android-metadata/src/lib.rs b/senbei-metadata/src/android/mod.rs similarity index 100% rename from senbei-android-metadata/src/lib.rs rename to senbei-metadata/src/android/mod.rs diff --git a/senbei-metadata/src/lib.rs b/senbei-metadata/src/lib.rs index 18e6fc7..a767c5d 100644 --- a/senbei-metadata/src/lib.rs +++ b/senbei-metadata/src/lib.rs @@ -1,5 +1,6 @@ -//! Unity il2cpp metadata de-obfuscation. +//! Unity il2cpp metadata restoration. -mod metadata; +pub mod android; +pub mod windows; -pub use metadata::*; +pub use windows::*; diff --git a/senbei-metadata/src/metadata.rs b/senbei-metadata/src/windows/metadata.rs similarity index 100% rename from senbei-metadata/src/metadata.rs rename to senbei-metadata/src/windows/metadata.rs diff --git a/senbei-metadata/src/windows/mod.rs b/senbei-metadata/src/windows/mod.rs new file mode 100644 index 0000000..88da0e8 --- /dev/null +++ b/senbei-metadata/src/windows/mod.rs @@ -0,0 +1,5 @@ +//! Windows metadata restoration. + +mod metadata; + +pub use metadata::*; diff --git a/senbei-pe/Cargo.toml b/senbei-pe/Cargo.toml index 8790c8b..01869b8 100644 --- a/senbei-pe/Cargo.toml +++ b/senbei-pe/Cargo.toml @@ -3,8 +3,7 @@ name = "senbei-pe" version.workspace = true edition.workspace = true license.workspace = true -description = "PE detection, unpacking, and validation for Senbei" +description = "PE format parsing and address mapping for Senbei" [dependencies] -senbei-crypto.workspace = true thiserror.workspace = true diff --git a/senbei-pe/src/lib.rs b/senbei-pe/src/lib.rs index 5db4e9a..783bb3c 100644 --- a/senbei-pe/src/lib.rs +++ b/senbei-pe/src/lib.rs @@ -1,5 +1,139 @@ -//! PE detection, unpacking, and structural validation. +//! Basic PE format parsing and address mapping. -mod engine; +use thiserror::Error; -pub use engine::*; +#[derive(Debug, Error, Clone, PartialEq, Eq)] +pub enum Error { + #[error("input is not a PE image")] + Invalid, + #[error("PE range is outside the input")] + OutOfBounds, +} + +pub type Result = std::result::Result; + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct Section { + pub virtual_address: u32, + pub virtual_size: u32, + pub raw_offset: u32, + pub raw_size: u32, + pub characteristics: u32, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct Headers { + pub pe_offset: usize, + pub is_pe32_plus: bool, + pub image_base: u64, + pub size_of_image: u32, + pub entry_rva: u32, + pub sections_offset: usize, + pub sections: u16, +} + +pub fn parse(data: &[u8]) -> Result { + if data.get(0..2) != Some(b"MZ") { + return Err(Error::Invalid); + } + let pe_offset = read_u32(data, 0x3c)? as usize; + if data.get(pe_offset..pe_offset + 4) != Some(b"PE\0\0") { + return Err(Error::Invalid); + } + let sections = read_u16(data, pe_offset + 6)?; + let optional_size = read_u16(data, pe_offset + 20)? as usize; + let optional = pe_offset.checked_add(24).ok_or(Error::OutOfBounds)?; + let magic = read_u16(data, optional)?; + let is_pe32_plus = magic == 0x20b; + if !is_pe32_plus && magic != 0x10b { + return Err(Error::Invalid); + } + let entry_rva = read_u32(data, optional + 16)?; + let image_base = if is_pe32_plus { + read_u64(data, optional + 24)? + } else { + read_u32(data, optional + 28)? as u64 + }; + let size_of_image = read_u32(data, optional + 56)?; + let sections_offset = optional + .checked_add(optional_size) + .ok_or(Error::OutOfBounds)?; + let table_size = usize::from(sections) + .checked_mul(40) + .ok_or(Error::OutOfBounds)?; + data.get(sections_offset..sections_offset + table_size) + .ok_or(Error::OutOfBounds)?; + Ok(Headers { + pe_offset, + is_pe32_plus, + image_base, + size_of_image, + entry_rva, + sections_offset, + sections, + }) +} + +pub fn sections(data: &[u8], headers: Headers) -> Result> { + (0..headers.sections) + .map(|index| { + let offset = headers + .sections_offset + .checked_add(usize::from(index) * 40) + .ok_or(Error::OutOfBounds)?; + Ok(Section { + virtual_size: read_u32(data, offset + 8)?, + virtual_address: read_u32(data, offset + 12)?, + raw_size: read_u32(data, offset + 16)?, + raw_offset: read_u32(data, offset + 20)?, + characteristics: read_u32(data, offset + 36)?, + }) + }) + .collect() +} + +pub fn rva_to_offset(data: &[u8], headers: Headers, rva: u32) -> Result { + if rva < headers.sections_offset as u32 { + return Ok(rva as usize); + } + for section in sections(data, headers)? { + let span = section.virtual_size.max(section.raw_size); + if rva >= section.virtual_address && rva < section.virtual_address.saturating_add(span) { + let offset = section + .raw_offset + .checked_add(rva - section.virtual_address) + .ok_or(Error::OutOfBounds)? as usize; + if offset < data.len() { + return Ok(offset); + } + } + } + Err(Error::OutOfBounds) +} + +fn read_u16(data: &[u8], offset: usize) -> Result { + let bytes: [u8; 2] = data + .get(offset..offset + 2) + .ok_or(Error::OutOfBounds)? + .try_into() + .map_err(|_| Error::OutOfBounds)?; + Ok(u16::from_le_bytes(bytes)) +} + +fn read_u32(data: &[u8], offset: usize) -> Result { + let bytes: [u8; 4] = data + .get(offset..offset + 4) + .ok_or(Error::OutOfBounds)? + .try_into() + .map_err(|_| Error::OutOfBounds)?; + Ok(u32::from_le_bytes(bytes)) +} + +fn read_u64(data: &[u8], offset: usize) -> Result { + let bytes: [u8; 8] = data + .get(offset..offset + 8) + .ok_or(Error::OutOfBounds)? + .try_into() + .map_err(|_| Error::OutOfBounds)?; + Ok(u64::from_le_bytes(bytes)) +} diff --git a/senbei-wasm/Cargo.lock b/senbei-wasm/Cargo.lock index a9c013f..8df0b4a 100644 --- a/senbei-wasm/Cargo.lock +++ b/senbei-wasm/Cargo.lock @@ -429,7 +429,7 @@ dependencies = [ ] [[package]] -name = "senbei-android-crypto" +name = "senbei-crypto" version = "1.2.0" dependencies = [ "aes", @@ -437,25 +437,12 @@ dependencies = [ ] [[package]] -name = "senbei-android-elf" -version = "1.2.0" -dependencies = [ - "memmap2", - "senbei-android-crypto", - "serde", - "serde_json", - "sha2", - "tempfile", - "thiserror", -] - -[[package]] -name = "senbei-android-engine" +name = "senbei-engine" version = "1.2.0" dependencies = [ "goblin", "memmap2", - "senbei-android-crypto", + "senbei-crypto", "serde", "serde_json", "sha2", @@ -463,21 +450,6 @@ dependencies = [ "thiserror", ] -[[package]] -name = "senbei-android-metadata" -version = "1.2.0" -dependencies = [ - "serde", - "thiserror", -] - -[[package]] -name = "senbei-crypto" -version = "1.2.0" -dependencies = [ - "thiserror", -] - [[package]] name = "senbei-io" version = "1.2.0" @@ -487,11 +459,8 @@ dependencies = [ "indicatif", "libc", "owo-colors", - "senbei-android-elf", - "senbei-android-engine", - "senbei-android-metadata", + "senbei-engine", "senbei-metadata", - "senbei-pe", "sha2", "tempfile", "walkdir", @@ -502,12 +471,8 @@ dependencies = [ [[package]] name = "senbei-metadata" version = "1.2.0" - -[[package]] -name = "senbei-pe" -version = "1.2.0" dependencies = [ - "senbei-crypto", + "serde", "thiserror", ] @@ -516,9 +481,9 @@ name = "senbei-wasm" version = "1.2.0" dependencies = [ "console_error_panic_hook", + "senbei-engine", "senbei-io", "senbei-metadata", - "senbei-pe", "wasm-bindgen", ] diff --git a/senbei-wasm/Cargo.toml b/senbei-wasm/Cargo.toml index 39a6a2c..d296dc9 100644 --- a/senbei-wasm/Cargo.toml +++ b/senbei-wasm/Cargo.toml @@ -11,7 +11,7 @@ crate-type = ["cdylib"] [dependencies] senbei-io = { path = "../senbei-io" } senbei-metadata = { path = "../senbei-metadata" } -senbei-pe = { path = "../senbei-pe" } +senbei-engine = { path = "../senbei-engine" } wasm-bindgen = "0.2" console_error_panic_hook = "0.1" diff --git a/senbei-wasm/src/lib.rs b/senbei-wasm/src/lib.rs index e135477..e4b5750 100644 --- a/senbei-wasm/src/lib.rs +++ b/senbei-wasm/src/lib.rs @@ -101,12 +101,12 @@ impl MetadataResult { } } -fn kind_str(kind: senbei_pe::Kind) -> &'static str { +fn kind_str(kind: senbei_engine::Kind) -> &'static str { match kind { - senbei_pe::Kind::NativeExe => "native-exe", - senbei_pe::Kind::ManagedExe => "managed-exe", - senbei_pe::Kind::NativeDll => "native-dll", - senbei_pe::Kind::ManagedDll => "managed-dll", + senbei_engine::Kind::NativeExe => "native-exe", + senbei_engine::Kind::ManagedExe => "managed-exe", + senbei_engine::Kind::NativeDll => "native-dll", + senbei_engine::Kind::ManagedDll => "managed-dll", } } @@ -120,7 +120,7 @@ pub fn detect(input: &[u8]) -> Option { if senbei_metadata::is_metadata(input) { return Some("metadata".to_string()); } - senbei_pe::detect(input).map(|d| kind_str(d.kind).to_string()) + senbei_engine::detect(input).map(|d| kind_str(d.kind).to_string()) } /// Unpack a protected module. diff --git a/web/README.md b/web/README.md index e8f8155..c952f19 100644 --- a/web/README.md +++ b/web/README.md @@ -1,76 +1,25 @@ -# Senbei web +# Senbei Web -Senbei running in the browser: the unpacker core compiled to WebAssembly, -wrapped in a small static page. Everything is client-side — files are read -into the page, unpacked locally, and offered back as downloads. Nothing is -uploaded; there is no server component. +Senbei runs in the browser through the `senbei-wasm` crate. Files are read locally, unpacked in a worker, and offered back as downloads; no server receives input bytes. ## Features -- A legal notice is shown as a blocking dialog on page open; the tool is - unusable until it is acknowledged. -- Dropped files land in a file list, not unpacked immediately: review the - batch, remove mistakes, then press **Unpack**. A module and its `._` - companion can be dropped in any order (or in separate drops) — companions - auto-pair by name (`Foo.dll._` → `Foo.dll`) and show as a badge on the - module's row; removing a module removes its companion too. -- Rows show state at a glance: black while staged, an animated blue bar - while unpacking, green on success (with a download button) and red on - failure. -- Drop one or more protected `.exe` / `.dll` modules → get `.unpack.*` - downloads. -- Drop an il2cpp `global-metadata.dat` → de-obfuscated - `global-metadata.unpack.dat` (only when tokens actually change). -- Each output passes the same static integrity check as the CLI; suspect - outputs are flagged with the specific defects found. +- Protected `.exe` and `.dll` files produce `.unpack.*` downloads. +- External `.exe._` and `.dll._` companions are paired by filename. +- `global-metadata.dat` produces `global-metadata.unpack.dat` when tokens change. +- Each output receives the same static integrity check as the CLI. -## Architecture notes +Every unpack uses a disposable Web Worker so a WebAssembly trap cannot freeze the page. A trapped DLL can be retried through the forced-EXE path, matching native routing. -- Every unpack runs in a **disposable Web Worker** (fresh wasm instance per - file): the UI stays responsive on 100 MB+ modules, and a wasm trap is - isolated to that worker. -- **Why workers matter for correctness:** the DLL-first routing probe relies - on `catch_unwind` to reject EXE-shell-layout DLLs, and panics cannot be - caught in WebAssembly — the probe traps the whole call. When a DLL unpack - traps, the app retries once in a new worker with the forced-EXE pipeline - (`unpack_file_force_exe`), reproducing the CLI's dll-first/exe-fallback - outcome. Spliced companion inputs skip the probe entirely (they are always - EXE-shell layout), exactly like the CLI. -- Rust panic messages are forwarded to the browser console - (`console_error_panic_hook`) — check devtools when reporting an issue. - -## Building - -Requires a Rust toolchain (`rust-toolchain.toml` in the repo root pins one, -including the `wasm32-unknown-unknown` target) and -[wasm-pack](https://rustwasm.github.io/wasm-pack/installer/). +## Build ```cmd cd senbei-wasm wasm-pack build --target web --release --out-dir ../web/pkg ``` -This produces `web/pkg/` (git-ignored). Then serve the `web/` directory with -any static file server and open `index.html`: - -```cmd -python -m http.server -d web 8000 -:: -> http://localhost:8000 -``` - -(Opening `index.html` via `file://` won't work — ES modules require HTTP.) +Serve `web/` with a static HTTP server, for example `python -m http.server -d web 8000`. Opening `index.html` with `file://` does not work because browser modules require HTTP. ## Layout -``` -senbei-wasm/ the senbei-wasm cdylib crate (own Cargo.lock, outside the - workspace; depends on the senbei-pe/-io/-metadata crates) -└── src/lib.rs #[wasm_bindgen] bindings: detect / unpack_file / - unpack_file_force_exe / deobfuscate_metadata -web/ -├── index.html the page -├── app.js dropzone, file list, worker orchestration, downloads -├── worker.js one-shot unpack worker (fresh wasm instance per file) -├── style.css -└── pkg/ wasm-pack output (git-ignored; build from senbei-wasm/) -``` +`senbei-wasm/src/lib.rs` contains the bindings. `web/app.js` manages the dropzone and downloads, `web/worker.js` runs one unpack job per worker, and `web/pkg/` contains ignored wasm-pack output.