Merge Android (AArch64) shared-library restoration, bump to 1.2.0

Adds the Android protection-scheme pipeline: hollowed ELF64/AArch64
libraries are restored statically (stage-1/stage-2 module extraction,
container decode, dynamic-linker table rebuild), with app-package
(.apk/.apks/.xapk) container handling, cross-source content dedup, and
il2cpp metadata support for the Android variants (seeded RID permutation;
embedded XOR-wrapped blob extraction).

The single senbei CLI now routes single .so files, packages, and folders
by content; outputs follow the existing .unpack-infix naming under
<root>/unpack or --out. PE behavior is unchanged (35/35 goldens).
This commit is contained in:
2026-09-02 03:09:21 +08:00
42 changed files with 7165 additions and 125 deletions
+83 -4
View File
@@ -18,10 +18,25 @@ Senbei is a Cargo workspace split into a pure core and thin shells around it:
primitives the core is built from. Same purity rules as `senbei-pe`.
- **`senbei-metadata/`** — il2cpp `global-metadata.dat` method-token
de-obfuscation (format version 31; other versions are left untouched).
- **`senbei-android-crypto/`** — container primitives of the Android
(AArch64) protection scheme: the word/record ciphers, the GF(2³²)
transform, the AES-augmented segment transform, and the Huffman/LZ decoder.
- **`senbei-android-engine/`** — stage-1/stage-2 extraction: finds the
appended payload section, decrypts the stage-1 header and stage-2 payload,
and walks the recursive record streams to decode every module. Native-only
(memory-maps the input, writes the module set to a workspace directory).
- **`senbei-android-elf/`** — the restore: replays the decoded target-image
and fixup containers onto a hollowed ELF and rebuilds the dynamic-linker
tables (hash tables, symbols, relocations) the protector stripped.
Native-only.
- **`senbei-android-metadata/`** — the Android metadata variants: the seeded
five-round MethodDef-RID permutation restore (v31), seed discovery, and the
embedded-metadata XOR unwrap (`keystream.rs`).
- **`senbei-io/`** — filesystem and orchestration: recursive folder scanning,
per-run log file, progress bar, Explorer-friendly exit pause, and the
per-run log file, progress bar, Explorer-friendly exit pause, the
single-file/folder orchestration in `job.rs` (incl. the wasm-safe in-memory
byte API used by the web frontend).
byte API used by the web frontend), and `android.rs` — the Android
single-library / folder / app-package orchestration.
- **`senbei-cli/`** — the `senbei` binary: argument parsing + dispatch. The
integration test suite (incl. the golden corpus test) lives in
`senbei-cli/tests/`.
@@ -33,7 +48,9 @@ senbei-io/src/
├── job.rs single-file + folder orchestration, out-naming,
│ companion splice, stub overlay/TLS restore,
│ pipeline routing (incl. the wasm-safe byte API)
├── scan.rs recursive Crackproof + metadata discovery
├── android.rs Android single-library / folder / package
│ orchestration, cross-source dedup
├── scan.rs recursive target discovery (PE + metadata + Android)
├── logfile.rs per-run timestamped log
├── ui.rs progress bar + status lines
└── pause.rs Explorer-friendly exit pause
@@ -58,6 +75,23 @@ senbei-pe/src/engine/ pure, panic-free, no-I/O core
│ └── pipeline/pe32.rs PE32-specific EXE restore
└── dll/
└── pipeline.rs native + managed DLL pipeline
senbei-android-crypto/src/
└── protector.rs container ciphers, GF(2^32), Huffman/LZ decoder
senbei-android-engine/src/
├── stage1.rs payload-section discovery + stage-1 header/payload
├── stream.rs record-stream parsing
├── extract.rs recursive module extraction (writes the workspace)
├── probe.rs protected-library content probe
└── report.rs machine-readable extraction report
senbei-android-elf/src/
├── restore.rs image restore + dynamic-table rebuild
├── layout.rs ELF layout parsing
├── artifact.rs module-workspace index loading
└── hash.rs SysV/GNU hash table rebuild
senbei-android-metadata/src/
├── method_tokens.rs seeded RID permutation restore + seed discovery
├── embedded.rs embedded-metadata blob locate + XOR unwrap
└── keystream.rs recovered keystream table (one observed build)
```
## Detection and routing
@@ -66,7 +100,11 @@ Detection is content-based (`unpacker::detect`), never extension-based: the
key table is derived from the file header and checked against the format
magic, then the PE characteristics classify the input as EXE or DLL and the
CLR data directory splits each into native vs managed (`NativeExe` /
`ManagedExe` / `NativeDll` / `ManagedDll`).
`ManagedExe` / `NativeDll` / `ManagedDll`). The folder scan additionally
classifies Android targets: an ELF64/AArch64 prefix promotes the file to a
full protection probe (`senbei_android_engine::is_protected_libil2cpp`), and a
package extension plus zip magic marks an app package for container
extraction.
`unpack_auto` then dispatches:
@@ -115,6 +153,47 @@ Several protected stages are themselves little bytecode programs. The core
includes a small VM (`bytecode.rs`) that generates and interprets those
programs rather than hardcoding each variant's constants.
## The Android pipeline
The Android scheme hollows an ELF64/AArch64 shared object: section bodies are
zeroed in the file and the original bytes move into an encrypted payload
appended as a `SHT_LOUSER` section (invisible to the dynamic loader). Restore
is two-phase:
1. **Extract** (`senbei-android-engine`): decrypt the stage-1 parameter block
and stage-2 payload from the payload section, then walk the recursive
record streams — each decoded module may interpret a further nested stream
— into a temporary module workspace with a JSON index.
2. **Restore** (`senbei-android-elf`): decode the target-image container onto
a copy of the hollowed file, apply the compact fixup database (the
relocations stripped from `.rela.dyn`), and rebuild the dynamic-linker
tables the loader needs (SysV/GNU hash, symbol and string tables,
`.rela.dyn`/`.rela.plt`). Validation is structural and total: mismatched
container sizes, descriptor bounds, or a rebuilt table overhanging its
section fail the restore rather than emit a broken image.
il2cpp metadata comes in three shapes, all routed through
`job::deobfuscate_metadata_to` / `android::restore_metadata_bytes`:
- **structural (Windows `-GMD`)**: sparse method tokens remapped to the
contiguous per-module range, keyless, idempotent (`senbei-metadata`).
- **seeded permutation (Android v31)**: MethodDef RIDs permuted by a keyed
five-round transform; the seed is recovered by intersecting per-image key
residues, and the restore validates every RID — a wrong seed errors and the
structural remap takes over (`senbei-android-metadata`).
- **embedded blob**: no metadata file in the app at all; a slim blob sits in
the library's data section under a per-word XOR layer. After a restore the
blob is located by content (two known plaintext header words against the
embedded keystream) and unwrapped to a standalone `global-metadata.dat`.
Key derivation is untraced — the shipped keystream covers the one observed
build, and other builds simply never match the probe.
Packages (`.apk`/`.apks`/`.xapk`) are containers, not targets: entries are
extracted to a temporary workspace and content-probed like loose files.
Cross-source duplicates (a library loose in the tree *and* inside its
package) are restored once, preferring the loose file, then the `.apk`, then
bundle splits.
## Integrity check
Every produced image passes through `integrity::check` — a static, execution-
+4
View File
@@ -57,6 +57,10 @@ since binaries are not committed).
- `SENBEI_THREADS` — cap the block-parallel fan-out (`1` forces the fully
sequential path).
- `SENBEI_SCAN_ALL` — same as `--scan-all` (probe every file in a folder).
- `SENBEI_ANDROID_SAMPLES` — override the Android corpus location (default
`samples/android/`; see `samples/README.md`). The Android corpus test pins
restored outputs with SHA-256 sidecar files next to each protected input
and documents known restore gaps with empty `<base>.restore-fails` markers.
## Conventions
+43 -6
View File
@@ -30,13 +30,44 @@ expects; the output is `global-metadata.unpack.dat`, written only when tokens
actually changed. Only metadata format version 31 is rewritten; other versions
are reported and left untouched.
## Android targets
Senbei also restores Android (AArch64) protected shared libraries and app
packages:
- **`.so`** — a protected library is hollowed out on disk: its original
sections live in an encrypted payload appended to the file, and senbei
rebuilds the static image from it. Output: `libil2cpp.unpack.so`.
- **`.apk`** — entries are extracted to a temporary workspace and
content-probed like loose files; protected libraries and metadata blobs
inside are restored to `<out>/<apk name>/<entry path>`.
- **`.apks` / `.xapk`** — split-package bundles; each nested `.apk` is opened
and searched the same way, under `<out>/<bundle name>/<split name>/...`.
When a restored il2cpp library carries its metadata embedded in its data
section (no standalone `global-metadata.dat` in the app at all), senbei
unwraps the blob and writes it next to the library as
`global-metadata.unpack.dat`. One observed packaging variant wraps the blob in
a per-word XOR layer whose keys are generated at runtime and stored nowhere;
senbei ships the keystream recovered from the one build known to use it and
content-probes for it — builds with a different keystream are silently
skipped (the library itself is still fully restored).
The same content may appear loose in a folder, in its `.apk`, and in a bundle
side by side: identical content is restored once, at the loose file's
destination. A restored library is validated structurally by the restore
itself (the rebuild refuses inconsistent layouts); a protected library that
fails validation counts as an error, not a suspect.
## Folder mode
Senbei walks the directory recursively, skips any subdirectory literally named
`unpack`, and unpacks every file it recognises as Crackproof-protected (by
content, not extension — renamed files and `.bak` backups are still found).
Results land under `<root>/unpack/` (or `--out DIR`), mirroring the input
tree's relative paths. The run log is written **in that same out directory**:
`unpack`, and unpacks every file it recognises as protected (by content, not
extension — renamed files and `.bak` backups are still found; packages are the
one exception, recognised by extension plus the zip magic because they are
containers). Results land under `<root>/unpack/` (or `--out DIR`), mirroring
the input tree's relative paths. The run log is written **in that same out
directory**:
```cmd
senbei "C:\Games\MyGame"
@@ -58,8 +89,14 @@ line, then duration:
done in 1234 ms
```
The `packages` count appears (as `· N packages`) only when Android app
packages were processed.
## Integrity check
(PE outputs only — Android restores carry their own structural validation; see
[Android targets](#android-targets).)
A successful unpack is not always a runnable one: a layout heuristic can pick
the wrong offset and leave the entry-point stub or import strings encrypted, so
the pipeline reports success but the OS loader faults at runtime (typically
@@ -98,7 +135,7 @@ line, adds a `SUSPECT` entry to the run log, and counts it in the summary's
| Flag | Behavior |
| --- | --- |
| `--out DIR` | Write outputs (and the log, unless `--no-log`) under `DIR`. |
| `-v`, `--verbose` | Print detailed `[N/9]` per-stage unpack progress (and the destination path) for each file. In folder mode this replaces the progress bar. |
| `-v`, `--verbose` | Print detailed per-stage progress (and the destination path) for each file`[N/9]` stages for PE targets, container/segment lines for Android libraries. In folder mode this replaces the progress bar. |
| `-q`, `--quiet` | Once: hide progress bar and per-file lines; keep banner, summary, and duration. Twice (`-q -q`): suppress all stdio (exit code only). |
| `--no-log` | Do not write `senbei-*.log`. Console output is unchanged by this flag alone. |
| `--scan-all` | Probe every file in a folder, including ones the scan pre-filter skips (under 4128 bytes, or a bulk-asset extension like `.ab`/`.xml`/`.acb`). Much slower on large game trees; finds the same targets in practice. |
@@ -115,7 +152,7 @@ process.
| Code | Meaning |
| --- | --- |
| `0` | Success (single file unpacked, or folder run with no errors). |
| `0` | Success (single file restored, or folder run with no errors). |
| `1` | At least one file failed, a scan probe was unreadable, or a single-file unpack errored. |
| `2` | Usage error: no path given, unknown option, missing `--out` value, or multiple input paths (help printed). |