mirror of
https://github.com/Momoko-Ayase/Senbei.git
synced 2026-09-19 03:57:59 -04:00
Merge Android (AArch64) shared-library restoration, bump to 1.2.0
Adds the Android protection-scheme pipeline: hollowed ELF64/AArch64 libraries are restored statically (stage-1/stage-2 module extraction, container decode, dynamic-linker table rebuild), with app-package (.apk/.apks/.xapk) container handling, cross-source content dedup, and il2cpp metadata support for the Android variants (seeded RID permutation; embedded XOR-wrapped blob extraction). The single senbei CLI now routes single .so files, packages, and folders by content; outputs follow the existing .unpack-infix naming under <root>/unpack or --out. PE behavior is unchanged (35/35 goldens).
This commit is contained in:
+83
-4
@@ -18,10 +18,25 @@ Senbei is a Cargo workspace split into a pure core and thin shells around it:
|
||||
primitives the core is built from. Same purity rules as `senbei-pe`.
|
||||
- **`senbei-metadata/`** — il2cpp `global-metadata.dat` method-token
|
||||
de-obfuscation (format version 31; other versions are left untouched).
|
||||
- **`senbei-android-crypto/`** — container primitives of the Android
|
||||
(AArch64) protection scheme: the word/record ciphers, the GF(2³²)
|
||||
transform, the AES-augmented segment transform, and the Huffman/LZ decoder.
|
||||
- **`senbei-android-engine/`** — stage-1/stage-2 extraction: finds the
|
||||
appended payload section, decrypts the stage-1 header and stage-2 payload,
|
||||
and walks the recursive record streams to decode every module. Native-only
|
||||
(memory-maps the input, writes the module set to a workspace directory).
|
||||
- **`senbei-android-elf/`** — the restore: replays the decoded target-image
|
||||
and fixup containers onto a hollowed ELF and rebuilds the dynamic-linker
|
||||
tables (hash tables, symbols, relocations) the protector stripped.
|
||||
Native-only.
|
||||
- **`senbei-android-metadata/`** — the Android metadata variants: the seeded
|
||||
five-round MethodDef-RID permutation restore (v31), seed discovery, and the
|
||||
embedded-metadata XOR unwrap (`keystream.rs`).
|
||||
- **`senbei-io/`** — filesystem and orchestration: recursive folder scanning,
|
||||
per-run log file, progress bar, Explorer-friendly exit pause, and the
|
||||
per-run log file, progress bar, Explorer-friendly exit pause, the
|
||||
single-file/folder orchestration in `job.rs` (incl. the wasm-safe in-memory
|
||||
byte API used by the web frontend).
|
||||
byte API used by the web frontend), and `android.rs` — the Android
|
||||
single-library / folder / app-package orchestration.
|
||||
- **`senbei-cli/`** — the `senbei` binary: argument parsing + dispatch. The
|
||||
integration test suite (incl. the golden corpus test) lives in
|
||||
`senbei-cli/tests/`.
|
||||
@@ -33,7 +48,9 @@ senbei-io/src/
|
||||
├── job.rs single-file + folder orchestration, out-naming,
|
||||
│ companion splice, stub overlay/TLS restore,
|
||||
│ pipeline routing (incl. the wasm-safe byte API)
|
||||
├── scan.rs recursive Crackproof + metadata discovery
|
||||
├── android.rs Android single-library / folder / package
|
||||
│ orchestration, cross-source dedup
|
||||
├── scan.rs recursive target discovery (PE + metadata + Android)
|
||||
├── logfile.rs per-run timestamped log
|
||||
├── ui.rs progress bar + status lines
|
||||
└── pause.rs Explorer-friendly exit pause
|
||||
@@ -58,6 +75,23 @@ senbei-pe/src/engine/ pure, panic-free, no-I/O core
|
||||
│ └── pipeline/pe32.rs PE32-specific EXE restore
|
||||
└── dll/
|
||||
└── pipeline.rs native + managed DLL pipeline
|
||||
senbei-android-crypto/src/
|
||||
└── protector.rs container ciphers, GF(2^32), Huffman/LZ decoder
|
||||
senbei-android-engine/src/
|
||||
├── stage1.rs payload-section discovery + stage-1 header/payload
|
||||
├── stream.rs record-stream parsing
|
||||
├── extract.rs recursive module extraction (writes the workspace)
|
||||
├── probe.rs protected-library content probe
|
||||
└── report.rs machine-readable extraction report
|
||||
senbei-android-elf/src/
|
||||
├── restore.rs image restore + dynamic-table rebuild
|
||||
├── layout.rs ELF layout parsing
|
||||
├── artifact.rs module-workspace index loading
|
||||
└── hash.rs SysV/GNU hash table rebuild
|
||||
senbei-android-metadata/src/
|
||||
├── method_tokens.rs seeded RID permutation restore + seed discovery
|
||||
├── embedded.rs embedded-metadata blob locate + XOR unwrap
|
||||
└── keystream.rs recovered keystream table (one observed build)
|
||||
```
|
||||
|
||||
## Detection and routing
|
||||
@@ -66,7 +100,11 @@ Detection is content-based (`unpacker::detect`), never extension-based: the
|
||||
key table is derived from the file header and checked against the format
|
||||
magic, then the PE characteristics classify the input as EXE or DLL and the
|
||||
CLR data directory splits each into native vs managed (`NativeExe` /
|
||||
`ManagedExe` / `NativeDll` / `ManagedDll`).
|
||||
`ManagedExe` / `NativeDll` / `ManagedDll`). The folder scan additionally
|
||||
classifies Android targets: an ELF64/AArch64 prefix promotes the file to a
|
||||
full protection probe (`senbei_android_engine::is_protected_libil2cpp`), and a
|
||||
package extension plus zip magic marks an app package for container
|
||||
extraction.
|
||||
|
||||
`unpack_auto` then dispatches:
|
||||
|
||||
@@ -115,6 +153,47 @@ Several protected stages are themselves little bytecode programs. The core
|
||||
includes a small VM (`bytecode.rs`) that generates and interprets those
|
||||
programs rather than hardcoding each variant's constants.
|
||||
|
||||
## The Android pipeline
|
||||
|
||||
The Android scheme hollows an ELF64/AArch64 shared object: section bodies are
|
||||
zeroed in the file and the original bytes move into an encrypted payload
|
||||
appended as a `SHT_LOUSER` section (invisible to the dynamic loader). Restore
|
||||
is two-phase:
|
||||
|
||||
1. **Extract** (`senbei-android-engine`): decrypt the stage-1 parameter block
|
||||
and stage-2 payload from the payload section, then walk the recursive
|
||||
record streams — each decoded module may interpret a further nested stream
|
||||
— into a temporary module workspace with a JSON index.
|
||||
2. **Restore** (`senbei-android-elf`): decode the target-image container onto
|
||||
a copy of the hollowed file, apply the compact fixup database (the
|
||||
relocations stripped from `.rela.dyn`), and rebuild the dynamic-linker
|
||||
tables the loader needs (SysV/GNU hash, symbol and string tables,
|
||||
`.rela.dyn`/`.rela.plt`). Validation is structural and total: mismatched
|
||||
container sizes, descriptor bounds, or a rebuilt table overhanging its
|
||||
section fail the restore rather than emit a broken image.
|
||||
|
||||
il2cpp metadata comes in three shapes, all routed through
|
||||
`job::deobfuscate_metadata_to` / `android::restore_metadata_bytes`:
|
||||
|
||||
- **structural (Windows `-GMD`)**: sparse method tokens remapped to the
|
||||
contiguous per-module range, keyless, idempotent (`senbei-metadata`).
|
||||
- **seeded permutation (Android v31)**: MethodDef RIDs permuted by a keyed
|
||||
five-round transform; the seed is recovered by intersecting per-image key
|
||||
residues, and the restore validates every RID — a wrong seed errors and the
|
||||
structural remap takes over (`senbei-android-metadata`).
|
||||
- **embedded blob**: no metadata file in the app at all; a slim blob sits in
|
||||
the library's data section under a per-word XOR layer. After a restore the
|
||||
blob is located by content (two known plaintext header words against the
|
||||
embedded keystream) and unwrapped to a standalone `global-metadata.dat`.
|
||||
Key derivation is untraced — the shipped keystream covers the one observed
|
||||
build, and other builds simply never match the probe.
|
||||
|
||||
Packages (`.apk`/`.apks`/`.xapk`) are containers, not targets: entries are
|
||||
extracted to a temporary workspace and content-probed like loose files.
|
||||
Cross-source duplicates (a library loose in the tree *and* inside its
|
||||
package) are restored once, preferring the loose file, then the `.apk`, then
|
||||
bundle splits.
|
||||
|
||||
## Integrity check
|
||||
|
||||
Every produced image passes through `integrity::check` — a static, execution-
|
||||
|
||||
@@ -57,6 +57,10 @@ since binaries are not committed).
|
||||
- `SENBEI_THREADS` — cap the block-parallel fan-out (`1` forces the fully
|
||||
sequential path).
|
||||
- `SENBEI_SCAN_ALL` — same as `--scan-all` (probe every file in a folder).
|
||||
- `SENBEI_ANDROID_SAMPLES` — override the Android corpus location (default
|
||||
`samples/android/`; see `samples/README.md`). The Android corpus test pins
|
||||
restored outputs with SHA-256 sidecar files next to each protected input
|
||||
and documents known restore gaps with empty `<base>.restore-fails` markers.
|
||||
|
||||
## Conventions
|
||||
|
||||
|
||||
+43
-6
@@ -30,13 +30,44 @@ expects; the output is `global-metadata.unpack.dat`, written only when tokens
|
||||
actually changed. Only metadata format version 31 is rewritten; other versions
|
||||
are reported and left untouched.
|
||||
|
||||
## Android targets
|
||||
|
||||
Senbei also restores Android (AArch64) protected shared libraries and app
|
||||
packages:
|
||||
|
||||
- **`.so`** — a protected library is hollowed out on disk: its original
|
||||
sections live in an encrypted payload appended to the file, and senbei
|
||||
rebuilds the static image from it. Output: `libil2cpp.unpack.so`.
|
||||
- **`.apk`** — entries are extracted to a temporary workspace and
|
||||
content-probed like loose files; protected libraries and metadata blobs
|
||||
inside are restored to `<out>/<apk name>/<entry path>`.
|
||||
- **`.apks` / `.xapk`** — split-package bundles; each nested `.apk` is opened
|
||||
and searched the same way, under `<out>/<bundle name>/<split name>/...`.
|
||||
|
||||
When a restored il2cpp library carries its metadata embedded in its data
|
||||
section (no standalone `global-metadata.dat` in the app at all), senbei
|
||||
unwraps the blob and writes it next to the library as
|
||||
`global-metadata.unpack.dat`. One observed packaging variant wraps the blob in
|
||||
a per-word XOR layer whose keys are generated at runtime and stored nowhere;
|
||||
senbei ships the keystream recovered from the one build known to use it and
|
||||
content-probes for it — builds with a different keystream are silently
|
||||
skipped (the library itself is still fully restored).
|
||||
|
||||
The same content may appear loose in a folder, in its `.apk`, and in a bundle
|
||||
side by side: identical content is restored once, at the loose file's
|
||||
destination. A restored library is validated structurally by the restore
|
||||
itself (the rebuild refuses inconsistent layouts); a protected library that
|
||||
fails validation counts as an error, not a suspect.
|
||||
|
||||
## Folder mode
|
||||
|
||||
Senbei walks the directory recursively, skips any subdirectory literally named
|
||||
`unpack`, and unpacks every file it recognises as Crackproof-protected (by
|
||||
content, not extension — renamed files and `.bak` backups are still found).
|
||||
Results land under `<root>/unpack/` (or `--out DIR`), mirroring the input
|
||||
tree's relative paths. The run log is written **in that same out directory**:
|
||||
`unpack`, and unpacks every file it recognises as protected (by content, not
|
||||
extension — renamed files and `.bak` backups are still found; packages are the
|
||||
one exception, recognised by extension plus the zip magic because they are
|
||||
containers). Results land under `<root>/unpack/` (or `--out DIR`), mirroring
|
||||
the input tree's relative paths. The run log is written **in that same out
|
||||
directory**:
|
||||
|
||||
```cmd
|
||||
senbei "C:\Games\MyGame"
|
||||
@@ -58,8 +89,14 @@ line, then duration:
|
||||
done in 1234 ms
|
||||
```
|
||||
|
||||
The `packages` count appears (as `· N packages`) only when Android app
|
||||
packages were processed.
|
||||
|
||||
## Integrity check
|
||||
|
||||
(PE outputs only — Android restores carry their own structural validation; see
|
||||
[Android targets](#android-targets).)
|
||||
|
||||
A successful unpack is not always a runnable one: a layout heuristic can pick
|
||||
the wrong offset and leave the entry-point stub or import strings encrypted, so
|
||||
the pipeline reports success but the OS loader faults at runtime (typically
|
||||
@@ -98,7 +135,7 @@ line, adds a `SUSPECT` entry to the run log, and counts it in the summary's
|
||||
| Flag | Behavior |
|
||||
| --- | --- |
|
||||
| `--out DIR` | Write outputs (and the log, unless `--no-log`) under `DIR`. |
|
||||
| `-v`, `--verbose` | Print detailed `[N/9]` per-stage unpack progress (and the destination path) for each file. In folder mode this replaces the progress bar. |
|
||||
| `-v`, `--verbose` | Print detailed per-stage progress (and the destination path) for each file — `[N/9]` stages for PE targets, container/segment lines for Android libraries. In folder mode this replaces the progress bar. |
|
||||
| `-q`, `--quiet` | Once: hide progress bar and per-file lines; keep banner, summary, and duration. Twice (`-q -q`): suppress all stdio (exit code only). |
|
||||
| `--no-log` | Do not write `senbei-*.log`. Console output is unchanged by this flag alone. |
|
||||
| `--scan-all` | Probe every file in a folder, including ones the scan pre-filter skips (under 4128 bytes, or a bulk-asset extension like `.ab`/`.xml`/`.acb`). Much slower on large game trees; finds the same targets in practice. |
|
||||
@@ -115,7 +152,7 @@ process.
|
||||
|
||||
| Code | Meaning |
|
||||
| --- | --- |
|
||||
| `0` | Success (single file unpacked, or folder run with no errors). |
|
||||
| `0` | Success (single file restored, or folder run with no errors). |
|
||||
| `1` | At least one file failed, a scan probe was unreadable, or a single-file unpack errored. |
|
||||
| `2` | Usage error: no path given, unknown option, missing `--out` value, or multiple input paths (help printed). |
|
||||
|
||||
|
||||
Reference in New Issue
Block a user