--- title: "Mach-O Support Enhancements" description: "Mach-O rewriting in LIEF: __LINKEDIT fixes, chained fixups and exports trie support, converting a binary into a dylib, adding exported symbols, and code injection." canonical_url: "https://lief.re/blog/2022-05-08-macho/" markdown_url: "https://lief.re/blog/2022-05-08-macho/index.md" authors: ["Romain Thomas"] date_published: "2022-05-08T00:00:00Z" date_modified: "2022-05-08T00:00:00Z" language: "en-US" section: "blog" tags: ["Mach-O","macOS","iOS","code-injection"] categories: [] --- # Mach-O Support Enhancements > Mach-O rewriting in LIEF: __LINKEDIT fixes, chained fixups and exports trie support, converting a binary into a dylib, adding exported symbols, and code injection. **tl;dr** The next release of LIEF (v0.13.0) is fixing several Mach-O layout issues when adding new sections/segments. I also added the support for the two new load commands: 1. `LC_DYLD_CHAINED_FIXUPS` 2. `LC_DYLD_EXPORTS_TRIE` The support of LIEF for modifying Mach-O binaries was mostly limited to adding new load commands and thus, extending the load commands table. The [tutorial #11](https://lief.re/doc/latest/tutorials/11_macho_modification.html) explains the technical details to extend the load commands table which consists in shifting the content right after the load commands table and patching the relocations accordingly. Nevertheless, the Mach-O binaries generated by LIEF after the modifications were somehow inconsistent regarding ``codesign``. As a consequence, the binaries generated by LIEF could not be signed and executed on iOS or -- more recently -- an Apple M1. ![Technical diagram](https://lief.re/img/stockholm/Layout/Layout-4-blocks.svg) ** LIEF is now able to generate Mach-O-modified files that can be signed and that follow a strict layout, enforced by dyld and codesign. ** To better understand what was wrong, let's consider the following script in which we add two new segments: ```python import lief target = lief.parse("mbedtls_selftest_arm64.bin") segment = lief.MachO.SegmentCommand("__NEW", [0] * 0x123) target.add(segment) segment = lief.MachO.SegmentCommand("__NEW", [0] * 0x456) target.add(segment) target.write("test.out") ``` Under the hood, LIEF was relocating the binary to add two new `LC_SEGMENT` commands and was allocating space **at the end** of the file to store the content of the new segments. In particular, the new segments data were located **after** the content of the `__LINKEDIT` segment which breaks the layout required by `codesign`. The following figure depicts the layout of a Mach-O file from the original layout to the layout generated by LIEF v0.13.0. ![Technical diagram](https://lief.re/blog/2022-05-08-macho/macho_layout.svg) In LIEF v0.13.0 we fixed this inconsistency to make sure that the content of the new segments are located **before** the content of the `__LINKEDIT` segment. We can perform this change without breaking the binary as `__LINKEDIT` is a kind of self-contained *blob of data*[^instrplace]. `codesign` requires the `__LINKEDIT` segment at the end of the file because the signature is appended at the end of the file. Otherwise, `codesign` would have to perform the similar *relocation* process done by LIEF. ## ``__LINKEDIT`` The ``__LINKEDIT`` segment plays an important role in the layout of the Mach-O format and its execution. This segment is used to store information about the exports, the symbols, the relocations, the signature, and more broadly, information used by the `dyld` loader to load the binary. This segment has a known layout which is described in the following figure: ![Technical diagram](https://lief.re/blog/2022-05-08-macho/linkedit_layout.svg) This layout is very strict and its content must follow the same order as mentioned in the previous figure. In addition, there are sanity checks that ensure all the `__LINKEDIT`'s chunks are contiguous within the `__LINKEDIT` content. If the layout is wrong, the executable **could** run but it won't likely pass the `codesign` checks. This strict layout can be seen -- at first sight -- as a major hurdle for modifying Mach-O files but since the `__LINKEDIT` segment is located **at the end** of the file, we can extend it or shrink it quite easily. ![Technical diagram](https://lief.re/img/stockholm/Layout/Layout-4-blocks.svg) ** LIEF v0.13.0 is able to regenerate the content of this segment from the LIEF objects stored in the LIEF::MachO::Binary object ** Completely regenerating the __LINKEDIT segment enables to perform advanced modifications like creating exports and adding or removing symbols as it is discussed in the next sections. ## `LC_DYLD_CHAINED_FIXUPS & LC_DYLD_EXPORTS_TRIE` Compared to the ELF and PE formats, the relocations and the exported functions of Mach-O binaries are not wrapped by a *table of entries* In the Mach-O format, the relocations are encoded either: 1. By a *bytecode* located in the `LC_DYLD_INFO` command 2. By a *chained fixups* located in the `LC_DYLD_CHAINED_FIXUPS` On the other hand, the exports are encoded in a [Trie](https://en.wikipedia.org/wiki/Trie) located either 1. In the `LC_DYLD_INFO` command 2. In the `LC_DYLD_EXPORTS_TRIE` `LC_DYLD_CHAINED_FIXUPS` appeared more recently compared to the `LC_DYLD_INFO` command for which the differences are described in the blog post: [*How iOS 15 makes your app launch faster*](https://www.emergetools.com/blog/posts/iOS15LaunchTime). The `LC_DYLD_EXPORTS_TRIE` has the same structure as `LC_DYLD_INFO[Export Trie]` but the export information has been moved in this dedicated load command. ## Converting a Mach-O Binary into a Library Converting a binary into a library can be useful to harness a fuzzed binary or to instrument/debug a specific function in a controlled environment (like an unknown cryptography function or a whiteboxed function) In the [tutorial #8](https://lief-project.github.io/doc/latest/tutorials/08_elf_bin2lib.html), we described the process to perform this transformation on an ELF binary and the transformation for a Mach-O binary is a bit more straightforward. Let's consider the following code: ```c #include #include #include static int X = 1; int compute() { return X++; } int main(int argc, const char** argv) { for (size_t i = 0; i < argc; ++i) { printf("compute(): %d\n", compute()); } return 0; } ``` It can be compiled with: ```bash romain@Mac-M1 % clang -O3 -fvisibility=hidden -Wl,-x -o bin2lib.bin bin2lib.c ``` Which produces this executable: [bin2lib.bin](https://lief.re/blog/2022-05-08-macho/bin2lib.bin) To convert this binary into a library, we first need to change its type in the Mach-O's header: ```python import lief bin2lib = lief.parse("bin2lib.bin") bin2lib.header.file_type = lief.MachO.FILE_TYPES.DYLIB bin2lib.write("bin2lib.dyld") ``` It's should be technically enough, but `dyld_info` raises some concerns: ```bash romain@Mac-M1 % dyld_info ./bin2lib.dylib dyld_info: './bin2lib.dylib' in './bin2lib.dylib' MH_DYLIB is missing LC_ID_DYLIB ``` This can be confirmed by looking at the source code of [dyld](https://github.com/apple-oss-distributions/dyld/blob/5c9192436bb195e7a8fe61f22a229ee3d30d8222/common/MachOAnalyzer.cpp#L775-L779). To fix this error, we just have to create a new `LC_ID_DYLIB` command: ```diff {style=pastie} import lief bin2lib = lief.parse("bin2lib.bin") bin2lib.header.file_type = lief.MachO.FILE_TYPES.DYLIB + bin2lib.add(lief.MachO.DylibCommand.id_dylib("bin2lib.dylib", 0, 1, 2)) bin2lib.write("bin2lib.dyld") ``` Which enables to dlopen `bin2lib.dyld` ```python import ctypes handler = ctypes.cdll.LoadLibrary("bin2lib.dyld") # ``` ## Adding Symbols Thanks to the improvements on the `__LINKEDIT` segment, we can now create new exports. If we consider the stripped function `int compute()` from the binary in the previous section, we can create a new export as follows: ![Technical diagram](https://lief.re/blog/2022-05-08-macho/function_to_export.svg) ```python address = 0x100003f18 original.add_exported_function(address, "_compute") ``` ## Code Injection Another use case of these improvements is the capability to inject code in Mach-O file **and to re-sign** the modified binary. Code signing is not required for x86-64 binaries but it becomes mandatory when targeting the arm64 architecture. Let's consider the library `_heapq.cpython-39-darwin.so` which is one of the first libraries dynamically loaded by the Python interpreter. The injection consists in: 1. Creating new segments in the library `_heapq.cpython-39-darwin.so` that will embed our shellcode 2. Changing the address of one of the exported functions to redirect the execution to the shellcode's entrypoint. By running the python interpreter with the environment variable ``DYLD_PRINT_APIS=1`` we can observe the following output: ```bash romain@Mac-M1 ~ % DYLD_PRINT_APIS=1 python3 -c "import io" dyld[76439]: _dyld_is_memory_immutable(0x1b3f8cea0, 26) => 1 dyld[76439]: dlopen("/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload/_heapq.cpython-39-darwin.so", 0x00000002) dyld[76439]: dlopen(_heapq.cpython-39-darwin.so) => 0x208f35800 dyld[76439]: dlsym(0x208f35800, "PyInit__heapq") dyld[76439]: dlsym("PyInit__heapq") => 0x104bcb824 ``` It suggests that ``PyInit__heapq`` is a suitable function for redirecting the execution to the shellcode's entrypoint. To create the shellcode, we can use [gdelugre/shell-factory](https://github.com/gdelugre/shell-factory) developed by a former colleague and which provides **no less than a C++ STL-like** to create shellcode. Thanks to this project, we can create the following shellcode: ```cpp volatile uintptr_t ORIGINAL_EP = 0xdeadc0de; volatile uintptr_t IMAGEBASE = 0x00c0de; using PyInit__heapq_t = void(*)(); inline uintptr_t imagebase() { /* * The value of IMAGEBASE is set by the injector. * After the patch, it contains the relative virtual address of &IMAGEBASE * in the final binary. */ return reinterpret_cast(&IMAGEBASE) - IMAGEBASE; } SHELLCODE_ENTRY { uintptr_t base = imagebase(); Pico::printf("LIEF says hello!\n"); Pico::printf("Time to jump on the real function: %p\n", ORIGINAL_EP); auto PyInit__heapq = reinterpret_cast(base + ORIGINAL_EP); return PyInit__heapq(); } ``` **`Pico::printf`** The attentive reader may have noticed the `Pico::printf("[...] %p")` which is correctly supported by shell-factory (see: [include/pico/format.h](https://github.com/gdelugre/shell-factory/blob/25639dd517ace9a9292db38f8ca423808317de65/include/pico/format.h)) The compiled shellcode can be downloaded here: [lief_demo_darwin_arm64.bin](https://lief.re/blog/2022-05-08-macho/lief_demo_darwin_arm64.bin). To inject the shellcode in `_heapq.cpython-39-darwin.so`, we first need to copy the shellcode's segments in the library: ```python shellcode = lief.parse("lief_demo_darwin_arm64.bin") heapq = lief.parse("_heapq.cpython-39-darwin.so") for segment in shellcode.segments: seg_name = segment.name.replace("__", "") seg = lief.MachO.SegmentCommand(f"__L{new_seg_name}", list(segment.content)) heapq.add(new_seg) ``` Then, we have to patch the Mach-O exports trie to change the address of `PyInit__heapq` to the shellcode's entrypoint: ```python shellcode_rva_entry = ... for exp in heapq.dyld_info.exports: if exp.symbol.name != "_PyInit__heapq": continue original = exp.address exp.address = shellcode_rva_entry return original ``` Finally, we can rewrite the library: ```python heapq.write("_heapq.cpython-39-darwin.so.patched") ``` and **sign it**: ```bash romain@Mac-M1 ~ % codesign -f --verbose -s - _heapq.cpython-39-darwin.so.patched ``` Now when running the Python interpreter, we can observe the execution of the shellcode: ```bash romain@Mac-M1 ~ % python3 LIEF says hello! Time to jump on the real function: 0x15f8 Python 3.9.5 (default, May 3 2021, 19:12:05) [Clang 12.0.5 (clang-1205.0.22.9)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> ``` **Injection** The script that contains the complete logic of the transformation is available [here](https://gist.github.com/romainthomas/16f384a21fe408c7d20e369d75e69588) and, `_heapq.cpython-39-darwin.so.patched` can be downloaded [here](https://lief.re/blog/2022-05-08-macho/_heapq.cpython-39-darwin.so.patched). Surprisingly, we open the **patched** version of the library ([`_heapq.cpython-39-darwin.so.patched`](https://lief.re/blog/2022-05-08-macho/_heapq.cpython-39-darwin.so.patched)) in IDA and we jump on the symbol `_PyInit__heapq`, it actually displays this function: [*IDA Version 7.7.211224*, January 18, 2022](https://hex-rays.com/products/ida/news/) ![IDA Output when jumping on _PyInit__heapq](https://lief.re/blog/2022-05-08-macho/IDA_patched_pyinit_heapq.png) Which is the original function and not the function associated with the shellcode whilst the patched library prints `LIEF says hello [...]` On the other hand, if we get the address of `_PyInit__heapq` with LIEF: ```python import lief patched = lief.parse("./_heapq.cpython-39-darwin.so.patched") symbol = patched.get_symbol("_PyInit__heapq") print(hex(symbol.export_info.address)) ``` The result is: ![Technical diagram](https://lief.re/img/stockholm/General/Thunder-move.svg) **`_PyInit__heapq: 0xf824`** Jumping on this address gives a better output (once manually disassembled): ![IDA Output when jumping on 0xf824](https://lief.re/blog/2022-05-08-macho/IDA_patched_pyinit_heapq_code.png) We recognize the shellcode's entrypoint function . What's happened in IDA since this is the function located at `0xf824` which is executed and thus, resolved by `dyld` and not IDA? IDA is confused because Mach-O's symbols can be stored in two different commands: 1. `LC_DYLD_INFO.export_trie` / `LC_DYLD_EXPORTS_TRIE` 2. `LC_SYMTAB` `LC_DYLD_INFO.export_trie` / `LC_DYLD_EXPORTS_TRIE` are used to store the **exported symbols** while `LC_SYMTAB` stores symbols for other purposes. The important point is that the same symbol can be duplicated in these two commands **with different addresses**. ![Technical diagram](https://lief.re/img/stockholm/General/Thunder-move.svg) ** IDA gives the priority to the `LC_SYMTAB` over the exports trie while the Mach-O loader uses the exports trie. ** The following figure illustrates why it can be confusing: ![Technical diagram](https://lief.re/blog/2022-05-08-macho/patched.svg) Actually, I intentionally took a shortcut in the LIEF script that resolves the address of `_PyInit__heapq` and we can programmatically access these two addresses as follows: ```diff {style=pastie} import lief patched = lief.parse("./_heapq.cpython-39-darwin.so.patched") symbol = patched.get_symbol("_PyInit__heapq") + print(hex(symbol.value)) print(hex(symbol.export_info.address)) + # 0x15f8 address from the LC_SYMTAB # 0xf824 address from the export trie ``` ![Technical diagram](https://lief.re/img/stockholm/General/Attachment-2.svg) ** We can observe a similar issue with BinaryNinja, Ghidra and, to a lesser extent, Radare2 ** ## BinaryNinja *Version 3.0* ![BinaryNinja Result](https://lief.re/blog/2022-05-08-macho/binaryninja_result.png) ## Ghidra [*Version 10.1.2*](https://github.com/NationalSecurityAgency/ghidra/releases/tag/Ghidra_10.1.2_build) - Jan 26, 2022 ![Ghidra Result](https://lief.re/blog/2022-05-08-macho/ghidra_result.png) ## Radare2 [*Version: 5.6.6*](https://github.com/radareorg/radare2/releases/tag/5.6.6) - Mar 22, 2022 ```bash $ r2 _heapq.cpython-39-darwin.so.patched [0x00000000]> aaa ... [0x00000000]> ia [Imports] nth vaddr bind type lib name ――――――――――――――――――――――――――――――――― 0 0x000021ec NONE FUNC PyErr_SetString 1 0x00000000 NONE FUNC PyExc_IndexError 2 0x00000000 NONE FUNC PyExc_RuntimeError 3 0x00000000 NONE FUNC PyExc_TypeError 4 0x000021f8 NONE FUNC PyList_Append 5 0x00002204 NONE FUNC PyList_SetSlice 6 0x00002210 NONE FUNC PyModuleDef_Init 7 0x0000221c NONE FUNC PyModule_AddObject 8 0x00002228 NONE FUNC PyObject_RichCompareBool 9 0x00002234 NONE FUNC PyUnicode_FromString 10 0x00002240 NONE FUNC _PyArg_CheckPositional 11 0x0000224c NONE FUNC _Py_Dealloc 12 0x00000000 NONE FUNC _Py_NoneStruct 13 0x00000000 NONE FUNC dyld_stub_binder [Exports] nth paddr vaddr bind type size lib name ――――――――――――――――――――――――――――――――――――――――――――――――――― 0 0x000015f8 0x000015f8 GLOBAL FUNC 0 _PyInit__heapq ``` On the other hand, the `afl` command outputs a better result: ```bash [0x00000000]> afl 0x000015f8 1 12 sym._PyInit__heapq 0x00001604 6 108 sym._heapq_exec 0x00002238 1 8 fcn.00002238 0x00002220 1 8 fcn.00002220 0x00002250 1 8 fcn.00002250 ... 0x0000f824 1 88 sym.imp._PyInit__heapq ``` ### Demo [Terminal recording](https://lief.re/blog/2022-05-08-macho/out.rec) ## Conclusion These changes strengthen LIEF to read and modify Mach-O binaries. It should enable to develop and create new reverse engineering and binary analysis techniques. For those who are interested in Mach-O (and ELF) *tricks* that could prevent static analysis tools from working correctly, I'll present *The Poor Man's Obfuscator* at [Pass The Salt](https://2022.pass-the-salt.org/) in July 2022 :) [^instrplace]: In the general case, we can't insert content between two arbitrary segments as it could break the binary. For instance, if the `__TEXT` segment references variables in the `__DATA` segment with relative addressing, inserting some data between these two segments will likely break the relative addressing.