FORTIGATE FIRMWARE ANALYSIS

IoT 1个月前 admin
108 0 0

TL; DR

  • Optistream provides the means (technology & expertise) to secure hybrid infrastructure and maintain an efficient segmentation
  • As part of our key activities, we monitor any new vulnerabilities affecting network and security appliances to conduct deep researches and strengthen our customers information systems
  • Fortinet products have been recently highly targeted and have been prone to many critical vulnerabilities, our R&D team has recently been focusing on FortiGate firmwares
  • In recent versions of FortiGate, ramdisk (rootfs.gz) is encrypted, we provide tools[1] for its decryption to help analysts
  • We present a methodology to modify FortiGate 7.4.3 firmware and debug it

Context

Optistream helps its clients improving the security of their infrastructure providing the means (technology & expertise) to build and maintain an efficient segmentation. As such, a key stake consists in a deep understanding of a wide variety of network technologies.

As a cybersecurity firm, beyond our core offerings, we are deeply committed to help our clients build and secure their IT landscape. We are therefore concerned by any security issues affecting appliances currently deployed by our customers. That’s why Optistream maintains a continuous monitoring of cyber threats and keeps a close eye on recent security flaws concerning several network and VPN solutions (e.g. Ivanti, Citrix, Fortinet).

We recently delved into analysis of FortiGate appliances, a network security equipment widely deployed and often Internet-facing making them a high-value target for attackers, and are pleased to share in this publication the key highlights of this R&D analysis.

Introduction

Fortinet products have recently encountered significant challenges marked by the disclosure of several highly critical vulnerabilities[2][3][4][5]. Prior to the release of proper updates, customers were left vulnerable to widespread exploitation campaigns targeting these products.

Recognizing the importance of understanding these vulnerabilities, Optistream researchers directed their efforts towards analyzing FortiGate firmware. During our research, we encountered difficulties in obtaining deep and unrestricted access to the FortiGate internals.

In response to these challenges, we present this analysis as a meticulous exploration of the integrity check mechanisms embedded within FortiOS. Additionally, we provide insights into establishing a debugging environment on the latest version of FortiGate-VM (version 7.4.3), which served as the foundation for our research endeavors.

FortiGate-VM

First requirement of our study is getting hold of FortiGate firmware.

Recently, Fortinet stopped to freely distribute its complete firmwares catalog, from now you need a properly registered equipment. However, for those who need to test Fortinet products (among FortiGate) you have possibility to download VMware images[6]. N.B.: In this way, you can only get the last updated release and cannot select a specific version.

Once downloaded, you can import the virtual machine into VMware. For our purposes, we configure port1 network interface with NAT enabled and set a static IP:

config system interface
edit port1
set mode static
set ip 172.16.62.100 255.255.255.0
end
config router static
edit 1
set device port1
set gateway 172.16.62.2
end

FortiGate-VM is now properly configured. We can stop the VM and start looking at the disk:

~/fortios$ apt install qemu-utils
~/fortios$ modprobe nbd max_part=16
~/fortios$ qemu-nbd -c /dev/nbd1 ~/vmware/FortiGate-VM64/FortiGate-VM64-disk1.vmdk
~/fortios$ mount /dev/nbd1p1 ~/fortios/fs
~/fortios$ tree -a -L 1 ~/fortios/fs
/home/user/fortios/fs
├── bin
├── boot
├── boot.msg
├── cmdb
├── config
├── datafs.tar.gz
├── datafs.tar.gz.chk
├── datafs.tar.gz.chk.bak
├── .db
├── .db.x
├── dhcp6s_db.bak
├── dhcpddb.bak
├── dhcp_ipmac.dat.bak
├── etc
├── extlinux.conf
├── filechecksum
├── flatkc
├── flatkc.chk
├── ldlinux.c32
├── ldlinux.sys
├── lib
├── log
├── lost+found
├── rootfs.gz
├── rootfs.gz.chk
└── web-ui
10 directories, 17 files

The files we are mainly interested in are flatkc and rootfs.gz. The former is FortiOS kernel that we’ll have to study, the latter is the ramdisk from which the appliance runs and contains main components making up FortiGate (including binaries historically affected by several vulnerabilities).

Until recently, it was possible to decompress this archive and have direct access to its content. But on most recent versions (those released around end of 2023), it seems that rootfs.gz is now encrypted:

~/fortios/fs$ ent rootfs.gz
Entropy = 7.999996 bits per byte.

The next part of this blog entry will follow the way this ramdisk is verified and decrypted before being loaded.

Ramdisk integrity

Having a look at the kernel command line confirms that rootfs.gz acts as the ramdisk:

DISPLAY boot.msg
TIMEOUT 10
TOTALTIMEOUT 9000
DEFAULT flatkc ro panic=5 endbase=0xA0000 console=ttyS0, root=/dev/ram0 ramdisk_size=65536 initrd=/rootfs.gz maxcpus=1
ExtLinux bootloader configuration file (extlinux.conf)

One of the key tasks of the bootloader is to load this ramdisk into memory at a specific location that kernel has knowledge of. Reading Linux kernel source code teaches us that populate_rootfs[7] makes use of global variables __initramfs_start and __initramfs_end to locate it and handle its loading.

populate_rootfs job is to unpack the GZIP-compressed CPIO image of ramdisk and mount rootfs.

FortiOS is Linux-based and uses a modified kernel 4.19.13 (as of version 7.4.3), thus we can disassemble flatkc (after conversion to ELF format using vmlinux-to-elf[8]) and look for the function mentioned above:

__int64 populate_rootfs()
{

if ( qword_FFFFFFFF81838080 )
{
printk((unsigned__int64)&unk_FFFFFFFF813CE3D0);
if ( !unpack_to_rootfs(qword_FFFFFFFF81838080, qword_FFFFFFFF81838078 – qword_FFFFFFFF81838080) )
{
LABEL_9:
free_initrd();
goto LABEL_10;
}
Excerpt of populate_rootfs

We can safely labelize the addresses 0xFFFFFFFF81838080 and 0xFFFFFFFF81838078 with their right symbols, respectively __initramfs_start and __initramfs_end. We then take a look at the cross-references to these variables and get two interesting functions: fgt_verify_initrd and fgt_verify_decrypt.

These functions names suggest some checks are firstly done on ramdisk before it is decrypted.

Interesting enough, fgt_verify_decrypt function seems recently added to FortiOS.

fgt_verify_initrd

This function is responsible for both ramdisk signature check and decryption:

__int64 fgt_verify_initrd()
{

ramdisk_size = (_DWORD)::_initramfs_end – (_DWORD)_initramfs_start;
if ( (unsigned __int64)(::_initramfs_end – _initramfs_start) <= 0x100 || (unsigned int)fgt_verifier_open(&pubkey))
{
v1 = -1;
LABEL_19:
machine_halt();
goto LABEL_20;
}
sha256_init(ctx);
sha256_update_0(ctx,_initramfs_start, (unsigned int)(ramdisk_size – 256));
sha256_final_0(ctx, pubkey.hash);
_initramfs_end = ::_initramfs_end;
v3 = ::_initramfs_end – 256;
pubkey.psig = ::_initramfs_end – 256;
rootfs_hash = pubkey.hash;
bg_sig_enc = mpi_read_raw_data(::_initramfs_end – 256, 256LL);

v1 = -12;
if ( bg_sig_enc )
{
if ( (signed int)mpi_cmp_ui(bg_sig_enc, 0LL) < 0 || (signed int)mpi_cmp(bg_sig_enc, pubkey.bg_n) >= 0 )
{
v1 = -22;
}
else
{
bg_sig_dec = mpi_alloc(0LL);

v1 = -12;
if ( bg_sig_dec )
{
v1 = mpi_powm(bg_sig_dec,bg_sig_enc, pubkey.bg_e, pubkey.bg_n);
if ( !v1 )
{
v8 = mpi_read_buffer(bg_sig_dec, pubkey.psig, 256u, (unsigned int *)&v16, 0LL);  // replace encrypted signature with “powm” result
v9 = v16;
LOBYTE(v9) = ~(_BYTE)v16;
v10 = v8 | v9 | (unsigned__int8)(*(_initramfs_end – 256) ^ 1);
psig = _initramfs_end – 255;
do
v10 |= (unsigned __int8)~*psig++;
while ( _initramfs_end – 53 != psig);  // check padding
v1 = v10 | v3[203];
i = 0LL;
do
{
v1 |= (unsigned__int8)(hash_type_ber[i] ^ _initramfs_end[i – 52]);
++i;
}
while ( i != 19 );  // check hash type
j = 0LL;
do
{
v1 |= (unsigned__int8)(rootfs_hash[j] ^ _initramfs_end[j – 33]);
++j;
}
while ( j != 32 );  // check hash
}
mpi_free(bg_sig_dec);
}
}
mpi_free(bg_sig_enc);
}
fgt_verifier_close(&pubkey);
if ( v1 )
goto LABEL_19;
LABEL_20:
::_initramfs_end -= 256;
fgt_verify_decrypt(); // decrypt ramdisk
return v1;
}
fgt_verify_initrd pseudo-code

RSA key decryption

First, we notice a call to fgt_verifier_open which is responsible for decrypting and loading a 2048-bits RSA public key that will be used further for signature check. Key decryption happens in a function called fgt_verifier_pub_key:

unsigned __int64 __fastcall fgt_verifier_pub_key(__int64 a1)
{

v1 = a1;

sha256_init(&v8);
sha256_update_0(&v8, &unk_FFFFFFFF817932E3, 29LL);
sha256_update_0(&v8, &unk_FFFFFFFF817932E0, 3LL);
sha256_final_0(&v8, &v9); // v9 = key
sha256_init(&v8);
sha256_update_0(&v8, &unk_FFFFFFFF817932E1, 31LL);
sha256_update_0(&v8, &unk_FFFFFFFF817932E0, 1LL);
sha256_final_0(&v8, &v10); // v10 = iv
v2 = &v6;
v3 = 8LL;
v4 = &v9;
while ( v3 )
{
*(_DWORD *)v2 = *(_DWORD *)v4;
v4 += 4;
v2 += 4;
–v3;
}
crypto_chacha20_init(&v7, &v6, &v10);
chacha20_docrypt(&v7, v1, &unk_FFFFFFFF817931C0, 270LL);
return v10 – __readgsqword(0x28u);
}
fgt_verifier_pub_key – RSA key decryption (0xFFFFFFFF817931C0)

Symbols reveal that ChaCha20 algorithm (stream-cipher) seems to be used for decryption. It is initialized with two parameters (key and IV) derived from a 32-bytes master key stored at 0xFFFFFFFF817932E0:

key = sha256(masterkey[3:32] + masterkey[:3])
iv  = sha256(masterkey[1:32] + masterkey[0])

As defined in RFC 8439, ChaCha20 uses a 256-bits key and a 96-bits IV. But it also uses a block counter initialized to 0 that is incremented after each 512-bits block processing. The implementation in FortiGate differs from RFC because it uses part of computed IV (first 4 bytes) to initialize this block counter to something different from 0.

FORTIGATE FIRMWARE ANALYSIS
ChaCha20 context (RFC 8439)

Once initialized, ChaCha20 is then used to decrypt an RSA public key (270 bytes).

We have implemented[9] this algorithm according to highlighted modification are now able to decrypt .

After having extracted master key (0x4CF7…) and encrypted blob (0x97CE…), we can then retrieve plain RSA public key used for signature verification:

~/fortios$ ./decrypt_rsapubkey \
4CF7A950B99CF29B0343E7BA6C609E49D9766F16C6D2F075F72AD400542F0765 \
97CE67A20E[…]

BER-encoded pub key:
3082010A02820101008D64BAC2CE5EBF82EDF58CA8C9E5B379D7C836E3F6ED0FEE2531A83286300F8A6…36BC071EAF6C7E3625E50203010001

We recognize last digits which represent the standard RSA exponent (65537). Further code analysis tells us that the RSA public key is encoded using BER[10].

FORTIGATE FIRMWARE ANALYSIS

Decoded RSA public key (e, n)

Signature check

Back in fgt_verify_initrd code, we analyze next implemented steps following key decryption:

  • SHA256 of ramdisk (minus block of signature) is computed
  • signature (last 256 bytes block) is loaded into a BigInt (bg_sig_enc)
  • signature decryption: modular exponentiation is calculated using mpi_pown (bg_sig_ence % n)
  • signature check: validate padding, hash type and compare computed SHA256 hash of ramdisk against the one embedded in signature.

In case all these tests pass, v1 = 0 (valid signature).

This is a PKCS#1 v1.5 RSA signature that follows the format:

FORTIGATE FIRMWARE ANALYSIS

PKCS#1 v1.5 signature structure

We can check it by ourself.

We retrieve last signature block of ramdisk:

~/fortios$ xxd -p -u -s -256 -c 256 rootfs.gz.x64.v7.4.3
3B6207103D6FA98110868213C59544B5…

We decrypt signature using few lines of Python:

>>> e = 0x10001
>>> n = 0x008D64BAC2CE5EBF82EDF58CA8C9E5B3…
>>> x = 0x3B6207103D6FA98110868213C59544B5…
>>> res = pow(x, e, n)
>>> print(“%X” % res)
1FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF

003031300D060960864801650304020105000420E2738E2E2C73A55CC6DFE49A1E2E6C904A7948E
996E79D01EEECEDEA967DF914

Finally, we decode part of signature following padding:

FORTIGATE FIRMWARE ANALYSIS

Decoded hash

Let’s check ramdisk SHA256:

~/fortios$ ls -l rootfs.gz.x64.v7.4.3
-rw-r–r–1 user user 71395325 Feb  8 09:11 rootfs.gz.x64.v7.4.3
~/fortios$ dd if=rootfs.gz.x64.v7.4.3 count=1 bs=$((71395325-256)) | sha256sum

e2738e2e2c73a55cc6dfe49a1e2e6c904a7948e996e79d01eeecedea967df914  –

Signature is verified.

Ramdisk decryption

Last function call within fgt_verify_initrd goes to fgt_verify_decrypt:

unsigned __int64 fgt_verify_decrypt()
{

v0 = _initramfs_start;
v1 = _initramfs_end – _initramfs_start;
fgt_verifier_key_iv((__int64)&v8, (__int64)&v9);
v2 = &v6;
v3 = 8LL;
v4 = &v8;
while ( v3 )
{
*(_DWORD *)v2 = *(_DWORD *)v4;
v4 += 4;
v2 += 4;
–v3;
}
crypto_chacha20_init(&v7, &v6, &v9);
chacha20_docrypt((__int64)&v7, v0, v0, v1);
return v10 – __readgsqword(0x28u);
}
fgt_verify_decrypt – ramdisk decryption

This is very similar to fgt_verifier_pub_key. The light difference resides in the way master key is derived inside fgt_verifier_key_iv to generate ChaCha20 parameters:

key = sha256(masterkey[4:32] + masterkey[:4])
iv  = sha256(masterkey[5:32] + masterkey[:5])

We also provide a tool[11] that implements decryption of ramdisk (rootfs.gz):

~/fortios$ ./decrypt_rootfs rootfs.gz.x64.v7.4.3 rootfs.gz.x64.v7.4.3.decrypted \
4CF7A950B99CF29B0343E7BA6C609E49D9766F16C6D2F075F72AD400542F0765

rootfs size: 71395069
Decrypting rootfs…
Writing to rootfs.gz.x64.v7.4.3.decrypted…
~/fortios$ file rootfs.gz.x64.v7.4.3.decrypted rootfs.gz.x64.v7.4.3.decrypted: gzip compressed data, last modified: Thu Feb  1 17:37:142024, from Unix, original size modulo 2^32 116932640

We now have access to main OS binaries:

~/fortios$ mkdir tmp
~/fortios$ gzip -dc -S .decrypted < rootfs.gz.x64.v7.4.3.decrypted > tmp/rootfs.cpio
~/fortios$ cd tmp; sudo cpio -idv < rootfs.cpio
~/fortios/tmp$ xz –check=sha256 -d bin.tar.xz
~/fortios/tmp$ tar -xf bin.tar
~/fortios/tmp$ ls -la bin/

-rwxr-xr-x1 user user  15K Jan 31 19:00 hotplug
lrwxrwxrwx1 user user    9 Jan 31 19:00 hotplugd -> /bin/init
lrwxrwxrwx1 user user    9 Jan 31 19:00 httpclid -> /bin/init
lrwxrwxrwx1 user user    9 Jan 31 19:00 httpsd -> /bin/init
lrwxrwxrwx1 user user    9 Jan 31 19:00 httpsnifferd -> /bin/init
lrwxrwxrwx1 user user    9 Jan 31 19:00 iflpd -> /bin/init
lrwxrwxrwx1 user user    9 Jan 31 19:00 ikecryptd -> /bin/init
lrwxrwxrwx1 user user    9 Jan 31 19:00 iked -> /bin/init
lrwxrwxrwx1 user user    9 Jan 31 19:00 imi -> /bin/init
lrwxrwxrwx1 user user    9 Jan 31 19:00 inat -> /bin/init
-rwxr-xr-x1 user user  75M Feb 20 05:42 init
lrwxrwxrwx1 user user    9 Jan 31 19:00 initXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX -> /bin/init

This directory contains binaries for FortiGate services (httpsdsslvpnd…). We can notice that most of them are actually symbolic links to /bin/init which is a big monolithic binary. Most of critical vulnerabilities target the latter and other services located in this directory.

Auxiliary tool: getrootfskey.py

To help analysts decrypting rootfs.gz, we developed a script[12] using Miasm framework[13] that helps automatically retrieving of master key from firmware in a generic way without need of disassembling.

The symbol address of fgt_verifier_pub_key is first retrieved. Then Miasm allows us to do symbolic execution on this function, the goal is to detect any call to sha256_update to which the master key is passed as an argument. Parts of master key are given to this function to compute hashes, so we register all these addresses and keep the minimum value that gives us the base address of master key in memory.

from argparse import ArgumentParser

import sys
import binascii
from miasm.analysis.binary import Container
from miasm.analysis.machine import Machine
from miasm.core.locationdb import LocationDB
from miasm.ir.symbexec import SymbolicExecutionEngine
from miasm.expression.expression import *

parser = ArgumentParser(“Get FortiGate rootfs encryption seed”)
parser.add_argument(“target_binary”, help=“Target binary path”)
options = parser.parse_args()

fdesc = open(options.target_binary, ‘rb’)
loc_db = LocationDB()
cont = Container.from_stream(fdesc, loc_db)
machine = Machine(cont.arch)

print(f”Architecture: {cont.arch})
ret_val_reg = None
arg_val_reg = None
match cont.arch:
case “x86_64”:
ret_val_reg = machine.mn.regs.RAX
arg_val_reg = machine.mn.regs.RSI
case “aarch64l”:
ret_val_reg = machine.mn.regs.X0
arg_val_reg = machine.mn.regs.X1
case _:
sys.stderr.write(“OS architecture not supported!”)
sys.exit(1)

mdis = machine.dis_engine(cont.bin_stream, loc_db=cont.loc_db)
addr = loc_db.get_name_offset(“fgt_verifier_pub_key”)
asmcfg = mdis.dis_multiblock(addr)
lifter = machine.lifter_model_call(mdis.loc_db)
ircfg = lifter.new_ircfg_from_asmcfg(asmcfg)

symb = SymbolicExecutionEngine(lifter)
all_seeds = list()
while True:
irblock = ircfg.get_block(addr)
if irblock is None:
break

addr = symb.eval_updt_irblock(irblock, step=False)
if ret_val_reg in symb.symbols.symbols_id:
reg_expr = symb.symbols.symbols_id[ret_val_reg]
if reg_expr.is_function_call():
target = reg_expr.args[0]
target_func = loc_db.get_offset_location(target.arg)
target_func = list(loc_db.get_location_names(target_func))[0]
if target_func == “sha256_update”:
all_seeds.append(symb.symbols.symbols_id[arg_val_reg].arg)

seed_addr = min(all_seeds)
print(f”Seed address: {hex(seed_addr)})

seed_data = cont.executable.get_virt().get(seed_addr, seed_addr + 32)
seed_data = binascii.hexlify(seed_data).upper()
print(f”Extracted seed: {seed_data})

getrootfskey.py

Usage example:

~/fortios$ python getrootfskey.py ~/fortios/flatkc.elf.x64.v7.4.3
Architecture: x86_64
Seed address: 0xffffffff817932e0
Extracted seed: b’4CF7A950B99CF29B0343E7BA6C609E49D9766F16C6D2F075F72AD400542F0765′

This tool has been tested on latest versions of FortiGate firmwares and supports x64 and aarch64 architectures.

Additional integrity checks

So far, we delved into ramdisk integrity verification handled by the kernel that is the earliest check happening during the boot process of the appliance.

In rootfs.gz, we’ve examined the content of bin.tar.xz archive that contains a binary called init. This is the first userland process executed by the kernel (PID 1).

We’ll see that this process also implements several integrity checks. We have to study what are the new mechanisms implemented in order to bypass them.

.db[x] files

The directory listing of .vmdk revealed two interesting files: .db and .db.x.

.db looks like this:

 …
{
“path”: “/rootfs.gz”,
“time”: “/2023-12-19 01:15:41.562618675 +0000”,
“size”: 60572953,
“digests”: [
{
“algorithm”: “SHA512”,
“digest”: “128997ed09453c778753228d6c7e7bcafd8466a719181f163d96a7127ca6c4e4a334cf2cbbaef8b60b0d4f21479bad8e5b121015a91464efa5c25d34672618ed”
}
]
},

This file contains hashes (no signatures, only plain SHA512 hashes) of firmware main components: rootfs.gzflatkc and datafs.tar.gz.

.db.x file contains blob data with strings related to certificates.

Among strings in /bin/init binary, we can find out an occurrence to “/data/.db”, we then locate under IDA Pro what is the only function referencing it:


v1 = sub_2D71C80(“/data/.db“, &var_20);
if ( !v1 )
return 0xFFFFFFFFLL;
_DB_content_1 = v1;
if ( var_20 <= 0 )
{
free(v1);
return 0xFFFFFFFFLL;
}
v2 = sub_2D71C80(“/data/.db.x“, &var_1C);
v3 = v2;
if ( v2 )
{
if ( var_1C <= 0 )
{
free(_DB_content_1);
free(v3);
result = 0LL;
}
else
{
result = sub_25029F0((__int64)_DB_content_1, var_20, (__int64)v2, var_1C);
}
}
Function located at 0x2D72240

These files appear to be read and their content passed to function sub_25029F0. Debug log functions calls let suppose this function is called ftnt_code_signing_verify_sig.


v8 = _parse_sig_data(__a_DB_X_content_1, __a_DB_X_content_size_1, (__int64)&BIO);
if ( v8 < 0 )
{
v9 = -(v8 != -2);
}
else
{
v7 = __a_DB_content_size_1;
if ( (signedint)_parse_raw_data(__a_DB_content, __a_DB_content_size_1, (__int64 *)&BIO)< 0 )
{
v9 = -1;
}
else if ( !(unsigned int)sub_2DD1AF0() || (v9 = 1, (unsigned int)sub_2DD1B20()) )
{
v9 = _cs_verify((__int64)&BIO, __a_DB_content_size_1);
}
}
ftnt_code_signing_verify_sig (after functions and variables renaming)

This code clearly shows us that .db file is checked against a signature stored in .db.x file. If this signature is validated, then .db JSON file is parsed and hashes of the three OS components are checked against the ones stored.

If we look at cross-references on this function, this leads us (several levels up) until the main function of /bin/init:


if ( (signed int)ftnt_code_signing_verify_sig_3() < 0 )
do_halt(v25, (__int64)”/bin/init”, v4, a4);
if ( (unsigned int)sub_2BC8AF0() )
{
sub_2CC6290();
if (sub_4543F0(“/bin/fips_self_test”) )
do_halt(v25, (__int64)”/bin/init”, v4, a4);
}
else
{
if ( sub_455770() )
do_halt(v25, (__int64)”/bin/init”, v4, a4);
sub_2C0A1F0();
}
Excerpt of /bin/init entrypoint

We can reasonably assume that we’ll have to disable this check to avoid OS shutdown during the boot process.

Another function has indirect usage of ftnt_code_signing_verify_sig:

__int64 __fastcall sub_D62C20(__int64a1)
{

result = ftnt_code_signing_verify_sig_2(0);
if ( (signed int)result <= 0 )
{
v2 = result;
result = message(“System file integrity monitor check failed!\n”, 2160000LL);
if ( v2 )
result = sub_2D73D90(“System file integrity check failed”);
}
return result;
}
forticron” task

It seems that /bin/init initializes some kind of cron tasks (“forticron”). This task might be a recent addition, and we’ll see that it is also mandatory to disable this check to avoid OS premature shutdown.

.chk files

In /bin/init entrypoint, we also notice a call to sub_455770:

__pid_t sub_455770()
{

v0 = fork();
if ( v0 < 0 ) {
sub_23338D0((__int64)”fork()failed\n”);
result = 0;
}
else
{
if ( !v0 )
{
v1 = sub_2C0ECD0(“Firmware integrity”);
if ( !v1 )
exit(0);
v3 = (*((__int64 (__fastcall**)(_QWORD))v1 + 2))(0LL);
exit(v3 == 0);
}
result = sub_454350(v0);
}
return result;
}
sub_455770
.data:0000000004E86AB8   dq offset aFirmwareIntegr ; “Firmware integrity”
.data:0000000004E86AC0   dq 1
.data:0000000004E86AC8   dq offset __cb_fw_integrity
Callbacks array

This function looks for a subroutine called “Firmware integrity”, once the entry found in the array it will call the linked callback (we dubbed it __cb_fw_integrity):

_BOOL8 __fastcall _cb_fw_integrity(int a1)
{

pubkey = d2i_RSAPublicKey(0LL, &v4, 270LL);
if ( pubkey && (pubkey_1 = pubkey, !(unsignedint)sub_2BC9CD0(“/data/rootfs.gz”, “/data/rootfs.gz.chk”, a1, pubkey)) )
result = (unsignedint)sub_2BC9CD0(“/data/flatkc”, “/data/flatkc.chk”, a1, pubkey_1) == 0;
else
result = 0LL;
return result;
}
__cb_fw_integrity

Without going further in this function analysis, we can guess that .chk files contains another kind of signature data for associated files. Again, rootfs.gz and kernel integrity is checked.

Customizing firmware

As analysts, our ultimate goal is to be able to deploy custom debugging tools on appliance.

Fortigate CLI interface provides a limited shell to the user with a scoped set of commands sufficient for appliance configuration.

Equipped with the knowledge aggregated so far, our goal is now to obtain full root access, get a Bash shell and deploy our tools (busyboxgdbserver…).

Methodology

Below steps are necessary:

  • mount .vmdk disk
  • retrieve ramdisk encryption key from kernel (flatkc) using getrootfskey.py and decrypt rootfs.gz
  • decompress bin.tar.xz to have access to /bin/init binary
  • patch /bin/init to disable the previously spotted integrity checks
  • deploy our tools into /bin
  • repack bin.tar.xz, replace it into rootfs.gz
  • debug FortiGate kernel to disable ramdisk signature check.

We’ve already done the first three steps, so let directly see how to disable /bin/init security mechanisms.

Patching /bin/init

We previously spotted three locations where integrity checks were performed by init process. At assembly level, we spot the instructions we need to disable by patching them with NOP instructions.

FORTIGATE FIRMWARE ANALYSIS

Example of init patches

Deploying tools

To get a proper shell we’ll use a popular tool often met in embedded devices: busybox. This will offer us basic Unix commands to interact with the OS. We’ll install a bind shell whose purpose is to attach busybox to a socket so we are able to remotely connect to the FortiGate.

/* gcc -o -static backdoor backdoor.c */
#include <stdio.h>

void shell() {
system(“/bin/busybox ls”, 0, 0);
system(“/bin/busybox id”, 0, 0);
system(“/bin/busybox killall sshd && /bin/busybox telnetd -l /bin/sh -b 0.0.0.0 -p 22”, 0, 0);
system(“/bin/busybox sh”, 0, 0);
}

int main(int argc, char const *argv[]) {
shell();
return 0;
}

Our bind shell

FortiGate forbids us to open arbitrary ports. The trick here is to kill legitimate (and non-necessary) services and re-use their port. For that, we configure the interface to enable ssh and telnet:

config system interface
edit port1
set allowaccess ssh telnet …
end

Now, we need a way to execute our bind shell. Several researchers[14][15] already gave a solution for that: it exists standalone binaries (i.e. not symbolic links to /bin/init) that it is possible to replace and to start from limited FortiGate CLI. One of them is /bin/smartctl that we can trigger using “diag hardware smartctl” command. We replace it with our bind shell and link busybox to /bin/sh:

~/fortios/tmp$ cp ~/fortios/busybox-1.36.1/busybox ./bin/
~/fortios/tmp$ cp ~/fortios/backdoor ./bin/smartctl
~/fortios/tmp$ rm ./bin/sh
~/fortios/tmp$ ln -s /bin/busybox ./bin/sh
~/fortios/tmp$ sudo chmod 755 ./bin/busybox ./bin/smartctl ./bin/sh

We’ll also need a static gdbserver for debugging purpose, we can also put in in /bin/.

We can now rebuild the customized ramdisk as a whole (and encrypt it!):

~/fortios/tmp$ tar -cf bin.tar ./bin
~/fortios/tmp$ xz -e bin.tar
~/fortios/tmp$ find . -path ‘./bin’ -prune -o -print | cpio -H newc -o > rootfs.cpio
~/fortios/tmp$ cat rootfs.cpio | gzip > rootfs.gz
~/fortios/tmp$ ~/fortios/encrypt_rootfs rootfs.gz rootfs.gz.enc \
4CF7A950B99CF29B0343E7BA6C609E49D9766F16C6D2F075F72AD400542F0765
~/fortios/tmp$ sudo mv rootfs.gz.enc ~/fortios/fs/rootfs.gz

We can now safely replace rootfs.gz within .vmdk and unmount it.

Debugging the kernel

Kernel is the first component that checks ramdisk integrity. We need to debug it to force this validation.

We take a closer look at the assembly of fgt_verify_initrd and locate the precise ASM instruction responsible for this test:

FFFFFFFF8170986B loc_FFFFFFFF8170986B:
FFFFFFFF8170986B                lea     rdi, [rbp+var_C0]
FFFFFFFF81709872                call    fgt_verifier_close
FFFFFFFF81709877                test    r12d, r12d
FFFFFFFF8170987A                jz      short loc_FFFFFFFF81709881
FFFFFFFF8170987C
FFFFFFFF8170987C loc_FFFFFFFF8170987C:
FFFFFFFF8170987C                call    machine_halt
FFFFFFFF81709881
FFFFFFFF81709881 loc_FFFFFFFF81709881:
FFFFFFFF81709881                sub     cs:__initramfs_end, 100h
FFFFFFFF8170988C                call    fgt_verify_decrypt
ASM excerpt of fgt_verify_initrd

We’d like to force call to decryption routine while skipping machine_halt call. We’ll set a breakpoint at 0xFFFFFFFF81709877 and force r12d = 0.

We first enable GDB stub for our VM (.vmx file):

debugStub.listen.guest64 = “TRUE”
debugStub.listen.guest64.remote = “TRUE”
debugStub.port.guest64 = “12345”

Then we set the breakpoint, start the VM and connect gdb (we have a few seconds window before init executes):

(gdb) file flatkc.elf.x64.v7.4.3
(gdb) b * 0xFFFFFFFF81709877
(gdb) commands
>set $r12=0
>end
(gdb) target remote 127.0.0.1:12345

(gdb) c

Finally, and despite warning messages, the boot process is completed and we got a privileged shell:

FORTIGATE FIRMWARE ANALYSIS

Conclusion

The security mechanisms outlined in this paper are designed to prevent arbitrary modifications to FortiGate firmware, thereby safeguarding against potential attackers willing to compromise the equipment.

However, while these redundant checks and encryption layers may be perceived as attempts to complicate the work of researchers and serve as obfuscation layers, they may not provide adequate protection against determined actors.

It is likely that Fortinet products, along with those of other vendors, will remain highly targeted. Therefore, Optistream advises maintaining updated customer appliances and applying security integrity check procedures based on IOCs provided by vendors.

Links

[1] https://github.com/optistream/fortigate-crypto
[2] https://www.fortiguard.com/psirt/FG-IR-23-097
[3] https://www.fortiguard.com/psirt/FG-IR-23-183
[4] https://www.fortiguard.com/psirt/FG-IR-24-015
[5] https://www.fortiguard.com/psirt/FG-IR-24-029
[6] https://support.fortinet.com/Download/VMImages.aspx
[7] https://elixir.bootlin.com/linux/v4.19.13/source/init/initramfs.c#L602
[8] https://github.com/marin-m/vmlinux-to-elf
[9] https://github.com/optistream/fortigate-crypto/blob/main/decrypt_rsapubkey.c
[10] https://en.wikipedia.org/wiki/X.690#BER_encoding
[11] https://github.com/optistream/fortigate-crypto/blob/main/decrypt_rootfs.c
[12] https://github.com/optistream/fortigate-crypto/blob/main/getrootfskey.py
[13] https://github.com/cea-sec/miasm
[14] https://ioo0s.art/2023/02/07/%E5%88%A9%E7%94%A8VMware%E8%8E%B7%E5%8F%96shell-%E8%BF%9B%E9%98%B6/
[15] https://forum.butian.net/share/2166

原文始发于optistream:FORTIGATE FIRMWARE ANALYSIS

版权声明:admin 发表于 2024年3月7日 下午8:39。
转载请注明:FORTIGATE FIRMWARE ANALYSIS | CTF导航

相关文章