Bypassing the gate: A closer look into Gatekeeper flaws on macOS

Author: Ferdous Saljooki

Some time ago, Jamf Threat Labs discovered a Gatekeeper vulnerability affecting Launch Services in macOS that may lead to the execution of an unsigned and unnotarized application without displaying appropriate security prompts to the user. We reported our findings to Apple, and in macOS Sonoma 14.0, Apple patched the vulnerability, assigning it CVE-2023-41067.

While exploring this issue, our research also identified similar issues in “The Unarchiver”, a popular application for handling many different archive formats, which completely bypasses all of Gatekeeper’s checks. We reported our findings to MacPaw and were subsequently assigned CVE-2023-46270 and CVE-2024-22405 due to a bug in the open-source XADMaster library.

Initial discovery

Launch Services is a core component of macOS that manages essential functions, such as the launching of applications, handling document types and URL schemes. We initially discovered this issue on macOS Ventura 13.1 (22C65), where we observed that when an unsigned application downloaded from the internet is placed within the /Applications/ folder and the user executes a .fileloc shortcut file pointing to the application, Gatekeeper allows the user to execute it without displaying the appropriate unsafe warning message. By default, Gatekeeper is designed to prevent the execution of unsigned or ad-hoc signed applications downloaded from a remote source unless the user intentionally overrides security controls.

The below prompt is what we expect to see in such a situation. The user has the option to either move the application to trash or cancel.

However, the prompt displayed below is what we observed when double-clicking a .fileloc shortcut file pointing to an unsigned or ad-hoc signed application.

As you can see, the Gatekeeper message does not indicate that the application is unsafe to open; it is actually the same prompt a user would receive when opening a validly signed application.

Jamf security efficacy note

From a detections perspective, we often monitor when Gatekeeper controls are overridden, disabled, or when quarantine attributes are cleared, as these threads can often lead back to potential attackers attempting to execute unsigned code.

The vulnerability

A .fileloc is an internet location shortcut file that can be used to open existing applications, binaries, or files on the system. It uses a plist format and provides quick access for opening files by double-clicking on the shortcut file. For example, the below .fileloc file will open a document in the temp directory called foo.doc.

The vulnerability we discovered exists when a user points this .fileloc URL at an unsigned or ad-hoc signed application.

We tested this idea with an app that we titled “DarkMode.app,” which we quickly hosted on a local web server and downloaded onto our device.

Upon trying to open the app, it was blocked by Gatekeeper as expected. However, we then attempted to open it using a .fileloc plist that we created that points at the downloaded application with the following content.

Upon clicking on the .fileloc file, we noticed Gatekeeper incorrectly prompts the user allowing them to open the application as if it were signed with a valid developer certificate.

The bug was quickly resolved in macOS 13.2 (22D49) before we had the chance to report it. We instead tried a slightly different technique that allows a user to launch the demo application from the internet using a .fileloc or .inetloc file by simply dragging and dropping the shortcut file over an application in the dock which, according to Apple’s macOS user guide, is another supported method for opening files within a specific application.

We conducted our testing with the Safari app, but this technique works with other apps as well, including many third-party applications, resulting in the same inaccurate Gatekeeper response.

Such a bypass may require some form of social engineering to ensure that the application first makes it into a fixed path like the Applications directory. Following the instructions provided within the disk image will result in a single prompt to the user allowing them to run unsigned or ad-hoc signed code.

Deeper vulnerability details

When a local application that has already been approved to run on the system, such as Safari.app, attempts to open an application downloaded from the internet, it can trigger unexpected behavior in how Launch Services handles the opening process. We used .fileloc and .inetloc shortcut files to trigger this behavior, as many applications like Safari are designed to natively open shortcut files.

Running the below command will also result in the same Gatekeeper prompt as seen with the vulnerable .fileloc approach.

open -a Safari.app /Applications/DarkMode.app

Investigating this issue in Safari triggers a function named isQuarantinedAndNotUserApproved, which incorrectly prompts the user and allows them to open the application even though it is unsigned.

Upon allowing the application, the quarantine flags are updated and the application will be allowed to launch. The com.apple.quarantine attribute is an extended attribute used to mark files that have been downloaded from the internet. This attribute signals that the application should be inspected by Gatekeeper to ensure it’s safe to open.

Quarantine attribute before launch:

Quarantine attribute after launch:

However, we already noted that the issue is not specific to Safari and relates to how Launch Services opens applications.

In a debugger attaching to the open command from earlier, we can identify some of the functions that may be responsible for this issue: _LSOpenItemsWithHandler_CFDictionaryApplier makes a call to _LSQuarantineMsgSetAllowUnsigned.

Taking a look at some of the disassembly for _LSOpenItemsWithHandler_CFDictionaryApplier, we see that the function sets a risk category such as LSRiskCategoryUnsafeExecutable before launching.

Apple has made several logic changes in macOS 14 with how Launch Services opens applications. One of the more notable changes is in _LSOpenItemsWithHandler_CFDictionaryApplier, which has improvements with the checks performed for unsigned applications.

Apple patched this vulnerability in macOS 14.0 to ensure the appropriate evaluation of the application, preventing the quarantine flags from being updated when opening an application using the .fileloc or .inetloc methods we discussed. Oddly, the user is still presented with the wrong Gatekeeper prompt when attempting to open unsigned applications from within a local app but selecting “open” does not modify any of the quarantine flags and therefore prevents the application from launching.

The Unarchiver vulnerability

The Unarchiver is an archiving application that allows users to create and extract archives. It supports many archive formats and is often installed on macOS systems as an alternative to the default Archive Utility by Apple.

At Jamf Threat Labs, we continue to explore ways in which Gatekeeper can be bypassed for unauthorized code execution. As the first line of defense on macOS, it continues to be an obstacle for malware authors where they often find creative ways to bypass these security controls.

When the native Archive Utility application extracts an archive, it propagates the quarantine attribute to all extracted items. This ensures that any application or executables opened thereafter will be checked by Gatekeeper. Similarly, Safari has a built-in feature enabled by default, “Open safe files after downloading,” that will automatically unzip applications held within a zip file after downloading them.

Previously, we reported vulnerabilities to Apple, such as CVE-2022-22616 (Safari) and CVE-2022-32910 (Archive Utility) where we found that we could craft a malicious archive in a way where it would not apply the quarantine attribute to the root of the application. During our research we discovered that Gatekeeper will only check an application if this quarantine attribute is applied to the root of the application bundle itself and does not seem to care if it’s applied to the files and folders within it.

For example, the below application bundle will be allowed to execute on macOS without any Gatekeeper checks performed due to the lack of quarantine attribute on the test.app folder.

After Apple patched the unarchiving bug that would lead to the above image, we explored the effects of this same bug on applications that were manually unarchiving files, one of which was the popular macOS tool “The Unarchiver” that gives users the ability unarchive many more additional archive formats than the built-in macOS utility allows.

The following bash script is a simple proof of concept (PoC) demonstrating how to craft a malicious archive using a zipx extension. This ensures that The Unarchiver application (if installed on the victim’s machine) extracts the archive instead of the default Archive Utility.

Upon downloading and extracting this archive file using The Unarchiver, the quarantine attribute is not applied to the root of the application, “foo.app”, only to the files and folders within it allowing us to bypass all Gatekeeper checks when opened.

We reported our findings to MacPaw’s product security team, who patched the vulnerability and assigned it CVE-2023-46270. The patched version was released in The Unarchiver version 4.3.6 and users are recommended to update to the latest available version.

The root cause of the vulnerability was in the XADMaster library, which The Unarchiver uses for creating and extracting archives. XADMaster is an open-source Objective-C library for archive creation, file unarchiving and extraction. It is also cross-platform, as command-line tools are available for Linux, Windows and other operating systems, but to our knowledge the vulnerability is specific to macOS.

The vulnerability existed in the XADUnarchiver and XADSimpleUnarchiver classes, where upon extracting our crafted archive, it would create the root directory at the path without propagating the quarantine attributes.

In the patched version, the code checks if a delegate is set and, if so, calls the delegate method [delegate unarchiver:self didCreateDirectory:path]. This method notifies the delegate that the directory has been successfully created.

It then triggers the function [XADPlatform writeCloneableMetadata:metadata toPath:directory], which is responsible for writing metadata such as the quarantine attribute to the newly created directory, ensuring that all quarantine attributes are applied when unarchiving.

MacPaw has also assigned CVE-2024-22405 for the vulnerability in the XADMaster library used by The Unarchiver. They created a GitHub Advisory GHSA-xg3c-r7w5-7xw2 and have patched this issue in version 1.10.8. Developers who are using this library are encouraged to update to the latest available version.

Conclusion

Gatekeeper is the first line of defense when it comes to ensuring that applications downloaded from the internet are blocked if they’re not signed by a developer with a valid developer ID. Power user’s may often override the quarantine function by right-clicking an application and selecting open, but Gatekeeper can still play a valid role in keeping many users from launching an application they shouldn’t be. All issues discussed in this blog post have been patched and Jamf Threat Labs remains active in monitoring malware that shows an interest in getting around the Gatekeeper feature. We greatly appreciate Apple and MacPaw product security teams for working with us to resolve these issues and improve macOS security.