Data Execution Prevention

Data Execution Prevention (DEP) is a security feature included in modern operating systems. It is known to be available in Linux, Mac OS X, iOS, Microsoft Windows and Android operating systems and is intended to prevent an application or service from executing code from a non-executable memory region. This helps prevent certain exploits that store code via a buffer overflow, for example.^[1] DEP runs in two modes: hardware-enforced DEP for CPUs that can mark memory pages as nonexecutable, and software-enforced DEP with limited protection for CPUs that do not have hardware support. Software-enforced DEP does not protect against execution of code in data pages, but instead counters SEH overwrite, another type of attack.

DEP was introduced on Linux in 2004 (kernel 2.6.8^[2]), on Windows in 2004 with Windows XP Service Pack 2,^[3] while Apple introduced DEP when they moved to x86 in 2006.^[1]

Mode of enforcement

Hardware enforcement

Hardware-enforced DEP enables the NX bit on compatible CPUs, through the automatic use of PAE kernel in 32-bit Windows and the native support on 64-bit kernels. Windows Vista DEP works by marking certain parts of memory as being intended to hold only data, which the NX or XD bit enabled processor then understands as non-executable.^[3] This helps prevent buffer overflow attacks from succeeding. In Windows, from version Vista, whether DEP is enabled or disabled for a particular process can be viewed on the Processes tab in the Windows Task Manager.

In some instances, enabling Data Execution Prevention can have the unintended consequence of preventing legitimate software from executing, typically exposing a hitherto unknown bug in the process. In these cases, the affected software needs to be remediated by its developer; in the short term it can be flagged as being allowed to execute code in those parts of memory, but this itself leads to a possible attack if the application isn't rigorous in validating data that is passed into a region of memory that is marked as being executable.

Microsoft Windows

If the x86 processor supports this feature in hardware, and if the system's BIOS provides support for this feature and it has been enabled, either by the manufacturer or the user,^[4] then the NX features are turned on in Windows, on a limited "OptIn" basis.^[5] This setting provides protection only for a limited set of Windows system and binary files. To achieve full protection, the user must choose either "OptOut", otherwise known as "turn on DEP for all program services except those I select" function: covering all programs and processes not specifically exempted, or "AlwaysOn", covering all without exemptions selected. These are configurable through the System Properties interface.^[6] If the feature is not supported by the x86 processor, then no protection is given. Outside of the x86 architecture, a version of NX also exists for Intel's IA-64 architecture that is supported by Windows.

Software enforcement

Software DEP, while unrelated to the NX bit, is what Microsoft calls their enforcement of "Safe Structured Exception Handling". Software DEP/SafeSEH simply checks when an exception is raised to make sure that the exception is registered in a function table for the application, and requires the program to be built with it. However, even though it creates an impression that software DEP is related to the prevention of executing code in data pages, it is a different form of protection.^[3]

Issues

Limitations

DEP provides some protection against attacks, but of course is not sufficient to protect against all forms of attack. Other security features such as address space layout randomization, structured exception handler overwrite protection (SEHOP) and Mandatory Integrity Control, can be used in conjunction with DEP.^[7]

Where code is executed at runtime—a JIT compiler is a prominent example—the compiler can potentially be used to produce exploit code (e.g. using JIT Spray) that has been flagged for execution and therefore would not be trapped by DEP.^[8]

Compatibility

DEP occasionally highlights software problems, usually with older software that was not compiled and tested to take advantage of the feature. Users have experienced problems using various command line commands that are a part of Microsoft's Services for Unix, which is included as part of Vista as well as Windows Server 2003 R2.^{[citation needed]}

These problems may be masked by disabling DEP, but at the risk of increasing exposure of the system to malware. DEP can be turned off on a per-application basis, or turned off entirely for all non-essential Windows programs and services.^[9] Microsoft recommends that DEP not be globally disabled where an application crashes due to a DEP error. Instead, the author or vendor of the offending software should firstly be contacted for an updated version that does not violate DEP, or failing that, that DEP be disabled on an exception basis for the offending application only.^[10]

DEP is applied to an entire process, so even if an application runs perfectly with DEP, it may need to be disabled if a non-DEP compliant extension is added that runs in the same process space. For example, DEP-related problems can occasionally occur with DEP-compliant core operating system components such as Windows Explorer, Internet Explorer and Windows Installer as they support in-process third party extensions or plugins that may not be DEP-compliant.^[11]

Configuration

See also

• NX bit – list of names of similar techniques used in other operating systems
• Executable space protection
• Buffer overflow
• Heap overflow
• Stack buffer overflow
• Stack-smashing protection

References

1. ^ ^{a b} Engler, Tobias (22 February 2011). "Schwere der Lücken ist besorgniserregend". Mac & i (in German). Heinz Heise. Retrieved 22 February 2011. 
2. ^ "Changelog for kernel 2.6.8". Retrieved 2012-02-28. 
3. ^ ^{a b c} "A detailed description of the Data Execution Prevention (DEP) feature in Windows XP Service Pack 2, Windows XP Tablet PC Edition 2005, and Windows Server 2003". Microsoft. 2006-09-26. Retrieved 2008-07-11. 
4. ^ "Microsoft Knowledge Base kb912923". 2007-10-07. Retrieved 2009-03-14. 
5. ^ "Microsoft Knowledge Base kb875352". 2006-09-26. Retrieved 2009-03-14. 
6. ^ "System-wide configuration of DEP". 2006-09-26. Retrieved 2009-03-14. 
7. ^ http://pax.grsecurity.net/docs/aslr.t xt elaborates
8. ^ Writing JIT-Spray Shellcode for fun and profit, Alexey Sintsov, (pdf) "interesting way to bypass DEP and ASLR in browsers (not only) and Just-In-Time compilers was presented."
9. ^ Marc Liron. "Adding Software Exceptions In Data Execution Prevention (DEP)". Windows XP Update. Retrieved June 8, 2006. 
10. ^ "Data Execution Prevention: frequently asked questions". Microsoft. Retrieved 2008-09-15. 
11. ^ "Windows Installer custom action code must be compatible with Data Execution Prevention in Windows Vista". Microsoft. 2007-03-15. Retrieved 2008-07-10. 

External links

• A description of DEP and how to configure it
• Guide to hardware-based DEP protection
• The "Understanding Data Execution Prevention" help topic incorrectly states the default setting for DEP in Windows Server 2003 Service Pack 1
• Easy steps to define DEP exceptions in Vista and XP