If you’re NOT using hugepages, you’re doing it wrong!

This is a recap and update to a blog I wrote more than 10 years ago. So, this is based on a not-so-recent discussion, here in the Oracle Database – General Questions Forum. But, it’s still highly relevant today.

I think my answer bears repeating, so, here is a slightly modified and expanded version:

So, I’m going to take a strong position on hugepages. I’m going to go as far as to say, for any non-trivial SGA size, if you’re not using hugepages, you’re doing it wrong. There are three main points to consider.

First, when allocating a large SGA, each page needs an entry in the page table. On x86-64 (a 64-bit architecture), each PTE (page table entry) is 8 bytes. So, let’s assume you are allocating a 20 GB SGA. Standard shared memory pages are 4 kb. So, to allocate 20 GB of SGA, you need 5,242,880 4 kb pages. Each of those pages requires an 8-byte entry in the page table. So, that’s 40MB of page table entries.

The second point is that, with standard 4 k pages, each process that attaches to the SGA needs it’s own copy of the page table. So, if you have, say, 200 dedicated server processes, that implies 8000 MB (or 8 GB) of page table entries. So, for a 20 GB SGA, you have an extra 8 GB (approximately) of overhead.

Compare that to the same 20 GB SGA, implemented with hugepages. 20GB, using 2MB sized hugepages, means 10,240 pages, and correspondingly, 10,240 page table entries. Again, a page table entry is 8 bytes. So, for the same 20GB SGA, the page table overhead is only 80 kb. However, the other big savings, is that with hugepages, the page table is shared. So, you only need that one copy of the 80 kb page table for your 20 GB SGA, regardless of how many dedicated server processes attach to the SGA!

Finally, hugepages are locked into memory, and cannot swap. So, that also can add significantly to stability.

So, depending on the size of your SGA, the number of dedicated server processes, and the total amount of RAM on the server, it’s not hard to imagine that without hugepages configured correctly, you could suffer from significant performance problems.

As I said before, if you’re not using hugepages, you’re doing it wrong!

Addition – 09/27/2025 – How do I setup hugepages?

Ok, so how do you allocate hugepges? Well, first, you need to know how many hugepages you need. Go to My Oracle Support and search for a script, hugepage_settings.sh. Download and run it on your server with all the databases up and running. This will tell you exactly what to set hugepages to. Now, the easiest way to set up hugepages is to edit /etc/sysctl.conf and add a parameter, vm.nr_hugepages. Simply set it to the number of hugepages you wish to allocate.

vm.nr_hugepages = 10240

The above code will allocate 10240 2 MB hugepages. for an SGA of 20 GB. Now, in order for this to take effect, you must either reboot the server (easiest way, if you can) or run the sysctl command. If you run this command:

sysctl -p

The O/S will attempt to allocate the number of hugepages you specified in /etc/sysctl.conf. Note, however, if there is not enough contiguous memory available, it may not allocate the total number you specified. Pay attention to the output of the command. It will return the number of hugepages that were actually allocated. If it does not return the full num you requested, you may try re-running the sysctl command one or more times, but it’s not guaranteed to ever fully allocate the number requested. I strpmgly suggest simply rebooting ther server, if at all possible.

If you would like to see how many hugepages are currently allocated, you can run:

grep HugePages_Total /etc/meminfo

Finally, you should consider setting the USE_LARGE_PAGES database parameter to ONLY. The parameter has three valid values: OFF|ON|ONLY. When set to OFF, Oracle will not attempt to allocate hugepages, even if they are available. When set to ON, Oracle will allocate hugepages, but if an insufficient number of hugepages have been reserved, the remainder of memory will be allocated in standard small pages, which will result in a hybrid SGA made up of both small and huge pages. This can be problematic, and cause weird and unpredictable perfomance problems. For this reason, I strongly suggest setting:

SQL> ALTER SYSTEM SET USE_LARGE_PAGES = ONLY SCOPE = SPFILE;

and bounce the database. This will ensure that, at startup, either there are a sufficient number of hugepages to allocate the entire SGA or Oracle will fail to start. In this way, you are guaranteed that your entire SGA is allocated in hugepages.

One final point: It’s important to note that memory allocated to hugepages is reserved for use as hugepages only! So, it’s important to allocate an adequate number of hugepages to satisfy your SGA size, but no more. If you get sloppy and over-allocate, you could severly hamstring other processes of the server.

In conclusion, I’ll say it again (come on, say it with me!):

If you’re not using hugepages, you’re doing it wrong!