In computing, the term “memory” refers explicitly to system RAM. While other forms of storage such as the CPU cache, HDDs, and SSDs are technically types of memory, they are typically referred to separately. System RAM has the critical task of ensuring that all the necessary data from the currently running programs are quickly available to the CPU when needed. You may think this should be relatively simple, with enough RAM allocated to each process. The reality, however, as it often is, is more complex than that, though.
Historical Memory Use
In early computers, system RAM usage was that simple. Applications requested the amount of RAM they needed. This approach ran into several issues, though. The first issue is cost vs. capacity. In those days, system RAM was costly, so computers tended to have minimal RAM. This causes issues if you have multiple applications running at once. It also causes issues if you have extensive programs that need more RAM than is physically available.
The second issue was one of efficient manageability. Large programs tended to need to manage their RAM usage. This added complexity to the software, making it more extensive and expensive to develop and providing more failure points. The final issue was one of complexity. Generally, the software would grab a continuous block of RAM. This makes it easy to define where everything is stored and makes prefetching instructions more efficient. If the software needs to request more RAM later, it’s entirely possible it will not be able to get the next piece of RAM as it may already be allocated to another process.
Paging Virtual Memory
Virtual memory solves all of these issues or at least facilitates the solution. Virtual memory adds an abstraction layer on top of the actual memory addresses. The operating system itself controls this abstraction. The operating system provides the requested virtual memory to an application in one continuous block. At least, it appears that way from the application’s view. The operating system completely hides the actual memory allocation from the application, allowing it to fit wherever there is space. This allows the operating system to manage the RAM as efficiently as possible for all programs.
The virtual memory space provided to a program can be expanded or contracted as needed. From the program’s point of view, this appears to be a simple expansion of its one continuous block of RAM. In reality, the operating system uses what space it has, wherever that may be.
Virtual memory incidentally offers a helpful security feature, memory isolation. By defining a distinct block of RAM for each process, it becomes easier to restrict access to areas of RAM belonging to other applications.
Paging is another critical feature enabled by the implementation of virtual memory. Virtual memory is split into units of allocatable memory called pages. There’s no actual requirement for virtual memory to have the same capacity as the physical RAM installed in the system. Instead, some virtual memory address space can point to disk storage.
If there is a particularly heavy RAM requirement, the operating system shuffles some of the least used memory pages into the disk storage portion of virtual memory. This can help to allow excessively RAM-heavy workloads to complete without hard crashing because of a lack of memory, which is particularly useful if RAM is very expensive.
Is Paging a Bad Idea?
Paging allows the operating system to trick the software into thinking it’s using system RAM when that data is sitting on disk storage instead. This can cause significant increases in latency and decreases in performance if that data is needed, as it first needs to be swapped into actual RAM. This generally requires something else to be swapped out. To minimize this effect, algorithms prefer moving the least used or least likely to be used data to disk storage.
Note: It’s important to understand that while virtual memory can include some disk storage space, that space cannot act as RAM. Programs are merely told it is RAM, while the operating system hides the fact that it’s disk storage and manages to transfer it back to system RAM when needed or if space frees up.
A page table is needed to translate the virtual memory address to the physical memory address. It is critically important that this paging table is never swapped out itself. If it were, the operating system would then be unable to find the page file to check. It may also be necessary for some other tasks not to be swapped out to disk storage from physical RAM. To enable this, a “Pinned” flag prevents certain pages from being swapped out to disk storage.
Thrashing
Under hefty RAM workloads, “thrashing” can occur. The computer spends unreasonable time swapping data to and from the disk storage. Thrashing prevents the computer from being able to perform much helpful work. Software tends to have a minimum set of pages it needs in actual RAM to make meaningful progress. If the combined minimum sets of pages of all running software exceed or approach the physical RAM capacity of the computer, the thrashing will occur.
Adjusting the code of the application can help to reduce its RAM requirements. However, this is only possible for first-party applications in active development. The only real solutions to thrashing are to install more physical RAM in the system or to reduce the number of applications running to minimize the total RAM requirement.
Conclusion
Virtual memory is an operating system abstraction of system RAM. Instead of assigning applications physical address spaces in RAM, the operating system gives them virtual address spaces. While this adds the requirement to track the translation of addresses, it offers multiple benefits. For example, the operating system can manage RAM allocation more efficiently. The operating system can also run more virtual memory than it has physical memory.
It does this by appropriating some disk storage space to store – unlikely to be used, but still required – data. This approach can appear to expand the overall available RAM of a system. It can undoubtedly prevent hard crashes from RAM exhaustion. It can, however, cause performance issues such as thrashing if it’s relied on too heavily. Virtual memory also requires hardware support in the CPU, which may need more die space than feasible on embedded processors.
