What Is Upscaling?

Many modern TVs now have a 4K resolution, that’s 3840 x 2160 pixels, although the “Full HD” or 1080p standard (1920 x 1080) is still very common too. One issue that you may have identified with these high-resolution screens, is that not all the content you watch is at 4K, especially older content. To be able to display lower resolution images and video on high-resolution screens TVs perform a function called upscaling.

A 4K screen is exactly four times the resolution of a 1080p screen, this means that if you wanted to do very simple upscaling you could just display each pixel of a 1080p video on a square of four pixels on a 4K screen. Unfortunately, there are two main issues with this approach. The first is that this would only work with certain resolutions where the content is being scaled up in exact multiples of four. The second issue is that it won’t look particularly good, each pixel of the content would be bigger than intended causing the image to appear blocky.

The real way upscaling works is by performing analysis with an interpolation algorithm. The interpolation algorithm works by interpreting the image as is at the higher resolution. This generally involves the device having to infer what the colours would be between multiple pixels. For example, if two pixels in the original image were red and the upscaled image’s pixel lies between those two, then it’s safe to assume it too should be red.

The difficulty comes with inferring the pixel colour when there is a colour gradient. For a smooth change in colour such as in a picture of a sunset, it’s relatively easy to infer the correct colour for an upscaled pixel. If there’s a sudden difference in colours, such as with dark clothes against a light background, then it’s a lot more difficult to accurately determine the correct pixel value.

To help determine the colour a pixel should be, a smoothing algorithm is used that determines the colour of a pixel by looking at the pixels around it and tries to blend the difference in colours. A sharpening algorithm is also used to help improve the clarity of the image.

The quality of the result

When upscaling is done well, it can be difficult to tell apart from content originally recorded at a higher resolution, the main giveaway is the lack of fine detail. The lack of detail can be relatively apparent specifically for people, details such as skin imperfections can often end up being smoothed out, which can cause a waxy or matte look.

Upscaling is almost always performed when the content is played, generally by the TV or other playback device. In the case of the remastering of older films, upscaling can be performed before playback and then combined with more time-consuming processing techniques to clean up the quality as much as possible.

Upscaling can only be taken so far. The Hollywood concept of “zoom and enhance” really doesn’t work, you can’t upscale an image to see and identify a distant figure as the image quality just isn’t there to work from.