Neural upscaling and restoration

This page as Markdown Whole site as Markdown

Neural upscaling and restoration use trained models, running on a Mac’s GPU, to enlarge or repair a game’s image. Ordinary upscalers fill in new pixels by fixed rules. A neural model, by contrast, is trained on many images and learns what images tend to look like, such as how a line in an anime drawing continues, what grass looks like up close, or which fine patterns are noise rather than detail. Given a small or damaged image, the model redraws it from what it learned.

MacLauncher runs these models on your Mac’s GPU through MLX and Metal, inside the same graphics chain as every other effect. They need an Apple Silicon Mac with macOS 14 or later. Intel games running translated under Rosetta or CrossOver can use them as well, because a small Apple Silicon helper runs the model on their behalf.

Upscaling drawn art

Neural upscalers do especially well on visual novels and anime-style games. This scene from CLANNAD was reduced to 640×480 and then doubled:

Nagisa under the cherry blossoms in CLANNAD, upscaled 2× from 640×480 (Bilinear 2×) — Real-CUGAN redraws line work and flat color cleanly where bilinear filtering only blurs.

Nagisa under the cherry blossoms in CLANNAD, upscaled 2× from 640×480 (Real-CUGAN 2×) — Real-CUGAN redraws line work and flat color cleanly where bilinear filtering only blurs.

The lighter shader upscalers from the shader collections aim at the same kind of image and cost far less:

The CLANNAD scene upscaled with Real-CUGAN 2× Conservative — Real-CUGAN 2× (Conservative), a neural model.

At 4× the differences grow. This is the same scene, starting from only 320×240:

The CLANNAD scene upscaled 4× with Real-ESRGAN Anime — Real-ESRGAN Anime 4×.

Upscaling everything else

Real-ESRGAN’s General model handles 3D games and other rendered or photographic images rather than line art:

Geralt from The Witcher 3, upscaled 4× from 480×270 (Bilinear 4×) — Geralt in The Witcher 3: Wild Hunt, reduced to 480×270 and enlarged 4×. Real-ESRGAN rebuilds hair, beard and chain mail where bilinear filtering only blurs them.

Geralt from The Witcher 3, upscaled 4× from 480×270 (Real-ESRGAN General 4×) — Geralt in The Witcher 3: Wild Hunt, reduced to 480×270 and enlarged 4×. Real-ESRGAN rebuilds hair, beard and chain mail where bilinear filtering only blurs them.

On pixel art, the same model behaves very differently. It treats the pixels as detail to smooth over and redraws the scene with soft, outlined shapes, which is why pixel-art games usually stay with Nearest or a pixel-art scaler:

Madeline among wind-blown flags in Celeste, enlarged 4× from 320×180 (Nearest 4×) — Celeste at its native 320×180, enlarged 4×.

Madeline among wind-blown flags in Celeste, enlarged 4× from 320×180 (Real-ESRGAN General 4×) — Celeste at its native 320×180, enlarged 4×.

Cleaning up an image

Not every model changes the size. NAFNet restores an image at its original resolution, with one model for noise and another for blur:

A close-up of Geralt from The Witcher 3 with heavy added noise (Noisy) — NAFNet's SIDD model removing heavy added noise.

A close-up of Geralt from The Witcher 3 with heavy added noise (NAFNet SIDD) — NAFNet's SIDD model removing heavy added noise.

A close-up of Ciri from The Witcher 3 with added motion blur (Motion blur) — NAFNet's GoPro model reversing added motion blur.

A close-up of Ciri from The Witcher 3 with added motion blur (NAFNet GoPro) — NAFNet's GoPro model reversing added motion blur.

The models

Model	What it does	Output size
Real-ESRGAN	Anime and General upscalers. General has a paired denoising model, and when both are installed, the Denoising control blends between them.	4×
Real-CUGAN	Upscaler trained on anime, in Conservative, No denoising and Denoise variants.	2×, 3× or 4×
RTX Video Super Resolution	NVIDIA’s video upscaler, through MLX-DLSS.	2×
DLSS neural rendering	The DLSS 5 neural rendering model through MLX-DLSS. It redraws the image toward a photoreal look, blended in with Intensity, and can keep image history with Temporal reconstruction.	Same size
DLSS SR (image)	DLSS super resolution working from a sequence of finished images.	2×
NAFNet	SIDD (denoising) and GoPro (deblurring) restoration models.	Same size

MacLauncher’s other neural models are RIFE and MLX interpolation for frame generation, and Depth Anything V2, which estimates depth from a flat image for the estimated depth fog effect.

Real-ESRGAN and NAFNet build on the existing Real-ESRGAN MLX and NAFNet MLX ports; Real-CUGAN runs its original models on MLX.

Getting models

MacLauncher doesn’t include model files. Runtimes → Graphics → Neural models lists the supported downloads: Install downloads one, and Import model… adds a model you already have. Installed models work offline and are shared by every game.

Each neural row in the chain has a Model menu. Automatic chooses an installed model for that row’s operation, starting with the default you set for the operation in Neural models. You can also pick a specific model for one row. If a row’s model is missing, the row shows Model needed and the rest of the chain keeps running.

Keeping it fast

A neural model’s cost grows with the number of pixels it processes, so the usual approach is to give it a smaller image. Put a Downsample row before the model, or lower the Game resolution, and fill the window afterward with a fast upscaler such as MetalFX. Building a chain shows a complete example. If the model still can’t keep up with the game, frame generation can fill in the missing frames.

The graphics guide’s neural models section lists every download and setting, and Neural models and packages describes the model architectures and import formats.