In LED-based virtual production, one of the most persistent challenges is getting foreground subjects to blend convincingly with the background — a problem caused by a combination of factors including lighting, art direction, camera settings, and the color calibration of the LED panels.
To address this, I designed an image code-value-based solution. It analyzes the foreground and background separately, extracts their color information, and then automatically adjusts the colors of both elements to achieve a better foreground/background composite.
The system draws on a stack of CV, color science algorithms, and ML techniques. By measuring the code-value difference between the foreground and background, quantifying that difference, and then iterating through machine learning, it converts that data into concrete adjustment parameters inside Unreal Engine — enabling fully automated color matching between the two layers.
Capabilities include:
- Automatically analyzing the color information of both foreground and background, computing the color deviation between them, translating those values into Unreal Engine adjustment parameters, and applying the corrections — all without manual intervention.
- Displaying the color information and adjustment results for both layers in real time, giving users an intuitive, perceptual window into the color-matching process as it happens.