Invented by KIM; Kyuwon, BANKAR; Akshay Janardan, SEN; Mrinmoy, SUBRAYA; Ravikiran Kundapur, KIM; Hyunsoo
Today, many people love to highlight one part of a photo in color and keep the rest in black and white. This makes the subject stand out and adds a creative touch. But what if you want to bring the rest of the photo back to life, or you want to fix mistakes from earlier edits? A new patent application offers a clever way to do just that. In this article, we’ll walk you through the background of this technology, explain the science behind it, and then break down exactly what makes this invention special and useful.
Background and Market Context
To truly appreciate this invention, it helps to know a little about how people use color in images today. On nearly every phone or computer, you’ll find apps that let you edit photos. One popular effect is called “color pop.” This is when most of the image is turned to black and white, but one person or object stays in full color. It’s eye-catching, and it’s everywhere—from social media to advertising.
People love this effect because it draws attention to what matters most in a picture. Maybe it’s a bright red balloon, a green apple, or a child’s yellow raincoat. By making the rest of the image plain, the colors seem even brighter.
But there’s a problem. Once you turn most of a photo to black and white, it’s hard to bring the original colors back. Sometimes, you might want to change your mind or fix a mistake, but the color details are already lost. Other times, you want to colorize the background without changing the special area you wanted to highlight. Current tools don’t make this easy. They might try to guess the colors, but results often look fake, flat, or just wrong.
The demand for smarter photo tools is growing fast. People want to create fun effects, but they also want to fix them or adjust them later. Businesses want to use these images in ads, posters, and online content. As photos become more central to communication, editing tools need to become more intelligent and flexible.
This is where the new patent application steps in. The invention is designed for modern electronic devices like smartphones, tablets, and computers. It aims to make it easy to tell if an image is a “color pop” (partly color, partly black and white), and then—if you want—restore color to the black and white areas, all while keeping the special, colorful part unchanged. The system works automatically, so anyone can use it, not just professional designers or photo editors.
Scientific Rationale and Prior Art
Let’s talk about the science behind the invention and how previous solutions have tried to tackle this problem.
First, computers need a way to figure out which parts of a photo are in color and which are not. This is harder than it sounds, because sometimes a picture looks gray, but it’s really just a very muted color. Other times, a photo might have a color “tint” over the whole thing, giving it a certain mood (like sepia or blue tone), even if it’s not black and white.
Earlier techniques often relied on simple rules. For example, they might look for pixels where the red, green, and blue values are equal—thinking those are gray. Or, they might just count how many pixels have strong color differences. But these methods have problems. They can’t always tell the difference between a real black-and-white image, a tinted photo, and a partly colorized image. Sometimes, they make mistakes, such as coloring parts that should stay gray or dulling the vivid colors in the main subject.
Other solutions use machine learning to guess what the original colors might have been, but these guesses are often poor if the background is truly black and white. The unique, rich colors in the highlighted area can get lost or become less detailed, making the whole effect less dramatic.
Some software tries to “paint” color back into black-and-white areas by using colorization techniques. These often use color histograms (charts that show how often each color appears), neural networks, or even manual user input. While these can help, they usually don’t preserve the special, colorful part of the image or blend the colors naturally. Often, the results have strange boundaries, smudges, or other visual errors, known as “artifacts.”
There is also a challenge with “tinted” images—photos where a filter gives the whole image a certain color cast. For instance, an old photo might have a sepia tone. Simple algorithms can’t tell if that’s a colorized photo or just a color-cast black-and-white image.
In summary, current methods for detecting and fixing partially colorized images are either too simple, make too many mistakes, or require too much effort from the user. They may also ruin the vivid colors that make “color pop” images so fun in the first place. There is a clear need for a smarter, more automatic tool that can handle all these cases and work well with modern devices.
Invention Description and Key Innovations
Now let’s dive into what this new patent application offers and why it’s a big improvement.
The invention describes an electronic device—like a phone, tablet, or computer—with special instructions stored in its memory. When you open a photo, the device’s processor uses these instructions to do several smart things, all aimed at making image editing easier and more natural.
Here’s how it works in simple terms:
1. Detecting Partially Colorized Images: The device first checks whether the photo is a “color pop” image. It does this by measuring two kinds of “colorfulness.” One is for the whole image, and the other is for specific spots (like the main subject). If the overall image has low color, but one area has high color, it’s probably a partially colorized image.
To get more precise, the device divides the image into parts and checks each one. It uses something called “semantic segmentation” to figure out what parts of the image are important (like a person, pet, or object). It then calculates how colorful each area is. If one area really stands out, it’s marked as special.
2. Handling Tinted and Grayscale Images: The system is also designed to tell the difference between a true black-and-white photo, a tinted photo (with a color cast), and a partially colorized image. It does this by looking at the numbers for red, green, and blue in each pixel and seeing how they relate. If all the numbers are similar, it’s likely grayscale. If the numbers follow a certain pattern (like higher red for sepia), it’s likely a tint. This helps avoid coloring areas that shouldn’t be colored.
3. Creating a Colorized Background: If the device finds that an image is partially colorized, it can automatically colorize the black-and-white areas. Importantly, it does this without changing the colors in the highlighted area. The device uses color histograms and other predictors to guess what colors would look natural in the background. It then creates a new color image for the whole photo.
4. Blending and Weighting: One of the biggest problems with colorizing is making the final image look natural. If you just slap color onto the background, you can get ugly lines or smudges where the colored area meets the black-and-white area. This invention solves this by using a “weighted sum.” For each pixel, it figures out how much to trust the original image and how much to use the new color. The weights depend on how close the pixel is to colored areas, how colorful it is, and several other factors.
For example, if a pixel is in a clearly gray area far from any color, it will get more of the new color. If it’s close to the special colored area, it will blend carefully to avoid making the border look strange. The device also lets users adjust settings like how sensitive the color detection is, how big the blending area is, and how strong the colors should be. This gives more control to the user, but the default settings work well for most cases.
5. Preventing Artifacts: The system is designed to stop common problems like smudges, bleeding, or unnatural borders. By adjusting blending weights and carefully measuring color differences, it keeps the main subject’s colors bright and clear, while smoothly adding color to the background. This results in images that look much more natural and professional.
6. Fast and Flexible: The invention can work on high-resolution images, but it can also speed things up by working on smaller, lower-resolution versions and then applying the results to the full image. This makes it fast enough for real-time editing on phones and tablets.
7. User Interface: The device can show sliders or indicators on the screen, letting users control where and how colorization is applied. For example, you can move a line to see the before-and-after side by side, or adjust sliders to change how sensitive the effect is. This makes the tool friendly for beginners and powerful for experts.
8. Smart Machine Learning: The system can use artificial intelligence to learn what people like. It can recognize faces, pets, or other important objects in the image and adjust the effect to make them stand out. Over time, it can get better at guessing which parts of an image should be colored and which should stay gray.
In summary, this invention brings together several clever ideas. It automatically detects if a photo is partly colorized, tells the difference between black-and-white and tinted images, restores color to the background without changing the subject, blends everything smoothly, prevents ugly artifacts, and gives users control—all in a way that works on modern devices. It’s a big leap forward for photo editing, making it easier for everyone to create eye-catching images or fix old edits.
Conclusion
Photos are a big part of how we share memories and tell stories. The “color pop” effect is popular, but until now, fixing or reversing it was tricky. This new patent application describes a smart, easy-to-use system that brings color back to black-and-white parts of a photo, all while keeping your special subject bright and bold. It uses smart ways to measure color, blends new colors in smoothly, and stops common mistakes that can ruin a photo. The result is a tool that lets anyone, from casual users to professionals, create stunning images with just a few taps. As photo editing becomes more important in daily life, inventions like this one will make creative tools more accessible and powerful for everyone.
Click here https://ppubs.uspto.gov/pubwebapp/ and search 20250218064.