Google RealFill Can Create the Perfect Picture From Reference Images

Google RealFill could be the tech giant’s latest bid to innovate artificial intelligence (AI)-powered image generation into a user-focused application. Recently, a research paper and a website by the name RealFill was spotted online which performs image completion and inpainting based on reference images and creates a target image. The company appears to have also applied for trademarks for its designed logo for the AI model-based product. Notably, the new AI model uses computer vision and pattern recognition algorithms and was trained using random masking techniques.

A Github page and a pre-print paper of the AI model were spotted recently by Android Authority. The publication also found trademark applications filed under the name of Google LLC in the US Patent and Trade Office (USPTO) and the European Union Intellectual Property Office (EUIPO) listings. Based on them, it appears the tech giant has not only reached the end of the research phase for the AI model but also has plans to introduce it as a commercial product.

According to its GitHub page, RealFill has been described as a “novel generative approach for image completion that fills in missing regions of an image with the content that should have been there.” Essentially, the AI model can scan through multiple images of a subject in the same setting and then use that reference to generate a pre-specified image. As a tool, it can be used when a user has clicked multiple pictures of an object but failed to get the perfect shot. AI can process those images and generate an image that does not even exist.

RealFill is a generative AI model that uses computer vision to understand the subject and the environment of reference photos and can process various aspects of it including technical specifications such as dimensions, colours, and shapes, as well as contextual understanding of the various objects. Using this information, it can then create a target image in a new plane and fill in details that might not have been present in the reference images.

While it is difficult to predict Google’s plans with RealFill, last year the company shipped a feature in the Google Pixel 8 series dubbed Best Take that could process multiple shots of a group photo and allow users to pick the best expression from each image to create the final photo. Compared to RealFill that feature appears very basic, but based on its application, the tech giant might be readying the AI model to turn it into a feature for Pixel phones.