Remove Module [mocha Pro]

The remove module is designed to remove foreground objects from your footage to create clean plates. A removed object can be anything: wires, poles, signs, people and so on. The only requirement for a good removal is enough background to use.

To understand how mocha Pro removes a foreground object consider how you would do it yourself. A common method is to select a source image where the foreground object does not obscure the background region you are trying to paint in the target frame. You would then clone or otherwise copy the pixels from the source frame to the target frame. If the background is not in the same position in the two frames you would have to track the patch of pixels into the frame, and if the lighting is different between the two frames you would have to adjust the brightness of the patch.

The remove module attempts to do this for you automatically using a method called motion keying. As it has tracked the background areas you have defined, it can move pixels of these background areas from one frame to another. Hence, if a part of a background area is obscured in one frame, mocha Pro can search the rest of the clip for a frame where the obscured background area is visible, and move the pixels correctly into the target frame. During this process mocha Pro will evaluate and compensate for the lighting differences between the source and the target frame.

Object removal requires a few simple steps to work correctly. The key to a good remove is adequate, tracked background and a correctly masked foreground.

1. Identify your foreground and background planes

First of all, scrub through your timeline and identify the planes of depth within your shot. Background often consists of multiple planes of motion, and there can also be obstructing foreground material. For example, in the shot below, the sign is in front of the ground, and the car door is in front of (but not obscuring) the sign.

2. Track and roto your foreground elements

We handle the foreground first as we need to mask them off from the background plane and from each other. You don’t actually need to worry too much about tracking the foreground: If the object is untrackable for whatever reason, just hand-animate a spline around it. You absolutely must track an object if it is going to be used as a background pass to replace foreground. For more information on effective tracking, please refer to the tracking section of the documentation.

In the example below, we have tracked and rotoscoped the sign and the door frame, which would otherwise obscure the ground plane.

3. Make sure your layer order is correct

Layer order matters in mocha. The closer to the camera an object is, the higher in the layer stack it should be. Far background should be on the bottom layer, foreground objects should be on top, in order of closeness.

In the example here, we show the door layer is above the sign layer, and the sign is above the ground layer in the stack.

4. Track your background plane

Once you have your foreground objects masked out across the timeline, draw as big a shape as possible to cover as much of the possible area you want to remove. Make sure the background layer is under the foreground layers, and track. For multiple planes of background, such as a floor and a wall, you will have to track two separate planes of motion. You must specifically track the background layer in order for the removal to work, even if you are working with a locked-off shot.

Here we can see the background drawn on a layer at the bottom of the layer stack, covering the large area of grass.

5. Select the layer you want to remove and adjust your removal parameters

Switch to the remove tab and adjust your parameters (see Remove Parameters below). Key things to focus on are your search range, whether or not you need to create or adjust a cleanplate, and if you need to use an illumination model.

We recommend trying a remove with Illumination modelling set to None first. This is the fastest setting for removals, and can quite often be all you need if there is not much light difference in the background over time.

6. Render

Once your layers are ready and the parameters are set, click the the render button or render forward.

A note on time: Removal can be a slow process depending on the resolution of the footage and the amount of frames you are removing on. You can optimise your search range (see Remove Parameters below) but keep in mind that removal quality can vary.

One or more background layers must be defined that include all areas that the foreground object moves across during the shot.

To get the result you expect you should observe the following rules:

Here we provide a breakdown of each remove tool parameter and how to use it effectively.

When changing parameters in mocha Pro the change only affects the currently selected layer. To change the parameters of a layer other than the currently selected layer that layer must first be selected.

Input Tab

Here you specify the input clip for the remove process and any cleanplates you wish to import.

Input Clip

You can choose from any of the mocha Pro result clips to be used as the source clip to fill the requested foreground layers, instead of the default, which is to use the originally imported clip as the input clip. This can be useful if you have to do multiple passes to get an effective remove.

Cleanplates

Here you can import cleanplates to replace frames in your footage. If you don’t have enough background to use in your shot, importing your own cleaned up version of a frame can assist the remove tool greatly.

To import one or more cleanplates:

NB The All option only applies when you are using a single cleanplate.

If you import two or more cleanplates, mocha Pro will try to guess the frame numbers from any numbering in the file name. When using the cleanplates between those frame numbers, mocha Pro will blend the nearest two cleanplates to produce a smooth transition through the clip.

If you want to change the cleanplate settings after exiting the cleanplate window, click on Edit... You would need to do this if you are using the frames on a new machine where the cleanplates are stored in a different location, or just to add new cleanplates. If you re-import files with the same name but different directory to existing cleanplate files, mocha Pro will update the file to the new directory.

Create (Cleanplate)

One useful option within mocha Pro is to create a cleanplate from the currently viewed frame.

To do this, make sure you are viewing the frame you want to use and click on the Create button. This will create a clip containing the frame you are viewing, and set the Cleanplates clip to the new clip. You can then touch up this cleanplate from your Results folder.

When you save your edits, it will automatically be updated in mocha Pro to be used in the cleanplate list.

Use Cleanplates Exclusively

If this option is checked, only the cleanplates will be used by Remove to remove the pixels in the selected layer. If it is unchecked, the normal Remove process will be used, pulling in pixels from other frames in the input clip. The cleanplates will then only be used to remove the remaining pixels.

Output Tab

This assigns an output clip for the removal render. You can create new output clips if needed here.

Search Range

Used to specify which frames should be used when removing a layer. The First Frame, Last Frame, # Frames Before and# Frames After settings can be keyframed.

Step

With this option you can specify that not every frame in the reference range is to be used. Setting it to three, for instance, means that only every third frame will be accessed. This feature can speed up the removal process for large projects, especially film projects, which are very memory intensive.

Illumination Model

This specifies how to model changes in illumination.

Smoothing Level

This controls the amount of smoothing applied in the Interpolated model. Increase the value if there are artefacts which might be resolved with more smoothing, either spatial variations or temporal variations.

Dissolve Width

Select either Blend or Randomize and increase the width value to reduce artifacts which sometimes can be seen when illumination modelling fails. This option causes pixels from different frames to be dissolved into each other to avoid tearing artifacts. Blend uses alpha blending from the replaced areas to either the original pixels or the recently replaced areas. Randomize mixes original and replaced pixels in a random way to achieve a similar effect.

3D Compensation

3D compensation can be switched on to try to remove artifacts due to the background layer not being planar. For example, if you have tracked a background that has subtle parallax it can cause removal in other frames to look incorrect. 3D compensation attempts to model the parallax change in the target removal frame.

Flood Fill

If part of the missing background has not been found anywhere in the clip, and the foreground object therefore cannot be completely removed, Flood Fill can be switched on to fill the remaining region using a flood fill method. This is especially useful when it is the matte you are interested in, as you then don’t care too much about the quality of the removal but require that the foreground object is completely removed to avoid holes in the matte. The Smoothing Level should be increased if you result is not as smooth as it should be or there are temporal variations in the results.

Some of the object is still visible after removal.

The replaced background region is brighter/darker than the surrounding image.

The background patches don't line up with the surrounding image.

Remove is slow.

If you have a long clip, especially working with HD or film, Remove can be slow because it has to search over a large number of images with a large memory footprint. Remove is the most memory intensive module in mocha Pro, and it will always benefit the performance to add more memory. If mocha Pro can fit all the images it needs in memory, performance will be dramatically accelerated when rendering Remove in multiple frames, because it will minimize the amount of disk accesses. Your aim should be where possible to change the settings to achieve this. Within mocha Prothere these options: