Photoshop Selection Strategies with Color Range and Refine Edge

By Seán Duggan Back to

Duggan_2_ND_2013

Making good selections is one of the essential core skills for using Adobe Photoshop effectively. This is especially true for those projects that require more localized precision than what is possible in either Camera RAW or Lightroom. In this article I’ll cover some important concepts and techniques for making accurate selections.

Manual vs. Assisted Selection Methods

Selection tools in Photoshop are roughly divided into two categories: manual and assisted. Manual methods involve tools such as the rectangular and elliptical marquees, the Lasso, Polygonal Lasso and the Pen tool that require you to carefully guide the cursor to define the selection. For certain types of selections, such as precise paths made with the Pen tool however, this manual and more methodical approach is often justified.

Assisted selection tools use qualities in the image such as color, contrast and tonal values, to create selections. The selection tools that make use of color or tonal differences are the Magic Wand, the Quick Selection Tool, the Magnetic Lasso and the Freeform Pen Tool (this tool has a Magnetic checkbox in the Options bar).

In addition to the aforementioned tools, commands such as Color Range and creating selections or masks based on the red, green or blue channels are also ways that you can leverage the existing color and tonal content of the image to assist in the creation of precise selections.

Color Range

Whenever there are distinct color or tonal differences between what you want to select and the rest of the image, using tools or methods that leverage these qualties can make your selection work easier.

The Color Range command (Select > Color Range) is very effective when the area you need to select is distinct from the rest of the image in terms of color or tone. It’s essentailly a deluxe version of the Magic Wand tool, but it does a much better job at creating transitional edges, partially selected areas, and it provides you with a preview of what the selection will look like before you press the OK button.

Under the preview window in the dialog the two but- tons determine what that preview shows: a mask-like preview of the selection or the image. The Selection Preview menu at the bottom of the dialog lets you choose how the image looks in the main document window. The grayscale preview can be useful for evaluating the quality of the selection along more intricate edges, and the black or white matte views are helpful if you will be compositing the selected area against a dark or light background.

By pre-selecting an area of the photo, you can determine what is shown in the Color Range preview.

The default Fuzziness value is 40 and this is usually a good value to begin with. Click on a color either in the main image window or the smaller preview in the dialog to begin the selection process. In the preview, white areas will be selected and black is not selected. Gray areas will be partially selected. You can expand the selected area by adjusting the Fuzziness slider. When this is adjusted to higher values it becomes more indiscriminate and the selection is harder to control. I prefer to leave the Fuzziness set to 40 (and sometimes less) and then use the plus or minus eye- droppers to add to or subtract from the selection mask as needed (the standard shortcuts keys of Shift for add and Option/Alt for subtract also work here).

The Localized Color Clusters check box limits the pixels that are selected by Color Range to within a certain distance from where you clicked (or Shift- clicked) to create a sample area. The distance is controlled by the Range slider, which is activated when you turn on this option. This can be useful if there are similar colors throughout the image but you only want to select the colors in a certain area.

Refine Edge

If you’ve never ventured into the Refine Edge dialog, then you’re missing out on one of the most powerful selection tools in Photoshop. Refine Edge is used once you’ve created a selection and is accessed either in the Select menu or via a button in the Options bar when a selection tool is active.

The controls in Color Range made it fairly easy to select the delicate structure of the red garment.

The Refine Edge dialog is divided into four sections: View Mode, Edge Detection, Adjust Edge and Output. Click the small disclosure triangle next to the View menu to see the different ways to view the selection in the main document window. Each view has a single- letter keyboard shortcut. Knowing these shortcuts makes switching back and forth between the different views much faster. For inspecting the quality of a se- lection edge I will usually use the On Black mode (B). The Overlay View (V) is similar to viewing a Quick Mask version of a selection and can be useful for some basic edits. The On Layers view (L) shows the selected area surrounded by transparency, as if it were a separate layer with a layer mask. If there are layers underneath the image, you’ll see how well the selection modifications are working in the entire composite.

Edge Detection

Edge Detection is truly the heart of the selection mojo offered by the Refine Edge dialog. For some selections, many of the edge adjustments you make can be handled entirely by the Radius slider and its accompanying Smart Radius check box.

• Radius slider. This determines the size of the selection border in which edge refinement occurs. Essentially, it tells Photoshop how far out from the current selection edge to look for additional areas that may need to be added to the selection. Low radius settings are best for objects with hard and sharply defined edges, whereas a larger radius setting works best for elements that have softer or more irregular edges.

The Refine Edge dialog.

• Smart Radius. Selecting this option will adjust the radius for the different characteristics found in a selection that has both hard and soft edges. It is less effective if the edge that you are selecting is uniformly hard or soft.

• Show Radius (J). This check box is located in the View Mode section. Turning on this option shows you the size of the radius in relation to the image.

Refine Radius Tools

To the left of the Radius slider and Smart Radius check box is a Brush tool icon. Click and hold on this to show the Refine Radius tool and the Erase Refinements tool. Both can be used to refine the edges of a selection by telling Photoshop where it should look for tricky edge transitions. These tools are useful if there were slight errors in the selection when you first entered the Refine Edge dialog. To change the size of either of the radius refinement brushes, you can use the Brush Size slider in the Options bar, or tap the left or right bracket keys to make the brush cursor smaller or larger.

Three of the seven ways for viewing a selection in Refine Edge.

• Refine Radius tool. This tool analyzes the area you brushed over and tries to determine where the edge should be. It does an excellent job of creating partial selections (gray areas) where there are subtle edge transitions, but is not as well suited for creating more distinct edges of fully selected pixels. If larger areas need to be added to the selection, it is probably best to exit the Refine Edge dialog and add them using one of the selection tools, and then re-enter Refine Edge to continue your edge enhancements.

• Erase Refinements tool. This tool removes any refinements created with the Refine Radius tool and restores the edge to what is was before you used either tool. It does not work like an eraser or as a regular brush when painting with the opposite color (a common task in mask editing); it merely restores the edge to what it was before the Refine Radius tool was used.

Adjust Edge

The Adjust Edge sliders let you modify the characteristics of the selection edge, but they do not analyze the selected pixels like the Edge Detection sliders do. Still, for some types of selections, these edge adjustments may be just what you need. One of the most useful is Shift Edge, which moves the entire edge of the selection inward or outward. Negative numbers will contract a selection, making it smaller. Positive numbers will expand the edge and make the selection bigger. This control can be useful in removing un- wanted background colors (“halos”) from the edges of a selection.

A: The selection mask seen in the Refine Edge dialog before refinement. B: The selection mask after brushing along the edges with the Refine Radius Tool.

Decontaminate Colors

The Decontaminate Colors option in the Output section replaces color fringes and color spill along the edge with the color of nearby pixels that are fully selected. The amount of the color replacement along the fringes is proportional to the softness of the selection edges. Due to the way this feature works, it cannot be applied to a selection or a layer mask on the original layer; a new layer must be created. The default Output option when Decontaminate Colors is selected is to create a new layer with a layer mask.

When this feature is turned on, the Amount slider becomes active. To evaluate the results, zoom in for a close view and try viewing the image on either black or white, depending on the color of the fringe. Decontaminate Colors often works well in conjunction with the Shift Edge slider as well as the Radius slider in the Edge Detection section.

The capabilities of Refine Edge created a very accurate selection of the sheep that was used to composite it with another image.


About the Author

Seán Duggan
SDuggan
Seán Duggan is a photographer, author and educator with a traditional, fine art photographic background combined with nearly two decades of extensive real world experience in the field of digital imaging. He is a co-author of The Creative Digital Darkroom (O’Reilly Media, 2008), Photoshop Artistry (New Riders, 2006) and Real World Digital Photography, 2nd Edition (Peachpit Press, 2004). You can see more of the Artifacts of an Uncertain Origin series at www.seanduggan.com.