Region Labeling Tool (October 1998 version)

The Hierarchical Image Segmentation algorithm produces a set of image segmentations that could potentially be very useful in many Earth science applications. However, most Earth scientists want a single segmentation to work with, rather than a set of segmentations. The problem, then, is selecting the appropriate image segmentation from the hierarchical set. At times this selection might be as simple as determining which single segmentation out of the hierarchical set that is suitable for the application. Most of the time, however, a single segmentation synthesized by combining segmentations from a number of the hierarchical segmentations would be more suitable.

An interactive "Region Labeling Tool" has been developed to facilitate the combining of segmentations from a number of the hierarchical segmentations into a single segmentation. This tool was developed as a so-called "xvroutine" under the Software Developer's Kit from the Khoros Pro 2000 software package from Khoral Research, Incorporated.

We will describe the Region Labeling Tool by working through an example labeling of the Landsat TM data set displayed in Figure 1. A threshold of 1.01 on the "convergence factor ratio" in the hierarchical image segmentation program produced a 20 level segmentation hierarchy (see Table 1). These hierarchical image segmentations are taken as an input into the Region Labeling Tool, along with an RGB representation of the original imagery data (such as shown in Figure 1.) and any ground reference data (formatted in image form and registered to the original imagery data).

Figure 1. RGB Representation of a Landsat TM scene.

Upon starting up the Region Labeling Tool, the analyst is presented with a control panel (see Figure 3). The analyst can select to display the RGB Image, Current Class Labeling, and/or Reference Class Labeling (e.g., ground reference data) if available. The initial class labeling is all "0" (unknown), with the current region selected from the segmentation hierarchy highlighted in purple (the region initially selected from the segmentation hierarchy is by default the region containing the image pixel at location (0,0) in the finest segmentation, i.e. level 0, from the segmentation hierarchy). In this example, the initially selected region (not shown) has region label "1" and contains 18553 pixels out of the 65536 pixels contained in the image. As shown in the Region Label Tool Informational Output display, the value of the criterion value (a measure of dissimilarity of the region mean versus the original image values) is 4.088795e-02 and the region mean vector is (68.8, 27.0, 21.0, 106.2) for the "0th" segmentation level. For the analyst's convenience, the number of pixels, criterion value and mean vector is also shown of the other levels of segmentation where region label "1" is present.

Figure 3. Region Labeling Tool control panel.

Inspecting the "Region Label Tool Informational Output," the analyst sees an number of steps in which the criterion value makes a significant jump. These are from level 19 to 18, level 18 to 17, level 14 to 12, level 10 to 6, level 6 to 5, and level 1 to 0. Examining the spatial coverage of the highlighted region at these steps, and comparing them to the RGB image and the ground reference data, the analyst determined that segmentation level 5 (or equivalently, level 4) is an appropriate level to label the highlighted region as "Wooded." After setting the "Segmentation Level" to 5, the analyst can easily label the region by clicking on the "Label Region" button on the control panel, and clicking on the "Label" button to the left of the green square, which has been given the name "Wooded" by the analyst as shown in Figure 4. Figure 5 shows the labeling to this point (with the "wooded" region colored green).

Figure 4. The analyst can easily label the selected region with
a color and an alphanumeric phrase using the Label Region panel.

Figure 5. Labeling of the first selected region as
"wooded" (colored green) with the Region Labeling Tool.

The analyst next decides to label the major roads, and selected a pixel in the middle of U. S. Highway 301 (running diagonally from center top to lower left). The analyst decided that segmentation level 8 provided the most complete labeling of the roads with minimal mislabeling of non-road pixels. The result after labeling the region so selected as "Urban/Residential" (yellow) is given in Figure 6.

Figure 6. Result after labeling U. S. Highway 301 (running diagonally from
center top to lower left) and other similar regions as "Urban/Residential" (yellow).

To label more of the roads and similar areas as "Urban/Residential," the analyst selected a pixel in a road in the southeast edge of the city of St. Charles. The analyst found that segmentation level 10 gave the best tradeoff between complete labeling of roads and urban/residential areas and mislabelings. The result of labeling the region so selected as "Urban/Residential" (yellow) is shown in Figure 7.

Figure 7. Result after labeling additional roads and
other similar regions as "Urban/Residential" (yellow).

A few small areas that have been label "Urban/Residential" should have been labeled as "Grasslands/Pasture." This can be corrected by selecting one of the mislabeled pixels, and selecting a segmentation level in which all of the mislabeled pixels in an area are highlighted. Then the analyst can click on the "Extract ROI" button on one of the image display panels (e.g., the Current Class Labeling Display or Zoom) and draw a circle around the mislabeled pixels. This will cause only the mislabeled pixels to remain highlighted. Then the mislabeled pixels can be relabeled to the correct class with the "Label Region" panel. The result is shown in Figure 8.

Figure 8. Some mislabelings corrected using the "Extract ROI" facility.

After a few more labelings with the Region Labeling Tool, the analyst can straightforwardly label some agricultural fields (colored red), some small ponds, and water in a water treatment plant (mid-right) (colored blue) and a some rock quarries and a land fill (mid-lower right) (colored white).

Figure 9. Partial labeling of the image into five classes: Agriculture (red),
Wooded (green), Water (blue), Urban/Residential (yellow), Grasslands/Pasture (turqouise),
and Bare Soil/Rock Quarry/Land Fill (white) using the Region Labeling Tool.

This process can be continued to produce a complete labeling of the image, if so desired. See Figure 10 an example of such a labeling. Compare this labeling to the original Landsat TM data, shown in Figure 1.

Figure 10. Completed labeling of the image into six classes: Wooded (green),
urban/residential (yellow), pasture/grassland (turquoise), water (blue)
bare soils (white), and agriculture (red) using the Region Labeling Tool.