Processing of the Test Data and Comparative Methodology

Many parameters are influential in the practical use of the filtering models summarized in the previous section. The correct determination of these parameters directly affects the filtering results. Hence, the aim of this study was determining the most suitable filtering algorithm and appropriate parameter values for each test area. Therefore, different combinations of parameters for each filtering algorithm were empirically tested, and optimal parameters were determined in a two-step procedure. First, qualitative analysis was performed by visual examination between the filtered DTM surface and reference DTM surface. If there were significant errors on the surface, detailed statistical analysis was not performed, and the filtering process was repeated by changing the parameters. For detecting a significant error, a visual examination was utilized between DTM surfaces and shaded relief maps generated by the reference and filtered point clouds. Based on visual examinations, obvious errors could be detected by investigating topographic changes, such as removing or preserving obvious features like mounds or buildings. If there was an apparent difference in the surfaces, the second step of the process was not initiated. However, if there was no significant difference between the filtered and reference surfaces, the second step was carried out.

As indicated above, there are different methods used for creating a reference DTM, such as employing a well-known algorithm (ATIN embedded in Terrasolid) for gathering ground data by field measurement (GNSS, Total Station). However, none of the filtering algorithms performs ideally in all terrain surfaces;

moreover, field measurements can only be performed in limited areas, and they are highly time-consuming and costly (Julge et al., 2014). For selecting the correct ground data, we used manual classification based on hand filtering via the detailed examination of satellite photographs, intensity views, and lidar point clouds. The reference DTM was generated using these correct ground data.

In the second step of this study, qualitative and quantitative analyses were conducted for different DTM surfaces (generated by extracting the filtered DTM from the reference DTM). Qualitative analysis was performed via the visual examination of different DTM surfaces. Quantitative assessment was done by

RECENZIRANI ČLANKI | PEER-REVIEWED ARTICLESSI | EN calculating statistical values, such as the minimum, maximum, RMS, mean, and median of different DTM surfaces. RMS is the most important statistical value for determining the parameters of filter-ing algorithms (Aguilar and Mills 2008; Liu et al., 2015). The RMS value was calculated accordfilter-ing to discrepancies between Z values of the reference grid nodes (Z_ref) and those at the same locations (x, y) of filtered DTM grid nodes (Z_flt-alg) in these two surfaces:

∆Z(x_i , y_i)  Z_fltalg(x_i , y_i)  Z_ref (x_i , y_i), (1) RMS 1

n

i Zi

n

= ∆

=

∑

_{, (2)}

Figure 2: The workflow applied to determine appropriate filtering parameters (Adopted from Sulaiman et al., 2010)

RECENZIRANI ČLANKI | PEER-REVIEWED ARTICLESSI | EN

where n is the number of observations and i = 1, 2, …, n. RMS values can be affected by gross errors.

The error distributions at certain intervals were determined, as shown in Table 4. A significant part of the results consists of the yellow component in the range of −20 to +20 cm, which is the acceptable error range. Positive errors were divided into three different ranges, which were as follows: 0.2 to 0.5 m, 0.5 to 1 m, and >1 m (areas where commission errors occurred). In addition, negative errors were divided into three different ranges, specifically, −0.2 to −0.5 m, −0.5 to −1 m, and <−1 m (areas where omission errors occurred). As a result of all these qualitative and quantitative analyses, the most suitable filtering algorithms for all the test areas and parameter values of this algorithm were determined. The parameter values for each filtering algorithm are shown in Table 2.

Table 2: Determined filtering parameters for all test area (Units are in meters) Adaptive TIN (ATIN) Cell Size Z Difference

(m)

Init TriGrid Size (m)

Tile X

Width (m) Tile Y Width (m) Tile Buffer

First Test Area 0.75 0.3 10 200 200 20

Second Test Area 0.6 0.2 25 200 200 20

Third Test Area 0.5 0.2 10 200 200 20

Fourth Test Area 1 0.8 10 20 20 2

Progressive Morphology (PM) Cell Size Window Base(m)

Maximum Local Slope Width(m) Height(m) Search Radius (m)

Expanding Window (ETEW) Width(m) Height(m) Slope (deg) Loop Times

(IPF) Cell Size Z Difference Outlier

Tolerance

Init Window Length

First Test Area 1.5 0.8 0.1 3

Second Test Area 1.5 1 0.5 20

Third Test Area 2 0.3 0.3 20

Fourth Test Area 2 0.2 0.3 20

RECENZIRANI ČLANKI | PEER-REVIEWED ARTICLESSI | EN 4 ANALYSIS OF COMPARISON RESULTS

Filtering was performed for all four test areas using six different filtering algorithms with specified pa-rameters, as shown in Table 2. The test studies indicated that all the filtering algorithms generated com-mission errors (incorrectly classifying nonground points as ground points) or ocom-mission errors (ground points mistakenly classified as nonground points) at different levels. The ∆Z value of the different DTM surfaces between the filtered DTM and reference DTM are shown in Figures 3 and 4. These ∆Z values were separated into seven different ranges. Each range was labelled in a different colour (positive values indicate commission errors and negative values indicate omission errors), and the distributions of these errors were visually examined. The statistical values of the different DTM surfaces are listed in Table 3.

The error distributions are shown in Table 4.

Table 3: The statistical values calculated for all test area (Units are in meters) Statistical

Measures ATIN-Ref MLS-Ref ETEW-Ref PM-Ref MCC-Ref IPF-Ref

First Test Area Max 3.47 1.77 12.2 3.25 7.9 2.33

Min -5 -3.2 -5.42 -3.54 -4.39 -4.37

RMS 0.17 0.17 0.17 0.16 0.21 0.14

Mean 0.045 0.048 0.018 0.04 0.087 0.03

Median 0.017 0.018 0.004 0.016 0.048 0.017

Second Test Area Max 14.11 9.40 8.68 7.04 18.23 5.36

Min -5.63 -5.98 -5.98 -5.68 -6.29 -5.97

RMS 0.51 0.57 0.55 0.47 0.86 0.33

Mean 0.11 -0.019 -0.024 0.12 0.1 0.054

Median 0.013 0.002 0.004 0.017 0.014 0.025

Third Test Area Max 0.52 0.57 0.56 0.52 1.45 0.40

Min -0.48 -0.48 -0.54 -0.48 -0.46 -0.55

RMS 0.077 0.079 0.083 0.075 0.088 0.074

Mean -0.016 -0.01 -0.042 -0.018 0.02 0.01

Median -0.019 -0.01 -0.043 -0.02 0.013 0.008

Fourth Test Area Max 3.65 9.41 7.37 2.98 3.24 2.86

Min -8.00 -12.47 -10.51 -7.75 -8.92 -15

RMS 0.22 0.27 0.17 0.18 0.18 0.25

Mean 0 -0.06 -0.04 -0.029 0.03 -0.004

Median -0.03 -0.07 -0.047 -0.04 0.015 -0.002

Since the first test area was composed of complex surfaces (hills, low and high vegetation, flat areas), multiple commission errors were observed to occur in the regions with sudden elevation or slope changes. Several tree measurements in a vegetation area were not removed, and commis-sion errors occurred at different rates. The minimum and maximum error values differed for each filtering algorithm in the first test area. Therefore, the mean value was found to be different for each algorithm. However, it was seen that similar RMS values were obtained with all the filtering algorithms except MCC.

RECENZIRANI ČLANKI | PEER-REVIEWED ARTICLESSI | EN

The ETEW filtering algorithm generally produced commission errors in regions where sudden slope and elevation changes occurred. Several tree objects in hilly areas were not removed, leading to small bumps in the shaded relief DTM, as shown in Figure 3b. The ETEW filtering algorithm generated omission errors on discontinuous surfaces, such as steep slopes (Figure 3b). An RMS value of 17 cm was obtained with the ETEW filtering algorithm, as shown in Table 3. The ATIN algorithm produced commission errors similar to those of the other algorithms. As shown in Figure 3a, it produced errors where eleva-tion differences occurred, such as in passages along rural roads to vegetaeleva-tion areas, vegetaeleva-tion in hilly areas, and trees located on the open landscape. Moreover, such low vegetation was found to generate erroneous results when filtering low objects near the ground. Thus, the threshold value was insufficient to filter such objects.

With the PM filtering algorithm, similar results were obtained to those of ATIN, as shown in Figure 3d. Omission and commission errors were produced in almost the same areas. The best result for the first test area was obtained using the IPF algorithm. An RMS error of 14 cm was obtained with IPF, and 89.9% of the errors were in the range of −20 to +20 cm, as illustrated in Tables 3 and 4; however, the IPF filtering algorithm produced the most omission errors. The omission errors generated by the MLS filtering algorithms occurred on discontinuous surfaces and sunken areas on the ground, as shown in Figure 3c. Like for the other filtering algorithms, commission errors were generated in areas where height and slope changes were experienced. The maximum commission error was generated with the MCC filtering algorithm. Commission errors were produced in nearly all the vegetation-covered hilly mountain areas where elevation changes were experienced.

The second test area consisted of dense settlement areas, large irregularly shaped buildings, and small hilly areas. The small hilly area located at the bottom right of our study area was preserved by all the filtering algorithms. In contrast, the algorithms all produced omission and commission errors at different rates in dense residential areas. As shown in Table 3, the maximum and minimum error values increased for each filtering algorithm. The error values at −20 to +20 cm were much lower than they were in other test areas, as illustrated in Table 4. Therefore, the highest RMS value for all the filtering algorithms was obtained in the second study area.

The best filtering algorithms in the area where the complex buildings were located were ATIN and IPF.

The PM algorithm preserved the small mounds in the upper left and lower right corners of the second test area. ATIN, ETEW, MCC, MLS, and PM produced few omission errors in the upper left corner.

Moreover, ETEW, PM, MCC, and MLS did not fully filter complex buildings, and thus, they produced commission errors. In the lower-left and upper right corners of the test area, in the region where the elevation suddenly changed, the ground surface was removed by the IPF algorithm and surface abrasion was generated in the shaded relief, as shown in Figure 3l. Such problems can be caused by a border effect.

To avoid this, filtering can be applied to an area that is larger than the test area, and then the image can be cropped to the test area to show the results.

ETEW could not fully filter complex buildings, leading to commission errors. The ETEW filtering algorithm removed the terrain point that caused omission errors throughout the settlement area. This led to distortions in the DTM, as shown in Figure 3(h,h). The highest omission error was obtained by the ETEW filtering algorithm, as shown in Table 4. An RMS value of 57 cm was obtained using the

RECENZIRANI ČLANKI | PEER-REVIEWED ARTICLESSI | EN MLS filtering algorithm, which produced high omission errors in dense residential areas, like the ETEW filtering algorithm. In addition, complex structures could not be filtered by the MLS algorithm, as shown in Figure 3 (k,k). The best result for the second test area was obtained by the IPF algorithm with an RMS value of 33 cm, as illustrated in Table 3; however, this algorithm caused erosion in the lower left and upper right corners of the work area. According to the statistical findings, the worst result for the second test area was obtained by the MCC filtering algorithm; here, an RMS error value of 86 cm was calculated, as shown in Table 3. Complex structures could not be filtered with this algorithm, causing intense commission errors in those regions.

Figure 3: Difference DTM for first test area: a) ATIN-Ref, b) ETEW-Ref, c) MLS-Ref, d) PM-Ref e) MCC-Ref f ) IPF-Ref Shaded Relief Map for first test area: a) ATIN, b) ETEW, c) MLS, d) PM, e) MCC, f ) IPF Difference DTM for second test area: g) ATIN-Ref, h) ETEW-Ref, i) PM-Ref, j) MCC-Ref, k) MLS-Ref, l) IPF-Ref Shaded Relief Map for second test area: g) ATIN, h) ETEW, i) PM, j) MCC, k) MLS, l) IPF

The lowest RMS values were observed in the third test area. All the filtering algorithms produced values of 7–8 cm, as shown in Table 3. In addition, it was found that the maximum and minimum error values were extremely low compared with those in the other test areas. Accordingly, the mean and median values were found to be close to each other. In Figure 4, it can be seen that almost all the filtering algorithms successfully filtered shrubs and crops. The error values in the −20 to +20 cm range were over 95% for all the filtering algorithms. Based on these results, all the algorithms can filter low vegetation on nearly flat surfaces.

The fourth test area was selected from a valley covered with dense vegetation, with riverbeds, transmission lines, and areas of mixed low and high trees. As shown in Table 4, the omission errors of all the filtering algorithms increased dramatically. Significant omission and commission errors were concentrated along

RECENZIRANI ČLANKI | PEER-REVIEWED ARTICLESSI | EN

Figure 4: Difference DTM for fourth test area: a) ATIN-Ref, b) MLS-Ref, c) PM-Ref, d) MCC-Ref e) ETEW-Ref f ) IPF-Ref Shaded Relief Map for fourth test area: a) ATIN, b) MLS, c) PM, d) MCC, e) ETEW, f ) IPF Difference DTM for third test area: g)

RECENZIRANI ČLANKI | PEER-REVIEWED ARTICLESSI | EN

ATIN-Ref, h) MLS-Ref, i) PM-Ref, j) MCC-Ref, k) IPF-Ref, l) ETEW-Ref Shaded Relief Map for third test area: g) ATIN, h) MLS, i) PM, j) MCC, k) IPF, l) ETEW

RECENZIRANI ČLANKI | PEER-REVIEWED ARTICLESSI | EN

the riverbeds and valley ridges, as shown in Figure 4a–f. The height difference is one of the basic elements used to separate ground and nonground objects. Since vegetation was located on a sloping surface in our fourth test area, the filtering algorithms were not effective in separating bare earth and vegetation, and they caused omission errors (Sithole and Vosselman, 2004). Since the omission errors were high, it can be the conclusion that the minimum error value had the highest value for all the algorithms.

Table 4: Percentage of error rates for all test area

Total Commission Error (%) Total Omission Error (%) ±20cm (%)

First Second Third Fourth First Second Third Fourth First Second Third Fourth

MLS-Ref 11.6 10.4 1.2 4.7 2.4 12.3 0.4 16.6 86 77.3 98.4 78.8

According to the RMS values in Table 3 and error values between −20 and +20 cm, as shown in Table 4, the MCC, ETEW, and PM algorithms performed better than the other filtering algorithms did.

Two filtering algorithms—ETEW and PM—were able to properly eliminate vegetation, as illustrated in Figure 4 c,c,e,e. However, the MLS and MCC algorithms failed to filter vegetation sufficiently and caused tiny divergences on the shaded relief, as can be seen in Figure 4 b,b,d,d. Compared with other filtering algorithms, the highest commission errors were brought about by the ATIN and MCC filtering algorithms. Although the most appropriate parameter values were selected, these algorithms could not separate low vegetation from the ground. MLS was sensitive to slope changes and had lower accuracy than the other filtering algorithms did in terms of its RMS value of 27 cm in Table 3 and 78.8% value between −20 and +20 cm, as illustrated in Table 4. As shown in Figure 4c,c some nonground data were falsely classified as ground points, which led to small bumps in the shaded relief map. In addition, the highest commission error was obtained by the MLS algorithm, and this was spread over almost the entire test area, although 84.4% of the errors were in the range of −20 and +20 cm, as can be observed in Table 4. An RMS value of 25 cm was obtained by the IPF filtering algorithm, representing the second-worst result. IPF has difficulty filtering local sharp surfaces, and it removed some sharp hilly areas, as illustrated in Figure 4f,f. Furthermore, IPF had edge effects that removed the edges of the test areas.

5 DISCUSSION

The worst result in the first test area among the six filtering algorithms was obtained from MCC, in which the smallest number of correctly filtered points (at −20 to +20 cm) was obtained. All the filtering algorithms except MCC performed well for the first test area. The best result was obtained with IPF, with an RMS of 14 cm. In this test area, commission errors were more common than omission errors were.

Commission errors generally occurred in areas where sudden slope and height changes were experienced in the topography and there were natural or artificial objects close to the ground surface. In this context, the determination of the optimal value of the slope or height difference parameters used by the filtering algorithms has a significant effect on reducing errors and improving the results.

RECENZIRANI ČLANKI | PEER-REVIEWED ARTICLESSI | EN As a result of the work done on the second test area, significant increases were seen in omission, com-mission, and total errors. Accordingly, there was a decrease in the number of points in the range of

−20 to +20 cm. Most omission errors were obtained with the MLS, MCC, and ETEW algorithms. In contrast, the least commission errors were obtained by MLS and ETEW. The complex structures were filtered well by the ATIN and IPF algorithms. However, IPF deleted some of the data in the top and bottom corners of the workspace, which corrupted the surface. The best result for the second study area was obtained with the IPF filtering algorithm, with an RMS value of 33 cm.

The best results were found in the third test area. All the filtering algorithms successfully filtered low vegetation. The maximum number of correctly filtered points (at −20 to +20 cm) and minimum RMS values were achieved by all the filtering algorithms for the third test area. Thus, the filtering algorithms can filter low vegetation in areas where the slope change is low.

The worst result in the fourth test area among the six filtering algorithms was obtained from MLS, which obtained the lowest number of correctly filtered points (at −20 to +20 cm). MCC, ETEW, and PM performed better than the other filtering algorithms did based on the RMS and number of points in the range of −20 to +20 cm. Significant proportions of omission and commission errors were concentrated along the riverbeds and valley ridges.

As a result of the filtering process in the four different test areas, it is clear that many parameters can affect the filtering accuracy. First, the filtering performance changes depending on the terrain topography and environmental conditions. Second, the correct selection of the filtering parameters of each algorithm is important for the accuracy and performance of the filtering results. The parameter values of the filtering algorithms, such as the slope, height threshold value, search ellipses, and initial window size, must be best defined to efficiently filter the work areas that have different characteristic features.

Different studies comparing the similar filtering algorithms to those used in this research have been published. There are some similarities and differences between the results. Although Julge et al. (2014) used the same filtering algorithms, the average RMS in their results was higher than ours. This may be due to the size of the work area or lower point density. Sulaiman et al. (2010) compared the filtering algorithms in only one test field, which was similar to our first test area. ATIN, ETEW, IPF, and MLS performed comparably in the two studies. However, the PM algorithm gave the worst results, unlike in our findings. In another work, Podobnikar and Vrečko (2012) conducted trials in two different test areas. The results showed significant similarities to those obtained from our study. Examining these findings more closely, it can be seen that the filtering algorithms have some common points about their performance and some differences. It is thought that the results may have been affected by the size and complexity of the test areas, point density, and parameter values of the filtering algorithms. To elucidate this possibility, more trial and test studies should be performed.

As a result of the test work done, a single filtering algorithm does not seem to be successful on any land surface. To improve the success of filtering algorithms in different study areas, it is necessary to develop integrated algorithms in which algorithms with different characteristics are used together (Chen et al., 2017). In future research, advanced ground-filtering methods can be developed with a combination of different filtering strategies. In addition, the integration of lidar data and other data sources, such as

In document Geodetski vestnik (Strani 92-105)