RVDEMINT

Description

RVDEMINT fills holes in raster digital elevation models (DEM) by interpolation, edits raster DEMs by 3-D and 2-D vector information, or both.

Parameters

rvdemint(fili, dbic, dbiw, failvalu, backelev, filv, dbvs, vectyp, fldnme, clip2pol, waterval, filo, dboc, demerase, maxiter, memsize)

Name	Type	Caption	Length	Value range
FILI *	str	Input DEM-raster file name	1 -
DBIC *	List[int]	Input elevation layer	1 - 1
DBIW	List[int]	Input window	0 - 4	x-offset, y-offset, x-size, y-size
FAILVALU	List[int]	DEM failure value	0 - 1	Default: -32767
BACKELEV	List[float]	Background value	0 - 1
FILV	str	Input vector file name	0 -
DBVS	List[int]	Input vector segments	0 -
VECTYP	List[int]	Type of vector segments	0 -	0 - 11
FLDNME	str	Field name for elevation	0 - 64	Default: ATTRIBUTE
CLIP2POL	str	Clip 3-D data to polygons	0 - 3	YES | NO Default: NO
WATERVAL	str	Elevations of water-polygon vertices	0 - 20	Default: INPUT
FILO	str	Output DEM file name	0 -
DBOC *	List[int]	Output edited elevation channel	1 - 1
DEMERASE	List[float]	Erase distance and sensitivity	0 - 2	0 - Default: 0,0
MAXITER	List[int]	Maximum number of iterations	0 - 1	1 - 10 Default: 4
MEMSIZE	List[int]	Memory size (MB)	0 - 1	1 - Default:

* Required parameter

Parameter descriptions

FILI

Specifies the file name of the input raster DEM with missing elevations to interpolate, edit by vector data, or both.

DBIC

Specifies the channel specified for the input raster DEM to edit.

DBIW

Specifies the raster window (x-offset, y-offset, x-size, y-size) to process. If no value is specified for this parameter, the entire DEM is processed by default.

X-offset and y-offset define the upper-left-starting-pixel coordinates of the window. X-size is the number of pixels that define the window width. Y-size is the number of lines that define the window height.

FAILVALU

Specifies the value used in the input raster DEM for areas (pixels) in which the elevation value is required, but missing. RVDEMINT fills these areas by interpolation from all available sources of information. If no value is specified for this parameter, a value of -32767 is assumed.

BACKELEV

Optionally specifies the value used in the input raster DEM for background areas (pixels) in which the elevation value is not required. RVDEMINT omits these pixels from interpolation.

If no value is specified for this parameter, the function checks for ELEVATION_BACKGROUND or NO_DATA_VALUE metadata tags, first at the channel level, and then at the file level. If the value is neither specified nor found in the metadata, the function uses the default value of -32768, and sets that value as the NO_DATA_VALUE metadata tag in the output elevation channel.

This parameter is optional.

FILV

Specifies the name of the file that contains the vector layers for DEM interpolation, editing, or both. If no value is specified for this parameter and the DBVS parameter, the input vector layers are not used. If no value is specified for this parameter, but a value is specified for the DBVS parameter, the input vector layers are read from the file specified for the FILI parameter.

DBVS

Specifies the vector segment or segments for DEM interpolation, editing, or both.

The segments can be in the input vector file or, if no file is specified, the segments can be in the input DEM-raster file. Each segment must be in the same file.

If no value is specified for this parameter, but a value is specified for the FILI parameter, RVDEMINT searches the input vector file for segments named POINTS, CONTOURS, BREAKLIN, VALLEYS, RIDGES, CLIFFS, OUTLINE, EXCLUDE, WATER, POLY3D, and POLY2D, and uses each segment found for interpolation. The segments can be produced with VDEMINGEST. If no such segments are found, no vector information will be used to process the raster DEM.

VECTYP

Specifies the type of each vector layer; that is, how the input vector layers are to be interpreted in processing the DEM.

The number of values specified for this parameter must be the same as the number of values specified for the input vector segments. A vector-segment type specified for this parameter applies to the vector segment at the same position in the list of input segments.

If no values are specified for the DBVS and VECTYP parameters, but a value is specified for the FILV parameter, RVDEMINT searches the corresponding file for segments named POINTS, CONTOURS, BREAKLIN, VALLEYS, RIDGES, CLIFFS, OUTLINE, EXCLUDE, WATER, POLY3D, and POLY2D, and automatically assigns the layer type based on each segment name. If no such segments are found, no vector information will be used to process the raster DEM.

Only one OUTLINE segment should be provided. When the selected OUTLINE segment has no shapes, the whole area specified by the Input window (DBIW) parameter is processed, except for the EXCLUDE polygons.

FLDNME

Specifies the name of the field that contains the elevation values.

If the value of this parameter is specified as ZCOORD, the actual z-coordinates of the vectors are used. Field names are not case-sensitive, and you need not specify them in complete form. If more than one match exists, the first name is used.

The Field name (FLDNME) parameter is required only for vector layers that contain 3-D points or contours. With 3-D lines and polygons, the z-coordinates are used automatically. The handling of water-polygon elevations is defined by the Water-polygon-vertex elevations (WATERVAL) parameter. With 2-D layers, the elevation values are not specified, and are ignored if present.

The same name applies to all contour and 3-D point segments in the input vector segments specified for the Input (DBVS) parameter. If necessary, you can convert segments that do not satisfy this requirement to the required format by using ZVALTRNS.

When a field name other than ZCOORD is specified, but not found, RVDEMINT attempts to use the z-coordinates of each vertex or point; this can, however, produce unexpected results.

CLIP2POL

Specifies where to apply 3-D conditioning data.

When NO is specified for this parameter, 3-D data is used everywhere in the area defined by the OUTLINE and EXCLUDE polygons. When YES is specified, 3-D data is clipped to the WATER, POLY3D, and POLY2D polygons and used to interpolate elevations only inside these polygons. The default value is NO.

WATERVAL

Specifies how to process elevations of water-polygon vertices.

The default value is INPUT, and it retains the elevation of each vertex. Any other value replaces vertex elevations of each shape by a single representative value.

The values provided for the STDDEV, ATTRIB, and VALUE options must be entered immediately after the colon and without any intervening spaces; for example: "STDDEV:-1.5", "ATTRIB:Height" or "VALUE:1245.8". The attribute name is case-insensitive.

FILO

Specifies the name of the GDB-supported raster image file to which to write the processed DEM. The file you specify must already exist before running RVDEMINT. If necessary, you need not specify a value. When no value is specified for FILO, the edited elevations are written to the specified input file (FILI).

However, when you specify a value for FILO, the corresponding database file must have the same number of lines and pixels, and the georeferencing of the files specified for FILI and FILO must be identical. Only the elevations in the image window selected for processing are overwritten.

DBOC

Specifies the output channel in the file specified for FILO to which to write the edited DEM. The channel must already exist. When the same file is used for the input and output DEM, the value of DBOC and DBIC can be the same; that is, an in-place processing is supported, but not recommended. The channel should have adequate pixel data type to support the range and precision of the edited elevation values. In general, either floating-point or 16-bit signed integer channels should be used.

DEMERASE

Specifies the DEM erase distance and sensitivity.

When you specify one value, it defines the distance in pixels to erase around every 3-D elevation point provided in the input DEM. The erased pixels are filled subsequently by interpolation from the surrounding 3-D data and non-erased DEM pixels. When the distance is zero, only pixels directly under 3-D data are set to their elevations.

When you specify two values, the first specifies the maximum allowable erase distance and the second specifies the sensitivity factor in pixels per elevation unit. The factor is multiplied by the elevation difference between the input and interpolated elevations at a pixel to derive the erase distance at the pixel. When the pixel is at or closer to a 3-D source than the computed distance, its value is erased. The sensitivity of 0 results in a static erase distance, as specified by the first value. There is a preset minimum erase distance of two pixels, unless the specified maximum value is lower.

The default setting results in no erasure of input data; therefore, it should be adjusted if vector data is present.

MAXITER

Specifies the maximum number of iterations for smoothing the DEM when interpolating failed values. The default value is 4; lower values may work, but, generally, are not recommended.

MEMSIZE

Specifies the physical memory, in megabytes, to use for image buffers. The default is 25 percent of the total system memory.

More allocated memory usually results in faster processing. For more information about this parameter, see the Details section, later in this topic.

Details

RVDEMINT interpolates holes in an existing raster DEM or edits an existing raster DEM using points, vectors and polygons. It can also combine the two functions, by filling the existing holes and using the provided 3-D and 2-D data to "condition" the surrounding pixels. The 3-D data provides much stronger constraints on the elevations than 2-D data, and should be used whenever possible. The polygons erase their interior areas from the DEM, and may provide vertex elevations for interpolation.

RVDEMINT processing is local in nature. The existing raster is adjusted to agree with the vector data provided, but the pixels far away from them are not modified. You can use ADJDEM to apply a global low-order-polynomial correction to all elevations in the DEM, based on a number of ground control points (GCP).

Non-contour types of 3-D vector data, such as TIN network, 3-D roads, 3-D rivers and valleys or 3-D ridges, should all be entered as 3-D structure lines. Note that 3-D cliffs are undefined. If the elevations on one side or both sides of the cliff are known, they can always be entered as another 3-D layer (in addition to a 2-D cliff layer).

The 2-D layers provide additional constraints for the editing of elevations. Specifically, a valley vector indicates a local minimum; a ridge vector indicates a local maximum; a cliff vector indicates a discontinuity in the DEM.

The polygon layers erase the existing DEM elevations within their interiors and set them according to their type. The EXCLUDE polygons fill their interiors with the background (no-data) value. The WATER and POLY3D polygons use elevations interpolated from the polygon vertices and any other 3-D data provided within their interiors. The vertices of every WATER polygon are expected to be at a constant elevation. If not, you can use the Water-polygon-vertex elevations (WATERVAL) parameter to replace them with a constant value, representative for each individual polygon.

The POLY2D polygons interpolate 3-D data provided within their interiors and the surrounding raster values, but discard their vertex elevations, even when they are available. The WATER and POLY3D polygons propagate outside their outlines according to the value of the Erase distance and sensitivity (DEMERASE) parameter.

The raster-DEM data specified for the FILI and DBIC parameters is mandatory, while all other data types are optional. The input DEM can contain failed and background values. The failed values will be filled by interpolation from all available data (surrounding pixels and all 3-D data within each failed area). The background pixels will be left unchanged.

A section of the input DEM will be processed when you specify a value for the Input window (DBIW) parameter. This can be useful when only a part of a larger DEM is needed or if different parts of the DEM must be processed with different parameter settings.

When you specify a value for the DBIW parameter, all pixels in the window specified by the FILO parameter will be overwritten with the processed values.

The 3-D data value sets the value of the DEM pixel containing it. When multiple 3-D data values occur in the same pixel, the minimum value of each is used. The 3-D values can also erase a small area around the pixel, as determined by the value of the Erase distance and sensitivity (DEMERASE) parameter.

With the FILV, DBVS, and VECTYP parameters, you can specify the location of 2-D and 3-D data to use and the type of each. When the default value is used for the FILV parameter; that is, you did not specify a value, but you did specify a value for the DBVS and VECTYP parameters, the value of FILV is assumed to be the same as that of FILI.

When you specify a value for FILV, but use the default value for DBVS and VECTYP, the data is assumed to reside in segments with standard names (POINTS, CONTOURS, BREAKLIN, VALLEYS, RIDGES, CLIFFS, OUTLINE, EXCLUDE, WATER, POLY3D, and POLY2D), and each is used. When no such segments are found, no vector information will be used in processing the raster DEM.

When you use the default value of FILV, DBVS and VECTYP, the vector data is not used, and only hole-filling is performed on the input DEM.

There can be multiple segments of any type, except the OUTLINE polygons segment. When multiple OUTLINE segments are specified or present in the vector file specified, only the first one is used.

When you specify the OUTLINE segment, it must contain a closed polygon or polygons surrounding the area or areas to process. The polygons can have interior rings. When polygons are not closed properly, the results will be unpredictable. Pixels outside the polygons will be set to the value specified for the Background elevation (BACKELEV) parameter.

When the outline polygons are present, they are converted internally to a temporary valid area bitmap. Bitmap pixels set to 1 correspond to the processed pixels; all other pixels in the processing window are set to the background value. When the input raster contains background values, they are transferred unchanged to the output raster.

The EXCLUDE segments behave similarly to the OUTLINE segment, except that the interior pixels are set to background. Their main use is when the outline polygons are not available, but some interior areas of the DEM must be set to the background elevation. There can be more than one EXCLUDE segment, and EXCLUDE and OUTLINE segments can be used simultaneously.

The OUTLINE and EXCLUDE segments can overlap. In such cases they are combined in a temporary "exclude" bitmap at the start of processing. During subsequent processing, the bitmap is examined to decide which pixels must be set to the background value.

The CLIP2POL parameter determines where the DEM conditioning data is used. When you specify YES, all conditioning data sets are clipped to the interiors of all WATER, POLY3D, and POLY2D polygons, and used only there. When you specify NO, the conditioning data is used throughout the entire DEM area, except outside the OUTLINE polygons and inside the EXCLUDE polygons.

The Water-polygon-vertex elevations (WATERVAL) parameter applies only to WATER polygons. Any value other than the default, INPUT, modifies vertex elevations before their use. When you specify VALUE followed by a number, the elevation given by the number is assigned to all polygons in all WATER segments. With any other option you specify, all vertex elevations of a polygon are replaced by a representative value derived from the original vertex elevations in each polygon.

The MODE option selects the most common vertex-elevation value. When several values have the same number of occurrences for a given polygon, the lowest such elevation is used. The number you specify after STDDEV is the number of standard deviations to add to (for positive numbers) or subtract from (for negative numbers) the mean to compute the representative elevation.

All WATER, POLY3D, and POLY2D polygons are combined into a single "erase" bitmap at the start of processing. The bitmap is subsequently used to erase all raster pixels underneath it, and to clip the DEM conditioning data if required.

The output raster file must be created prior to running RVDEMINT. When the input raster DEM is not in the same database file as the output raster, the two files must have the same number of pixels, lines, and pixel size, and their projection must be identical.

With the FILO and DBOC parameters, you specify the destination of the edited raster. If you do not specify a value for FILO, the value of FILI is used. The output channel DOBC can be same as DBIC, but such use is not recommended.

If the DEM is very large, it is recommended that the files be tiled, if the format supports such a file structure. This can significantly reduce disk I/O during the run.

With the Erase distance and sensitivity (DEMERASE) parameter, you specify one or two values to control the erasing of input raster pixels around each 3-D point or line in the vector file you specified. When you specify only one value, it defines the fixed erasure radius in DEM pixels around each 3-D point. Note that the radius is interpreted internally to be in the Manhattan (city-block) metric, so that the erased areas may not be exactly circular around isolated points.

When you specify two values, the "dynamic radius" mode is used, which computes the effective radius at every pixel. The first value specifies the maximum allowable erase radius. The second specifies the sensitivity (in pixels per elevation unit) of the erase radius to the difference at a pixel between the input and interpolated elevation values:

        Radius [pixels] = ABS(DEMERASE[2] * (raster - interpolated))
      

In the preceding example, ABS is the absolute value of a number, and the computed value is rounded to the nearest integer before use. For example, if DEMERASE[2] is 0.2, the input raster elevation at a pixel is 127.4 meters and the elevation from 3-D data is 143.6 meters, then the computed erase radius is rounded to three pixels.

The computed value is thresholded to be not smaller than two and not larger than DEMERASE[1]. Next, the actual distance of the pixel to the nearest 3-D data source is compared with the computed radius. When the distance is smaller than or equal to the erase radius, the input DEM elevation at the pixel is erased. Each pixel erased by this process is filled subsequently by values interpolated from the surrounding retained pixels, and any 3-D data present in each erased area.

With lines, the erase distances are perpendicular to the dominant direction of the line (either left-right or up-down).

The erase distances depend on the accuracy and smoothness of the input DEM, its pixel size and elevation units, and the quantity and accuracy of the vector and point data.

The default setting (0,0) for the Erase distance and sensitivity (DEMERASE) parameter results in no erasure. This is a required setting if no 3-D data is used. However, if any 3-D data is available, each parameter should be adjusted according to its data characteristics. When you want to use the static mode, the erase distance of 5 to 10 might work in many cases. It will result in a zone of five to 10 adjusted DEM pixels around each 3-D data point and along scanned lines.

The value you specify for the Maximum iterations (MAXITER) parameter determines the number of iterations during pyramid interpolation. The requested number of iterations should not be too low, because the resulting surface may be rough.

With the Working memory size (MB) (MEMSIZE) parameter, you specify the amount of memory to use for image buffers. The buffer requirement for RVDEMINT is 32 bytes per pixel. The memory is allocated statically for the duration of the run. At most 16,383 MB of buffer memory will be allocated, plus the memory required for scanning of vector data, if used.

RVDEMINT attempts to divide the DEM area specified for the Input window (DBIW) parameter into tiles such that each tile can be processed in memory. Each tile has approximately the same size, and is extended by 256 pixels or lines on each side, so that there is overlap in input data used for interpolation of adjacent tiles. The result is that the whole DEM generates faster, and there are no visible artifacts along tile boundaries, except in large areas void of input data.

RVDEMINT must perform a pass over all input layers for each tile. Therefore, it is recommended that you allocate as much memory as possible, to minimize the number of tiles. A suitable limit is approximately 60 percent of your available system memory, unless you anticipate intensive use of the system by another process. Only the required memory is allocated: depending on the tile size, less memory than the amount specified for the Working memory size (MB) (MEMSIZE) parameter can be used.

When the individual layers do not cover the entire DEM area, their intersection with each tile is checked. Layers that do not intersect the current tile are skipped, which saves additional processing time. Having the vector segments and the DEMs all in the same projection also increases processing speed.

When the image cannot be tiled due to insufficient memory, it is processed as a single tile. In this case, RVDEMINT uses image-buffer files. A small part of each processing buffer is kept in memory, and the rest is written to and read from a temporary file, as required. The results are the same as in the tiled case, but the heavy disk I/O significantly slows down the DEM-editing process. The temporary files are created automatically by RVDEMINT and deleted in the latter stages of processing.

After completing interpolation, the current processing tile is written to the output file, and the process continues with each subsequent tile. When the outline or exclude polygons, or both are provided, each pixel outside the requested area is set to the background value.

RVDEMINT supports a mixture of raster, vector, and point data. Data sets without the input DEM raster should be processed using VDEMINT.

In most cases, RVDEMINT produces a smooth and pleasing DEM. The void areas of tens or low hundreds of pixels are effectively filled and smoothed. However, in rare cases, when the processing window contains large areas that are failed or void of any input data, the final surface may show some artifacts, and may need to be further edited manually. This effect may be particularly visible at the margins of the Input window (DBIW), or the tiles generated internally. A closed contour or 3-D polygon fully within a failed area will properly set its interior pixels, and could be used to address such cases.

Example

The test file rvdemtest.pix contains an existing DEM raster with holes and background areas, and three vector segments: mountain peak elevations as 3-D points (segment 2), and manually digitized 2-D valleys (segment 4) and 2-D ridges (segment 5). A DEM will be filled and edited using the raster and vector layers, and stored in another channel on the file.

from pci.rvdemint import rvdemint


fili = "rvdemtest.pix"
dbic = [1]
dbiw = []         # process entire DEM
failvalu = [-100]
backelev = [-150]
filv = ""         # use vectors from fili
dbvs = [2,4,5]
vectyp =[0,8,10]  # 3-D points and exclude and 3-D polygons
fldnme = "ZCOORD" # elevations are extracted from z-coordinates
clip2pol = "YES"  # use points only inside the 3-D polygons
waterval = ""     # use water vertices as-is
filo =    ' '     # use fili for output
dboc = [2]
demerase = [5]    # static erase of five pixels
maxiter = [4]     # perform at most four iterations
memsize = []      # use default memory size (25 percent of system memory or less)

rvdemint( fili, dbic, dbiw, failvalu, backelev, filv, dbvs, vectyp, fldnme, clip2pol, waterval, filo, dboc, demerase, maxiter, memsize )

Algorithm

For more information on the 3-D and 2-D data-scan and interpolation steps, follow the link under Related functions to VDEMINT at the end of this topic.