Photogrammetric Processing Algorithms

Core Processing

Camera Calibration

Guidelines and Best Practices

It is strongly recommended to use basic setting templates, and only do adjustment to particular settings if required. See calib.Settings.TEMPLATES.

Recommendations for the Settings.pipeline parameter

The calibration uses a selection of four main algorithms represented by the pipeline parameter. It is the main factor influencing the calibration processing outcome. Each one of the algorithms has a specific purpose, advantages and limitations. None of the pipelines is “one size fits all”.

Possible pipeline values:

• Standard Designed for all types of flight plans (nadir, oblique, free flight, etc.). Its main advantage is its robustness in terms of input data quality (less organized flight plans, image overlaps that are not constant, missing images, variable image accuracies, etc.). Its disadvantages are the performance, the scalability and the accuracy of the calibration when comparing it to the other pipelines. It is unfit for processing large and corridors projects.

• Scalable This calibration pipeline has been designed for high processing speed, scalability, large projects and corridors. It requires a high degree of flight precision and planning, sufficient and steady overlaps. It is therefore suitable for flights performed using a professional flight planning software, nadir flights, corridors and for specific oblique flight plan patterns. This pipeline should not be used for processing any rig camera projects.

• LowTexturePlanar Designed for scenes with low number of features on images, uniform surfaces (e.g. fields), flat-like scenes, and for multispectral rig camera captures. The image input must come from nadir flights with minimal camera tilt angle: no more than 10 degrees (absolute maximum) from down pointing direction.

• TrustedLocationOrientation This calibration pipeline is intended for processing projects with accurate relative locations and IMU (Inertial Measurement Unit) data, including PIX4Dcatch indoor and outdoor captures or captures using RTK drones/devices or PPK operations. All images must include information about position and orientation.

Recommendations for Settings.ImagePairSettings

These are the settings used for image pair matching.

Image pair matching defines the strategy to select the set of pairs to be used by calibration in the feature matching algorithm step. Feature matching is the process of finding the same feature in both images of an image pair.

It is possible to try matching features on all possible image pairs, matching each image with each other within the dataset by setting match_all=True. However, the processing time of this approach does not scale well with input data size, resulting in long processing times for even moderately sized data sets.

Reoptimization

Reoptimization is a calibration feature which updates the internal and external camera parameters when new inputs (such as marks on images or GCPs, MTPs) have been added after calibration has been run.

It is a faster method for updating calibration and improving accuracy when compared to re-running calibration.

Currently, the only known usage of reoptimization is after AutoGCP automatic GCP mark detection.

See calib.reoptimize().

Point Cloud Densification

Usage

A densified point cloud is obtained by calling dense.densification().

The area of the sparse point cloud subject to densification can be limited by using a region of interest (ROI) constructed with the help of the geometry utilities.

Certain segments on images can be masked, thus excluding the 3D points projecting onto these areas from being considered for densification. Masks excluding sky and water segments can be obtained using the sky and water segmentation algorithm.

The densification can also produce sparse synthetic depth maps if this is enabled by the dense.Settings.compute_depth_maps configuration parameter. These can be further densified and used in mesh geometry generation.

In order to visualize a point cloud in a third-party software, it has to be exported as a file of one of the following formats, LAS, Cesium and SLPK, using the respective functions io.las.write_pcl(), io.cesium.write_pcl() and io.slpk.write_pcl().

Guidelines and Best Practices

In order to configure the densification algorithm, it is strongly recommended to use the configuration templates that are available for the common use cases. See dense.Settings.TEMPLATES. Adjustment to individual configuration parameters can be done if necessary. More details on the parameters are given in the API reference for dense.Settings.

DSM/DTM

Two types of DSM generation are available: Triangulation and IDW. At the API level, this choice is done by choosing one of the two available settings classes:

• dsm.Settings.Triangulation

  Recommended for rural areas, agriculture or low texture captures.

  Provides less accurate DSM, but generates no empty cells.
• dsm.Settings.IDW

  Recommended for urban areas, construction sites and buildings.

  Provides good accuracy, but can generate empty cells and outliers.

Recommendations for dsm.Settings.IDW

It is recommended to use preset and default settings for the parameters:

• gsd

  The GSD value as generated by calibration.

  It is recommended to use the value computed by calib.analytics.compute_gsd(scene).
• interpolation_nn_count

  When set, gaps that might appear in the DSM are completed by interpolation. The value sets the number of nearest neighbors to be used for interpolation. Since the surface is approximated with neighboring points, some inaccuracies can be expected. Larger value allows for larger gaps to be filled, increasing the likelihood for the surface to be gap-less, but allows for larger inaccuracies in the output. Lower value ensures higher accuracies in the output, but increases the risk of generating surfaces with gaps.

  Recommended value: 10 [default value]

  Typical range: 10-30
• smoothing_median_radius

  When set, the algorithm performs post-processing surface smoothing of the DSM. When using low values, the final DSM contains sharper features and edges and vice versa. For urban areas (constructions, parks, roads), where the heights of the surface change often and rapidly, it is recommended to use a value in the lower range. For surveying hilly areas, slopes or scenes with large objects, higher values can be used.

  Recommended value: 12 [default value]

  The values should be in range 0-20:

  • 0 No smoothing is applied.

  • 1 Tries to keep sharp features such as corners and edges of buildings. Value 1 is considered as a very low value.

  • 20 Tries to smooth areas considering a large surrounding area. Sharp edges will be smoothed out. Areas that are not too planar are smoothed and may become planar. Value 20 is considered as a high value.

Orthomosaic

Guidelines and Best Practices

• The NADIR capture pattern is best suited for nadir aerial photography with the cameras pointing approximately 90 degrees down.

• The OBLIQUE capture pattern is best suited for oblique aerial and terrestrial photography, as well as for nadir aerial photography when the cameras are tilted, typically in the direction of the flight.

• For the same data set, changing the capture pattern from NADIR to OBLIQUE, will increase the processing time.

• The FAST algorithmic pipeline gives good quality results for NADIR photography.

• The DEGHOST algorithmic pipeline can be used when moving objects (cars, people, etc.) are present in the scene and are desired to be removed from the orthomosaic.

• The FULL and DEGHOST algorithmic pipelines require more processing time and more temporal disk space than the FAST pipeline.

• Combination of an algorithmic pipeline and a capture pattern for typical uses cases:

  • Inspection of runways, surveying, mapping, large cadasters: FAST + NADIR.

  • Inspection of motorways, inspection of busy streets for road work quality: DEGHOST + NADIR.

  • Inspection of tower structures: FULL + OBLIQUE.

  • 3D modeling of objects in busy traffic locations: DEGHOST + OBLIQUE.

  • A car crash scene: FULL + OBLIQUE.

  • Busy construction sites (using a crane camera): DEGHOST + NADIR.

  • Terrestrial photography: FULL + OBLIQUE.

  • Agricultural surveying: FAST + NADIR.

3D Mesh

Usage

The two components of a 3D mesh model, geometry and texture, are generated using mesh.gen_mesh_geometry() and mesh.gen_mesh_texture(). The geometry obtained by the former function serves as an input to the latter.

When a level-of-detail (LOD) 3D mesh is needed, it is obtained by calling mesh.gen_textured_mesh_lod() taking a geometry and the corresponding texture as its inputs.

In order to visualize a 3D mesh in a third-party software, it has to be exported as a file of one of the following formats, OBJ, Cesium and SLPK, using the respective functions io.obj.write_mesh(), io.cesium.write_mesh() and io.slpk.write_mesh(). The Cesium and SLPK exports require an LOD mesh as an input.

Guidelines and Best Practices

• For configuring the mesh geometry and texture generation algorithms, a number of configuration templates are provided for each of them as mesh.Settings.TEMPLATES and mesh.TextureSettings.TEMPLATES. Choose the one suitable for your use case.

• When masks were used in point cloud densification, it is recommended to provide the same masks to the geometry generation.

• Constructing input cameras with densified synthetic depth maps and using them to constrain the geometry can help achieving a better quality of reconstruction of thin structures such as power lines and transmission towers.

More Processing

AutoGCP

Automatic GCP detection tools.

AutoGCPs consists of a set of tools for the automatic detection in images of control point targets with pixel level accuracy.

It supports three types of targets with black and white Haar-like features: square, diagonal and Aeropoint.

Despite its name, AutoGCPs imposes no restrictions on the use of targets. The functionality only concerns itself with the detection of targets in images and the obtainment of an accurate estimate of the position of their marker. The user can then use the information as ground control points (GCPs), check-points (CPs) or anything else.

For more information on control and check points see Control Points.

Usage

The basic usage of the AutoGCPs functionality is straight forward: the user calls a function to obtain marks. The usage within the context of a processing pipeline may be more involved, since it requires specific processing to obtain the inputs to AutoGCPs, and typically the results are used in further processing, either to improve or measure camera calibration accuracy.

Guidelines and Best Practices

• Although only nadir flights are supported the detection might work well (with no guarantees) with oblique captures without excessive image angles and very precise geolocation and orientation.

• The detection is supported only on RGB images.

• Images should not be excessively blurry.

• Targets should be clean, unobstructed by objects above or nearby from the camera PoV, and should not be crossed by shadow lines.

• If an obstruction cannot be avoided, the CP targets should be moved to another location so that a sufficient minimum number of them is always available for re-calibration or re-optimization.

• The CP target must have a size in pixels within an optimal range in order to be detected reliably. This depends on the image resolution but, as a rule of thumb, should be roughly within the range of 20 to 100 pixels.

• Configuring the correct horizontal and vertical uncertainty values can speed up the target detection if georeferencing of images is accurate. The default values cover most cases but are in meters and must be converted if a different unit of measurement is being used (refer to Settings for more information). Using a lower uncertainty than the default values is generally not recommended.

• The targets on the ground should be well placed at a sufficient distant from each other. The spacing must be larger than the camera georeferencing horizontal position uncertainty otherwise the image marks may get a wrong physical target to CP assignment.

• Good image overlap ensures the AutoGCPs results to be more reliable with each target being visible on multiple images. Minimum three reliable and confirmed marks (by the algorithm) are needed to generate the AutoGCPs output for a CP.

Depth Processing

Depth processing uses LiDAR depth maps to create depth point clouds. It can also merge depth point clouds with dense point clouds obtained through photogrammetry, thus producing point clouds of a superior quality. Densification of sparse synthetic depth maps is another functionality that can be useful in specific 3D mesh use cases.

Usage

A depth point cloud is generated by calling depth.gen_pcl() with input cameras containing LiDAR depth maps and a scene reconstructed by calibration as its inputs. A region of interest defined with the help of the geometry utilities can limit the area of the point cloud if necessary.

Merging of a depth and a dense point clouds is performed by calling depth.pcl_merge().

Sparse synthetic depth maps optionally obtained by dense.densification() can be densified (completed) using depth.densify(). The resulting depth maps can be further used as a constraint in mesh geometry generation.

Point Cloud Alignment and Transformation

Sky and Water Segmentation

Usage

Sky and water segmentation allows to identify sky segments, water segments or both on images (in this context, an image segment is an image region/area).

The output may be used to provide additional input to point cloud densification. Computing depth from sky or water segments in images is unreliable. Thus, allowing densification to exclude such segments greatly reduces or eliminates noise in the point cloud originating from these segments

Guidelines and Best Practices

• For images where water is present and to be removed from the images, the full segmentation mode must be used. (i.e. When the masking_type argument is WATER or SKY_WATER, then the mode argument must be FULL.)

• For images where only sky is present and to be removed from the images, the fast segmentation mode is recommended. (i.e. When the masking_type argument is SKY, then the mode argument is recommended to be FAST.)