Advanced usage¶
Project validation during processing¶
The Pix4Dengine enables performing checks on the quality report after each processing step is
completed. This is done by constructing a Pipeline with
a validators argument.
Define a validator¶
The validator is a callable which must accept as argument an instance of
Report. Here we show how this can be used for
checking calibration quality and stopping processing if the calibration is judged insufficient
based on some user-defined criterion.
First, we define a callable that returns False if the calibration quality in the project is too
poor, and True otherwise. The callable also defines a message, providing a description of why
the validation does not succeed.
class CalibrationChecker:
def __init__(self):
self.message = "calibration quality is insufficient"
def __call__(self, report):
from pix4dengine.utils.report import Quality
return all(q == Quality.SUCCESS for q in report.calibration_quality_status())
Use a validator with Pipeline¶
Similar behaviour can be accomplished with pipelines.
Pipeline may contain multiple tasks,
therefore one has to define which task will be validated.
This can be done by constructing a Pipeline
with the validators argument. This argument is a dictionary where keys are task names and values are previously
defined validators.
from pix4dengine.pipeline import Pipeline, apply_template
from pix4dengine.utils.errors import FailedValidation
from pix4dengine.pipeline.templates import RapidMaps3D
pipeline = Pipeline(
project=project, algos=["CALIB", "DENSE", "ORTHO"], validators={"CALIB": CalibrationChecker()}
)
apply_template(pipeline, RapidMaps3D) # fast processing
try:
pipeline.run()
except FailedValidation as stop:
print("Stopping processing, %s." % str(stop))
Note that CalibrationChecker will be called only after finishing the "CALIB" task.
Filtering outliers from a densified point cloud¶
The standard densification task can be configured to flag outliers (noise) in the generated point cloud. The outlier points are flagged and can later be filtered out by the meshing algorithm thus producing a better mesh.
In order to produce a point cloud where the outlier points are flagged, we need to do the following.
As before, we create a pipeline for processing a project. This pipeline will run two
tasks, the calibration task and the point cloud densification task. The latter will also run the
meshing algorithm. Next, we update the configuration of the densification task to make it flag the
outliers in the generated point cloud. Finally, we run the pipeline. If after densifying the
point cloud any outliers are detected, they will be filtered out by the meshing algorithm.
# Define a pipeline that includes a densification task
pipeline = Pipeline(project=project, algos=("CALIB", "DENSE"))
# Configure the "DENSE" task to flag outliers in the generated point cloud
dense_with_pointcloud_outliers_task = pipeline.get_task("DENSE")
dense_with_pointcloud_outliers_task.update_config(
{AlgoOption.Densification.PCL_FLAG_OUTLIERS: True}
)
# Run the pipeline
pipeline.run()
The outlier detection algorithm runs a statistical analysis of a point’s neighborhood and flags the points that don’t meet a given criterion as outliers. The statistical analysis computes the average distance from each point to its neighbors and flags a point as an outlier if that average distance is larger than the overall mean of all average distances plus some margin. The hypothesis is that the density of a point cloud around outlier points is much lower as fewer measurements agree together, thus the distance between an outlier point and its neighbors is bigger than the distance between a valid point and its neighbors.
The algorithm takes two input parameters, in addition to the point cloud itself, which define the neighborhood of a point, and the distance threshold. These parameters have to be tuned for each particular case depending on the nature of the noise in a point cloud. The first parameter is the number of neighbors of a point to consider when computing the average distance from that point to its neighbors. The second parameter controls the threshold on that distance, and is defined as the mean of all average distances plus a certain number of times the standard deviation of all average distances, i.e. t = x + SIGMA_COEF * σ.
The two parameters have reasonable default values for the case when the noise in the point cloud is random, i.e. there are single randomly distributed noisy points that don’t follow a pattern. These default parameters are 6 for the number of neighbors, and 1.0 for SIGMA_COEF. When the nature of the noise is such that the noisy points form clusters, the neighborhood has to be large enough to identify the clusters as noise. When the amount of noise is important, the overall mean of average distances between all points and their respective neighbors is going to increase. In order to filter out noisy points in that case, the threshold should be made smaller. In general, a smaller threshold means a more aggressive filtering.
The tuning of the algorithm with different values of the neighborhood and the threshold can be done by giving the size of the neighborhood and SIGMA_COEF to the standard densification task as part of its configuration.
# Tune the outlier flagging algorithm for:
# (1) the number of neighbors of a point to consider when computing a local average distance from a point to its neighbours
dense_with_pointcloud_outliers_task.update_config(
{AlgoOption.Densification.PCL_FLAG_OUTLIERS_NEIGHBOR_COUNT: 50}
)
# (2) the threshold on the standard deviation of the distribution of average local distances to neighbours
dense_with_pointcloud_outliers_task.update_config(
{AlgoOption.Densification.PCL_FLAG_OUTLIERS_SIGMA_COEF: 0.1}
)
# Run the pipeline
pipeline.run()
Processing multispectral projects¶
Pix4Dengine supports multispectral cameras and can thus be used for algriculture projects. The following code sample shows how to generate an NDVI map using multispectral images:
from pix4dengine.pipeline import Pipeline
from pix4dengine.constants.index import Indices
from pix4dengine.options import AlgoOption, ExportOption
# Define a pipeline where we run only CALIB and ORTHO
pipeline = Pipeline(project=project, algos=("CALIB", "ORTHO"))
# Get the ortho task
ortho = pipeline.get_task("ORTHO")
# Produce only the reflectance maps
ortho.update_config(
{
ExportOption.Ortho.MOSAIC_TIFF: False,
ExportOption.Ortho.MOSAIC_TIFF_MERGED: False,
ExportOption.Ortho.DSM_TIFF: False,
ExportOption.Ortho.DSM_TIFF_MERGED: False,
ExportOption.Index.REFLECTANCE: True,
ExportOption.Index.REFLECTANCE_MERGED: True,
}
)
# Add the NDVI index
ortho.update_config({AlgoOption.Index.INDICES: [Indices.NDVI.value]})
# Run the pipeline
pipeline.run()
We could have used the pix4dengine.pipeline.templates.AgriMultispectral template. The
main difference with the code above is that the AgriMultispectral template sets non-default
calibration options such as CALIBRATION_METHOD
and MATCH_GEOMETRICALLY_VERIFIED.
CUDA support¶
CUDA is NVidia proprietary technology for General-Purpose computing on Graphics Processing Units (GPUs). CUDA is used during calibration and usually brings significant speedups.
pix4dengine requires an NVidia GPU supporting CUDA 9.1, which corresponds to a compute capability version
of 3.0, and driver version 390.46 or greater. Further information can be found on
Wikipedia or on the NVidia
websites.
CUDA is supported in pix4dengine on both Ubuntu 18.04 and Windows and is used automatically if a compatible GPU
is available.
The function number_of_cuda_gpus() returns the number of compatible GPUs that can be leveraged.
When creating a Pipeline, the argument enable_cuda can be set to False in order
not to use CUDA.
Custom cameras¶
A custom camera configuration can be passed to the create_project() function.
Only this camera configuration shall be used, and it will not be saved to the database
pix4dengine uses two databases to store calibrated cameras.
We refer to them as “internal camera database” (ICDB) and “user camera database” (UCDB)
for cameras calibrated by Pix4D and users, respectively.
The UCDB is initially empty. Users may calibrate custom cameras and save them there for later use.
The purpose of this section is to illustrate how to do this.
Functions and types required to modify the user camera database are present in camera module.
User camera database¶
It is possible to access or change the location of the UCDB. The database can be erased.
from pix4dengine.camera import user_database_location, change_user_database_location
path_to_db = user_database_location()
print(f"System user configuration: {path_to_db}")
new_user_db = Path(tempfile.mkdtemp()) / "user_db.xml"
change_user_database_location(str(new_user_db))
new_path_to_db = user_database_location()
print(f"Temporary user db: {new_path_to_db}")
System user configuration: /home/user/.local/share/pix4d/common/19/ucmdb.xml Temporary user db: /tmp/tmp74ok79t7/user_db.xml
Remember that user camera database location is saved in the system configuration, per system user.
To temporarily change the location of the UCDB location
user_camera_database() managed context can be used,
as follows:
from pix4dengine.camera import user_camera_database, clean_userdb
with user_camera_database(new_user_db):
print(f"Just created temporary user db: {user_database_location()}")
with user_camera_database() as location:
print(f"Another temporary db: {location}")
with user_camera_database(path_to_db):
print(f"Saved before system configuration: {user_database_location()}")
clean_userdb()
Just created temporary user db: /tmp/tmp74ok79t7/user_db.xml Another temporary db: /tmp/pix4dengine_db_4vbkb738/user_db.xml Saved before system configuration: /home/user/.local/share/pix4d/common/19/ucmdb.xml
As can be seen, it is possible to clean up user database using clean_userdb().
Example¶
Before adding a custom camera to the database let’s have a look at an example where this is crucial: a wrong camera may be the cause of a failing pipeline.
project = create_project("test_project", image_directory, work_dir=tempfile.mkdtemp())
pipeline = Pipeline(project=project, algos=["CALIB"])
try:
pipeline.run()
except Exception:
print(f"Failed because cannot find calibrated camera.")
Failed because cannot find calibrated camera.
Retrieve a camera¶
A camera configuration (CameraConfig) can be retrieved from pix4dengine database
by name or with use of an image.
It will be the best camera that pix4dengine can use for that image.
from pix4dengine.camera import get_camera_config_from_image
path_to_image = glob(image_directory + "/*.JPG")[0]
camera_config, camera_id = get_camera_config_from_image(path_to_image)
print(f"Camera identifier {camera_id}.")
print(camera_config)
Camera identifier CanonIXUS220HS_4.3_4000x3000.
Camera:
Serial number=
Image width= 4000
Image height= 3000
Pixel size in micrometers= 1.5494
Maker name= Canon
Model name= CanonIXUS220HS
Bands=
Spectral band name= Red
Central wavelength in microns= 660
Width of the band in microns= 0
Band weight for grayscale mapping= 0.2126
Spectral band name= Green
Central wavelength in microns= 550
Width of the band in microns= 0
Band weight for grayscale mapping= 0.7152
Spectral band name= Blue
Central wavelength in microns= 470
Width of the band in microns= 0
Band weight for grayscale mapping= 0.0722
Shutter type= Global
Sensor=
Perspective Sensor:
Focal length= 35
Principal point x-cord= 3.12942
Principal point y-cord= 2.39692
First radial distortion parameter= 18
Second radial distortion parameter= 0.0259073
Third radial distortion parameter= -0.00608853
First tangential distortion parameter= 0.00119999
Second tangential distortion parameter= 0.00169852
Number of nonzero distortion parameters= 5
Lens model= None
Pixel values=
Vignetting=
Velocity number= 0
Rolling shutter readout time= 0
Camera can be retrieved by identifier. A camera identifier has a pattern
CameraModel_FocalLength_WidthxHeight or CameraModel_LensModel_FocalLength_WidthxHeight.
from pix4dengine.camera import get_camera_config_from_name
camera_config = get_camera_config_from_name("CanonIXUS220HS_4.3_4000x3000")
print(f"Retrieved camera: {str(camera_config)[:100]}...")
Retrieved camera: Camera:
Serial number=
Image width= 4000
Image height= 3000
Pixel size in micrometers= 1.5494
...
Add a camera to the user database.¶
A camera can be saved to the user database. Note that it has to be saved under a correct identifier
(CameraModel_FocalLength_WidthxHeight or CameraModel_LensModel_FocalLength_WidthxHeight).
In the code below it is camera_id from previous paragraph.
from pix4dengine.camera import add_camera_to_userdb
camera_config.sensor.focal_length_in_mm = 4.39974
camera_config.sensor.radial_K1 = -0.042563
add_camera_to_userdb(camera_config, camera_id)
project = create_project("test_project", image_directory, work_dir=tempfile.mkdtemp())
pipeline = Pipeline(project=project, algos=["CALIB"])
apply_template(pipeline, RapidMaps3D)
pipeline.run()
Note
pix4dengine will try to find the best camera while creating a project.
Therefore the camera should be added before project creation.
Remove a camera from the user database.¶
The camera configuration can be removed from the user database.
Do do so just use remove_cameras_from_userdb().
Note that there can be more than one camera that is mapped to this identifier.
from pix4dengine.camera import remove_cameras_from_userdb
removed_ids = remove_cameras_from_userdb(camera_id)
print(f"Removed: {removed_ids}.")
Removed: ['CanonIXUS220HS_4.4_4000x3000'].
After this operation our example will fail again because calibration cannot success without calibrated camera.
project = create_project("test_project", image_directory, work_dir=tempfile.mkdtemp())
pipeline = Pipeline(project=project, algos=["CALIB"])
apply_template(pipeline, RapidMaps3D)
try:
pipeline.run()
except Exception:
print(f"Failed.")
Failed.
Create a camera configuration manually¶
To add a new camera we can start by creating one from scratch, by creating an object of CameraConfig.
All required structures are documented in camera.
from pix4dengine.camera import CameraConfig, PerspectiveSensorConfig, BandConfig
camera_config = CameraConfig()
camera_config.image_width = 4000
camera_config.image_height = 3000
camera_config.pixel_size_in_um = 1.5494
camera_config.maker_name = "Canon"
camera_config.model_name = "CanonIXUS220HS"
bR = BandConfig(name="Red", central_wavelength=660, width=0, weight=0.2126)
bG = BandConfig(name="Green", central_wavelength=550, width=0, weight=0.7152)
bB = BandConfig(name="Blue", central_wavelength=470, width=0, weight=0.0722)
camera_config.bands = [bR, bG, bB]
camera_config.shutter_type = "Global"
sensor = PerspectiveSensorConfig()
sensor.focal_length_in_mm = 4.39974
sensor.principal_point_x_in_mm = 3.12942
sensor.principal_point_y_in_mm = 2.39692
sensor.radial_K1 = -0.042563
sensor.radial_K2 = 0.0259073
sensor.radial_K3 = -0.00608853
sensor.tangential_T1 = 0.00119999
sensor.tangential_T2 = 0.00169852
camera_config.sensor = sensor
add_camera_to_userdb(camera_config, camera_id)
project = create_project("test_project", image_directory, work_dir=tempfile.mkdtemp())
pipeline = Pipeline(project=project, algos=["CALIB"])
apply_template(pipeline, RapidMaps3D)
pipeline.run()
Use a custom camera configuration¶
A custom camera configuration can be specified explicitly when creating a project. In this case the camera database will not be used.
project = create_project(
"test_project", image_directory, work_dir=tempfile.mkdtemp(), camera_config=camera_config
)
pipeline = Pipeline(project=project, algos=["CALIB"])
apply_template(pipeline, RapidMaps3D)