AI Detection

This page describes how to execute AI-detectors on reality data to automatically extract relevant information

Main Toolbar > Extract > AI Detection

A detection job will use an AI-Detector to classify images or point clouds.

1. Input:
   • Collection of images (in JPG format)
   • Point cloud
2. Output:
   • Object in images
   • Region in images (class is attributed to each pixel of original image resulting in segmentation masks)
   • Classified point clouds
   • 3D Objects

A detector is specific to a certain job type. The quality of the detection will depend on the similarity between your dataset and the training dataset’s description.

A detector is a frozen model. Running on your reality data through an iTwin Capture Analysis job, it will automatically recognize elements of interest.

Personal detectors can be trained from AI Training sidebar and a list of generic/pre-trained detectors is available here:
https://communities.bentley.com/products/itwin-capture/w/itwin-capture-wiki/66882/itwin-capture-detectors-download-page

To enable sidebar, Analysis Engine must be launched

Define your main AI-detectors repository. This will list all AI-detectors available to execute AI-detection jobs. All currently production Detectors are available from the “Download Detectors” link.¹⁾

Select the reality you want your AI-detection job to be processed on. Old AI-detection job can also be opened from this section.

• Choose the resource type you want to work on (images, point cloud or orthophoto)
• Choose the AI-detection job-type you want to submit
• Choose AI-detector from list

Launch AI detection job. Processing automatically starts and terminates after completion. Processing time depends on input data size and detector type.

Results of AI detection can be imported after job completion

• 3D Segmentation: A classified point cloud (OPC) and possibly 3D-Objects (OVF) are generated and can be imported in main canvas for future operations
• 2D Objects / 2D Segmentation: A series of annotations is processed and can be imported to overlay original images after manage image annotations in dedicated wizard