====== iTwin Capture Analysis ======

This page describes how to use the "iTwin Capture Analysis Engine" in iTwin Capture Manage & Extract. 

iTwin Capture Analysis Engine requires a separate installation but does not require separate entitlement. \\
For iTwin Capture Analysis Engine installation guidelines, see [[237:technology:platforms:install_contextinsights|]].

{{orbit_desktop:how_to_find.png?25&nolink  |}} Main Toolbar > Extract > iTwin Capture Analysis

===== Concepts =====

==== iTwin Capture Analysis ====

iTwin Capture Analysis extension enables automatic detection on mapping and point cloud resources. \\
It relies on detectors to identify objects or regions of interest.

Running iTwin Capture Analysis detectors requires Python, iTwin Capture Analysis Engine, and original images or point cloud from any mapping resource. The results can be directly imported in the mapping resources via the Annotations procedure. 

==== Jobs ==== 
 
An iTwin Capture Analysis job will leverage machine learning to execute different types of detection in your reality data. \\
There are various types of jobs available:
  * Image 2D Objects: Will detect elements as regular 2D-bounding boxes in images. Regular 2D boxes around assets.
  * Image 2D Segmentation: Will detect regions in images and deliver segmented raster. Regions to segmented raster.
  * Point Cloud 3D Segmentation: Attribute a class to each point of laser scan data. Classify each element of a point cloud.

Each job type requires a suited detector to be executed on reality data. E.g: a 2D segmentation detector will only be used for image-based segmentation jobs

==== Image jobs ====

Special notes from Image-based jobs:
  * Only original jpg images are used, optimized omi images are not supported today.
  * A mask can be used to ignore image pixels from jobs. Place the image mask (TIF format) in <mapping resource>/<camera>/mask/mask.tif.

==== Detectors ====

A detector is a frozen model that is pre-trained by Bentley Systems. Running on your reality data through an iTwin Capture Analysis job, it will automatically recognize elements of interest. List of pre-trained detectors is available here: \\
https://communities.bentley.com/products/itwin-capture/w/itwin-capture-wiki/66882/itwin-capture-detectors-download-page

A detector is specific to a certain job type. The quality of the detection will depend on the similarity between your dataset and the training dataset’s description.

===== Launch iTwin Capture Analysis Engine =====

Launch iTwin Capture Analysis Engine \\

{{  :237:desktop_ext:context_insights_engine.png?nolink&600  |}}
===== iTwin Capture Analysis Sidebar =====

==== Preferences ====

Configure the directory to the iTwin Capture Analysis Engine detectors.
The detectors and specs are listed underneath. 

==== Action ====

Define whether you want to create a new job or open existing one
  * Create a new job and enter the target location of the results. 
  * Open an existing job and browse to the location of the results. 

==== Target or Analysis Engine Job ====

Define the target directory for the new job, or select the existing job.

==== Annotation Type ==== 

Select the type of annotation you want to detect. List list will be filled with annotations available for the added detectors.

=== Label Job === 

Define optional label for the job.

==== Detector ====

Select the detector to be used. Depending on the annotation type selected, this list will be filled with detectors available.

==== Reality Data ==== 

Select the source data: mapping run or point cloud resource, depending on the annotation type for the job.
If job is based on images, a mask can be selected in a second dropdown list.

==== Analyze ====

Delegate the process to the task manager or start now. Running the iTwin Capture Analysis Engine is required before starting the process. 

==== Results ====

=== Import Image Annotations ===

Open the Manage Image Annotations procedure where the result json file is already pre-filled as source object annotation file. \\
Import 2D Objects or 2D Segmentations, depending on the chosen Annotation Type.

=== Add Segmented Point Cloud ===

Add the result segmented point cloud to the data set list.

=== Import Objects ===

Open the ContextScene to Objects tool to import objects from the ContextScene result file. \\
The file path is already pre-filled. Use Source Options to select the type of objects you want to import and define target file location. 

=== Open File Location ===

Open the target directory to verify the results.