.. index:: 
  !Portrait processing

.. _portrait_processing:

====================
Portrait processing
====================

The :ref:`PortraitProcessor<id3_face_portrait_processor_class>` module facilitates the capture and analysis of portraits in various scenarios such as enrolment or face verification.

The module is fed with images and can perform the following operations:

* Face detection and tracking with prediction models
* Landmark detection (68 points)
* Head pose estimation
* Subject's position verification
* Template encoding and updating
* Age estimation
* Background uniformity computation
* Expression detection
* Age estimation
* Occlusion detection
* Presentation attack detection
* Background uniformity estimation
* Background removal
* Expression estimation
* Face attribute estimation (eye opening, eye gaze, face mask, gender, glasses, head covering, make-up, mouth opening, smile)
* Photographic attributes estimation
* Verification of compliance on a number of :ref:`quality check points<portrait_quality_check_points>`
* Unified quality score computation

.. important:: 
  The following AI models are required by the :ref:`PortraitProcessor<id3_face_portrait_processor_class>` module:

  - ``FaceDetector4B`` or another preferred face detector (see :ref:`face_detection` for details).
  - ``FaceEncoder9B``, or another preferred face encoder (see :ref:`face_encoding` for details).
  - ``FaceLandmarksEstimator2A``
  - ``FacePoseEstimator1A``
  - ``FaceAttributesClassifier2A``
  - ``FaceOcclusionDetector2A``
  - ``FaceColorBasedPad3A``

Creating and updating a portrait
================================

* **Creating a Portrait (Single Image):**

  For analysis based on a single image, use the :ref:`PortraitProcessor.createPortrait<id3_face_portrait_processor_create_portrait_class_method>` method. This method generates a :ref:`Portrait<id3_face_portrait_class>` object from a static image, enabling subsequent analysis (e.g., liveness detection or biometric processing).

* **Updating a Portrait (Live Capture):**

  For real-time or live video capture, the :ref:`PortraitProcessor.updatePortrait<id3_face_portrait_processor_update_portrait_class_method>` method is preferred. This method continuously updates the :ref:`Portrait<id3_face_portrait_class>` object with each new frame, ensuring a more dynamic and accurate analysis during live sessions.

* **Checking Portrait Status:**

  The current state of the portrait (e.g., whether it's initialized, being processed, or completed) can be monitored through the :ref:`Portrait.status<id3_face_portrait_status_class_member>` property. This property provides key status updates throughout the portrait's lifecycle, which can be used to control workflow logic.


.. _portrait_quality_check_points:

Quality check points
====================

The :ref:`PortraitProcessor<id3_face_portrait_processor_class>` module verifies the compliance to the :ref:`quality check points<id3_face_portrait_quality_checkpoints_struct>` defined below.

.. important:: The quality check points are verified only if the associated processing option is enabled.

Photographic quality
--------------------

* The brightness is well balanced.
* The image is coloured, not grayscale.
* The skin looks natural.
* The image resolution is correct.
* The dynamic range of the image is correct.
* The image is sharp.
* No flash reflection is visible.
* No noise is present in the image.
* The image is not pixelated.
* No red-eye is present.
* The background is uniform.

Facial attributes
-----------------

* The subject's face is frontal.
* The face expression is neutral.
* The eyes are visible.
* The eyes are open.
* The subject does not wear glasses.
* The subject looks straight towards the camera.
* The mouth is visible.
* The mouth is closed.
* The subject is not smiling.
* The subject does not wear a hat.
* The nose is visible.
  
Geometry
--------

* The height of the head in the image is correct.
* The width of the head in the image is correct.
* The horizontal position of the head in the image is correct.
* The vertical position of the head in the image is correct.

.. note:: 
  Vertical position is the distance from the bottom edge of the image to the imaginary line passing through the center of the eyes, in percentage of the total vertical length of the image.

Presentation attack detection
=============================

Presentation Attack Detection (PAD) is enabled through the :ref:`PortraitProcessor.detectPresentationAttack<id3_face_portrait_processor_detect_presentation_attack_class_method>` method. This method implements a passive liveness detection algorithm that analyzes multiple frames to differentiate between bona-fide users and potential spoofing attempts (such as photos or masks). For optimal performance, the user must position their face centrally within the frame.

The result of the detection is reflected in the :ref:`Portrait.padStatus<id3_face_portrait_pad_status_class_member>` property, which will indicate whether the portrait corresponds to a bona-fide subject or a presentation attack.

.. note:: 
  Liveness detection typically requires the analysis of several frames to ensure robustness. To enhance user experience and guide proper interactions, the :ref:`Portrait.instruction<id3_face_portrait_instruction_class_member>` property provides real-time feedback, containing specific instructions that help users adjust their position or actions.

.. important:: In certain conditions, such as low image quality or challenging environmental factors, accurate liveness detection may be difficult. It is crucial to assess the photographic quality of input images (e.g., lighting, focus) before invoking the PAD method to minimize the risk of false detections and improve the feedback provided to users.


Example
=======

The example below demonstrates how to analyze a portrait on an image:

.. tab-set::

  .. tab-item:: Python
    :sync: Python

    .. literalinclude:: /../samples/sample.py
      :language: python
      :start-after: [portrait_processing]
      :end-before: [portrait_processing]
      :dedent: 4



See also
========

- :ref:`id3_face_portrait_processor_class`
- :ref:`id3_face_portrait_class`
- :ref:`id3_face_portrait_quality_checkpoints_struct`