What can I do with it? | API Reference | BRFv4 on Github | JS Demo

BRFv4 - What you can do with it.

Overview

Beyond Reality Face is a multi face tracker. It detects faces and tracks them continuously. It is a two step process using face detection and face tracking. This page shows what you can do with BRFv4 and what results to expect from each step.

face detection

(1st step)

It scans the image systematically to detect faces. A found candidate is marked by a (blue) rectangle. This produces lots of candidates and if enough (blue) rectangles are located in the same spot, this spot is marked as an actual found face by merging the blue rectangles into a yellow one.

Many blue candidates are merged into a yellow rectangle.
Many blue candidates are merged into a single yellow rectangle.
Restrict the region of interest (green rectangle) to set a specific detection area.
Restrict the region of interest (green rectangle) to set a specific detection area.
Detect small faces.
Detect small faces.
Detect large faces.
Detect large faces.

A merged (yellow) rectangle is used as a starting point for the actual face tracking.

face tracking

(2nd step)

A detected face is being analysed in detail and provides facial data: 68 facial landmarks, eg. the location of eyes, eyebrows, nose and mouth, also the 3D position, scale and rotation.

Provides the location of 68 facial landmarks.
Provides the location of 68 facial landmarks (points).
It is accurate even at an angle.
It is accurate even at an angle.
It is possible to calculate 6 additional points that also cover the forehead as well as...
It is possible to calculate 6 additional points that also cover the forehead (estimated, not tracked) or ...
... visualize the candide3 model based on the 3D position values.
... visualize the candide3 model based on the 3D position values.

These 68 facial landmarks may be utilized in many different ways, eg. ...

smile detection
... smile detection ...
yawn detection
... yawn detection ...
face tracking: blink detection
... (static) blink detection ...
face tracking: color triangles, eg. lips
... color triangles, eg. lips ...

... or for a bit more advanced stuff, like ...

Place image masks over a face.
... place masks on top of a face ...
Place other masks over a face.
... place other image masks on top of a face ...
Place 3d objects on a face.
... place 3d objects on top of a face ...
Place a face texture, using UV mapping.
... place a face texture, using UV mapping ...

point tracking

(optical flow, optional)

Apart from tracking the face you can also track points.

Track lots of points in a camera stream.
Track lots of points in a camera stream.
Track it simultaneously with face tracking.
Track them simultaneously with face tracking.
↑ top