Face Dataset Creation
Module Description
Face Dataset Creation reads the names displayed on screen text-inserts and associates them to the corresponding person. Create an individual and unique dataset of your own repository of personalities.
Module ID: dataset_creation
Module Parameters
| Name | Type | Default | Description |
|---|---|---|---|
| dataset_id | string | null | The dataset ID to store the extracted faces (See Dataset Resource). |
| detect_single_name | boolean | false | Face Dataset Creation is optimized on detecting full names. Enable this to detect single names as well. |
| apply_generic | boolean | true | Use the generic method to find names |
| name_dictionary_list | List of strings | null | You can provide a custom names dictionary to detect your custom names. Only names from this list will be detected. Example: ["Angela Merkel", "Markus Söder"] |
| min_face_size | integer | 112 | Minimum size of the smallest side of the face in pixels |
| sharpness_threshold | integer | 40 | Minimum quality of the face (lower number is more blurry) |
| name_heuristic | string | v1 | Method used to detect names in the lower thirds. Possible values are: v1 (classic method), v2 (line-based detection with higher accuracy; supports hyphenated and multi-part names, academic titles and party affiliations after the name, and ignores headlines, credits and text in the scene). |
| allow_unknown_surnames | boolean | false | Only with name_heuristic v2: also detect names whose surname is not contained in the name dictionaries, if the layout of the lower third clearly identifies the name. Detects more names, but may add false detections. |
| max_text_tilt | number | 2.0 | Only with name_heuristic v2: maximum tilt of the name text in degrees (range from 0 to 90). More tilted text is treated as text in the scene (e.g. name plates or signs) and ignored. Increase the value if your lower thirds are slightly rotated or animated. |
| min_text_height | number | 0 | Only with name_heuristic v2: minimum height of the name text in percent of the frame height (range from 0 to 100). Smaller text, e.g. captions or text in the scene, is ignored. 0 disables this filter. |
| extract_subline | boolean | false | Add the text directly below the detected name, usually the role or function of the person (e.g. Spokesperson of the Federal Government), to the result as field subtitle. Works with both values of name_heuristic. |
| multi_line_subline | boolean | false | Only with extract_subline: also include further lines below the first subtitle line, separated by commas. Can include unrelated text directly below the lower third, such as channel logos. |
| report_rejected_names | boolean | false | Only with name_heuristic v2: also return names that were rejected during post-processing (e.g. detected in a single frame only, or a misread variant of another name) as results, with face_extracted set to false and the reason in face_status. Useful for troubleshooting; no faces are extracted for these names. |
Module Result
With name_heuristic set to v2, every detected name is returned as a result, also if no face could be extracted for it. Each result then contains these additional fields:
| Name | Type | Description |
|---|---|---|
| face_extracted | boolean | Whether a face was extracted to the dataset for this result. |
| face_status | string | extracted, or the reason why no face was extracted. Possible values are: no_face (no face found), face_too_small (smaller than min_face_size), multiple_faces (no single dominant face), face_blurry (below sharpness_threshold), image_limit_reached (enough images of this person extracted already). With report_rejected_names: name_single_frame, name_ocr_variant, name_text_moved, name_on_screen_too_long for names rejected during post-processing. |
Example
Send the following JSON as request body via POST to the /jobs/ endpoint:
{
"sources": [
"{url-to-your-image}"
],
"modules": {
"dataset_creation": {
"dataset_id": "{ID-of-your-dataset}"
}
}
}
In order to get the results you can request the /jobs/{JOB_ID}/detailed-results/ endpoint. For a job with "name_heuristic": "v2", "extract_subtitle": true and "report_rejected_names": true, the response looks like this:
{
"limit": 100,
"next": null,
"offset": 0,
"prev": null,
"total": 3,
"data": [
{
"detections": [
{
"data": {
"x": 0.412,
"y": 0.181,
"w": 0.142,
"h": 0.297
},
"frame_index": 1050,
"type": "face_bbox"
},
{
"data": {
"x": 0.415,
"y": 0.183,
"w": 0.141,
"h": 0.295
},
"frame_index": 1100,
"type": "face_bbox"
},
{
"data": {
"x": 0.413,
"y": 0.18,
"w": 0.143,
"h": 0.298
},
"frame_index": 1150,
"type": "face_bbox"
}
],
"frame_end": 1175,
"frame_start": 1050,
"id": "c2a7e5f1-8d4b-4a3e-b9c6-0e1f2d3a4b5c",
"media_type": "video",
"meta": {
"face_extracted": true,
"face_status": "extracted",
"from_dictionary": false,
"label": "Max Mustermann",
"subline": "Mayor of Sampletown"
},
"module": "dataset_creation",
"source": "storage://VSY8ihIGhR5qehVjWEWC",
"tc_end": "00:00:47:00",
"tc_start": "00:00:42:00",
"thumbnail": null,
"time_end": 47.0,
"time_start": 42.0
},
{
"detections": [],
"frame_end": 2850,
"frame_start": 2750,
"id": "5e9d3b2a-1f7c-4d6e-8a0b-c4e2f6a8d0b1",
"media_type": "video",
"meta": {
"face_extracted": false,
"face_status": "face_too_small",
"from_dictionary": false,
"label": "Erika Musterfrau",
"subline": "Head of Press Office"
},
"module": "dataset_creation",
"source": "storage://VSY8ihIGhR5qehVjWEWC",
"tc_end": "00:01:54:00",
"tc_start": "00:01:50:00",
"thumbnail": null,
"time_end": 114.0,
"time_start": 110.0
},
{
"detections": null,
"frame_end": 4100,
"frame_start": 4100,
"id": "7d4a1c9e-3b6f-4e2a-9c8d-2f5e7b1a0c36",
"media_type": "video",
"meta": {
"face_extracted": false,
"face_status": "name_single_frame",
"from_dictionary": false,
"label": "John Doe"
},
"module": "dataset_creation",
"source": "storage://VSY8ihIGhR5qehVjWEWC",
"tc_end": "00:02:44:00",
"tc_start": "00:02:44:00",
"thumbnail": null,
"time_end": 164.0,
"time_start": 164.0
}
]
}