NAV Navbar
c# http
  • Facial Recognition SDK
  • Windows .NET
  • Linux Java
  • Web API
  • Facial Recognition SDK

    The SDK allows you to capture faces, evaluate gender, age group, emotions, and create face templates to compare them and find similarity percentages.

    Face templates can be compared to each other, one-to-one, or one-to-many using a collection of templates.

    Windows .NET

    Description

    The SDK allows you to capture faces, evaluate gender, age group, emotions, and create face templates to compare them and find similarity percentages.

    Face templates can be compared to each other, one-to-one, or one-to-many using a collection of templates.

    The sdk also includes a video frame processor that simplifies detection, tracking, estimation, and facial recognition.

    Recognition Type

    There are two types of recognition:

    Fast

    Fast Recognition type is recommended in cases where the fastest recognition is needed and the accuracy of recognition is not as important. For example, this recognition type is great for video processing.

    Quality

    Quality Recognition type requires more processing time but the result is more accurate.

    Installation

    Download fast or quality recognition package from the developer portal, run setup, and follow installation steps.

    Requirements

    Software

    Facial Recognition SDK require .NET Framework v. 4.6.2.

    .NET Framework v. 4.6.2 can be download at microsoft.com

    Hardware

    Minimal

    Operating system: Windows 7 SP1 / 8 / 10 (64-bit)

    Processor: Intel Core i5-7500

    RAM: 8 GB

    Hard drive space: 16 GB

    Operating system: Windows 10 (64-bit)

    Processor: Intel Core i7-8700 / AMD Ryzen 7 2700

    RAM: 16 GB

    Hard drive space: 32 GB

    Ultimate performance (Hight density video processing)

    Operating system: Windows 10 (64-bit)

    Processor: Intel Core i9-7960X / AMD Ryzen Threadripper 1950X

    RAM: 32 GB

    Hard drive space: 32 GB

    Facial Recognition API usage

    SDK Initialization

    To initialize SDK create Settings object and pass it to constructor of FacialRecognitionServiceBuilder. Then call Build() method to create IFacialRecognitionService object

    var service = new FacialRecognitionServiceBuilder(new Settings
    {
      LicenseFilePath = path to idscan.facesdk.license,
      SdkDataDirectoryPath = path to SDK Data
    }).Build())
    

    IFacialRecognitionService inherit from IDisposable. SDK must be initialized only once at startup and disposed at the end. All the following calls of FacialRecognitionServiceBuilder.Buld() will throw an exception InvalidOperationException

    Set LicenseFilePath to path of idscan.facesdk.license file

    License file may be obtained from official store

    If SdkDataDirectoryPath field does not set SDK will look at default installation path.

    IFacialRecognitionService after initialization contain a list of sub services:

    Settings references:

    Properties Description
    LicenseFilePath Path to file containing license key (idscan.facesdk.licanse)
    SdkDataDirectoryPath SDK Data directory path
    TemplatesSearchThreadsCount Threads count for search in TemplateCollection
    ProcessingThreadsCount Threads count for processing
    IsEstimateEmotions Is estimate emotions in IFrameProcessor
    IsEstimateAge Is estimate age in IFrameProcessor
    IsEstimateGender Is estimate gender in IFrameProcessor
    IsEstimateLiveness Is estimate liveness in IFrameProcessor
    LivenessThresholdValue Liveness estimation threshold value (Accepts values from 0.0f to 0.75f)
    RecognizerSearchThreshold Recognizer search confidence (Accepts values from 0.0f to 1.0f)

    IFacialRecognitionService references:

    Properties Type Description
    Capturer ICapturer Face Capturer, used to captured faces from image file or video frame
    Recognizer IRecognizer Face Recognizer, used to create and process FaceTemplates
    FrameProcessor IFrameProcessor Video Frame Processor, simplify video processing workflow
    Templates TemplateCollection Templates collection, used in IRecognizer and IFrameProcessor
    Settings Settings Service settings

    Face Capturing

    First create service:

    //  create service
    Service = new FacialRecognitionServiceBuilder(new Settings
    {
      LicenseFilePath = ...,
      SdkDataDirectoryPath = path to SDK Data
    }).Build());
    
    //  use prebuiled ICapturer object
    Capturer = Service.Capturer;
    

    Get FaceSamples by calling Capturer.CaptureFaces(image1)

    var image1 = File.ReadAllBytes(imagePath1);
    var faceSamples = Capturer.CaptureFaces(image1);
    

    ICapturer used to detect and capture faces on image file or video frame.

    ICapturer references:

    Method Return type Description
    CaptureFaces(byte[] imageBytes) List<FaceSample> Capture FaceSamples on given image or videoframe. Supported encoded image with JPEG, PNG, TIF and BMP formats with 8 or 24 bits per pixel.
    CaptureFacesAsync(byte[] imageBytes) Task<List<FaceSample>> Capture FaceSamples async on given image or videoframe. Supported encoded image with JPEG, PNG, TIF and BMP formats with 8 or 24 bits per pixel.

    FaceNotDetectedException thrown when no one face founded.

    Face Sample

    To estimate various parameters of captured face use this code snippet:

    _service = new FacialRecognitionServiceBuilder(new Settings
    {
      LicenseFilePath = ...,
      SdkDataDirectoryPath = path to SDK Data
    }).Build());
    
    var imageData = File.ReadAllBytes(imagePath);
    var faceSamples = _service.Capturer.CaptureFaces(imageData);
    
    foreach(var faceSample in faceSamples)
    {
      var gender = faceSample.EstimateGender();
      var ageInfo = faceSample.EstimateAge();
      var emotionsInfo = faceSample.EstimateEmotions();
      var faceQualityInfo = faceSample.GetFaceQuality();
    }
    

    FaceSample incapsulate all functions related to captured faces.

    FaceSample references:

    Method Return type Description
    Face Face Face parameters
    Track int Track number (> 0 if created by IFrameProcessor)
    Template FaceTemplate Face template
    Confidence float Match confidence (used with IFrameProcessor)
    ComputeTemplate() FaceTemplate Compute face template
    ComputeTemplateAsync() Task<FaceTemplate> Compute face template async
    EstimateGender() Gender Estimate gender
    EstimateGenderAsync() Task<Gender> Estimate gender async
    IsRecognized bool Indicate recognition state (used with IFrameProcessor)
    MatchedTemplate FaceTemplate Matched FaceTemplate from current TemplateCollection recognized in IFrameProcessor
    EstimateAge() AgeInfo Estimate Age
    EstimateAgeAsync() Task<AgeInfo> Estimate Age async
    EstimateEmotions() EmotionsInfo Estimate emotions
    EstimateEmotionsAsync() Task<EmotionsInfo> Estimate emotions async
    CroppFace() byte[] Cropp face (cropped face encoded in PNG format)
    CroppFaceAsync() Task<byte[]> Cropp face async (cropped face encoded in PNG format)
    GetFaceQuality() FaceQualityInfo Estimate face quality
    GetFaceQualityAsync() Task<FaceQualityInfo> Estimate face quality async
    SearchTemplate() SearchResult Search for the most similar template from the TemplateCollection. Templates with a matching result below the Settings.RecognizerSearchThreshold are ignored.
    SearchTemplateAsync() Task<SearchResult> Search async for the most similar template from the TemplateCollection. Templates with a matching result below the Settings.RecognizerSearchThreshold are ignored.

    Face references:

    Properties Return type Description
    LeftEye Point Left eye coordinates
    RightEye Point Right eye coordinates
    Rectangle Rectangle Face rectangle on origin image
    Gender Gender Gender
    Age float Age as float value
    Liveness Liveness Liveness value (used with IFrameProcessor when Settings.IsEstimateLiveness is set to true)
    QualityInfo FaceQualityInfo Quality parameters

    Face Template

    To create FaceTemplate use this code snippet:

    var image1 = File.ReadAllBytes(imagePath1);
    var faceTemplate = _service.Recognizer.CreateTemplate(image1);
    faceTemplate.Id = 1;
    faceTemplate.Info = "Face 1";
    faceTemplate.Tag = "group 1";
    

    To create FaceTemplate use this code snippet:

    var faceTemplate = faceSample.ComputeTemplate();
    faceTemplate.Id = 2;
    faceTemplate.Info = "Face 2";
    faceTemplate.Tag = "group 2";
    

    FaceTemplate references:

    Properties Return type Description
    Id long User defined id
    Data byte[] Template data
    RecognizerType RecognizerType Recognizer Type - Fast or Quality - corresponding to SDK redistributable
    Info string User defined text info
    Tag string User defined tag

    IRecognizer references:

    Method Return type Description
    CreateTemplate(byte[] imageBytes) FaceTemplate Create FaceTemplate from given image. Supported encoded image with JPEG, PNG, TIF and BMP formats with 8 or 24 bits per pixel. The image must contain only one face, in other case InvalidOperationException would be thrown.
    CreateTemplateAsync(byte[] imageBytes) Task<FaceTemplate> Create FaceTemplate async from given image. Supported encoded image with JPEG, PNG, TIF and BMP formats with 8 or 24 bits per pixel. The image must contain only one face, in other case InvalidOperationException would be thrown.
    CompareTemplates(FaceTemplate first, FaceTemplate second) float Compare two previously created FaceTemplate
    CompareTemplatesAsync(FaceTemplate first, FaceTemplate second) float Compare async two previously created FaceTemplate
    SearchTemplate(FaceTemplate template) SearchResult Search for the most similar template from the TemplateCollection. Templates with a matching result below the Settings.RecognizerSearchThreshold are ignored.
    SearchTemplateAsync(FaceTemplate template) Task<SearchResult> Search async for the most similar template from the TemplateCollection. Templates with a matching result below the Settings.RecognizerSearchThreshold are ignored.

    Face Templates comparison

    To compare two FaceTemplate use this code snippet:

    var image1 = File.ReadAllBytes(imagePath1);
    var image2 = File.ReadAllBytes(imagePath2);
    var faceTemplate1 = _service.Recognizer.CreateTemplate(image1);
    var faceTemplate2 = _service.Recognizer.CreateTemplate(image2);
    var percent = _service.Recognizer.CompareTemplates(faceTemplate1, faceTemplate2);
    

    InvalidOperationException thrown when detected more then one face.

    FaceNotDetectedException thrown when image does not contain any face.

    CreateTemplateException thrown when somethong goes wrong.

    CompareTemplatesException

    To search captured face in TemplateCollection use this code snippet:

    _service = new FacialRecognitionServiceBuilder(new Settings 
    {
      LicenseFilePath = ...,
      SdkDataDirectoryPath = path to SDK Data
    }).Build());
    
    var imageData = File.ReadAllBytes(imagePath);
    var faceSamples = _service.Capturer.CaptureFaces(imageData);
    
    foreache(var faceSample in faceSamples)
    {
      var searchResult = faceSample.SearchTemplate();
    }
    

    SearchTemplateException thrown when FaceTemplate RecognizerType mismatch

    To search FaceTemplate in TemplateCollection use this code snippet:

    var imageData = File.ReadAllBytes(imagePath);
    var template = _service.Recognizer.CreateTemplate(imageData);
    var searchResult = _service.Recognizer.SearchTemplate(template);
    

    SearchTemplateException thrown when FaceTemplate RecognizerType mismatch

    SearchResult references:

    Properties Return type Description
    TemplateId long Searched FaceTemplate id
    Percent float Search confidence percent
    IsMatched bool True if matched with one of FaceTemplate in TemplateCollection
    Template FaceTemplate Searched FaceTemplate

    Face Templates Collection

    To store FaceTemplate use this code snippet:

    var faceTemplate = faceSample.ComputeTemplate();
    faceTemplate.Id = 2;
    faceTemplate.Info = "Face 2";
    faceTemplate.Tag = "group 2";
    
    _service.Templates.AddTemplate(faceTemplate);
    

    TemplateCollection used to store FaceTemplates and for templates searching and comparison.

    TemplateCollection references:

    Method Return type Description
    DeleteTemplate(long id) bool Delete template with corresponding id. Returns true if template successfully deleted.
    DeleteTemplateAtIndex(int index) - Delete template at index
    AddTemplate(FaceTemplate template) - Add FaceTemplate
    AddTemplates(IEnumerable<FaceTemplate> templates) - Add range of FaceTemplate
    DeleteAllTemplates()) - Delete all FaceTemplates
    GeTemplates()) List<FaceTemplate> Get all FaceTemplate
    SetTemplates(List<FaceTemplate> templates) - Reset all FaceTemplate and set new list
    GetTemplateAtIndex(long index) FaceTemplate Get FaceTemplate at given index
    GetTemplatesCount() int Get FaceTemplate count
    GetTemplate(Func<FaceTemplate, bool> func) FaceTemplate Get FaceTemplate

    Video Frame Processor

    Basic usage of IFrameProcessor :

    var settings = new Settings();
    _service = await new FacialRecognitionServiceBuilder(settings)
      .BuildAsync();
    _service.FrameProcessor.Tracking += OnTracking;
    _service.FrameProcessor.TemplatedMatched += OnTemplatedMatched;
    ...
    
    _service.FrameProcessor.Process(image, width, height);
    
    private void OnTemplatedMatched(object sender, FaceSample e)
    {
      ... do stuff ...
    }
    
    private void OnTracking(object sender, TrackingInfo trackingInfo)
    {
      foreach (var face in trackingInfo.FaceSamples)
      {
        ... do stuff ...
      }
    }
    

    IFrameProcessor simplifies face detection, tracking, estimation, and facial recognition.

    IFrameProcessor references:

    Method Return type Description
    Stop() - Stop frame processing.
    Start() - Start frame processing
    Process(byte[] bytes, int width, int height) - Process frame. Supported encoded image with JPEG, PNG, TIF and BMP formats with 8 or 24 bits per pixel.
    ProcessRawPixels(byte[] bytes, int width, int height, FramePixelFormat pixelFormat) - Process frame represented as raw pixel data. Accept raw RGB 24 bit per pixel data.
    event EventHandler<TrackingInfo> Tracking TrackingInfo Tracking event, called after frame processed
    event EventHandler<List<FaceSample>> TrackingLost List<FaceSample> TrackingLost event, called if FaceSample lost in last processed frame. Return list of all missed FaceSamples
    event EventHandler<FaceSample> TemplatedMatched FaceSample TemplatedMatched event, called when FaceSample processing at current frame matched with one from TemplateCollection
    event EventHandler<long> NewTrack long NewTrack event, called if new face detected at last processed frame
    IsProcessing bool Determines the processing state

    TrackingInfo references:

    Properties Return type Description
    FrameWidth int Frame width
    FrameHeight int Frame height
    ProcessingTime long Processing time in ms
    FrameId long Frame id

    Samples

    SDK Contain a set of samples with source code. Source code can be founded at [SdkInstallationPath]/Samples

    FaceMatchingSample

    FaceMatchingSample created to show basic API usage such as creating FaceSamples, creating FaceTemplates, storing FaceTemplates, usage of TemplateCollection and FaceTemplate comparison.

    FaceTrackingWpfSample

    FaceTrackingWpfSample shows the result of IFrameProcessor work such as face tracking, creating FaceTemplate from file and realtime face recognition.

    Linux Java

    Description

    The SDK allows you to capture faces, evaluate gender, age group, emotions, and create face templates to compare them and find similarity percentages.

    Face templates can be compared to each other, one-to-one, or one-to-many using a collection of templates.

    Recognition Type

    There are two types of recognition:

    Fast

    Fast Recognition type is recommended in cases where the fastest recognition is needed and the accuracy of recognition is not as important. For example, this recognition type is great for video processing.

    Quality

    Quality Recognition type requires more processing time but the result is more accurate.

    Installation

    Download SDK package from developer portal. The archive contains following parts:

    Add FaceSDK-fast-x86-64-1.0.0.jar or FaceSDK-quality-x86-64-1.0.0.jar to the Java project as a regular java library.

    Requirements

    Software

    Facial Recognition SDK requires 64-bit JDK 8 or newer.

    Information about installation of JDK 8 on Linux can be found at oracle.com

    Hardware

    Minimal

    Facial Recognition API usage

    Analyze faces

    // 1. Create service instance.
    FacialRecognitionService srv = FacialRecognitionService.createInstance("LICENSE_KEY");
    // 2. Find faces.
    List<Face> faces = srv.findFaces(input);
    // 3. Analyze faces.
    for(Face f : faces) {
        FaceAnalytics analytics = srv.analyzeFace(f);  
        ...
    }
    

    For analyze faces on the image you need:

    1. Create instance of the Facial Recognition Service by calling FacialRecognitionService.createInstance(String licKey). As a result, you will get instance of the Facial Recognition Service that can be used to analyze faces on images.
    2. Find faces on the image by calling FacialRecognitionService.findFaces(Image image). As a result, you will get the set of faces that have been detected on the Image. Each entity in the set contains the basic information about the face such as position of eyes, position the face on the image and set of characteristic points.
    3. Analyze the faces by calling either FacialRecognitionService.analyzeFace(Face face) or FacialRecognitionService.analyzeFace(Face face, int flags). As a result, you will get advanced information about faces on the image such as emotions, age and gender.

    Matching faces

    For matching faces you need build Templates and TemplateIndexes. With Templates you can match faces each other. With TemplateIndexes you cat search one face in set of faces.

    // 1. Create service instance.
    FacialRecognitionService srv = FacialRecognitionService.createInstance("LICENSE_KEY");
    // 2. Find faces on the first photo.
    List<Face> faces1 = srv.findFaces(photo1);
    // 3. Find faces on the second photo.
    List<Face> faces2 = srv.findFaces(photo2);
    //4. Compare two faces.
    double distance = srv.compareFaces(faces1.get(0), faces2.get(0));
    

    For matching two faces each other you can either build Template for each face by calling FacialRecognitionService.buildTemplate(Face f) and then compare templates by calling compareFaces(Template first, Template second) or call compareFaces(Face first, Face second). In the second case templates for the faces will be build implicitly. As a result, you will get normal distance between two faces in range [0.0, 1.0]. The 0.0 value means the faces are similar, 1.0 value means the faces are different.

    // 1. Create service instance.
    FacialRecognitionService srv = FacialRecognitionService.createInstance("LICENSE_KEY");
    // 2. Create index builder.
    TemplateIndexBuilder<String> indexBuilder = srv.buildIndex();
    // 3. Add faces to index.
    List<Face> faces = ...;
    for(Face f : faces) {
      indexBuilder.add(f, "Some Name");
    }
    // 4. Build index.
    TemplateIndex<String> index = indexBuilder.build();
    // 5. Search face in index.
    Face face = ...;
    FaceMatch<String> match = index.search(f);
    

    For searching face in set of faces you need:

    1. Create TemplateIndexBuilder by calling FacialRecognitionService.buildIndex();
    2. Add one or more faces to the index by calling either TemplateIndexBuilder<T>.add(Face face, T data) or TemplateIndexBuilder<T>.add(Template template, T data). In the second case template will be build implicitly.
    3. Build TemplateIndex by calling TemplateIndexBuilder<T>.build().
    4. Search the face in the index by calling either TemplateIndex<T>.search(Face face) or TemplateIndex<T>.search(Template template). As a result, you will get FaceMatch structure.

    API references

    Samples

    SDK contains samples with source code. Runnable example can be found at [SdkPath]/example/bin. Source code of the example can be found at [SdkPath]/example-sources.

    Matching faces on group photo

    ./facesdk-example match --faces ./data/faces --photo ./data/group.jpg > ./data/group.json
    
    • --faces - directory with photos.
    • --photo - group photo.
    • stdout - result of matching.
    ./facesdk-example visualize --photo ./data/group.jpg  --input ./data/group.json --output ./data/result.png
    
    • --photo - group photo.
    • --input - result of match command.
    • --output - result image.

    match command matches faces from the specific directory with faces on the group photo. As a result, you will get description of matching faces in JSON format. You can visualize this JSON data by using visualize command.

    Comparing faces

    ./facesdk-example distance first_photo.jpg second_photo.jpg
    
    • first_photo.jpg - photo with first face.
    • second_photo.jpg - photo with second face.

    distance command compares two faces each other. As a result, you will get distance between two faces in range [0.0, 1.0], where 0.0 value means the faces are similar and 1.0 value means the faces are different.

    Web API

    Endpoint: http://faces.idscan.net/upload.php

    Token is required: “token”: “[Authentication Token]”

    Methods

    Analytics

    Upload one image in query body. Returns the analytics after uploading the image.

    POST-query example:

        POST /upload.php HTTP/1.1
        Host: faces.idscan.net
        Referer: http://faces.idscan.net/upload.php
        User-Agent: Mozilla
        Content-Type: multipart/form-data; boundary=-------------573cf973d5228
        Content-Length: 288
        Connection: keep-alive
        Keep-Alive: 300
    
        ---------------573cf973d5228
        Content-Disposition: form-data; name="file"; filename="1.jpg"
        Content-Type: image/jpeg
    
        Content file
        ---------------573cf973d5228--
    

    Compare 2 images

    Upload two images in query body. Returns match of the 2 images.

    POST-query example:

        POST /upload.php HTTP/1.1
        Host: faces.idscan.net
        Referer: http://faces.idscan.net/upload.php
        User-Agent: Mozilla
        Content-Type: multipart/form-data; boundary=-------------573cf973d5228
        Content-Length: 288
        Connection: keep-alive
        Keep-Alive: 300
    
        ---------------573cf973d5228
        Content-Disposition: form-data; name="file" filename="1.jpg"
    
        Content-Type: image/jpeg
    
    
        Content file
        ---------------573cf973d5228
        Content-Disposition: form-data; name="file2"; filename="2.jpg"
        Content-Type: image/jpeg
    
        Content file
        ---------------573cf973d5228--
    

    Live demo

    Live demo can be found at http://faces.idscan.net