Documents - iOS SDK V2

Binah’s HRV SDK descripti upon

(bnhhrvlib)

Status: Update

Version: 1.2

Date: March 01, 2020

Category: Released

©Bianh.AI LTD. 2016-2020

Contents

  1. Introduction        4
  1. Overview        4
  2. Background        4
  1. Requirements and dependencies iOS        5
  1. Basic tools        5
  1. Additional tools        5
  1. External API functions        6
  1. Description        6
  2. Function for continuous operation        6
  1. Bnh_hrv_sdk        6
  2. StartHRV        6
  3. getHRVStatus        7
  4. StopHRV        8        
  5. onResume        8
  6. onPause        9
  1.  Handling the mathematical engine messages        9
  1. Heart rate variability json string
  1.  References        14        

  1. Introduction
  1. Overview

Binah.AI Software Development Kit (SDK) supply a tool to extract a person’s heart rate, photo-plethysmography pulse trace, inter-beat interval, heart rate variability, and stress index. The pulse signal is extracted using remote photo-plethysmography (rPPG) from a video feed of a person’s face. The rPPG trace is reconstructed and the heart rate is calculated. Utilizing statistical modeling the heart rate variability and stress index are extracted.

We developed a second high-level SDK appropriate for use on an Android-based mobile device. This SDK controls both camera and computation of the heart rate parameters. This documentation describes only this SDK.

In subsection 1.2 we give the background of how the pulse rate is extracted from video. In section 2 we describe the requirements and installation procedure. Finally, in section 3 we describe in detail the library functions, allowing the user to build his own android based application.

  1. Background

Photo-Plethysmography (PPG) detects the optical absorption variations of the human skin due to blood volume variations during the cardiac cycle [1]. There are two categories of devices measuring the PPG: contact-based and remote (rPPG). In the latter configuration one measures the light reflection from a skin surface.

The main peaks in the rPPG trace are the systolic peaks [2]. The interval between successive systolic peaks is called the RR interval. From the RR intervals we extract the

instantaneous heart rate, in beats per minute, by the following 60/RR

The fluctuations in the instantaneous heart rate, indicate how the cardiovascular system is adjusted to sudden physical and psychological challenges to homeostasis. The measure of these fluctuations is referred to as heart rate variability (HRV). HRV corresponds to a collection of statistical data derived from the measured RR intervals [3].

The HRV parameters relate to the status of the sympathetic nervous system (SNS) and parasympathetic nervous system (PNS) activities. The SNS and PNS are indicators for the individual stress level, allowing to estimate the stress index.

  1. Requirements and dependencies iOS

The library was designed as standard IOS framework and implemented using Objective C/ C++.This gives ability to be used within all IOS configurations

  1. Basic tools:
  • Xcode 11
  • iOS Deployment target 12.4
  • The C++ flag used c++17 [-std=c++17]

  1. Additional tools:
  • We use render scripts for GPU usage
  • iFollowing framework required:
  • MessageUI
  • CoreMedia
  • UIKit
  • AVFoundation
  • Accelerate

  1. External API functions
  1. Description, how to link with application:
  • To run our demo app as simple as possible, go to the root folder and double click on the hrv_binah.xcworkspace. Then you can build, archive and upload to the store.

The binah_health.framework consist of three parts:

  • Header-header files to import
  • Binah SDK static library
  • Models.bundle

To link library with custom application:

  1. In the project navigator, select your project.
  2. Select your target.
  3. Select the "Build Phases" tab.
  4. Open "Link Binaries With Libraries" expander.
  5. Click the + button.
  6. Click “Add Other…” and navigate to binah_health.framework
  7. Select your binah_health.framework.

Next, the models.bundle which can be found inside the binah_health.framework folder must be added to application:

  1. In the project navigator, select your project.
  2. Select your target.
  3. Select the "Build Phases" tab.
  4. Open “Copy Bundle resources" expander.
  5. Open finder window and navigate to binah_health.framework
  6. Find and locate models.bundle and then drag it over “Copy Bundle Resources”

The final stage is quite simple

        In the project navigator, select your project.

  1. Select your target.
  2. Select the "Build Settings" tab.
  3. Navigate to “Search Paths” section
  4. In subsection “Framework Search Paths” add path for binah_health.framework

  1. Function for continuous operation

When using with ViewController following must be added:

#import <binah_health/binah_health.h>

The SDK object has to be defined within ViewController derived class

bnh_hrv_sdk *Hrv_Object_Name

And  delegate must be added

@interface ViewController :UIViewController<run_sdk_delegate,”additional delegates”>

  1. bnh_hrv_sdk initialization

-(void)initWithDelegate:(id)delegate withPreview:(UIImageView*) preview aspectRatio:(AspectRatio) aspect;

Description: starting the mathematical engine responsible for analyzing/ calculating the heart rate and RR interval calculation procedure. From the extracted data the heart rate variability is calculated.

Input parameters:

(id)delegate - where delegate (receiver)

(UIImageView*)preview - preview for camera stream

(AspectRatio) aspect – 16x9 or 4x3

Return: there is no return parameter.

Example run:

-(void) ViewDidLoad {

[super viewDidLoad];

Hrv_Object_Name=[[bnh_hrv_sdk alloc] init];

//Objective C:

[Hrv_Object_Name initWithDelegate:self withPreview:mImageView aspectRatio:Aspect_ratio16x9];

//Swift:

hr.initWithDelegate(self, withPreview: mImageView, aspectRatio: AspectRatio.ratio16x9)

  1. StartHRV

-(void) StartHRV:(BOOL) report withUserName:(nullable NSString*)username

Description: starting the mathematical engine responsible for analyzing/calculating the heart rate and RR interval calculation procedure. From the extracted data the heart rate variability is calculated.

Input parameters:

  • Boolean report. If ‘report’ value is true then once sufficient RR-intervals where acquired an HRV report will be saved. The report text file is saved in standard iOS container (documents). The report includes three columns: time, RR interval, and pulse rate.
  • String username. The user name is given in the subject filled. The input can not be null, in the app.

Return: there is no return parameter.

Example run:

In the ViewController we use this procedure as part of the function - (IBAction)clickOrientation:

(id)sender.

- (IBAction)clickOrientation:(id)sender {

UIButton *button = (UIButton *)sender;

if(button==bStartStop)

{

NSString *txt= bStartStop.currentTitle;

if([txt isEqualToString:@"Play"])

{

[email protected]"";

[pStress setEnabled:false];

report_string=nil;

[bStartStop setTitle:@"Stop" forState:UIControlStateNormal];

_timer = [NSTimer scheduledTimerWithTimeInterval:0.1 target:self

selector:@selector(timerTick:) userInfo:nil repeats:YES];

[pReport setEnabled:false];

[hr StartHRV:true withUserName:[pSubject text]];//GK

}

else{

[self stopHRV:FALSE];

}

}

}

See sample code.

  1. getHRVStatus

-(bool)getHRVStatus:(int) i

Description:  starting the mathematical engine responsible for analyzing/calculating the heart rate and RR interval calculation procedure. From the extracted data the heart rate variability is calculated.

Input parameters:

  • int i. Currently this input must be set to 1. It’s holding parameter for future versions.

Return: true/false boolean parameter. True if there are sufficient RR intervals for calculating the hear rate variability, otherwise the function returns false.

Example run:

-(void) stopHRV:(BOOL)received

{

if(received==FALSE)

{

bool complete=[hr getHRVStatus:1];

[hr StopHRV];

if(complete)

{

[pReport setEnabled:true];

}

else{

[pReport setEnabled:false];

}

}else{

[hr StopHRV];

[pReport setEnabled:true];

}

}

  1. StopHRV

-(void) StopHRV

Description:  stopping the mathematical engine processes, responsible for the heart rate and RR interval calculation procedure.

Input parameters: There are no input parameters.

Return: There is no return parameter.

Example run:

- (IBAction)clickOrientation:(id)sender {

UIButton *button = (UIButton *)sender;

if(button==bStartStop)

{

NSString *txt= bStartStop.currentTitle;

if([txt isEqualToString:@"Play"])

{

[email protected]"";

[pStress setEnabled:false];

report_string=nil;

[bStartStop setTitle:@"Stop" forState:UIControlStateNormal];

_timer = [NSTimer scheduledTimerWithTimeInterval:0.1 target:self

selector:@selector(timerTick:) userInfo:nil repeats:YES];

[pReport setEnabled:false];

[hr StartHRV:true withUserName:[pSubject text]];//GK

}

else{

[self stopHRV:FALSE];

}

}

}

  1. stopMeasures

-(void) stopMeasures

Description:  stopping the face recognition job in order to raise performance.

Input parameters: There are no input parameters.

Return: There is no return parameter.

Example run:

func SetScreenSleepMode(enableSleep : Bool) {

        if (hr != nil) {

            if (enableSleep) {

                hr.stopMeasures()

            } else {

                hr.restart(false)

            }

        }

  1. restart

-(bool) restart: (bool) report

Description:  Turning back face detection job to work

  • Input parameters:
  • Boolean report. If ‘report’ value is true then once sufficient RR-intervals where acquired an HRV report will be saved. The report text file is saved in standard iOS container (documents).

Return: There is no return parameter.

Example run:

func SetScreenSleepMode(enableSleep : Bool) {

        if (hr != nil) {

            if (enableSleep) {

                hr.stopMeasures()

            } else {

                hr.restart(false)

            }

        }

  1. Handling the mathematical engine messages

The standard delegate method used to provide messages between SDK and application.

-(void)didRecieveFace:(CGRect)img size:(CGSize)size

rotation:(CGFloat)angle ;

-(void)didRecieveNoFace;

-(void)didRecieveHeartRate:(double)bpm;

-(void)didRecieveHeartTrace:(double)trace;

-(void)didRecieveHeartReport:(NSString *)info;

-(void)didRecieveReportName:(NSString *)name;

-(void)didRecieveStressCategory:(NSString *)category;

-(void)didRecievedProgress:(float)progress;

-(void)didRecievespo2Trace:(double)spo;

-(void)didRecievespo2:(double)spo;

-(void)didRecieveBreathValue:(double)val;

The delegate -(void)didRecieveFace :(CGRect)img size:(CGSize)size rotation:(CGFloat)angle

Some additional explanations:

(CGRect)img represents the face rectangle, (CGSize)size is size of the view in which

the rectangle will be placed and (CGFloat)angle is the rotation angle of the whole resulting view.

See implementation in example code

  1. Heart rate variability json string

Once there are sufficient RR intervals, the heart rate variability information is calculated and transferred with a didRecieveHeartReport. In the following we present an example json format which includes both the json fields and sample values:

{

"timestamp": "22-08-2019 10-36-04",

"Report": {

"hr": {

"meanPulseRate": {

"value": 84,

"measurementUnit": "bpm"

},

"minPulseRate": {

"value": 78,

"measurementUnit": "bpm"

},

"maxPulseRate": {

"value": 100,

"measurementUnit": "bpm"

}

},

"stress": {

"value": 246,

"measurementUnit": "N.A."

},

"respiration": {

"value": 18,

"measurementUnit": "rpm"

},

"oxygen": {

"saturationLevel": {

"value": 98,

"measurementUnit": "%"

}

},

"bloodPressure": {

"systolic": 0,

"diastolic": 0

}

},

"FullData": {

"hr": {

"meanPulseRate": {

"value": 84,

"measurementUnit": "bpm"

},

"minPulseRate": {

"value": 78,

"measurementUnit": "bpm"

},

"maxPulseRate": {

"value": 100,

"measurementUnit": "bpm"

}

},

"stress": {

"value": 246,

"measurementUnit": "N.A."

},

"respiration": {

"value": 18,

"measurementUnit": "rpm"

},

"oxygen": {

"saturationLevel": {

"value": 98,

"measurementUnit": "%"

}

},

"bloodPressure": {

"systolic": 0,

"diastolic": 0

},

"TimeParameters": {

"meanRRInterval": {

"value": 718.37993500203606,

"measurementUnit": "ms"

},

"rmssd": {

"value": 23.06334681492342,

"measurementUnit": "ms"

},

"lnRmssd": {

"value": 3.1382446393889092,

"measurementUnit": "ms"

},

"sdrr": {

"value": 35.13932783049551,

"measurementUnit": "ms"

},

"pRr50": {

"value": 4.225352112676056,

"measurementUnit": "%"

},

"rr50": {

"value": 3,

"measurementUnit": "N.A."

},

"amo50ms": {

"value": 59.722222222222221,

"measurementUnit": "%"

},

"mxDmn": {

"value": 167.62961234502848,

"measurementUnit": "ms"

},

"si": {

"value": 246,

"measurementUnit": "N.A."

}

},

"FrequencyParameters": {

"totalPower": {

"value": 1377.1501387984904,

"measurementUnit": "ms2"

},

"vlfPower": {

"value": 409.77943082996745,

"measurementUnit": "ms2"

},

"lfPower": {

"value": 626.87297297714952,

"measurementUnit": "ms2"

},

"hfPower": {

"value": 340.49773499137405,

"measurementUnit": "ms2"

},

"lfToHf": {

"value": 1.8410488780286021,

"measurementUnit": "ms2"

}

}

},

"ExtraData": [

{

"time": {

"value": 24.771304574550534,

"measurementUnit": "s"

},

"value": {

"value": 0.76538211626086605,

"measurementUnit": "s"

},

"mean": {

"value": 0.74040135395043738,

"measurementUnit": "s"

}

},

…….

{

"time": {

"value": 26.309516916124231,

"measurementUnit": "s"

},

"value": {

"value": 0.76766885234236004,

"measurementUnit": "s"

},

]

}

  1. W. Verkruysse, L. O. Svaasand, and J. S. Nelson, ”Remote plethysmographic imaging using ambient light”, Opt. Express 16(26), pp. 21434-21445 (2008); G. de Haan, and V. Jeanne, IEEE Trans. Biomed. Eng. 60, 2878-2886 (2013).
  2. D. McDuff, S. Gontarek, R. W. Picard, “Remote detection of photoplethysmographic systolic and diastolic peaks using a digital camera”, IEEE Trans. Biomed. Eng. 61, 2948–2954 (2014).
  3. F. Shaffer, J. P. Ginsberg , “An Overview of Heart Rate Variability Metrics and Norms”, Frontiers in Public Health 5, 258 (2017).

Was this article helpful?
0 out of 0 found this helpful

Comments

0 comments

Please sign in to leave a comment.