Documents - Android SDK V2

Binah’s HRV SDK description

(bnhhrvlib)

Status: Update

Version: 1.2

Date: March 01, 2020

Category: Released

©Bianh.AI LTD. 2016-2020

Contents

  1. Introduction        4
  1. Overview        4
  2. Background        4
  1. Requirements and dependencies Android        5
  1. Basic tools        5
  1. Additional tools        5
  1. External API functions        6
  1. Description        6
  2. Function for continuous operation        6
  1. Bnh_hrv_sdk        6
  2. StartHRV        6
  3. getHRVStatus        7
  4. StopHRV        8        
  5. onResume        8
  6. onPause        9
  1.  Handling the mathematical engine messages        9
  1. Heart rate variability json string
  1.  References        14        

  1. Introduction
  1. Overview

Binah.AI Software Development Kit (SDK) supply a tool to extract a person’s heart rate, photo-plethysmography pulse trace, inter-beat interval, heart rate variability, and stress index. The pulse signal is extracted using remote photo-plethysmography (rPPG) from a video feed of a person’s face. The rPPG trace is reconstructed and the heart rate is calculated. Utilizing statistical modeling the heart rate variability and stress index are extracted.

We developed a second high-level SDK appropriate for use on an Android-based mobile device. This SDK controls both camera and computation of the heart rate parameters. This documentation describes only this SDK.

In subsection 1.2 we give the background of how the pulse rate is extracted from video. In section 2 we describe the requirements and installation procedure. Finally, in section 3 we describe in detail the library functions, allowing the user to build his own android based application.

  1. Background

Photo-Plethysmography (PPG) detects the optical absorption variations of the human skin due to blood volume variations during the cardiac cycle [1]. There are two categories of devices measuring the PPG: contact-based and remote (rPPG). In the latter configuration one measures the light reflection from a skin surface.

The main peaks in the rPPG trace are the systolic peaks [2]. The interval between successive systolic peaks is called the RR interval. From the RR intervals we extract the

instantaneous heart rate, in beats per minute, by the following 60/RR

The fluctuations in the instantaneous heart rate, indicate how the cardiovascular system is adjusted to sudden physical and psychological challenges to homeostasis. The measure of these fluctuations is referred to as heart rate variability (HRV). HRV corresponds to a collection of statistical data derived from the measured RR intervals [3].

The HRV parameters relate to the status of the sympathetic nervous system (SNS) and parasympathetic nervous system (PNS) activities. The SNS and PNS are indicators for the individual stress level, allowing to estimate the stress index.

  1. Requirements and dependencies Android

Here we describe requirements and dependencies of Binah’s android SDK (Android).
The library was designed and developed for Android version 8.1(Oreo), using Android Studio.

2.1 Basic tools:

  • Android Studio 3.5.2
  • Compile SDK version 27
  • Minimum SDK version 27
  • Target SDK version 28

2.2 Additional tools:

  • implementation 'com.android.support:appcompat-v7:26.0.0'
  • We use render scripts for GPU usage.


  1. External API functions

3.1 Description:

The API is decomposed to two levels: video camera data processor and frame rendering; and message handling protocol which is responsible for data exchange between mathematical engine and host application. On the other hand, the API also invokes the implementation of Binah’s heart rate computation which is called mathematical engine.  

The mathematical engine is a collection of methods used to support the computation of the heart rate and the HRV parameters and are described in ‎3.2. The message handling protocol is used to transfer the heart rate, heart rate variability, and stress index by standard Java technique (Handler and Message classes). This is described in ‎3.3.

Binah have prepared a sample android app using the library, which is given upon request. This app's purpose is to ease the process of preparing a new Android-based app. The app code is written in JAVA, the underlying calculations are calculated using c++.

An example of how to use the SDK can be found in the provided example app.  Specifically, the file HrvActivity.java contains the usage of the SDK.

3.2 Function for continuous operation

Here we describe the usage of the video camera data processor and frame rendering by applying an instance of the class named ‘bnh_hrv_sdk’.  First import our aar as a module to the project.

You can read here:

https://developer.android.com/studio/projects/android-library

Under: "Add your library as a dependency".

These are the steps on a new project:

1. Add to build.gradle , under dependencies

implementation project(':bnhhrlib')

2. Add to settings.gradle :

include ':bnhhrlib'

3. In testing activity:

import ai.binah.hrv.*;

To use SDK the following lines must be added

package ai.binah.example;

import ai.binah.hrv.bnh_hrv_sdk;

and instance must be initialized using class constructor described in ‎3.2.1.  The example name will be used throughout this section.

3.2.1 bnh_hrv_sdk

public bnh_hrv_sdk(Context cnt, Activity activity, AutoFitTextureView miTextureView, Handler miSDKResponeHadler, bnh_hrv_sdk.AspectRatios ar)

Description: The class constructor sets up the camera and initializes the mathematical engine.

Input parameters: 

Context cnt - The application Context.

Activity activity - The application Activity is required.  

AutoFitTextureView miTextureView - preview for camera stream

Handler miSDKResponeHadler - The Handler to which the messages with the heart rate information will be sent to.

bnh_hrv_sdk.AspectRatios ar - Aspect ratio (16X9 or 4X3)

Example run:

mAlgo = new bnh_hrv_sdk(getApplicationContext(),this, mImageDisplay,mSDKResponeHadler, bnh_hrv_sdk.AspectRatios.Aspect_ratio4x3);

The constructed call can be found in the function protected void onCreate(Bundle savedInstanceState)

3.2.2 StartHRV

public void StartHRV(boolean report)

Description: starting the mathematical engine responsible for analyzing/calculating the heart rate and RR interval calculation procedure.  From the extracted data the heart rate variability is calculated.  

Input parameters:

Boolean report - If ‘report’ value is true then once sufficient RR-intervals where acquired an HRV report will be saved.  The report text file is saved in downloads/binah/username. The report includes three columns: time, RR interval, and pulse rate.  

Return: there is no return parameter.

Example run:

In the HrvActivity.java we use this procedure as part of the function public void startHRV(View v). A sample code:

String subject_str = mEditSubject.getText().toString();
if(subject_str.isEmpty())
   
subject_str = "no_subject";
mSDK.StartHRV(false);

3.2.3 getHRVStatus

public boolean getHRVStatus(int i)

Description: Checking if there are enough RR intervals for calculating the heart rate variability and stress index.  

Input parameters:

int i - Currently this input must be set to 1. It’s holding parameter for future versions.

Return: true/false boolean parameter. True if there are sufficient RR intervals for calculating the hear rate variability, otherwise the function returns false. 

Example run:

In the HrvActivity.java we use this procedure as part of the function public boolean getHRVStatus(). The function is used as part of StopHrv() function:

boolean x= mSDK.getHRVStatus();
mSDK.StopHRV();
mHRVRunning=
false;
if (x) {
   enablePlayButton();
   
return;
}


3.2.4 StopHRV

public void StopHRV()

Description: stopping the mathematical engine processes, responsible for the heart rate and RR interval calculation procedure.   

Input parameters: There are no input parameters.

Return: There is no return parameter.

Example run:

In the HrvActivity.java we use this procedure as part of the function public void StopHRV(). The function is used as part of StopHrv() function:

boolean x= mSDK.getHRVStatus();
mSDK.StopHRV();
mHRVRunning=false;
if (x) {
   enablePlayButton();
   
return;
}

3.2.5 onResume

public void onResume()

Description: Resumes or starts the camera operation (listener).  In addition, it uses Java internal Message class to transfer the results of the heart rate and RR interval. In section ‎3.3, we describe the details of the Messages.

Input parameters: There are no input parameters.

Return: There is no return parameter.

Example run:

In the HrvActivity.java we use this procedure as part of the function protected void onResume(). 

@Override
protected void onResume() {
   
mSDK.onResume();
   
super.onResume();
}

3.2.6 onPause

public void onPause()

Description: Closes the camera operation (listener).

Input parameters: There are no input parameters.

Return: There is no return parameter.

Example run:

In the HrvActivity.java we use this procedure as part of the function protected void onPause(). 

@Override
protected void onPause() {
   
mSDK.onPause();
   
super.onPause();
}

3.3 Handling the mathematical engine messages 

In this section we describe the different messages using the Java Handler and Message classes.  The Handler is defined in the bnh_hrv_sdk constructor (subsection ‎3.2.1): Handler miSDKResponeHadler.

In the example app, file HrvActivity.java, we show how to receive the messages using the overloaded function public void handleMessage(Message msg).  

Each message structure includes at least two fields: what, and obj.  The what field is an integer number to specify the type of information.  The obj filed contains the actual data.  

The different possible messages (what fields) are categorized as follows:

  1. The heart rate value.
  2. The rPPG heart rate trace value (ranges between -1.0 and 1.0).
  3. The json string holding all of the heart rate variability and stress fields.  The detail information of the json is given in ‎3.3.1.
  4. The stress level string.
  5. The current frame (Bitmap format) acquired from mobile android front camera.

3.3.1 Heart rate variability json string

Once there are sufficient RR intervals, the heart rate variability information is calculated and transferred with a didRecieveHeartReport. In the following we present an example json format which includes both the json fields and sample values:

{

"timestamp": "22-08-2019 10-36-04",

"Report": {

"hr": {

"meanPulseRate": {

"value": 84,

"measurementUnit": "bpm"

},

"minPulseRate": {

"value": 78,

"measurementUnit": "bpm"

},

"maxPulseRate": {

"value": 100,

"measurementUnit": "bpm"

}

},

"stress": {

"value": 246,

"measurementUnit": "N.A."

},

"respiration": {

"value": 18,

"measurementUnit": "rpm"

},

"oxygen": {

"saturationLevel": {

"value": 98,

"measurementUnit": "%"

}

},

"bloodPressure": {

"systolic": 0,

"diastolic": 0

}

},

"FullData": {

"hr": {

"meanPulseRate": {

"value": 84,

"measurementUnit": "bpm"

},

"minPulseRate": {

"value": 78,

"measurementUnit": "bpm"

},

"maxPulseRate": {

"value": 100,

"measurementUnit": "bpm"

}

},

"stress": {

"value": 246,

"measurementUnit": "N.A."

},

"respiration": {

"value": 18,

"measurementUnit": "rpm"

},

"oxygen": {

"saturationLevel": {

"value": 98,

"measurementUnit": "%"

}

},

"bloodPressure": {

"systolic": 0,

"diastolic": 0

},

"TimeParameters": {

"meanRRInterval": {

"value": 718.37993500203606,

"measurementUnit": "ms"

},

"rmssd": {

"value": 23.06334681492342,

"measurementUnit": "ms"

},

"lnRmssd": {

"value": 3.1382446393889092,

"measurementUnit": "ms"

},

"sdrr": {

"value": 35.13932783049551,

"measurementUnit": "ms"

},

"pRr50": {

"value": 4.225352112676056,

"measurementUnit": "%"

},

"rr50": {

"value": 3,

"measurementUnit": "N.A."

},

"amo50ms": {

"value": 59.722222222222221,

"measurementUnit": "%"

},

"mxDmn": {

"value": 167.62961234502848,

"measurementUnit": "ms"

},

"si": {

"value": 246,

"measurementUnit": "N.A."

}

},

"FrequencyParameters": {

"totalPower": {

"value": 1377.1501387984904,

"measurementUnit": "ms2"

},

"vlfPower": {

"value": 409.77943082996745,

"measurementUnit": "ms2"

},

"lfPower": {

"value": 626.87297297714952,

"measurementUnit": "ms2"

},

"hfPower": {

"value": 340.49773499137405,

"measurementUnit": "ms2"

},

"lfToHf": {

"value": 1.8410488780286021,

"measurementUnit": "ms2"

}

}

},

"ExtraData": [

{

"time": {

"value": 24.771304574550534,

"measurementUnit": "s"

},

"value": {

"value": 0.76538211626086605,

"measurementUnit": "s"

},

"mean": {

"value": 0.74040135395043738,

"measurementUnit": "s"

}

},

…….

{

"time": {

"value": 26.309516916124231,

"measurementUnit": "s"

},

"value": {

"value": 0.76766885234236004,

"measurementUnit": "s"

},

]

}

  1. W. Verkruysse, L. O. Svaasand, and J. S. Nelson, ”Remote plethysmographic imaging using ambient light”, Opt. Express 16(26), pp. 21434-21445 (2008); G. de Haan, and V. Jeanne, IEEE Trans. Biomed. Eng. 60, 2878-2886 (2013).
  2. D. McDuff, S. Gontarek, R. W. Picard, “Remote detection of photoplethysmographic systolic and diastolic peaks using a digital camera”, IEEE Trans. Biomed. Eng. 61, 2948–2954 (2014).
  3. F. Shaffer, J. P. Ginsberg , “An Overview of Heart Rate Variability Metrics and Norms”, Frontiers in Public Health 5, 258 (2017).

Was this article helpful?
0 out of 0 found this helpful

Comments

0 comments

Please sign in to leave a comment.