Date 11/01/22

Alert Vest – connected tinyML safety vest by Swanholm Technology

When former Scania intrapreneur Jonas Svanholm saw the statistics on how prone truck drivers were to accidents and injury at work, he and his team decided to take on the challenge of making such workplaces safer for all professionals, by making the safety vest the drivers are obliged to wear outside the vehicle into something smarter and more helpful in hazardous situations.

Being a professional truck driver tops the most hazardous work environments both in Sweden and across the globe; out of 404 fatal work accidents, truck drivers make up 66 deaths during the period 2011-2020 according to the Swedish Work environment authority.

tinyML and fall detection

The need of connected safety goes well beyond just truck drivers, and the sheer number of workers in hazardous occupations became the driving force to found Swanholm Technology AB, with the mission to offer connected safety products for exposed professionals. The main purpose is very straight-forward: to make the end-user safer and provide the owners with insight how to build a safer work environment in close co-operation with companies to find new business models and new products to sell.

This objective was not without its challenges, as we needed to introduce a new security layer, without adding additional tasks to perform or things to bring along for the user. 

Truck drivers often experience long hours, regularly with potentially stressful situations as well as demanding weather conditions, and it was vital to work out of the platform they already had. The vest needs to comply with the EN 471 class 2 safety standard, also when equipped with smart technology and become the Alert Vest. Needless to say, the questions were many and a lot of practical work needed to be done to reach the product range Swanholm today offer the market, with 100+ units sold and with operations in 5 European countries.

Some practical issues about such a device needed to be addressed, laying the foundation of a new, augmented type of safety product. This raised an array of practical questions such as;

  • How is a safety vest used today, what is its actual purpose?
  • What sensors are key, where are sensors best placed for optimal reading?
  • How are the electronics best embedded in the fabric and sewing?
  • How often would a user need to recharge the embedded lithium batteries?
  • What is the best way to connect the smart vest to a smartphone, or even tablet?
  • How would the device withstand years of dirt, rain, normal wear and tear?
  • Can the electronics be easily removed when the vest needs washing?
  • Will workers feel watched, even though the vest only alerts its position via the smartphone when the fall detection alarm is triggered?

This product line was developed in close collaboration with the vest manufacturers, embedded engineers, transport and construction companies and most importantly the end-users themselves. In parallel with these interviews and workshops, the software performing the detection of hazardous falls needed to be developed and trained, and this is when Jonas Svanholm contacted the team at Imagimob in order to explain the needs and wants of end-users,  and thereby giving the assignment which initiated a string of development iteration cycles, leading to the final product and market introduction in 2020.

Fig 1. Collaboration is key, workers in a mining operation, the Alert Vest and private cellular networks allow workers to be safe even deep down in a mine 

Getting back to the theories on sensing a person falling; the fall itself is registered as a split-second event; usually starting off with a small change in the motion vector, followed by milliseconds of apparent weightlessness, and then a sudden, greater rate of change in the resulting acceleration vector, such are the characteristic of a true fall. 

Using a safety vest as platform, the nape of the neck proved to be a good placement of the sensor to get consistent readings, while everything else like the batteries and the embedded computer can be placed on either side, or both for weight symmetry.  In machine learning there are the traditional series of steps to be taken, each affecting the efficiency and accuracy of the resulting model. 

Fig. 2 Graph of typical pre-fall, fall and post-fall readings during collection of training data, curtesy of Yu et al [2]

tinyML and datasets

Machine learning projects start with gathering data one way or the other, and in many application areas there are open datasets available for training. In specific applications, whereas a dataset covering the exact use case may be hard to find, hence additional training data has to be captured. This was done  using a Bluetooth-connected accelerometer with concurrent video capture. The video capture, even though not used itself in the final application, allows labeling of the data both in real-time as well as in post-production, using the Imagimob mobile capture app for Android.

One of the more known available dataset for fall detection, is the UR fall detection set, developed by a team at University of Rzeszow in Poland [1]. The accelerometer used in their study, however, was placed at the waist and not at the neck as with the Connected Safety Vest’s sensor mount point, rendering such a third-party dataset less accurate for this purpose, but still useful to some degree. The sensor was also of another make, potentially resulting in that the characteristics of a person falling may still be as accurate, while the actual readings can differ from one sensor make to another. It’s naturally preferred to use a similar sensor element for the commercial product, as during the collection of training and testing data respectively. The conclusion was that more training data was needed.

Being a conscientious team, a large number of exercises and tests have been carried out by the Imagimob team, in the parking lot at the Swanholm Technology offices and on various customer sites in several countries. 

Falling out from the driver’s seat of a truck, from the back of it when unloading, slipping on the ground, falling into a roadside ditch; the possibilities of falling are near endless and it’s important to cover as many cases as possible. At one point, a professional judo practitioner was hired to perform a large number of pre-determined falls, but just couldn’t trigger the algorithms of the smart vest, as the falls performed were too soft (judo meaning “the soft way” may be part of the explanation). Subsequently, a 70kg crash test dummy was acquired to be a less-skilled fall agent, and suffered through a number or ordeals, only too numerous to mention from a wide range of starting positions and fall heights.

Figure 3. One of many falls performed by the development team, here simulating tripping off the lowest in-step of a truck, here with the mobile app in the foreground.

Imagimob tinyML platform

The sensor used for this product is the Bosch BMI-160, which carries both 3-axis accelerometer and 3-axis gyroscope in a small form factor. It handles acceleration up to ±16g and its power consumption is 180 μA for the accelerometer in full operation and 3µA in suspended mode, making it a good candidate for embedded applications when powered by batteries.

After data collection and labeling, the model is generated and trained followed by verification using testing data. It’s a fancy way of saying there is parts of the training data that the AI hasn’t seen before, where it’s predictions is compared to the ground truth, i.e. the labels set during data capture. The last step is deployment into a executable binary library, which can be introduced as part of a new or existing application, see figure 4.

Fig 4. Order of events for the preamble, training and deployment of a model using Imagimob AI, several of which are carried out with a high degree of automation, allowing an iteration of the process to take hours rather than days or weeks to complete.

A false positive – fall detected when there isn’t one – is annoying at least, eventually leading to a Peter-and-the-wolf effect where eventually no one reacts to alarms from an ill-constructed system. An actual fall occurring undetected is considerably worse, and only needs one false negative occurrence to invoke distrust as well as dire consequences for the individual.

Naturally, a user doesn’t need to be lone worker in order to benefit from using the Connected Safety Vest. Many other groups of professional workers carry the vest today, construction sites, storage facilities, road workers, dock loaders et cetera. 

One of Swanholm’s current customers had an incident prior to outfitting their crews with the smart vests. It was a most unfortunate event, which occurred at a construction site in the middle of a team operation with plenty of people nearby; one of the wheel loader operators gets dizzy and falls down over 2 meters height from his vehicle and after landing unconscious it takes the others a full hour to realize his situation with him lying behind one of the big wheels. So even when working together, it’s good to be able to continuously monitor everyone in case of sudden incidents.

Fig 5. Construction sites are the perfect match for the Alert Vest

Far from all falls, stumbles or jumps actually results in severe injury. Therefore, the device is equipped with a timer which starts right when a fall is detected, allowing for the user to reset the vest during a grace time, during which a set of warning beeps are heard. It only after a typical 30 seconds has passed, when an actual alarm is issued, sending notifications to peers, management and/or safety staff. If there’s an actual accident, the user doesn’t need to do or perhaps can’t do anything, so just waiting for the timer to time-out is sufficient and the alarm will execute as expected.

The connected vest also is equipped with an explicit pushbutton switch on the front, to raise attention should an operator become stuck, wedged or otherwise pinned in a work situation without being able to call someone using their phone. There might not be a fall at all, and this makes up a secondary way of calling for assistance to compliment the AI detection algorithm.

One other important standard function of the Swanholm product series, is the ability to securely modify the firmware over the air, FOTA, using the smartphone, allowing for new tinyML models and applications to be updated as the fall detection algorithms evolve without having to hand them in for an upgrade.

Learn more about the Alert Vest here.

[1] Bogdan Kwolek, Michal Kepski, Human fall detection on embedded platform using depth maps and wireless accelerometer, Computer Methods and Programs in Biomedicine, Volume 117, Issue 3, December 2014, Pages 489-501, ISSN 0169-2607

[2] Yu X, Jang J and Xiong S (2021) A Large-Scale Open Motion Dataset (KFall) and Benchmark Algorithms for Detecting Pre-impact Fall of the Elderly Using Wearable Inertial Sensors. Front. Aging Neurosci. 13:692865. doi: 10.3389/fnagi.2021.692865

Date 01/27/23

Deploying Quality SED models in a week

Imagimob has the expertise, software tools and methods to de...

Date 11/17/22

An introduction to Sound Event Detection (SED)

This article is an introduction to Sound Event Detection (SE...

Date 11/14/22

Imagimob condition monitoring AI-demo on Texas Ins...

Date 11/01/22

Alert Vest – connected tinyML safety vest by Swanh...

Date 10/21/22

Video recording from tinyML AutoML Deep Dive

Date 10/19/22

Edge ML Project time-estimates

Date 10/05/22

An introduction to Fall detection - The art of mea...

Date 04/20/22

Imagimob to exhibit at Embedded World 2022

Date 03/12/22

The past, present and future of edge ML

Date 03/10/22

Recorded AI Tech Talk by Imagimob and Arm on April...

Date 03/05/22

The Future is Touchless: Radical Gesture Control P...

Date 01/31/22

Quantization of LSTM layers - a Technical White Pa...

Date 01/07/22

How to build an embedded AI application

Date 12/07/21

Don’t build your embedded AI pipeline from scratch...

Date 12/02/21

Imagimob @ CES 2022

Date 11/25/21

Imagimob AI in Agritech

Date 10/19/21

Deploying Edge AI Models - Acconeer example

Date 10/11/21

Imagimob AI used for condition monitoring of elect...

Date 09/21/21

Tips and Tricks for Better Edge AI models

Date 06/18/21

Imagimob AI integration with IAR Embedded Workbenc...

Date 05/10/21

Recorded Webinar - Imagimob at Arm AI Tech Talks o...

Date 04/23/21

Gesture Visualization in Imagimob Studio

Date 04/01/21

New team members

Date 03/15/21

Imagimob featured in Dagens Industri

Date 02/22/21

Customer Case Study: Increasing car safety through...

Date 12/18/20

Veoneer, Imagimob and Pionate in joint research pr...

Date 11/20/20

Edge computing needs Edge AI

Date 11/12/20

Imagimob video from tinyML Talks

Date 10/28/20

Agritech: Monitoring cattle with IoT and Edge AI

Date 10/19/20

Arm Community Blog: Imagimob - The fastest way fro...

Date 09/21/20

Imagimob video from Redeye AI seminar

Date 05/07/20

Webinar - Gesture control using radar and Edge AI

Date 04/08/20

tinyML article with Nordic Semiconductors

Date 02/18/20

What is tinyML?

Date 12/11/19

Edge AI for techies, updated December 11, 2019

Date 12/05/19

Article in Dagens Industri: This is how Stockholm-...

Date 09/06/19

The New Path to Better Edge AI Applications

Date 07/01/19

Edge Computing in Modern Agriculture

Date 04/07/19

Our Top 3 Highlights from Hannover Messe 2019

Date 03/26/19

The Way You Collect Data Can Make or Break Your Ne...

Date 03/23/18

AI Research and AI Safety

Date 03/11/18

What is Edge AI?

Date 01/30/18

Imagimob and Autoliv demo at CES 2018

Date 05/24/17

Wearing Intelligence On Your Sleeve

LOAD MORE keyboard_arrow_down