DEEPCRAFT™ Studio now supports sensor fusion. You can combine signals from several sensors into a single model input, directly in our Graph UX interface. In this blog post, ML Engineer Gustav Nilsson looks at what sensor fusion is and why it matters. He also shows how to combine audio and motion data in your own Edge AI projects.
One Sensor, One Perspective
Every sensor sees the world from one angle. A microphone hears what a machine sounds like. An IMU (inertial measurement unit) feels how it moves and vibrates. Each gives you useful information, but each has blind spots. A microphone next to a motor also picks up background noise, nearby machines and people talking, so a fault can be buried in sound that has nothing to do with it. An IMU feels vibration very precisely, but some faults show up in the sound long before they show up in the movement.
An IMU and a microphone recording at the same time, shown as two separate tracks on the DEEPCRAFT™ Studio timeline. The IMU captures movement on several axes, while the microphone captures sound.
Sensor Fusion: Seeing the Whole Picture
Sensor fusion combines signals from several sensors into one view of what is happening. When the model gets both the sound and the vibration of the motor at the same time, it can learn patterns that neither sensor shows on its own. For example, a specific sound might only mean a fault when it comes with a certain vibration. This makes the model more robust against noise and more accurate on the cases that matter.
The difficulty is that sensors don't run at the same pace. A microphone captures thousands of samples per second, while an IMU typically captures around a hundred. To feed both into one model, the signals must be aligned in time and merged into a single stream.
The microphone delivers many samples per second, the IMU only a few. Sensor fusion aligns both to the higher rate and joins them into one stream.
The Sensor Fusion Node: Combining Signals in One Step
The new Sensor Fusion node in DEEPCRAFT™ Studio handles this for you. You'll find it in the node library under Signal Processing → Time Domain.
It takes two input signals, resamples both to the higher of the two rates, and joins them into one combined signal that your model can train on.
Where to find the node: search for “sensor” in the graph, or open Signal Processing/Time Domain in the Node Explorer. In this graph, a built-in IMU (50 Hz) and microphone (16 kHz) record into a Dataset.
The Sensor Fusion node wired up: the IMU goes to Input A and the microphone to Input B. The output runs at the microphone’s 16 kHz and holds seven values per sample; six from the IMU and one from the microphone.
There is one practical point when fusing audio with motion data. Because the node resamples to the higher rate, fusing raw audio directly with an IMU would stretch the slow IMU signal to the audio rate, about 160 times more samples. A better approach is to first turn the audio into features, such as a mel spectrogram, at a frame rate close to the IMU's. The fused signal then stays compact, and each part of it carries meaningful information.
A Showcase Project: Built Entirely in Graph UX
Our condition monitoring showcase project is built entirely in the Graph UX interface. A microphone and an IMU record a running motor. The audio is turned into features, fused with the IMU signal, and fed into a classification model. You can record, view, fuse and train in one workflow, without writing any code. The project can be built and trained in DEEPCRAFT™ Studio today. Deploying fused models to hardware is coming soon.
The showcase recording after fusion: one IMU+Microphone track in the Dataset, holding both sensors’ signals in a single lane, ready for training.
More News: Smoother Multi-Sensor Recording
Working with several sensors means more than fusing their signals. We've also improved the experience around it:
Get Started with Sensor Fusion in DEEPCRAFT™ Studio
With sensor fusion in DEEPCRAFT™ Studio, your models no longer have to rely on a single sensor. Whether you're monitoring machines, recognizing activities or detecting events that only show up across several signals, DEEPCRAFT™ Studio gives you the tools to combine your data and build more robust Edge AI models. Stay tuned for more updates and tutorials on how to use these features in your own projects.
Don't miss out on the latest blogs, news and more from Imagimob!
Subscribe to our monthly newsletter to stay up-to-date on all the latest blogs, news, events, webinars and more from Imagimob.