Access Sound Amplitude of Microphone In Real Time

[hiraz01]

What is the best way to access real time amplitude of microphone for displaying or control of a graphics object? For now, I'm trying to accomplish something relatively simple where I speak in the microphone and have an amplitude indicator (like what you see with a program like OBS studio). I have read through the examples of simple capture, simple playback, and simple duplex.

I have been successful loading a wav file from disk with ma_decoder_read_pcm_frames using miniaudio with a simple C++ class. I use ma_decoder m_decoder and a std::vector m_FramesVec like so, ma_decoder_read_pcm_frames(&m_decoder, &m_FramesVec[0], n_samples, &framesread). I get n_samples from ma_decoder_get_length_in_pcm_frames and resize the std::vector before the read.

I seem to struggle with the device and callback part of the minaudio API the most. Specifically, the pDevice->pUserData seems to be what I need to fully grasp. From what I have done so far, It's like I want to use the ma_encoder_write_pcm_frames to something like m_FramesVec where the encoder would write frames from the microphone frames grabbed by miniaudio rather than the ma_decoder reading those frames as I do from sounds loaded from disk, if that makes sense. I thought the simple duplex example would help me understand what was going on better, but I am confused as to how to utilize the pOutput or pInput without an encoder to write the frames to my own std::vector, for example.

[mackron]

The simple_capture example is what you want. The playback and duplex examples won't help you, nor will ma_decoder. The pUserData member is just a way for you to associate an arbitrary pointer from your program with the device. That way you can access said pointer wherever you have access to a ma_device object, such as in the data callback. In the capture example it just happens to be that I designed it to use the pUserData pointer with an ma_encoder object. You can assign it to whatever you want, or nothing at all - it just depends on how you want to design your program. For example, I could have instead declared the ma_encoder object globally in which case I wouldn't need to use the pUserData pointer at all.

The way it works is that you receive audio data from the microphone via the data callback. Once you have that data you can do whatever you want with it. To simplify it, set the format to ma_format_f32 in the device config (deviceConfig.capture.format in the example). That way the data callback will give you samples in floating point format such that pInput will just be an array of floats that you can iterate over in a for loop. Each sample is between -1 and 1. The absolute value of the sample represents the amplitude. I would just take the average amplitude for each sample that the data callback gives you and use that as the amplitude that you display on the UI. You'll get a fresh amplitude every 10 milliseconds by default (can be configured by changing the period size in the device config).

[hiraz01]

Thank you very much for the help. I somehow struggled to understand the pInput of the data_callback. Up until now, I had thought the only way to access the data was by passing a reference of my own buffer(in this case m_FramesVec) to ma_encoder so it could write the data in memory rather than to a file. Your previous discussion (How to get the raw pcm data when capture? #525) helped, but I didn't quite get it. I tried the following within the callback,

const float* floatBuffer = (float*)(pInput);
std::vector vecBuffer(floatBuffer, floatBuffer + frameCount);

So I had to cast to a const float* and then constructed the std::vector directly from the data. It seems to be working, and I believe this is reasonable solution. In the callback, I did a for loop to sum the magnitude and with an average of the 441 frames (sample rate of 44100, i.e. 10 ms). Unless, I am missing something, I think this answers my question.

[mackron]

Yeah the data callback and the encoder/decoder are completely decoupled. I just happen to use them for the sake of the example to demonstrate one possible way you could use the data from the callback.

There's an improvement you can make. You're assuming the number of floats in the buffer is equal to frameCount which is not quite right. It's actually equal to frameCount * pDevice->capture.channels. A float in your case is a "sample". A "frame" in miniaudio, is actually a group of samples totalling one sample for each channel. A mono device will have just one channel, a stereo device will have two channels (left and right), a 5.1 surround system will have 6 channels, etc. Most microphones will capture only a single channel so your code will work just fine most of the time. If, however, you had a microphone that captured more than one channel, you'd want to update your code just to make sure each sample is accounted for. If you wanted to get really advanced you could detect how many channels the microphone is capturing, and then do a UI indicator for each of those channels independently. But for now, just change floatBuffer + frameCount to floatBuffer + (frameCount * pDevice->capture.channels) and you'll be fine.

[hiraz01]

Thank you very much for the help and clarification. I will definitely make the change to account for the channels. Cheers.