Accessing the voice buffer

The TeamSpeak Client Lib allows users to access the raw playback and capture voice data and even modify it, for example to add effects to the voice. These callbacks are also used by the TeamSpeak client for the voice recording feature.

Note

Using these low-level callbacks is not required and should be reserved for specific needs. Most SDK applications won’t need to implement these callbacks.

Playback

Before effects or mixing

The following event is called when a voice packet from a client (not own client) is decoded and about to be played over your sound device, but before it is 3D positioned and mixed with other sounds. You can use this function to alter the voice data (for example when you want to apply effects to it) or to simply get voice data. The TeamSpeak client uses this function to record sessions.

struct ClientUIFunctions

Defines available callbacks that you can receive.

Set the members of this struct to a function to call when the specific event happens.

Public Members

void (*onEditPlaybackVoiceDataEvent)(uint64 serverConnectionHandlerID, anyID clientID, short *samples, int sampleCount, int channels)

called before any effects are applied, allows access to individual client raw audio data

Param serverConnectionHandlerID:: specifies on which connection the callback was called
Param clientID:: id of the source client for the audio
Param samples:: buffer of audio data for the client as 16 bit signed at 48kHz
Param sampleCount:: how many audio frames are available in the buffer
Param channels:: number of audio channels in the audio data

After effects but before mixing

The following event is called when a voice packet from a client (not own client) is decoded and 3D positioned and about to be played over your sound device, but before it is mixed with other sounds. You can use this function to alter/get the voice data after 3D positioning.

struct ClientUIFunctions

Defines available callbacks that you can receive.

Set the members of this struct to a function to call when the specific event happens.

Public Members

void (*onEditPostProcessVoiceDataEvent)(uint64 serverConnectionHandlerID, anyID clientID, short *samples, int sampleCount, int channels, const unsigned int *channelSpeakerArray, unsigned int *channelFillMask)

called before audio data is mixed together into a single audio stream for playback, but after effects (3D positioning for example) have been applied.

Param serverConnectionHandlerID:: specifies on which connection the callback was called
Param clientID:: id of the source client for the audio
Param samples:: buffer of audio data for the client as 16 bit signed at 48kHz
Param sampleCount:: how many audio frames are available in the buffer
Param channels:: number of audio channels in the audio data
Param channelSpeakerArray:: Array with an entry for each channel in the buffer, defining the speaker each channel represents. see SPEAKER_* defines in public_definitions.h
Param channelFillMask:: a bit mask of SPEAKER_* that defines which of the channels in the buffer have audio data. Be sure to set the corresponding flag when adding audio to previously empty channels in the buffer.

Example

For example, this callback reports:

channels = 6
channelSpeakerArray[0] = SPEAKER_FRONT_CENTER
channelSpeakerArray[1] = SPEAKER_LOW_FREQUENCY
channelSpeakerArray[2] = SPEAKER_BACK_LEFT
channelSpeakerArray[3] = SPEAKER_BACK_RIGHT
channelSpeakerArray[4] = SPEAKER_SIDE_LEFT
channelSpeakerArray[5] = SPEAKER_SIDE_RIGHT  // Quite unusual setup
*channelFillMask = 1

This means “samples” points to 6 channel data, but only the SPEAKER_FRONT_CENTER channel has data, the other channels are undefined (not necessarily 0, but undefined).

So for the first sample, samples[0] has data and samples[1], samples[2], samples[3], samples[4] and samples[5] are undefined.

If you want to add SPEAKER_BACK_RIGHT channel data you would do something like:

*channelFillMask |= 1<<3;  // SPEAKER_BACK_RIGHT is the 4th channel (is index 3) according to *channelSpeakerArray.
for (int i=0; i<sampleCount; ++i) {
    samples[3 + (i*channels) ] = getChannelSoundData(SPEAKER_BACK_RIGHT, i);
}

After effects and mixing

The following event is called when all sounds that are about to be played back for this server connection are mixed. This is the last chance to alter/get sound.

struct ClientUIFunctions

Defines available callbacks that you can receive.

Set the members of this struct to a function to call when the specific event happens.

Public Members

void (*onEditMixedPlaybackVoiceDataEvent)(uint64 serverConnectionHandlerID, short *samples, int sampleCount, int channels, const unsigned int *channelSpeakerArray, unsigned int *channelFillMask)

called after mixing individual client audio together but before sending it to playback device.

Last chance to access/modify audio data before it gets sent to the playback device.

Param serverConnectionHandlerID:: specifies on which connection the callback was called
Param samples:: buffer of audio data as 16 bit signed at 48kHz
Param sampleCount:: how many audio frames are available in the buffer
Param channels:: how many audio channels are available in the buffer
Param channelSpeakerArray:: Array with an entry for each channel in the buffer, defining the speaker each channel represents. See SPEAKER_* defines in public_definitions.h
Param channelFillMask:: a bit mask of SPEAKER_* that defines which of the channels in the buffer have audio data.

Capture

Before preprocessing

struct ClientUIFunctions

Defines available callbacks that you can receive.

Set the members of this struct to a function to call when the specific event happens.

Public Members

void (*onEditCapturedVoiceDataPreprocessEvent)(uint64 serverConnectionHandlerID, short *samples, int sampleCount, int channels, int *flags)

called after audio data was aquired from the capture device, without any pre processing applied. Allows access to raw audio data.

Param serverConnectionHandlerID:: specifies on which connection the callback was called
Param samples:: buffer of audio data
Param sampleCount:: how many audio frames are available in the buffer
Param channels:: how many audio channels are available in the buffer
Param flags:: allows to mute the audio stream, set LSB to 1 to mute the audio.

After preprocessing

The following event is called after sound is recorded from the sound device and is preprocessed. This event can be used to get/alter recorded sound. It can also be used to determine if this sound will be transmitted to the server, or muted. This is used by the TeamSpeak client to record sessions.

If the sound data will be transmitted, (*edited | 2) is true. If the sound data is changed, set bit 1 (*edited |= 1). If the sound should not be transmitted, clear bit 2. (*edited &= ~2)

struct ClientUIFunctions

Defines available callbacks that you can receive.

Set the members of this struct to a function to call when the specific event happens.

Public Members

void (*onEditCapturedVoiceDataEvent)(uint64 serverConnectionHandlerID, short *samples, int sampleCount, int channels, int *edited)

called after pre processing has been applied to recorded voice data, before it is sent to the server.

This allows access to or modification of captured data from the recording device.

Param serverConnectionHandlerID:: specifies on which connection the callback was called
Param samples:: buffer of audio data as 16 bit signed at 48kHz
Param sampleCount:: how many audio frames are available in the buffer
Param channels:: how many audio channels are available in the buffer
Param edited:: bitMask indicating whether you modified the buffer. Set LSB to 1 if you modified the buffer. Bit 2 indicates whether or not this buffer will be sent to the server.

Voice recording

When using the above callbacks to record voice, you should notify the server when recording starts or stops with the following functions:

unsigned int ts3client_startVoiceRecording(uint64 serverConnectionHandlerID)

Flags the client as recording received audio transmissions.

This does NOT cause any recording to take place, it merely informs other clients that this client is actually recording the conversation.

Parameters:

serverConnectionHandlerID – the connection handler on which to flag this client for recording.

Returns:

An Error code from the Ts3ErrorType enum indicating either success or the failure reason

unsigned int ts3client_stopVoiceRecording(uint64 serverConnectionHandlerID)

Flags the client as no longer recording audio transmissions.

Unsets the flag set by ts3client_startVoiceRecording causing other clients to no longer mark this client as recording the conversation.

Parameters:

serverConnectionHandlerID – the connection handler on which to unset the recording flag for this client.

Returns:

An Error code from the Ts3ErrorType enum indicating either success or the failure reason