Android Synthesizer App Tutorial Part 6: C++ Wavetable Synthesizer

The epic conclusion to our Android synthesizer app!

Android Wavetable Synthesizer Tutorial Series

Introduction

Welcome to the final part of our Android wavetable synthesizer app tutorial!

Figure 1. Graphical user interface of the synthesizer app we are building.

We have already:

Created the UI of our app using Jetpack Compose,
Connected the UI to the model using a ViewModel.
Connected the Kotlin code with the C++ code using Java Native Interface (JNI),
Created an OboeAudioPlayer class that uses the Oboe library to handle Android’s audio device.

The last thing left to do is to implement the logic of the synthesizer itself in C++.

In this, we will closely follow the principles of wavetable synthesis. If you are not familiar with this synthesis technique, please, consider reviewing these materials first:

Classes Overview
Moving the Wavetable Enum Definition To a Separate Header
WavetableFactory
WavetableOscillator Class
Changes to WavetableSynthesizer
Testing the Synthesizer
Android Wavetable Synthesizer App Summary
Bibliography

Classes Overview

In this final part of the tutorial, we will introduce 2 new classes:

WavetableFactory for generating the wavetables of sine, triangle, square, and saw and
WavetableOscillator for playing back the given wavetable with the given amplitude and frequency.

Both of these will be controlled by the WavetableSynthesizer class that we already have.

Moving the Wavetable Enum Definition To a Separate Header

To make some things easier, we will move the definition of the Wavetable enum class from include/WavetableSynthesizer.h to the new include/Wavetable.h header:

Listing 1.

// include/Wavetable.h
#pragma once

namespace wavetablesynthesizer {
    enum class Wavetable { SINE, TRIANGLE, SQUARE, SAW };
}

WavetableFactory

To generate the wavetables, we will need a class that accepts an instance of the Wavetable enum class and generates a fixed-length vector (std::vector<float>) with values of the wavetable.

Listing 2.

// include/WavetableFactory.h
#pragma once
#include <vector>

namespace wavetablesynthesizer {
enum class Wavetable;

class WavetableFactory {
public:
  std::vector<float> getWaveTable(Wavetable wavetable);
  std::vector<float> sineWaveTable();
  std::vector<float> triangleWaveTable();
  std::vector<float> squareWaveTable();
  std::vector<float> sawWaveTable();

private:
  std::vector<float> _sineWaveTable;
  std::vector<float> _triangleWaveTable;
  std::vector<float> _squareWaveTable;
  std::vector<float> _sawWaveTable;
};
}  // namespace wavetablesynthesizer

The generated wavetables will be stored internally so that they are generated only once. One could also think of generating them all upon WavetableFactory instantiation.

Sine Generation

To simplify things, let’s consider generating a sine first. Below, I demonstrate how it could be done.

As a reminder, the include/MathConstants.h header file contains the PI constant.

Listing 3.

// WavetableFactory.cpp
#include "WavetableFactory.h"
#include <cmath>
#include <vector>
#include "Wavetable.h"
#include "MathConstants.h"

namespace wavetablesynthesizer {
static constexpr auto WAVETABLE_LENGTH = 256;

std::vector<float> generateSineWaveTable() {
  auto sineWaveTable = std::vector<float>(WAVETABLE_LENGTH);

  for (auto i = 0; i < WAVETABLE_LENGTH; ++i) {
    sineWaveTable[i] =
        std::sinf(2 * PI * static_cast<float>(i) / WAVETABLE_LENGTH);
  }

  return sineWaveTable;
}
//...

std::vector<float> WavetableFactory::getWaveTable(Wavetable wavetable) {
  switch (wavetable) {
    case Wavetable::SINE:
      return sineWaveTable();
  // TODO: The remaining wavetables
    default:
      return {WAVETABLE_LENGTH, 0.f};
  }
}

template <typename F>
std::vector<float> generateWaveTableOnce(std::vector<float>& waveTable,
                                         F&& generator) {
  if (waveTable.empty()) {
    waveTable = generator();
  }

  return waveTable;
}

std::vector<float> WavetableFactory::sineWaveTable() {
  return generateWaveTableOnce(_sineWaveTable, &generateSineWaveTable);
}
//...
}  // namespace wavetablesynthesizer

The scenario is as follows:

WavetableSynthesizer calls getWaveTable() with the given wavetable type.
getWaveTable() calls the appropriate member function based on the wavetable type. In the case of the sine, it is sineWaveTable().
sineWaveTable() calls the template function generateWaveTableOnce() passing it a reference to its member (to hold the wavetable’s values) and the appropriate generator function. In this case, we pass in the reference to the _sineWaveTable member vector and to the generateSineWaveTable() generator function.
generateWaveTableOnce() checks if the given reference does not already contain a wavetable. If it does, than it is simply returned. If not, the desired waveform is generated using the supplied generator function and then assigned to the member. In this case, generateSineWaveTable() is called and its result assigned to the _sineWaveTable member.
generateSineWaveTable() uses the std::sin function from the standard library to generate the values of the sine wavetable of the desired length.

With the basic structure in place, we can proceed to implementing the remaining wavetable generators.

Triangle, Square, Saw Generation

In the code below, I generate the above-mentioned wavetables by summing their harmonics (not using the time-domain formulas). I do this to reduce the number of harmonics and, thus, make the wavetables less prone to aliasing (although they will alias still at high frequencies).

If you are unsure what these formulas do, please, check out my tutorial on basic waveforms in sound synthesis and the supporting Python code.

The formulas are taken from a great book by professor Marek Pluta [Pluta2019].

Listing 4.


// WavetableFactory.cpp
//...
std::vector<float> generateTriangleWaveTable() {
  auto triangleWaveTable = std::vector<float>(WAVETABLE_LENGTH, 0.f);

  constexpr auto HARMONICS_COUNT = 13;

  for (auto k = 1; k <= HARMONICS_COUNT; ++k) {
    for (auto j = 0; j < WAVETABLE_LENGTH; ++j) {
      const auto phase = 2.f * PI * 1.f * j / WAVETABLE_LENGTH;
      triangleWaveTable[j] += 8.f / std::pow(PI, 2.f) * std::pow(-1.f, k) *
                              std::pow(2 * k - 1, -2.f) *
                              std::sin((2.f * k - 1.f) * phase);
    }
  }

  return triangleWaveTable;
}

std::vector<float> generateSquareWaveTable() {
  auto squareWaveTable = std::vector<float>(WAVETABLE_LENGTH, 0.f);

  constexpr auto HARMONICS_COUNT = 7;

  for (auto k = 1; k <= HARMONICS_COUNT; ++k) {
    for (auto j = 0; j < WAVETABLE_LENGTH; ++j) {
      const auto phase = 2.f * PI * 1.f * j / WAVETABLE_LENGTH;
      squareWaveTable[j] += 4.f / PI * std::pow(2.f * k - 1.f, -1.f) *
                            std::sin((2.f * k - 1.f) * phase);
    }
  }

  return squareWaveTable;
}

std::vector<float> generateSawWaveTable() {
  auto sawWaveTable = std::vector<float>(WAVETABLE_LENGTH, 0.f);

  constexpr auto HARMONICS_COUNT = 26;

  for (auto k = 1; k <= HARMONICS_COUNT; ++k) {
    for (auto j = 0; j < WAVETABLE_LENGTH; ++j) {
      const auto phase = 2.f * PI * 1.f * j / WAVETABLE_LENGTH;
      sawWaveTable[j] += 2.f / PI * std::pow(-1.f, k) * std::pow(k, -1.f) *
                         std::sin(k * phase);
    }
  }

  return sawWaveTable;
}

std::vector<float> WavetableFactory::getWaveTable(Wavetable wavetable) {
  switch (wavetable) {
    case Wavetable::SINE:
      return sineWaveTable();
    case Wavetable::TRIANGLE:
      return triangleWaveTable();
    case Wavetable::SQUARE:
      return squareWaveTable();
    case Wavetable::SAW:
      return sawWaveTable();
    default:
      return {WAVETABLE_LENGTH, 0.f};
  }
}

//...

std::vector<float> WavetableFactory::triangleWaveTable() {
  return generateWaveTableOnce(_triangleWaveTable, &generateTriangleWaveTable);
}

std::vector<float> WavetableFactory::squareWaveTable() {
  return generateWaveTableOnce(_squareWaveTable, &generateSquareWaveTable);
}

std::vector<float> WavetableFactory::sawWaveTable() {
  return generateWaveTableOnce(_sawWaveTable, &generateSawWaveTable);
}

//...

Changes to CMakeLists.txt

For WavetableFactory to work, we need to add its source file to the compiled files in CMakeLists.txt file.

Listing 5.

# CMakeLists.txt
add_library( wavetablesynthesizer

             SHARED

             wavetablesynthesizer-native-lib.cpp
             WavetableSynthesizer.cpp
             WavetableOscillator.cpp
             WavetableFactory.cpp # NEW!
             OboeAudioPlayer.cpp
        )

You can now try syncing and building your project 🙂

WavetableOscillator Class

We have just implemented the wavetable generation. Now, we need a class to play back those wavetables!

In the previous part of the tutorial, we have implemented an A4Oscillator class that plays back a sine at 440 Hz. We did that to test our audio output.

Now, we need a class that is able to play back any wavetable at any desired frequency (in a reasonable range) and with an adjustable amplitude. We’ll do it in a class called WavetableOscillator.

Class Declaration with Members

In Listing 6, there is the class declaration of WavetableOscillator, which should be put in the include/WavetableOscillator.h file.

Listing 6.

// include/WavetableOscillator.h
#pragma once

#include <vector>
#include "AudioSource.h"

namespace wavetablesynthesizer {

class WavetableOscillator : public AudioSource {
 public:
  WavetableOscillator() = default;
  WavetableOscillator(std::vector<float> waveTable, float sampleRate);

  float getSample() override;

  virtual void setFrequency(float frequency);

  virtual void setAmplitude(float newAmplitude);

  void onPlaybackStopped() override;

  virtual void setWavetable(const std::vector<float> &wavetable);

 private:
  float interpolateLinearly() const;
  void swapWavetableIfNecessary();

  float index = 0.f;
  std::atomic<float> indexIncrement{0.f};
  std::vector<float> waveTable;
  float sampleRate;
  std::atomic<float> amplitude{1.f};

  std::atomic<bool> swapWavetable{false};
  std::vector<float> wavetableToSwap;
  std::atomic<bool> wavetableIsBeingSwapped{false};
};

// The declaration of the A4Oscillator class may stay here
// ...

}  // namespace wavetablesynthesizer

As you can see, the WavetableOscillator inherits from AudioSource which is our interface that the AudioPlayer uses to retrieve samples from (getSample() member function) and to notify that the playback has stopped (onPlaybackStopped() callback).

The rest of the public member functions are needed by the WavetableSynthesizer class to control the playback. I have made them virtual (i.e., they can be overridden in any class inheriting from the WavetableSynthesizer) so that we can fake the class in tests.

The constructor of WavetableOscillator takes a wavetable and the sample rate as its arguments.

interpolateLinearly() member function is used in the wavetable synthesis algorithm. It has already been presented in the tutorial on the wavetable synthesizer JUCE plugin.

swapWavetableIfNecessary() is a helper function for changing the wavetable if the one chosen by the user is different from the one currently played back.

index member is a real-valued index into the wavetable.

indexIncrement is the amount by which we increase the index after every sample generation. It depends on the frequency, sample rate, and the buffer size. It is atomic so that it can be set in a thread-safe way.

sampleRate and amplitude are self-explanatory.

The remaining members are used to implement the thread-safe wavetable swapping.

Included Headers and the Constructor

In Listing 7, you can see the headers included in the WavetableOscillator.cpp file and the definition of the constructor.

Listing 7.

// WavetableOscillator.cpp
#include "WavetableOscillator.h"
#include <cmath>
#include "MathConstants.h"

namespace wavetablesynthesizer {

WavetableOscillator::WavetableOscillator(std::vector<float> waveTable,
                                         float sampleRate)
    : waveTable{std::move(waveTable)}, sampleRate{sampleRate} {}

The constructor simply moves or copies the given arguments to its member variables.

Sample Generation

In Listing 8, the getSample() method is shown along with interpolateLinearly().

Listing 8.

// WavetableOscillator.cpp
float WavetableOscillator::getSample() {
  swapWavetableIfNecessary();

  index = std::fmod(index, static_cast<float>(waveTable.size()));
  const auto sample = interpolateLinearly();
  index += indexIncrement;
  return amplitude * sample;
}

float WavetableOscillator::interpolateLinearly() const {
  const auto truncatedIndex =
      static_cast<typename decltype(waveTable)::size_type>(index);
  const auto nextIndex = (truncatedIndex + 1u) % waveTable.size();
  const auto nextIndexWeight = index - static_cast<float>(truncatedIndex);
  return waveTable[nextIndex] * nextIndexWeight +
         (1.f - nextIndexWeight) * waveTable[truncatedIndex];
}

At the beginning, we check if need to swap the wavetable and we do so if it’s necessary. More on this functionality below.

The rest of the function is the same algorithm as in the wavetable synthesizer JUCE plugin tutorial:

We read the index into the wavetable. Modulo operation always returns a value within the [0, waveTable.size()) range (we loop over the wavetable).
We interpolate linearly the value of the wavetable at that real-valued index using the nearest neighbors (upper and lower integer indices into the wavetable). This is a simple proportional weighing of samples from the wavetable.
We increment the index by the current index increment.
We return the sample scaled by the current amplitude.

Wavetable Swapping

In Listing 9, the wavetable swapping logic is shown. These functions are quite complicated so allow me a more detailed explanation. However, if you are not interested in this part, you can skip to the next.

Listing 9.

void WavetableOscillator::swapWavetableIfNecessary() {
  wavetableIsBeingSwapped.store(true, std::memory_order_release);
  if (swapWavetable.load(std::memory_order_acquire)) {
    std::swap(waveTable, wavetableToSwap);
    swapWavetable.store(false, std::memory_order_relaxed);
  }
  wavetableIsBeingSwapped.store(false, std::memory_order_release);
}

void WavetableOscillator::setWavetable(const std::vector<float> &wavetable) {
  // Wait for the previous swap to take place if the oscillator is playing
  swapWavetable.store(false, std::memory_order_release);
  while (wavetableIsBeingSwapped.load(std::memory_order_acquire)) {
  }
  wavetableToSwap = wavetable;
  swapWavetable.store(true, std::memory_order_release);
}

First of all, we need to understand that swapWavetableIfNecessary() is called from within the audio callback, which means it runs on the audio thread. On the other hand, setWavetable() is called from some other thread (remember, setWavetable() is invoked by user’s actions while the audio callback is invoked by the Android audio driver).

Therefore, we cannot simply alter the waveTable member from setWavetable(); we could run into audio glitches if we did that. For example, some waveTable values could be altered and some not while we are reading them out. std::vector is not thread-safe.

We also cannot use thread locks because these require system calls and must never be called from within the audio callback.

What we do here is that we use livelocks, a mechanism that does not put the waiting-for thread to sleep.

For this we use the following member variables:

std::vector<float> waveTable, which stores the wavetable that we are currently playing back.
std::vector<float> wavetableToSwap, which stores the wavetable that should be played back if swapWavetable is true.
std::atomic<bool> swapWavetable, which is a boolean flag that indicates that the waveTable member should be swapped with wavetableToSwap.
std::atomic<bool> wavetableIsBeingSwapped, which is a boolean flag that indicates that a swap is being performed and, thus, wavetableToSwap should not be used.

With that being said, swapWavetableIfNecessary() performs the following steps:

It indicates that a wavetable swap is taking place so the wavetableToSwap member should not be accessed.
It checks if there is a wavetable to swap. If the swapWavetable flag is true, then we should perform the swap.
In the body of the if-statement, it:
1. swaps the wavetables (exchanges these variables),
2. sets the swapWavetable flag to false (because the swap is not needed anymore).
Indicates that the swap has finished and wavetableToSwap can be accessed again.

The setWavetable() method performs the following steps:

It indicates that the wavetable swap is not needed at the moment. After all, we are about to set a new wavetable so it doesn’t make sense to swap the to-be-replaced wavetableToSwap member with the waveTable.
It actively waits until the current swap (if any) completes. This is the livelock part.
It sets the wavetableToSwap member to the new value given by the user.
It indicates that there now is a wavetable that should be swapped with the waveTable member.

Explaining memory ordering (std::memory_order_acquire, etc.) is beyong the scope of this article. If you are interested in this topic, there’s a great explanation by Herb Sutter on YouTube.

Otherwise, feel free to ask any questions in the comment section below the article.

With these two functions explained, the rest is actually quite simple 😄

Setting the Amplitude and Frequency

In Listing 10, methods setFrequency() and setAmplitude() are presented.

Listing 10.

// WavetableOscillator.cpp
void WavetableOscillator::setFrequency(float frequency) {
  indexIncrement = frequency * static_cast<float>(waveTable.size()) /
                   static_cast<float>(sampleRate);
}

void WavetableOscillator::setAmplitude(float newAmplitude) {
  amplitude.store(newAmplitude);
}

setFrequency() calculates the current index increment based on the given frequency, wavetable size, and the sample rate. If you do not understand this formula, I explained it in the wavetable synthesis algorithm tutorial.

setAmplitude() is a setter of the amplitude member.

Remember that amplitude and indexIncrement are both atomic so they can be safely changed from various threads.

onPlaybackStopped() Callback

In onPlaybackStopped() (Listing 11), we simply reset the index member to 0 so that it starts playing from the beginning of the wavetable upon the next playback start.

Listing 11.

// WavetableOscillator.cpp
void WavetableOscillator::onPlaybackStopped() {
  index = 0.f;
}

And that’s it for the WavetableOscillator class! The last class to change is the WavetableSynthesizer and then we’re done!

Changes to WavetableSynthesizer

To make it clear what happened to the WavetableSynthesizer class, I put its whole declaration in Listing 12.

Listing 12.

// include/WavetableSynthesizer.h
#pragma once

#include <memory>
#include <mutex>
#include "Wavetable.h"
#include "WavetableFactory.h"

namespace wavetablesynthesizer {
class WavetableOscillator;

class AudioPlayer;

constexpr auto samplingRate = 48000;

class WavetableSynthesizer {
 public:
  WavetableSynthesizer();

  ~WavetableSynthesizer();

  void play();

  void stop();

  bool isPlaying() const;

  void setFrequency(float frequencyInHz);

  void setVolume(float volumeInDb);

  void setWavetable(Wavetable wavetable);

 private:
  std::atomic<bool> _isPlaying{false};
  std::mutex _mutex;
  WavetableFactory _wavetableFactory;
  Wavetable _currentWavetable{Wavetable::SINE};
  std::shared_ptr<WavetableOscillator> _oscillator;
  std::unique_ptr<AudioPlayer> _audioPlayer;
};
}  // namespace wavetablesynthesizer

We have

3 new includes,
WavetableOscillator forward declaration replacing the AudioSource forward declaration (we changed the type of the _oscillator member.),
3 new members: _mutex, _wavetableFactory, and _currentWavetable, and
_isPlaying made atomic.

Below I have put the implementation of methods that changed since the last tutorial episode. That includes all but the destructor and isPlaying().

Included Files and the Constructor

In Listing 13, you can see the included files and the constructor of WavetableSynthesizer.

Listing 13.

// WavetableOscillator.cpp
#include "WavetableSynthesizer.h"
#include <cmath>
#include "Log.h"
#include "OboeAudioPlayer.h"
#include "WavetableOscillator.h"

namespace wavetablesynthesizer {

WavetableSynthesizer::WavetableSynthesizer()
    : _oscillator{
        std::make_shared<WavetableOscillator>(
          _wavetableFactory.getWaveTable(_currentWavetable),
          samplingRate)},
      _audioPlayer{
          std::make_unique<OboeAudioPlayer>(_oscillator, samplingRate)} {}

The important bit is the initialization of the _oscillator member. As you can see, we pass a generated wavetable to its constructor. We generate it by calling _wavetableFactory.getWaveTable(_currentWavetable) , where _currentWavetable is the value of the Wavetable enum class.

We can use _wavetableFactory and _currentWavetable here because they are placed before _oscillator in the member declaration list of the class (Listing 12).

play() and stop()

In Listing 14, there are play() and stop() member functions.

These functions now contain locks. The locks are introduced to protect the access to the _audioPlayer because these member functions may be called simultaneously from different threads and _audioPlayer is not thread-safe. _isPlaying is atomic so it should be safe to use. However, we change it under the mutex to ensure that its state is consistent, in other words, the _mutex ensures that subsequent calls to play() and stop() are sequentially consistent.

Listing 14.

// WavetableOscillator.cpp
void WavetableSynthesizer::play() {
  std::lock_guard<std::mutex> lock(_mutex);
  const auto result = _audioPlayer->play();
  if (result == 0) {
    _isPlaying = true;
  } else {
    LOGD("Could not start playback.");
  }
}

void WavetableSynthesizer::stop() {
  std::lock_guard<std::mutex> lock(_mutex);
  _audioPlayer->stop();
  _isPlaying = false;
}

Setting Amplitude and Frequency

In Listing 15, there are frequency and amplitude setters presented.

While setFrequency() is straightforward, in setVolume we convert the volume in decibels to an amplitude value using the dBToAmplitude helper function. This is a standard conversion that could eventually be put in a different header file.

Listing 15.

// WavetableOscillator.cpp
void WavetableSynthesizer::setFrequency(float frequencyInHz) {
  _oscillator->setFrequency(frequencyInHz);
}

float dBToAmplitude(float dB) {
  return std::pow(10.f, dB / 20.f);
}

void WavetableSynthesizer::setVolume(float volumeInDb) {
  const auto amplitude = dBToAmplitude(volumeInDb);
  _oscillator->setAmplitude(amplitude);
}

Setting the Wavetable

Finally, we set our wavetable. First, we check if a change is really necessary. Then, we assign the new value of the wavetable to the current one, generate the new wavetable using the WavetableFactory instance and pass it to the WavetableOscillator instance.

Listing 16.

// WavetableOscillator.cpp
void WavetableSynthesizer::setWavetable(Wavetable wavetable) {
    if (_currentWavetable != wavetable) {
        _currentWavetable = wavetable;
        _oscillator->setWavetable(_wavetableFactory.getWaveTable(wavetable));
    }
}
}  // namespace wavetablesynthesizer

And that’s it! That’s the end of our wavetable synthesizer Android app implementation!

Testing the Synthesizer

You should now be able to compile the whole project and install it on your Android device of choice or an emulator.

While pressing the buttons and using the slider, you should be able to

change the wavetable being played,
change the frequency being played,
adjust the volume of the playback,
stop and resume the playback.

If you have trouble getting one of these to work, be sure to check your code against mine. I have put it in full on GitHub.

Android Wavetable Synthesizer App Summary

In this tutorial series, you have learned:

how modern Android apps are structured,
how to build a user interface (UI) using Jetpack Compose,
how to connect your UI with your Model using a ViewModel,
how to call C++ code from Kotlin on Android using Java Native Interface, Gradle, and CMake,
how to play back sound on Android using the Oboe library, and
how to generate sound using the wavetable synthesis in C++.

Congratulations on completing this mini-series! You are now prepared to tackle more complicated Android musical applications.

If you would like to learn more about audio programming, be sure to sign up on my newsletter, where I share all the newest content as well as useful tips and tricks regarding learning audio programming.

Congratulations once again and see you soon!

Bibliography

[Pluta2019] Marek Pluta, Sound Synthesis for Music Reproduction and Performance, monograph, AGH University of Science and Technology Press, Kraków 2019.