title:: Writing Unit Generators
summary:: Get started with writing unit generators
categories:: Internals
related:: Reference/ServerPluginAPI

SuperCollider has a vast library of unit generators that can be assembled in unlimited ways, but sometimes even those
aren't sufficient. You may have a need for an unusual signal processing algorithm, or you're running into efficiency
problems that can be solved by condensing parts of your SynthDef into a single UGen.

UGens are defined in server plugins written in C++. Server plugins are not to be confused with quarks, which extend the
SuperCollider language. UGens exist more or less independently of the interpreter and you don't need much familiarity
with SC to write them.

Writing UGens is not too difficult, but it's arguably far less convenient and intuitive than the high-level tools that
SuperCollider provides. You'll need a build system and a good amount of boilerplate code -- even fairly basic signal
processing operations can require a lot of code. You don't have an instant live coding environment, and mistakes can
easily crash the server. SuperCollider's UGens are stable and well-tested, and custom UGens are best viewed as a last
resort for when the limitations of SC are impassable.

section:: Pseudo-UGens

Before we proceed to the real UGens, we'll take a quick detour for the sake of completeness. A pseudo-UGen is a bit of
SuperCollider code that abbreviates a certain configuration of UGens that gets used repeatedly. A pseudo-UGen is a class
that superficially resembles a UGen class, but it only returns a composition of existing UGens. It has no efficiency
savings, but it does save typing.

The below example has only a code::.ar:: method, but you can just as easily have both code::.ar:: and code::.kr::
methods.

code::
BoringMixer {
    *ar { arg left, right;
        ^(left + right) * 0.5;
    }
}
::

Examples of pseudo-UGens found in SC include link::Classes/BLowPass4:: and link::Classes/BHiPass4::, which break down
into link::Classes/SOS:: UGens.

There are very few restrictions on what these classes can contain, but you should keep the following in mind:

list::
## It is courteous to leave a note in your class's help file that identifies it as a pseudo-UGen.
## To avoid confusion, pseudo-UGens should not inherit from the base class link::Classes/UGen::.
## A SynthDef can only have two link::Classes/LocalIn::/link::Classes/LocalOut:: pairings — one for control rate and one
for audio rate. Putting LocalIn and LocalOut in a pseudo-UGen is a bad idea, as it could interfere with other feedback
loops in the SynthDef.
::

section:: Basics of UGens

A (real) UGen needs two components: a plugin for the server, and a class for the language. The class goes in an ordinary
code::*.sc:: file, and defines the interface for your UGen in the language. This class is generally just a few lines of
code that ultimately call the class method link::Classes/UGen#-multiNew::.

The server plugin is where the actual UGen behavior is defined. It is given by a dynamically loaded library written in
C++, whose format is platform-dependent:

list::
## code::*.so:: for *nix
## code::*.dll:: for Windows
## code::*.scx:: for macOS
::

A plugin file can contain more than one UGen, and it can also define things other than UGens such as buffer fill
("/b_gen") commands.

When the server boots, it will look for plugin files in code::Platform.userExtensionDir::. Since sclang also looks for
class files in the same location, the class file and the library file can go in the same place.

Plug-ins are loaded during the startup of the server, so it will have to be restarted after (re-)compiling a
plugin. If you modify the plugin file but not the class file, you don't need to reboot the interpreter.

section:: FAUST

FAUST footnote::http://faust.grame.fr/:: is an open source DSP language that describes real-time audio units. It can
compile to SuperCollider plugins, providing an easy way to create UGens in SuperCollider.

FAUST provides a shell script useful for SuperCollider users called code::faust2supercollider::. This compiles a .dsp
file into a class file and server plugin, which you can then drop into your extensions directory.

FAUST plugins are often quick to develop and can be painlessly ported to other environments. Unfortunately, they can't
take advantage of all of the server's features, such as accessing Buffers or random number generators, and some UGens
featuring very complex logic are difficult or impossible to write in FAUST. Furthermore, the FAUST compiler is quite
intelligent but it might not always offer the best efficiency in its results. If a UGen you are developing hits these
limitations, it is time to move on to handwritten C++.

section:: Example Plugins

To get an idea of the necessary ingredients for writing UGens, it's often best to poke around at complete examples.
We've set up a GitHub repository at https://github.com/supercollider/example-plugins, which contains some example
plugins numbered roughly by complexity. Each directory in that repository is self-contained with its own build system,
so you can copy out a directory to form a starting point for your own UGens. The source codes of these plugins are
heavily commented.

The first example, BoringMixer, is very minimal. The UGen is stateless and has only one calculation function, which is
audio rate.

MySaw introduces states and multiple calculation functions. AnalogEcho introduces real-time memory management through
internal buffers, and demonstrates how to do cubic interpolation from an array of samples.

section:: Anatomy of a UGen

The SC source code has a header file, code::include/plugin_interface/SC_PlugIn.h::, that gives you your interface to the server
architecture as well as a bunch of helper functions. These are documented at link::Reference/ServerPluginAPI::.

subsection:: The Entry Point

When the library is loaded the server calls a function in the library, which is defined by the code::PluginLoad()::
macro.  This entry point has two responsibilities:

list::
## It needs to store the passed in pointer to the InterfaceTable in a global variable.
## It registers the unit generators.
::

Unit Generators are defined by calling a function in the InterfaceTable and passing it the name of the unit generator,
the size of its C data struct, and pointers to functions for constructing and destructing it. There are 4 macros, which
can be used to simplify the process.

definitionList::
## DefineSimpleUnit || Define a `simple' unit generator
## DefineDtorUnit || Define a unit generator with a destructor
## DefineSimpleCantAliasUnit || Define a `simple' unit generator, whose input and output buffers cannot alias
## DefineDtorCantAliasUnit || Define a unit generator with a destructor, whose input and output buffers cannot alias
::

These macros depend on a specific naming convention:
list::
## The unit generator struct is named like the plug-in.
## The unit generator constructor is named code::PluginName_Ctor::
## The unit generator destructor is named code::PluginName_Dtor::
::

subsection:: The Calculation Function

The meat of the UGen is its calculation function, which gets called every control period with the UGen object as an
argument. (This is for control-rate and audio-rate UGens -- demand-rate is different.) In this function, the UGen reads
from its inputs and writes to its outputs.

The calculation function is selected in the code::PluginName_Ctor:: function with the code::SETCALC:: macro. You can
name the calculation function whatever you want, but the convention is code::PluginName_next::.

UGens often have multiple calculation functions, depending on the rate of the UGen itself and the rate of its inputs.
For example, Phasor can be .ar or .kr, and its argument can be either .ar or .kr. So it has four calculation functions:
Phasor_next_aa, Phasor_next_ak, Phasor_next_ka, and Phasor_next_kk. You don't need to be this thorough for your own
UGens, however. For example, link::Classes/FreeVerb:: has only one calculation function. Who would want a control-rate
reverb?

subsection:: Building Unit Generator Plugins

The most portable way to build plugins is using cmake footnote::http://www.cmake.org::, a cross-platform build
system.

The examples in the example repository contain code::CMakeLists.txt:: files.

section:: Coding Guidelines

Unit generator plugins are called from the real-time context, which means that special care needs to be taken in order
to avoid audio dropouts.

definitionList::
## STL Containers || It is generally not recommended to use STL containers, since they internally allocate memory. The only
                     way the STL containers can be used is by providing an Allocator, which maps to the allocating functions of
                     the server.
## Blocking API Calls || Unit generators should not call any code, which could block the execution of the current thread. In
                         particular, system calls should be avoided. If synchronization with other threads is required, this has to be
                         done in a lock-free manner.
::

subsection:: Memory Allocation
Do not allocate memory from the OS via code::malloc:: / code::free:: or code::new::/ code::delete::. Instead you should use the real-time memory allocator via code::RTAlloc:: / code::RTFree::.

definitionList::
## Remember to code::RTFree:: the memory you code::RTAlloc:: ||

Like with code::malloc/free::, you are reponsible for freeing all the memory you allocate. Remember to include code::RTFree:: calls in your destructor functions.

## Ensure your pointers are initialized to code::nullptr:: ||

Not initialized pointers can have a garbage address, and when RTFree tries to free such a pointer, it can crash the server.
To prevent this, ensure your member pointers are initialized to code::nullptr:: as early as possible in your constructor functions.

## Use code::ClearUnitIfMemFailed:: ||
This macro is used to check your pointers after memory allocation. If any of them is still code::nullptr::, it means that RTAlloc failed to allocate memory for it.
The macro will then print an error message, set the UGen's calculation function to a no-op, and return from the calling function immediately.
Since this can cause early exit from your constructor function, it is fundamental that all pointers are initialized to code::nullptr:: as early as possible, as stated above.

code::ClearUnitIfMemFailed:: can be passed a single pointer, or it can check multiple pointers at the same time, by chaining them with the code::&&:: operator (see examples below).

## For PV and FFT UGens ||
UGens in FFT chains (such as the ones listed in LINK::Guides/FFT-Overview#PV and FFT UGens in the Standard Library::) should use code::ClearFFTUnitIfMemFailed:: instead. This is because, on a failed allocation, code::ClearUnitIfMemFailed:: would make them output code::0::, which would be interpreted by the next UGen in the FFT chain as "FFT data is ready to be processed on buffer number 0", which is not the case. code::ClearFFTUnitIfMemFailed:: will set their output to code::-1:: instead, meaning that FFT data is not ready, and thus blocking further processing for the rest of the FFT chain. For more informations please see LINK::Guides/FFT-Overview#How FFT UGens communicate::.

code::ClearFFTUnitIfMemFailed:: is defined in code::FFT_UGens.h::
::

Minimal example, C style:
code::
struct MyUnit : public Unit {
  float* m_values;
  float* m_moreValues;
}
void MyUnit_Ctor(MyUnit* unit) {
  // 1. Ensure pointers are initialized to nullptr
  unit->m_values = unit->m_moreValues = nullptr;
  // 2. Allocate memory
  unit->m_values = (float*) RTAlloc(unit->mWorld, 64 * sizeof(float));
  unit->m_moreValues = (float*) RTAlloc(unit->mWorld, 128 * sizeof(float));
  // 3. Clear unit if any allocation failed
  ClearUnitIfMemFailed(unit->m_values && unit->m_moreValues);
  // 4. Feel free to access memory now
  memset(unit->m_values, 0, 64 * sizeof(float));
}

void MyUnit_Dtor(MyUnit* unit) {
  // 5. Free your allocated memory
  RTFree(unit->mWorld, unit->m_values);
  RTFree(unit->mWorld, unit->m_moreValues);
}
::
Or, in C++ class style:
code::
namespace MyUnit {
  // 1. Use initializer list to ensure pointers are initialized to nullptr
  MyUnit::MyUnit():
    mValues(nullptr),
    mMoreValues(nullptr) {
    // you'll need to define unit in order to use ClearUnitIfMemFailed
    Unit* unit = (Unit*) this;
    // 2. Allocate memory
    mValues = (float*) RTAlloc(unit->mWorld, 64 * sizeof(float));
    mMoreValues = (float*) RTAlloc(unit->mWorld, 128 * sizeof(float));
    // 3. Clear unit if any allocation failed
    ClearUnitIfMemFailed(mValues && mMoreValues);
    // 4. Feel free to access memory now
    memset(mValues, 0, 64 * sizeof(float));
  }

  MyUnit::~MyUnit() {
    // 5. Free your allocated memory
    RTFree(mWorld, mValues);
    RTFree(mWorld, mMoreValues);
  }
}
::

subsection:: Thread Safety

There are two different implementations of the SuperCollider server. strong::scsynth:: is the traditional server and
strong::supernova:: is a new implementation with support for multi-processor audio synthesis. Since the plugins in
strong::supernova:: can be called at the same time from multiple threads, write access to global data structures needs
to be synchronized.

definitionList::
## Shared Global Data Structures || Unit generators should not share data structures, which are written to. While it it safe to use
    global data structures for read-only purposes (e.g. different unit generators could use the same constant wavetable),
    the data structures that are modified by the unit generators should not be shared among different instances.

## Resource Locking || SuperCollider's buffers and busses are global data structures, and access needs to be synchronized.
    This is done internally by using reader-writer spinlocks. This is done by using the code::ACQUIRE_::, code::RELEASE_::, and
    code::LOCK_:: macros, which are defined in SC_Unit.h. As exception, buffers in the wavetable format are not required to be
    locked.
::

subsection:: Deadlock Prevention

In order to prevent deadlocks, a simple deadlock prevention scheme is implemented, based on the following constraints.

list::
## Lock resources only when required: few unit generators actually require the access to more than one resource at the same time.
   The main exception of this rule are the FFT Chain UGens, which access multiple buffers at the same time. There is no known unit
   generator, which accesses both buffers and busses at the same time.
## Acquire reader locks if possible. Since multiple UGens can acquire a reader lock to the same resource at the same time, their
   use reduces contention.
## Resources have to be acquired in a well-defined order: busses should be acquired before buffers and resources with a high index
   should be acquired before resources with a low index.
::
