Add a subchapter on cue markers

[ardour-manual] / include / lua-scripting.html

<p>
Starting with version 4.7.213, Ardour supports Lua scripts.
</p>


<p class="warning">
This Documentation is Work in Progress and far from complete. Also the documented API may be subject to change.
</p>

<h2 id="Preface">Preface</h2>
<p>
There are cases that Ardour cannot reasonably cater to with core functionality alone, either because they're session specific
or user specific edge cases.
</p><p>
Examples for these include voice-activate (record-arm specific tracks and roll transport depending on signal levels),
rename all regions after a specific timecode, launch an external application when a certain track is soloed, generate
automation curves or simply provide a quick shortcut for a custom batch operation.
</p><p>
Cases like this call for means to extend the DAW without actually changing the DAW itself. This is where scripting comes in.
</p><p>
"Scripting" refers to tasks that could alternatively be executed step-by-step by a human operator.
</p><p>
Lua is a tiny and simple language which is easy to learn, yet allows for comprehensive solutions.
Lua is also a glue language it allows to tie existing component in Ardour together in unprecedented ways,
and most importantly Lua is one of the few scripting-languages which can be safely used in a real-time environment.
</p><p>
A good introduction to Lua is the book <a href="http://www.lua.org/pil/">Programming in Lua</a>. The first edition is
available online, but if you have the means buy a copy of the book, it not only helps to support the Lua project, but provides
for a much nicer reading and learning experience.
</p>

<h2 id="Overview">Overview</h2>
<p>
The core of Ardour is a real-time audio engine that runs and processes audio. One interfaces with an engine by sending it
commands. Scripting can be used to interact with or modify the active Ardour session, just like a user uses the Editor/Mixer
GUI to modify the state or parameters of the session.
</p><p>
Doing this programmatically requires some knowledge about the objects used internally.
Most Ardour C++ objects and their methods are directly exposed to Lua and one can call functions or modify variables:
</p>

<div style="width:80%; margin:.5em auto;">
        <div style="width:45%; float:left;">
                C++<br/>
                <code class="cxx">
                        session-&gt;set_transport_speed (1.0);
                </code>
        </div>
        <div style="width:45%; float:right;">
                Lua<br/>
                <code class="lua">
                        Session:set_transport_speed (1.0)
                </code>
        </div>
</div>
<div style="clear:both;"></div>

<p>
You may notice that there is only a small syntactic difference in this case.
While C++ requires recompiling the application for every change, Lua script can be loaded, written or modified while the
application is running. Lua also abstracts away many of the C++ complexities such as object lifetime, type conversion and
null-pointer checks.
</p><p>
Close ties with the underlying C++ components is where the power of scripting comes from.
A script can orchestrate interaction of lower-level components which take the bulk of the CPU time of the final program.
</p>
</p><p>
At the time of writing Ardour integrates Lua 5.3.5: <a href="http://www.lua.org/manual/5.3/manual.html">Lua 5.3 reference
manual</a>.
</p>

<h2 id="Integration">Integration</h2>
<p>
Like Control surfaces and the GUI, Lua Scripts are confined to certain aspects of the program. Ardour provides the framework
and runs Lua (not the other way around).
</p>
<p>
In Ardour's case Lua is available:
</p>

<table class="dl">
        <tr><th>Editor Action Scripts</th><td>User initiated actions (menu, shortcuts) for batch processing</td></tr>
        <tr><th>Editor Hooks/Callbacks</th><td>Event triggered actions for the Editor/Mixer GUI</td></tr>
        <tr><th>Session Scripts</th><td>Scripts called at the start of every audio cycle (session, real-time)</td></tr>
        <tr><th>DSP Scripts</th><td>Audio/Midi processor - plugins with access to the Ardour session (per track/bus, real-time)</td></tr>
        <tr><th>Script Console</th><td>Action Script commandline</td></tr>
</table>

<p>
There are is also a special mode:
</p>
<table class="dl">
        <tr><th>Commandline Tool</th><td>Replaces the complete Editor GUI, direct access to libardour (no GUI) from the
        commandline.<br/>
        <em>Be aware that the vast majority of complex functionality is provided by the Editor UI.</em></td></tr>
</table>

<h2 id="Managing Scripts">Managing Scripts</h2>

<p>
Ardour searches for Lua scripts in the <code>scripts</code> folder in <code>$ARDOUR_DATA_PATH</code>,
Apart from scripts included directly with Ardour, this includes</p>
<table>
        <tr><th>GNU/Linux</th><td><code>$HOME/.config/ardour6/scripts</code></td></tr>
        <tr><th>Mac OS X</th><td><code>$HOME/Library/Preferences/Ardour6/scripts</code></td></tr>
        <tr><th>Windows</th><td><code>%localappdata%\ardour6\scripts</code></td></tr>
</table>

<p>Files must end with <code>.lua</code> file extension.</p>

<p>Scripts are managed via the GUI</p>
<table class="dl">
        <tr><th>Editor Action Scripts</th><td>Menu &rarr; Edit &rarr; Scripted Actions &rarr; Manage</td></tr>
        <tr><th>Editor Hooks/Callbacks</th><td>Menu &rarr; Edit &rarr; Scripted Actions &rarr; Manage</td></tr>
        <tr><th>Session Scripts</th><td>Menu &rarr; Session &rarr; Scripting &rarr; Add/Remove Script</td></tr>
        <tr><th>DSP Scripts</th><td>Mixer-strip &rarr; context menu (right click) &rarr; New Lua Proc</td></tr>
        <tr><th>Script Console</th><td>Menu &rarr; Window &rarr; Scripting</td></tr>
</table>

<h2 id="Script Layout">Script Layout</h2>
<ul>
        <li>Every script must include an <code>ardour</code> descriptor table. Required fields are "Name" and "Type".</li>
        <li>A script must provide a <em>Factory method</em>: A function with optional instantiation parameters which returns the
        actual script.</li>
        <li>[optional]: list of parameters for the "factory".</li>
        <li>in case of DSP scripts, an optional list of automatable parameters and possible audio/midi port configurations, and a
        <code>dsp_run</code> function, more on that later.</li>
</ul>


<p>A minimal example script looks like:</p>
<div>
<pre><code class="lua">
        ardour {
          ["type"]    = "EditorAction",
          name        = "Rewind",
        }

        function factory (unused_params)
          return function ()
           Session:goto_start()  -- rewind the transport
          end
        end
</code></pre>
</div>

<p>
The common part for all scripts is the "Descriptor". It's a Lua function which returns a table (key/values) with the following
keys (the keys are case-sensitive):
</p>
<table class="dl">
        <tr><th>type [required]</th><td>one of "<code>DSP</code>", "<code>Session</code>", "<code>EditorHook</code>",
        "<code>EditorAction</code>" (the type is not case-sensitive)</td></tr>
        <tr><th>name [required]</th><td>Name/Title of the script</td></tr>
        <tr><th>author</th><td>Your Name</td></tr>
        <tr><th>license</th><td>The license of the script (e.g. "GPL" or "MIT")</td></tr>
        <tr><th>description</th><td>A longer text explaining to the user what the script does</td></tr>
</table>

<p class="note">
Scripts that come with Ardour (currently mostly examples) can be found in the
<a href="https://github.com/Ardour/ardour/tree/master/share/scripts">Source Tree</a>.
</p>

<h3 id="Action Scripts">Action Scripts</h3>
<p>Action scripts are the simplest form. An anonymous Lua function is called whenever the action is triggered. A simple action
script is shown above.</p>
<p>There are 10 action script slots available, each of which is a standard GUI action available from the menu and hence can be
bound to a keyboard shortcut.</p>

<h3 id="Session Scripts">Session Scripts</h3>
<p>Session scripts similar to Actions Scripts, except the anonymous function is called periodically every process cycle.
The function receives a single parameter - the number of audio samples which are processed in the given cycle</p>
<div>
<pre><code class="lua">
ardour {
  ["type"]    = "session",
  name        = "Example Session Script",
  description = [[
  An Example Ardour Session Script.
  This example stops the transport after rolling for a specific time.]]
}


-- instantiation options, these are passed to the "factory" method below
function sess_params ()
  return
  {
    ["print"]  = { title = "Debug Print (yes/no)", default = "no", optional = true },
    ["time"] = { title = "Timeout (sec)", default = "90", optional = false },
  }
end

function factory (params)
  return function (n_samples)
    local p = params["print"] or "no"
    local timeout = params["time"] or 90
    a = a or 0
    if p ~= "no" then print (a, n_samples, Session:frame_rate (), Session:transport_rolling ()) end -- debug output (not rt safe)
    if (not Session:transport_rolling()) then
      a = 0
      return
    end
    a = a + n_samples
    if (a &gt; timeout * Session:frame_rate()) then
      Session:request_transport_speed (0.0, true, ARDOUR.TransportRequestSource.TRS_UI)
    end
  end
end
</code></pre>
</div>

<h3 id="Action Hooks">Action Hooks</h3>
<p>Action hook scripts must define an additional function which returns a <em>Set</em> of Signal that which trigger the
callback (documenting available slots and their parameters remains to be done).</p>
<div>
<pre><code class="lua">
ardour {
  ["type"]    = "EditorHook",
  name        = "Hook Example",
  description = "Rewind On Solo Change, Write a file when regions are moved.",
}

function signals ()
  s = LuaSignal.Set()
  s:add (
    {
      [LuaSignal.SoloActive] = true,
      [LuaSignal.RegionPropertyChanged] = true
    }
  )
  return s
end

function factory (params)
  return function (signal, ref, ...)
    -- print (signal, ref, ...)

    if (signal == LuaSignal.SoloActive) then
      Session:goto_start()
    end

    if (signal == LuaSignal.RegionPropertyChanged) then
      obj,pch = ...
      file = io.open ("/tmp/test" ,"a")
      io.output (file
      io.write (string.format ("Region: '%s' pos-changed: %s, length-changed: %s\n",
        obj:name (),
        tostring (pch:containsFramePos (ARDOUR.Properties.Start)),
        tostring (pch:containsFramePos (ARDOUR.Properties.Length))
        ))
      io.close (file)
    end
  end
end
</code></pre>
</div>

<h3 id="DSP Scripts">DSP Scripts</h3>
<p>See the scripts folder for examples for now.</p>
<p>Some notes for further doc:</p>
<ul>
        <li>required function: <code>dsp_ioconfig ()</code>: return a list of possible audio I/O configurations - follows Audio
        Unit conventions.</li>
        <li>optional function: <code>dsp_dsp_midi_input ()</code>: return true if the plugin can receive midi input</li>
        <li>optional function: <code>dsp_params ()</code>: return a table of possible parameters (automatable)</li>
        <li>optional function: <code>dsp_init (samplerate)</code>: called when instantiation the plugin with given samplerate.</li>
        <li>optional function: <code>dsp_configure (in, out)</code>: called after instantiation with configured plugin i/o.</li>
        <li>required function: <code>dsp_run (ins, outs, n_samples)</code> OR <code>dsp_runmap (bufs, in_map, out_map, n_samples,
        offset)</code>: DSP process callback. The former is a convenient abstraction that passes mapped buffers (as table). The
        latter is a direct pass-through matching Ardour's internal <code>::connect_and_run()</code> API, which requires the caller
        to map and offset raw buffers.</li>
        <li>plugin parameters are handled via the global variable <code>CtrlPorts</code>.</li>
        <li>midi data is passed via the global variable <code>mididata</code> which is valid during <code>dsp_run</code> only.
        (dsp_runmap requires the script to pass raw data from the buffers according to in_map)</li>
        <li>The script has access to the current session via the global variable Session, but access to the session methods are
        limited to realtime safe functions</li>
</ul>

<h2 id="Accessing Ardour Objects">Accessing Ardour Objects</h2>
<p>
The top most object in Ardour is the <code>ARDOUR::Session</code>.
Fundamentally, a Session is just a collection of other things:
Routes (tracks, busses), Sources (Audio/Midi), Regions, Playlists, Locations, Tempo map, Undo/Redo history, Ports, Transport
state and controls, etc.
</p><p>
Every Lua interpreter can access it via the global variable <code>Session</code>.
</p><p>
GUI context interpreters also have an additional object in the global environment: The Ardour <code>Editor</code>. The Editor
provides access to high level functionality which is otherwise triggered via GUI interaction such as undo/redo, open/close
windows, select objects, drag/move regions. It also holds the current UI state: snap-mode, zoom-range, etc.
The Editor also provides complex operations such as "import audio" which under the hood, creates a new Track, adds a new
Source Objects (for every channel) with optional resampling, creates both playlist and regions and loads the region onto the
Track all the while displaying a progress information to the user.
</p>

<p class="note">
Documenting the bound C++ methods and class hierarchy is somewhere on the ToDo list.
Meanwhile <a href="https://github.com/Ardour/ardour/blob/master/libs/ardour/luabindings.cc">luabindings.cc</a> is the best we
can offer.
</p>

<h2 id="Concepts">Concepts</h2>
<ul>
        <li>There are no bound constructors: Lua asks Ardour to create objects (e.g. add a new track), then receives a reference
        to the object to modify it.</li>
        <li>Scripts, once loaded, are saved with the Session (no reference to external files). This provides for portable
        Sessions.</li>
        <li>Lua Scripts are never executed directly. They provide a "factory" method which can have optional instantiation
        parameters, which returns a Lua closure.</li>
        <li>No external Lua modules/libraries can be used, scripts need to be self contained (portable across different systems
        (libs written in Lua can be used, and important c-libs/functions can be included with Ardour if needed).</li>
</ul>
<p>
Ardour is a highly multithreaded application and interaction between the different threads, particularly real-time threads,
needs to to be done with care. This part has been abstracted away by providing separate Lua interpreters in different contexts
and restricting available interaction:
</p>
<ul>
        <li>Editor Actions run in a single instance interpreter in the GUI thread.</li>
        <li>Editor Hooks connect to libardour signals. Every Callback uses a dedicated Lua interpreter which is in the GUI thread
        context.</li>
        <li>All Session scripts run in a single instance in the main real-time thread (audio callback)</li>
        <li>DSP scripts have a separate instance per script and run in one of the DSP threads.</li>
</ul>
<p>
The available interfaces differ between contexts. For example, it is not possible to create new tracks or import audio from
real-time context; while it is not possible to modify audio buffers from the GUI thread.
</p>

<h2 id="Current State">Current State</h2>
Fully functional, yet still in a prototyping stage:

<ul>
        <li>The GUI to add/configure scripts is rather minimalistic.</li>
        <li>The interfaces may change (particularly DSP, and Session script <code>run()</code>.</li>
        <li>Further planned work includes:
                <ul>
                        <li>Built-in Script editor (customize/modify Scripts in-place)</li>
                        <li>convenience methods (wrap more complex Ardour actions into a library). e.g set plugin parameters, write
                        automation lists from a Lua table</li>
                        <li>Add some useful scripts and more examples</li>
                        <li>Documentation (Ardour API), also usable for tab-expansion, syntax highlighting</li>
                        <li>bindings for GUI Widgets (plugin UIs, message boxes, etc)</li>
                </ul>
                <li>
</ul>

-<h2 id="Examples">Examples</h2>
-<p>Please see the example <a href="https://github.com/Ardour/ardour/tree/master/share/scripts">scripts included with the source-code</a>.
All the files that start with a leading underscore are not inlcluded with releases, but are intended as example snippets.</p>

<h2 id="Commandline Session">Commandline Session</h2>
<p>The standalone tool <code>luasession</code> allows one to access an Ardour session directly from the commandline.
It can also be used as #! interpreter for scripted sessions.
Interaction is limited by the fact that most actions in Ardour are provided by the Editor GUI.
</p><p>
<code>luasession</code> provides only two special functions <code>load_session</code> and <code>close_session</code> and
exposes the <code>AudioEngine</code> instance as global variable.
</p>

<div>
<pre><code class="lua">
for i,_ in AudioEngine:available_backends():iter() do print (i.name) end

backend = AudioEngine:set_backend("ALSA", "", "")
print (AudioEngine:current_backend_name())

for i,_ in backend:enumerate_devices():iter() do print (i.name) end

backend:set_device_name("HDA Intel PCH")
backend:set_buffer_size(1024)

print (backend:buffer_size())
print (AudioEngine:get_last_backend_error())

s = load_session ("/home/rgareus/Documents/ArdourSessions/lua2/", "lua2")

assert (s)

s:request_transport_speed (1.0, true, ARDOUR.TransportRequestSource.TRS_UI)
print (s:transport_rolling())

s:goto_start()

ARDOUR.LuaAPI.usleep (10 * 1000000) -- 10 seconds

close_session()

</code></pre>
</div>