re-organize lua scripting doc

[ardour-manual] / include / lua-scripting.html

<p>
Starting with version 4.7.213, Ardour supports Lua scripts.
</p>


<p class="warning">
This Documentation is Work in Progress and far from complete. Also the documented API may be subject to change.
</p>

<h2 id="Preface">Preface</h2>
<p>
There are cases that a Ardour cannot reasonably cater for with core functionality by itself, either because they're session specific or user specific edge cases.
</p><p>
Examples for these include voice-activate (record-arm specific tracks and roll transport depending on signal levels),
rename all regions after a specific timecode, launch an external application when a certain track is soloed, generate automation curves
or simply provide a quick shortcut for a custom batch operation.
</p><p>
Cases like this call for means to extend the DAW without actually changing the DAW itself. This is where scripting comes in.
</p><p>
"Scripting" refers to tasks that could alternatively be executed step-by-step by a human operator.
</p><p>
Lua is a tiny and simple language which is easy to learn, yet allows for comprehensive solutions.
Lua is also a glue language it allows to tie existing component in Ardour together in unprecedented ways,
and most importantly Lua is one of the few scripting-languages which can be safely used in a real-time environment.
</p><p>
A good introduction to Lua is the book <a href="http://www.lua.org/pil/">Programming in Lua</a>. The first edition is available online,
but if you have the means buy a copy of the book, it not only helps to support the Lua project,
but provides for a much nicer reading and learning experience.
</p>

<h2 id="Overview">Overview</h2>
<p>
The core of ardour is a real-time audio engine that runs and processes audio. One interfaces with than engine by sending it commands.
Scripting can be used to interact with or modify active Ardour session. Just like a user uses the Editor/Mixer GUI to modify the state or parameters of the session.
</p><p>
Doing this programmatically requires some knowledge about the objects used internally.
Most Ardour C++ objects and their methods are directly exposed to Lua and one can call functions or modify variables:
</p>

<div style="width:80%; margin:.5em auto;">
        <div style="width:45%; float:left;">
                C++<br/>
                <code class="cxx">
                        session-&gt;set_transport_speed (1.0);
                </code>
        </div>
        <div style="width:45%; float:right;">
                Lua<br/>
                <code class="lua">
                        Session:set_transport_speed (1.0)
                </code>
        </div>
</div>
<div style="clear:both;"></div>

<p>
You may notice that there is only a small syntactic difference, in this case.
While C++ requires recompiling the application for every change, Lua script can be loaded, written or modified while the application is running.
Lua also abstracts away many of the C++ complexities such as object lifetime, type conversion and null-pointer checks.
</p><p>
Close ties with the underlying C++ components is where the power of scripting comes from.
A script can orchestrate interaction of lower-level components which take the bulk of the CPU time of the final program.
</p>
</p><p>
At the time of writing Ardour integrates Lua 5.3.2: <a href="http://www.lua.org/manual/5.3/manual.html">Lua 5.3 reference manual</a>.
</p>

<h2 id="Integration">Integration</h2>
<p>
Like Control surfaces and the GUI, Lua Scripts are confined to certain aspects of the program. Ardour provides the framework and runs Lua (not the other way around).
</p>
<p>
In Ardour's case Lua is available:
</p>

<dl>
        <dt>Editor Action Scripts</dt><dd>User initiated actions (menu, shortcuts) for batch processing</dd>
        <dt>Editor Hooks/Callbacks</dt><dd>Event triggered actions for the Editor/Mixer GUI</dd>
        <dt>Session Scripts</dt><dd>Scripts called at the start of every audio cycle (session, real-time)</dd>
        <dt>DSP Scripts</dt><dd>Audio/Midi processor - plugins with access to the Ardour session (per track/bus, real-time)</dd>
        <dt>Script Console</dt><dd>Action Script commandline</dd>
</dl>

<p>
There are is also a special mode:
</p>
<dl>
        <dt>Commandline Tool</dt><dd>Replaces the complete Editor GUI, direct access to libardour (no GUI) from the commandline.<br/>
        <em>Be aware that the vast majority of complex functionality is provided by the Editor UI.</em></dd>
</dl>

<h2 id="Managing Scripts">Managing Scripts</h2>

<p>
Ardour searches for Lua scripts in the <code>scripts</code> folder in <code>$ARDOUR_DATA_PATH</code>,
Apart from scripts included directly with Ardour, this includes</p>
<table>
        <tr><th>GNU/Linux</th><td><code>$HOME/.config/ardour5/scripts</code></td></tr>
        <tr><th>Mac OS X</th><td><code>$HOME/Library/Preferences/Ardour5/scripts</code></td></tr>
        <tr><th>Windows</th><td><code>%localappdata%\ardour5\scripts</code></td></tr>
</table>

<p>Files must end with <code>.lua</code> file extension.</p>

<p>Scripts are managed via the GUI</p>
<dl>
        <dt>Editor Action Scripts</dt><dd>Menu &rarr; Edit &rarr; Scripted Actions &rarr; Manage</dd>
        <dt>Editor Hooks/Callbacks</dt><dd>Menu &rarr; Edit &rarr; Scripted Actions &rarr; Manage</dd>
        <dt>Session Scripts</dt><dd>Menu &rarr; Session &rarr; Scripting &rarr; Add/Remove Script</dd>
        <dt>DSP Scripts</dt><dd>Mixer-strip &rarr; context menu (right click) &rarr; New Lua Proc</dd>
        <dt>Script Console</dt><dd>Menu &rarr; Window &rarr; Scripting</dd>
</dl>

<h2 id="Script Layout">Script Layout</h2>
<ul>
        <li>Every script must include an <code>ardour</code> descriptor table. Required fields are "Name" and "Type".</li>
        <li>A script must provide a <em>Factory method</em>: A function with optional instantiation parameters which returns the actual script.</li>
        <li>[optional]: list of parameters for the "factory".</li>
        <li>in case of DSP scripts, an optional list of automatable parameters and possible audio/midi port configurations, and a <code>dsp_run</code> function, more on that later.</li>
</ul>


<p>A minimal example script looks like:</p>
<div>
<pre><code class="lua">
        ardour {
          ["type"]    = "EditorAction",
          name        = "Rewind",
        }

        function factory (unused_params)
          return function ()
           Session:goto_start()  -- rewind the transport
          end
        end
</code></pre>
</div>

<p>
The common part for all scripts is the "Descriptor". It's a Lua function which returns a table (key/values) with the following keys (the keys are case-sensitive):
</p>
<dl>
        <dt>type [required]</dt><dd>one of "<code>DSP</code>", "<code>Session</code>", "<code>EditorHook</code>", "<code>EditorAction</code>" (the type is not case-sensitive)</dd>
        <dt>name [required]</dt><dd>Name/Title of the script</dd>
        <dt>author</dt><dd>Your Name</dd>
        <dt>license</dt><dd>The license of the script (e.g. "GPL" or "MIT")</dd>
        <dt>description</dt><dd>A longer text explaining to the user what the script does</dd>
</dl>

<p class="note">
Scripts that come with Ardour (currently mostly examples) can be found in the <a href="https://github.com/Ardour/ardour/tree/master/scripts">Source Tree</a>.
</p>

<h3 id="Action Scripts">Action Scripts</h3>
<p>Action scripts are the simplest form. An anonymous Lua function is called whenever the action is triggered. A simple action script is shown above.</p>
<p>There are 10 action script slots available, each of which is a standard GUI action available from the menu and hence can be bound to a keyboard shortcut</p>

<h3 id="Session Scripts">Session Scripts</h3>
<p>Session scripts similar to Actions Scripts, except the anonymous function is called periodically every process cycle.
The function receives a single parameter - the number of audio samples which are processed in the given cycle</p>
<div>
<pre><code class="lua">
ardour {
  ["type"]    = "session",
  name        = "Example Session Script",
  description = [[
  An Example Ardour Session Script.
  This example stops the transport after rolling for a specific time.]]
}


-- instantiation options, these are passed to the "factory" method below
function sess_params ()
  return
  {
    ["print"]  = { title = "Debug Print (yes/no)", default = "no", optional = true },
    ["time"] = { title = "Timeout (sec)", default = "90", optional = false },
  }
end

function factory (params)
  return function (n_samples)
    local p = params["print"] or "no"
    local timeout = params["time"] or 90
    a = a or 0
    if p ~= "no" then print (a, n_samples, Session:frame_rate (), Session:transport_rolling ()) end -- debug output (not rt safe)
    if (not Session:transport_rolling()) then
      a = 0
      return
    end
    a = a + n_samples
    if (a &gt; timeout * Session:frame_rate()) then
      Session:request_transport_speed(0.0, true)
    end
  end
end
</code></pre>
</div>

<h3 id="Action Hooks">Action Hooks</h3>
<p>Action hook scripts must define an additional function which returns a <em>Set</em> of Signal that which trigger the callback (documenting available slots and their parameters remains to be done).</p>
<div>
<pre><code class="lua">
ardour {
  ["type"]    = "EditorHook",
  name        = "Hook Example",
  description = "Rewind On Solo Change, Write a file when regions are moved.",
}

function signals ()
  s = LuaSignal.Set()
  s:add (
    {
      [LuaSignal.SoloActive] = true,
      [LuaSignal.RegionPropertyChanged] = true
    }
  )
  return s
end

function factory (params)
  return function (signal, ref, ...)
    -- print (signal, ref, ...)

    if (signal == LuaSignal.SoloActive) then
      Session:goto_start()
    end

    if (signal == LuaSignal.RegionPropertyChanged) then
      obj,pch = ...
      file = io.open ("/tmp/test" ,"a")
      io.output (file
      io.write (string.format ("Region: '%s' pos-changed: %s, length-changed: %s\n",
        obj:name (),
        tostring (pch:containsFramePos (ARDOUR.Properties.Start)),
        tostring (pch:containsFramePos (ARDOUR.Properties.Length))
        ))
      io.close (file)
    end
  end
end
</code></pre>
</div>

<h3 id="DSP Scripts">DSP Scripts</h3>
<p>See the scripts folder for examples for now.</p>
<p>Some notes for further doc:</p>
<ul>
        <li>required function: <code>dsp_ioconfig ()</code>: return a list of possible audio I/O configurations - follows Audio Unit conventions.</li>
        <li>optional function: <code>dsp_dsp_midi_input ()</code>: return true if the plugin can receive midi input</li>
        <li>optional function: <code>dsp_params ()</code>: return a table of possible parameters (automatable)</li>
        <li>optional function: <code>dsp_init (samplerate)</code>: called when instantiation the plugin with given samplerate.</li>
        <li>optional function: <code>dsp_configure (in, out)</code>: called after instantiation with configured plugin i/o.</li>
        <li>required function: <code>dsp_run (ins, outs, n_samples)</code> OR <code>dsp_runmap (bufs, in_map, out_map, n_samples, offset)</code>: DSP process callback. The former is a convenient abstraction that passes mapped buffers (as table). The latter is a direct pass-through matching Ardour's internal <code>::connect_and_run()</code> API, which requires the caller to map and offset raw buffers.</li>
        <li>plugin parameters are handled via the global variable <code>CtrlPorts</code>.</li>
        <li>midi data is passed via the global variable <code>mididata</code> which is valid during <code>dsp_run</code> only. (dsp_runmap requires the script to pass raw data from the buffers according to in_map)</li>
        <li>The script has access to the current session via the global variable Session, but access to the session methods are limited to realtime safe functions</li>
</ul>

<h2 id="Accessing Ardour Objects">Accessing Ardour Objects</h2>
<p>
The top most object in Ardour is the <code>ARDOUR::Session</code>.
Fundamentally, a Session is just a collection of other things:
Routes (tracks, busses), Sources (Audio/Midi), Regions, Playlists, Locations, Tempo map, Undo/Redo history, Ports, Transport state &amp; controls, etc.
</p><p>
Every Lua interpreter can access it via the global variable <code>Session</code>.
</p><p>
GUI context interpreters also have an additional object in the global environment: The Ardour <code>Editor</code>. The Editor provides access to high level functionality which is otherwise triggered via GUI interaction such as undo/redo, open/close windows, select objects, drag/move regions. It also holds the current UI state: snap-mode, zoom-range, etc.
The Editor also provides complex operations such as "import audio" which under the hood, creates a new Track, adds a new Source Objects (for every channel) with optional resampling, creates both playlist and regions and loads the region onto the Track all the while displaying a progress information to the user.
</p>

<p class="note">
Documenting the bound C++ methods and class hierarchy is somewhere on the ToDo list.
Meanwhile <a href="https://github.com/Ardour/ardour/blob/master/libs/ardour/luabindings.cc">luabindings.cc</a> is the best we can offer.
</p>

<h2 id="Concepts">Concepts</h2>
<ul>
        <li>There are no bound constructors: Lua asks Ardour to create objects (e.g. add a new track), then receives a reference to the object to modify it.</li>
        <li>Scripts, once loaded, are saved with the Session (no reference to external files). This provides for portable Sessions.</li>
        <li>Lua Scripts are never executed directly. They provide a "factory" method which can have optional instantiation parameters, which returns a lua closure.</li>
        <li>No external lua modules/libraries can be used, scripts need to be self contained (portable across different systems (libs written in Lua can be used, and important c-libs/functions can be included with ardour if needed).</li>
</ul>
<p>
Ardour is a highly multithreaded application and interaction between the different threads, particularly real-time threads, needs to to be done with care.
This part has been abstracted away by providing separate Lua interpreters in different contexts and restricting available interaction:
</p>
<ul>
        <li>Editor Actions run in a single instance interpreter in the GUI thread.</li>
        <li>Editor Hooks connect to libardour signals. Every Callback uses a dedicated lua interpreter which is in the GUI thread context.</li>
        <li>All Session scripts run in a single instance in the main real-time thread (audio callback)</li>
        <li>DSP scripts have a separate instance per script and run in one of the DSP threads.</li>
</ul>
<p>
The available interfaces differ between contexts. e.g. it is not possible to create new tracks or import audio from real-time context; while it is not possible to modify audio buffers from the GUI thread.
</p>

<h2 id="Current State">Current State</h2>
Fully functional, yet still in a prototyping stage:

<ul>
        <li>The GUI to add/configure scripts is rather minimalistic.</li>
        <li>The interfaces may change (particularly DSP, and Session script <code>run()</code>.</li>
        <li>Further planned work includes:
                <ul>
                        <li>Built-in Script editor (customize/modify Scripts in-place)</li>
                        <li>convenience methods (wrap more complex Ardour actions into a library). e.g set plugin parameters, write automation lists from a lua table</li>
                        <li>Add some useful scripts and more examples</li>
                        <li>Documentation (Ardour API), also usable for tab-exansion, syntax highlighting</li>
                        <li>bindings for GUI Widgets (plugin UIs, message boxes, etc)</li>
                </ul>
                <li>
</ul>


<h2 id="Examples">Examples</h2>
<p>Apart from the <a href="https://github.com/Ardour/ardour/tree/master/scripts">scripts included with the source-code</a> here are a few examples without further comments...

<h3 id="Editor Console Examples">Editor Console Examples</h3>
<div>
<pre><code class="lua">
print (Session:route_by_remote_id(1):name())

a = Session:route_by_remote_id(1);
print (a:name());

print(Session:get_tracks():size())

for i, v in ipairs(Session:unknown_processors():table()) do print(v) end
for i, v in ipairs(Session:get_tracks():table()) do print(v:name()) end

for t in Session:get_tracks():iter() do print(t:name()) end
for r in Session:get_routes():iter() do print(r:name()) end


Session:tempo_map():add_tempo(ARDOUR.Tempo(100,4), Timecode.BBT_TIME(4,1,0))


Editor:set_zoom_focus(Editing.ZoomFocusRight)
print(Editing.ZoomFocusRight);
Editor:set_zoom_focus(1)


files = C.StringVector();
files:push_back("/home/rgareus/data/coding/ltc-tools/smpte.wav")
pos = -1
Editor:do_import(files, Editing.ImportDistinctFiles, Editing.ImportAsTrack, ARDOUR.SrcQuality.SrcBest, pos, ARDOUR.PluginInfo())

#or in one line:
Editor:do_import(C.StringVector():add({"/path/to/file.wav"}), Editing.ImportDistinctFiles, Editing.ImportAsTrack, ARDOUR.SrcQuality.SrcBest, -1, ARDOUR.PluginInfo())

# called when a new session is loaded:
function new_session (name) print("NEW SESSION:", name) end


# read/set/describe a plugin parameter
route = Session:route_by_remote_id(1)
processor = route:nth_plugin(0)
plugininsert = processor:to_insert()

plugin = plugininsert:plugin(0)
print (plugin:label())
print (plugin:parameter_count())

x = ARDOUR.ParameterDescriptor ()
_, t = plugin:get_parameter_descriptor(2, x) -- port #2
paramdesc = t[2]
print (paramdesc.lower)

ctrl = Evoral.Parameter(ARDOUR.AutomationType.PluginAutomation, 0, 2)
ac = plugininsert:automation_control(ctrl, false)
print (ac:get_value ())
ac:set_value(1.0, PBD.GroupControlDisposition.NoGroup)

# the same using a convenience wrapper:
route = Session:route_by_remote_id(1)
proc = t:nth_plugin (i)
ARDOUR.LuaAPI.set_processor_param (proc, 2, 1.0)

</code></pre>
</div>

<h3 id="Commandline Session">Commandline Session</h3>
<p>The standalone tool <code>luasession</code> allows one to access an Ardour session directly from the commandline.
Interaction is limited by the fact that most actions in Ardour are provided by the Editor GUI.
</p><p>
<code>luasession</code> provides only two special functions <code>load_session</code> and <code>close_session</code> and exposes the <code>AudioEngine</code> instance as global variable.
</p>

<div>
<pre><code class="lua">
for i,_ in AudioEngine:available_backends():iter() do print (i.name) end

backend = AudioEngine:set_backend("ALSA", "", "")
print (AudioEngine:current_backend_name())

for i,_ in backend:enumerate_devices():iter() do print (i.name) end

backend:set_input_device_name("HDA Intel PCH")
backend:set_output_device_name("HDA Intel PCH")

print (backend:buffer_size())
print (AudioEngine:get_last_backend_error())

s = load_session ("/home/rgareus/Documents/ArdourSessions/lua2/", "lua2")
s:request_transport_speed (1.0)
print (s:transport_rolling())
s:goto_start()
close_session()

</code></pre>
</div>