Multimedia Programming Using Max-MSP and TouchDesigner
上QQ阅读APP看书,第一时间看更新

Setting things up

Max comes in two flavors: 32-bit and 64-bit. What you choose really depends on what you want to do with Max, given that you have a 64-bit OS and therefore have a choice. A major advantage of the 32-bit version is that there are way many more externals (third-party Max objects) compiled for this version. In this book, we will make use of some externals that are not yet compatible with the 64-bit version, so I actually recommend that you go for the 32-bit one.

Getting help

So, I guess you've installed Max and have tested it out, but as the first step, we actually need you to know where you can get help if you are stuck in any way. There are several places where you can get help.

The Max-integrated help system

Max comes with an exceptionally good built-in help system. Use it whenever you have any doubt about what you are doing. There are the tutorials, the reference pages, and the object help patches. The first two can, for example, be accessed via the help menu of Max, while the latter one can be accessed simply by pressing Alt/option + clicking on an object. These help files are patchers themselves, and therefore, it's a common practice to copy-paste parts of them when working with an unfamiliar object or when we simply need exactly what's built in there already. A slightly hidden help are the examples. In our Max application folder, we find a folder called examples, which is a huge repository of partly very advanced patches.

Also, there is the clue window. You can find it in the menu by navigating to window | clue window. A floating window opens and offers you hints about many things if you roll over them with the cursor.

The forums

The Max/MSP forums, which can be found at to help. To increase your chances of getting help, there are at least three things you have to consider:

  • Search the forum before you post.
  • Be concrete and, if possible, post a tidy patch that illustrates your problem or question as isolated as possible.
  • If you post a patch, select all the objects in your patch, use the copy-compressed option from the Edit menu in Max, and paste the code as text into your post. This is the easiest and most common method of sharing patches.

Externals

Externals are Max objects made by third parties. We can often download them for free, or sometimes pay for them. Shortly, in this chapter, we are going to cover how to install externals. After installation, they behave just like normal Max objects. Externals can help us if we are confronted with a higher-level problem that we assume somebody has solved already, for example pitch tracking, an ambisonics toolset, or classic music notation tools and GUI objects. A good place to look for externals is the C '74 forum; another place is http://www.maxobjects.com/.

Other resources

It might be obvious, but if you are programming with Max/MSP, you often need additional information on the topic in which you are programming. Don't forget that whether you want to find out how Fast Fourier Transform (FFT) works, render OpenGL content, or do algorithmic composition, there are tons of resources out there that are completely independent of Max but more or less ready to be applied in it. For example, the open online courses that many universities provide also offer Computer Science classes, courses on digital signal processing, and even audio engineering. What one learns there can help a lot when dealing with Max.

Note

The following are some articles that help us with Max more directly (although some of these use a different language or no specific programming language at all):

Setting up Max

So after installing Max, we should go through a couple of things. Depending on how deep you want to go with this, feel free to jump over some parts, but audio is going to be an important part for us, so let's start here.

The audio status window

Let's first check whether audio is set up correctly. This won't be such a big deal; you'll typically have audio running in no time, but you might think about tuning it at some point, optimizing it for a certain patch, and this section should support you if you do so. If you just want to get audio up and running quickly, then proceed with the following steps:

  1. Navigate to Extras | Audiotester, so you can check all your input and output immediately while setting it up.
  2. Then, while leaving the other window open, which provides us with test signals and metering, configure audio and navigate to Options |Audio Status.

You'll get this window:

Here, you'll quickly find what to do, but let's look at it in more detail. The audio status window actually is a Max patcher, and you can press command/control + e to get into edit mode, and then press command/control + click on the view toggle box that suddenly appears to inspect the patcher (we will go through these mysterious keystrokes more slowly and in a structured manner later in the Hello World section of this chapter). The audio status window being a patch means that we can access and reconfigure all these settings within our patchers also if we like. Let's have a look at all the settings for a moment. Some of these are simple while some are quite complex. For a thorough description, refer to http://cycling74.com/docs/max6/dynamic/c74_docs.html#mspaudioio. Here, we'll just try to get an overview and go through the more important ones:

  • Audio: Here, we can switch global audio processing in Max on and off.
  • Driver: This is the audio driver of course. On a Mac, we'll typically use Core Audio; on Windows, we'll strive to use ASIO (your sound card needs to support ASIO for this to be available).
  • Input Device: This is your audio input device.
  • Output Device: This is your audio output device.
  • I/O Vector Size: This is also referred to as the buffer size or block size. It's the number of samples that are passed between the audio interface and Max at a time (as a block). Increasing this number not only gives more input/output latency but also more CPU cycles left for your computations. Decreasing it naturally causes the opposite; the input/output delay decreases at the cost of some CPU processing power. Often, you might encounter clicks in your audio output if you demand too much of your system. You can check whether this is the problem here in the status window during the CPU utilization, and if it's the case, you can try to increase the buffer size and/or decrease the sampling rate if possible to treat the problem. However, this is a bit of a simplification; too large vector sizes can also cause clicks. For more detailed information, refer to http://cycling74.com/docs/max6/dynamic/c74_docs.html#mspaudioio. As it's also stated there, optimizing a system's parameters for a certain patcher is a process of trial and error. Also, this setting affects the timing accuracy of Max messages if the audio is on, overdrive is on, and the scheduler in audio interrupt is on (which is a very typical situation). Decreasing this value will improve the timing accuracy in this case; the details of this are explained later in the In Audio interrupt point of this list.
  • Signal Vector Size: This is the size of the buffers that MSP is processing at a time. MSP objects don't process each sample individually, but chunks of samples and this is the amount of samples. Decreasing the number costs more CPU cycles, but this setting has nothing to do with audio latency. Some applications in audio demand the use of very short delays and feedback, and in MSP, our shortest delay (with feedback) is equal to this number of samples (luckily, there is gen~ in which we can have single sample delays that allow for feedback, but since this is an optional package and we might want to stay out of gen~ sometimes, this number is still important). If you are running patches with very small delays that are really at these limits, changing this number can effectively change how your patches sound. Also, this setting can be set locally for parts of our patches using [poly~]. Therefore, we will have a tendency to put in the largest number possible here.
  • Sampling rate: This is the sample rate of both Max and your audio device. The sampling theorem tells us that we can represent any frequency up to half of the sampling rate (the Nyquist frequency), so with a sampling rate of 44.1 kHz, we can work with frequencies up to 22.05 kHz. Many processes in synthesis and processing generate frequencies beyond this (which, in short, causes noise or aliasing), so we tend to either use higher sampling rates for the overall processing or up sample locally and only those processes where we think it's important. We'll see how this works using [poly~] later. The Nyquist frequency, however, will be the highest frequency you'll be able to input into Max or output from your sound card. To put it simply, here, we can trade audio quality against CPU usage.
  • Scheduler in Overdrive: This setting is pretty important. If active, Max prioritizes timing accuracy and MIDI input over user interaction, such as mouse clicks and GUI redraw. If your work will concentrate on audio and MIDI processing, you should activate overdrive. If you are mainly using Jitter, it should be turned off.
  • In Audio interrupt (Scheduler): This setting will force Max to compute the max event scheduler right before processing each signal sample block (only available when Scheduler In Overdrive is active). This can greatly improve the timing of audio triggered by the Max scheduler (not only Max processing but also MIDI), but the improvement is dependent on your I/O vector size setting. The actual resulting scheduler interval can be calculated as follows:

Here, I is the scheduler interval in seconds vs the I/O vector size in samples, and fs is the sampling rate in Hz.

Anyway, you can always open the Timing_accuracy.maxpat patcher that is also depicted in the following screenshot, to inspect the timing accuracy with different settings:

The following are important points worth mentioning:

  • CPU utilization: CPU utilization is calculated by taking the amount of time required to calculate a signal vector's worth of audio divided by the amount of real time taken up by the signal vector. So this only takes the MSP portion of your patches into account.
  • Signals used: This simply is the amount of internal buffers needed to send signals between MSP objects.
  • Function calls: This number gives an approximate idea of how many calculations are needed for one sample of audio in the current network.
  • Parallel processing: This enables Max to distribute the audio processing of multiple top-level patchers over multiple CPU cores.

Setting up MIDI

Setting up MIDI is quite straightforward. Go to Options | MIDI Setup and you will be presented with the window shown in the following screenshot:

If you are on a Mac and have no MIDI device connected, it's likely to look exactly like what is shown in the previous screenshot. Max automatically created some virtual MIDI ports for you to be used in other applications that can communicate with Max. On Windows, this doesn't happen automatically, but you can use applications such as loopMIDI to achieve similar setups. I highly recommend that you assign abbreviations for your MIDI devices and use them; beware, though, as Max tends to forget them if it's started without your device being connected. The offset parameter simply allows us to add an offset to the outgoing or incoming MIDI channel. This way, we can identify the incoming MIDI messages sent from different devices just by the channel number, regardless of them being sent on the same channel. This, therefore, is an alternative to the abbreviation/naming method.

Other preferences

So now, let's take a moment to configure Max in order to work with it more efficiently. Go to Max | Preferences on a Mac and Options | Preferences on Windows.

The default preferences are fine for most things, but here are some settings that might come in handy:

  • Debug | Probing | enable: This enables little help bubbles to pop up if you hover over an MSP cord. This again is very helpful for debugging signal processing. We can very easily detect Not a Number (NaN), infinity values, or DC offset at any point in the signal flow.
  • Font | Native Text Rendering | enable: This is particularly helpful on Windows machines. Often, Max's text is rendered in a strange way on Windows, and switching to native text rendering fixes these problems. So only enable this if you encounter problems.
  • Patching | Grid Size: Experiment with this setting over time. We will soon see how we can move around objects via a keyboard, and a good value here can make us a lot faster. I personally use a grid size of 18 x 15.
  • Patching | Show Grid (default for new patchers) | enable: Try this out; it has a tendency to encourage a tidier patching.
  • Patching | Segmented patch cords | enable: This may be a question of taste, so try it out. Instead of having to drag each patch cord, you can just click on an inlet or outlet, route it in a tidy way while connecting, and lead it to its end with another click (cancel the connecting process with command + click on Mac and Ctrl + click on Windows)
  • Scheduler | Refresh Rate: This only applies if the Refresh option under Scheduler is enabled and is Macintosh only. Play around with this setting too. It determines how often Max redraws the GUI. The parameter is in ms, although the clue window tells us it's in Hertz. Anyway, if you concentrate on audio and want to squeeze out more processing power one can easily live with 10 FPS or less, therefore less than 100 ms. If you prefer a very snappy interface, you can even lower the default value of 33 ms. Since your computer screen probably has a refresh rate of 60 Hz, values above 17 ms make sense.

Object defaults

Last but not least, there are the object defaults. Under Options, we find the object defaults. Here, we can define, save, and load color schemes. Of course, you can configure this to your personal preferences, but there is one preference I want to recommend; put the background color of the [multislider] option to an opacity of zero. We often layer [multislider] options for complex signal displays, and therefore, this makes a lot of sense. Also, beware that you can always switch these object defaults and embed them in a saved patcher. Therefore, if you are creating a user interface that should look somewhat coherent and consistent, you might define colors here instead of defining them for each object in your patcher. Beware though that these settings will also affect the already created objects. Therefore, you can globally change your colors when you're done creating a GUI. Finally, to fix the colors, you have to go to the patcher inspector (Shift + command / Ctrl + I) and enable Save Default-Valued Object Attributes to bake your colors. This cannot be reverted, so do a backup beforehand!

File preferences

File references is where we can tell Max where to search for things we want it to find. The folders included in this list are part of the Max search path. It does make a lot of sense to think not only about the file and folder structures for all our Max-related work, but also for the individual projects. We'll talk about this later anyway, but if you already have in mind to create some sort of toolset of your own, which consists of lots of Max patches, you might go ahead and create a folder and add it there. Also, media such as WAVE files and so on are easier to work with if you have them somewhere Max can see them. It's quite typical to add a folder called samples in here where we store some files we regularly work with. If this list of folders gets too crowded over time, or if you decide to add huge sample libraries in here, it can happen that Max starts up quite slow, so beware. We can always load any media file by using its full path, and with the new projects feature in Max 6, we can also have project-specific search paths.

The global search path should mainly point to your abstractions, externals, and frequently used files.

Installing externals

We have mentioned externals already; we typically find them online, install them, and then they behave like Max objects. There are many reasons for using them; they help us a lot if we don't want to reinvent the wheel, if processes are too complex for us and somebody else solved them already, or if their greater efficiency makes a real difference for a certain task. We also saw in Chapter 1, Getting Started with Max, that sometimes, it makes more sense to solve a problem with text programming rather than in Max, and in these cases, we can embed the code in Max as an external.

Installing these externals is simple; we just follow the instructions that (hopefully) come with the package. Now, let's walk-through one installation.

Go to that includes tools used for impulse response creation and convolution. The main reason why we are downloading it at this early stage is that it also contains a very flexible spectrum display that will find great usage throughout this book.

After downloading, we have to extract the files from the ZIP file and locate the actual externals (the .mxo files for Mac or .mxe or .mxe64 for Windows) and their help files (the .maxhelp files). In the case of HIRT, everything is really tidy and we'll just take the corresponding folders and put them into the Max search path. For example, we know from the file menu that the Max 6.1/patches folder is in our search path. We'll just create a folder there that we'll call externals for example, and put everything in there, the externals folder as well as the help files folder we downloaded, that's the whole installation (you can consider using the packages feature discussed later instead).

Some other highly recommended additions (not necessarily externals) to Max are as follows:

  • The Max package downloader
  • The Max toolbox
  • The CNMAT library
  • The BEAP library
  • [shell]/[DOSHack]

You can easily find all of them by doing a quick search on the Cycling '74 forums.

Other setup tips

Now that we have configured Max, we have an idea of how to handle Max files, where to put them, and how everything should be set up. Max and TouchDesigner are often used in audiovisual installations to increase maintainability and stability, and to improve the overall workflow on site. There is a lot we can consider to install or configure to give us and Max/TD a better environment to work with. Here is a small list of things we might consider in this case. You can treat it as a checklist and an overview; some of these are highly recommended to be done, and others relate to personal preference, particularity, or exaggeration. I tried to put them in order of the most important things to rather negligible ones:

  • Make an image of your disk when everything is up and running finally. You never know; maybe one day the PC running your installation suddenly explodes for no reason after you are finally finished.
  • Install a VNC server; remote maintenance saves our lives.
  • Restart On Crash and ReStartMe: These are both little programs that check whether our Max patch just crashed and will restart it, or even restart our system if it hangs.
  • Consider exporting our Max patch(es) as an application(s). We will learn more about this topic in a later chapter.
  • Often, we will want our system to start automatically at a certain time of the day, so don't forget to configure the BIOS that way.
  • Get the [shell] or [DOSHack] external. This way, you can give Max a lot more control over your system and also shut it down in the evening from within Max.
  • Uninstall everything you don't need. Do this before everything else because you might run into problems if you notice that you do need some of those things after all and something stops working. If you do it beforehand, you'll have probably more time to cope with these problems.
  • Deactivate unnecessary services, processes, and applications that start with the system. Keep an eye on automatic software and OS updates. You know those public screens used for advertisements that show a window about software updates being ready to install; we'd like to avoid that. Also, our system should just stay the same; never change a running system, they say; letting itself change automatically over time is like running around with a blindfold.
  • Sometimes, we really need to emulate user interaction (keystrokes, mouse clicks, and so on) if there is additional software in use that isn't scriptable. For these situations, there are externals such as [aka.keyboard], the Autobot Java class for Max (as mxj), and applications such as Autolt and AutoHotkey.
  • Install bginfo/geektool/bginfo4x or similar software. It allows the IP address (and other information) of our system to be shown on the desktop wallpaper. This is incredibly handy with multi-machine setups.
  • Install SoX and ffmpeg. These are both command-line conversion and editing tools that can do things really quickly. If you suddenly find that all the hundred samples you use in your installation should be EQed and a little more compressed to meet on-site room acoustics and so on and there is no CPU overhead to do it at runtime, you can batch-preprocess them with SoX. Finally, if you need a different video format, you will love the ffmpeg video format.
  • Install Wireshark. We often deal with information transmitted via networks. Wireshark is a great debugging tool for these things.
  • This is some useful hardware advice; for bigger jobs or in environments where you can't know whether somebody has pulled the power plug out of your system (so nearly everywhere!), you might consider getting a battery backup/uninterruptible power supply.
  • Having a good text editor at hand will speed up things a lot. We can also edit Max patches with text editors, for example, to search for and replace a lot of objects. Sublime Text and Notepad++ are very powerful text editors for instance.
  • Last but not least, sometimes, we have to deal a lot with moving files around and so on. There are a lot of alternatives to Windows Explorer and Apple's Finder that can help us do things a lot faster. muCommander is one of those.

Organizing finished code

When installing externals or other additional material, you might stumble upon one of these concepts. However, they are all very useful even if we just coded some small Max patch ourselves that we plan to use regularly, so let's see what they are exactly.

Abstractions

Abstractions will become increasingly important for us over the time we work with Max. Suppose we want an object to square an incoming number. Now, there is [pow], but this object is capable of doing more (taking the nth power) than just squaring, and we just want to square the input and nothing more. We'll need that a lot and having an object that just does this might be handy.

Alright, so we'll make a new patcher (jump ahead to basic patching if you don't feel ready and revisit this section afterwards). We'll create an [inlet] object, a [* 0.] object (don't be too confused about the zero now; we'll learn about this in arguments and attributes), and an [outlet] object. Inlet and outlet magically transform into the boxes you see on the top and bottom of the screenshot after writing, for example, inlet in an empty object box and hitting Enter. So we'll arrive here:

Now, we'll save this patcher in a folder within the Max search path; ideally, we have/create one called abstractions (it's really up to you how you organize your files and folder structures, but you might consider taking the advantage of the packages system we'll go through in this chapter) and save it there with the name square.maxpat. We close that patcher, forget about it, and make a new patcher as follows:

You can get these float boxes by just hitting the f key, or by typing flonum into an empty object box. Since Max is aware of the fact that there is a patcher called square somewhere in its search path, it will automatically insert it for us. If you double-click on [square], you can actually see the contents of it, finding our previously created abstraction.

Tip

In order to prevent ambiguity, try to make sure there is no object or abstraction that already has the name you choose for a new abstraction. Just try it out before you save; make a new patcher and try creating an instance of your new abstraction. If you get an error, you are fine; if an object or abstraction gets loaded, you should make up a different name!

We said we shall forget about it; this is one advantage of abstraction. We can hide complexity we don't care about. You can easily imagine that if our abstraction contains more than one object, we are making things a lot tidier using abstraction. There is an easier way (encapsulation/subpatchers/bpatchers) of doing this, which we will learn in the section about advanced patching, but abstraction also has another advantage; assume we use our new square abstraction in multiple patches many times. Then, it strikes us that there is an error in the abstraction all over the place. However, we don't have to (in fact, can't) correct the mistake a hundred times if we use abstractions. We just have to correct our mistake in the original file and all instances get updated!

So you see, making lots of abstractions might seem like an advanced topic, but for beginners and intermediates working with Max and fighting with bugs, it's especially useful because we can reduce the time we spend on debugging.

So the takeaway message is start using abstractions early, use them a lot, and build up a library of things you need often.

One more thing about abstractions: we should follow all the guidelines of tidy patching (and so on) that apply for normal patching; we'll go through them later on. But one of them is particularly recommended for abstractions; remember the inlets and outlets we created? And remember that we do want to forget what's in there? So what about an abstraction with multiple inlets and outlets, or just one that's a bit more complex? If we don't want to look inside what's actually in the abstractions, we should give ourselves some hints like this:

The hints appears when we hover over the input of our abstraction; my cursor just vanished while taking the screenshot. It's what I typed into the comment field in the inspector of the [inlet] object (select the inlet and press command/Ctrl + I). This may be a bit overdone in this case, but it's a very good practice to state all of these in this place:

  • What parameter do I control when I send something in the specific inlet? Take a filter, for example; is this the frequency or resonance?
  • What datatypes are accepted? Int, Float, Signal, Strings, Lists, and so on? What range makes sense? What are the units? For example, a delay might be controlled in samples, milliseconds, seconds, or even Hertz. So, which one is it? Or if we control the amplitude with this inlet, is it dB? Is it linear? Is it linear from 0 to 1 or did we decide it should be from 0 to 127 to stick to MIDI conventions?

This will make our abstractions a lot more usable because obviously if we don't do it, we will be forced to go through the hidden code again to see what all these properties could be. Of course, all this is true for outlets as well.

Extras

You might have noticed the Extras menu in Max. We can find a lot of helpful things there, and they are all patchers. The nice thing is you can add your own patchers there. It's quite simple; just save your patcher in Max 6.1/patchers/extras, restart Max, and it should be there. This can be quite handy; for example, you could put your patch that we used to measure our system's timing accuracy there in order to be able to tweak performance slightly. Also, CPU and RAM measurements are a good thing to have there, and we'll go through how to measure these in a more advanced chapter later.

Note

Exercise

You may often need to just send the input of your audio interface out to your speakers. You may need a certain MIDI connection between a keyboard and outboard equipment all the time. Try to think of something useful that will make your life easier when you get it at hand in the Extras menu and place it there!

Clippings

Clippings are great. Essentially, they are also just patchers we can save somewhere and access in a specific way, but this is something special. We'll create one right away that you'll need a lot and make the development of audio effects and processors a lot more convenient.

Make a new patcher. Create a new [degrade] object. We are going to steal something from its help file and afterwards delete the object again. I just happen to know that degrade is one of the many objects whose help file we want. Press Alt + click on [degrade] to get its help patcher. Here you'll find this:

Unlock the help patch (command/Ctrl + e) and copy this little field (it's called bpatcher, and we'll also learn about these soon). Paste this into our patcher. Delete the [degrade] object as we don't need it anymore. Now, we'll save our patcher under Max 6.1/patchers/clippings/AudioTestInput.maxpat. Restart Max. Make a new patcher, and while it's unlocked, right-click on an empty area of your patcher:

In the context menu, you can find our patch; choose it and you'll see that it's pasted into our new patch. The neat thing about this really is this bpatcher has output (and can have input), so we can actually build small modules that can be interconnected and have their own GUI. This little patch will serve us well since it provides a variety of test signals that will come in handy when we'll be designing signal-processing algorithms.

Although we now created a clipping out of an existing bpatcher object, we can make our own also, and that way build up, for example, modular synth-like environments very conveniently. You can take a look at the VIZZIE clippings that come with Max and are suited for simple video processing, or try out the BEAP library (the BEAP library can be downloaded at https://github.com/stretta/BEAP/wiki/BEAP-Modular---Overview-and-Install), which works in a similar way with the aim of replicating a modular synth-style workflow.

Tip

Consider deleting some of the preinstalled clippings to make your Paste from menu cleaner. It's about accessing things quickly right? So we don't want to have all kinds of stuff lying around that we don't need. Some of the ones that come with Max are really not very useful in the long run. Have a look through them and see which ones make sense for you. In this occasion, you might notice that some menu items are somewhat mysteriously just there, although there are no according files in the clippings folder this has to do with packages.

Packages

So now, we've seen externals, abstractions, extras, and clippings that moved to special places in our filesystem, and after a year, we really lost track of everything. We probably have two or three different abstraction folders by now; we've had multiple updates of Max each time the standard clippings get restored that we delete each time because we want to keep our clippings menu clean. Also, at a certain point, you'd like to share a patch that contains all sorts of references to files and patches on your system. All this sounds pretty messy and confusing right? Here come packages to our rescue!

Go to Max 6.1/packages and take a look inside the Vizzie folder there. We find some folders called extras, clippings, and patchers, and some other stuff. These folders just work the same way as the folders in the Max 6.1/ directory. So if we create a folder called My first project, my general stuff, or something similar, inside the packages folder and create these subfolders, we can use these instead of having our files spread out over the system. Within the packages folder, you'll also find an about packages.txt file, which describes what types of folders are supported and how to use them.

Projects

Projects are a feature that allows us both to keep track of all the dependencies a patcher might have and also to easily pack those together. We can view it as the dependant of the packages feature. If we plan a complex patcher, we might save it as a project, consolidate everything (media, abstractions, code, and so on) automatically, and open the whole thing on another computer without worrying about dependencies.

Prototypes

Prototypes are saved versions of a specific object. Most of its attributes are saved and we can load them at any time. Take, for example, a [led] object; if we create it, we can right-click on it and select prototypes and can choose different styles. Since we can also save our own, and many objects have many attributes that define their overall behavior, we might consider building up our own library of prototypes that show up more complex configurations than just a single color. Also, for a consistent GUI design, this comes in handy.