Sending OSC messages from SC3 to Processing

SuperCollider has OSC support in its core, as it is based in a client-server model for communicating between sclang (client-side) and scsyth (server-side). Processing supports OSC communication via the oscP5 library which is one of the standard libraries that Processing is shipped.

I found a bit tricky to send messages from SC3 to Processing. I was looking the oscP5 methods checkTypetag and getTypetagAsBytes though the problem is that the incoming data in Processing are in a hexadecimal which I was not able to decode using unhex function. I found in sccode a script by Fredrik Olofsson which implements communication from scsynth output to processing via sclang.

The script by Fredrik Olofsson is a generalization of the scripts below. I modified the Processing script in order to be able to send one-by-one float numbers from SC3.

s.boot;

n = NetAddr("127.0.0.1", 47120);  // open 47120 on localhost server

(
SynthDef(\blip, { | freq = 440, amp = 0.85, att = 0.01, rel = 0.06, ffreq = 1000 |
    var sig, env, lfo;
    sig = SinOsc.ar(freq, 0, amp);
    env = EnvGen.ar(Env.perc(att, rel), doneAction:2);
    lfo = SinOsc.kr(rel * ffreq);

    Out.ar(0, Pan2.ar(RHPF.ar(sig*env, ffreq), SinOsc.kr(211*lfo)))
}).add;
)

Synth(\blip);

(
f = fork {
    loop {
        256 do: { |i|
            n.sendMsg("/sc3p5", i.asFloat); // send OSC message to P5
            Synth(\blip, [\freq, 440+i, \ffreq, 1000+i*2]);
            ((i+1).reciprocal*2).wait;
        }
    }
};
)

f.stop;

Processing script.

import oscP5.*;
import netP5.*;
OscP5 oscP5;

float x; // global variable

void setup() {
  size(400, 300);
  frameRate(24);
  background(0);
  smooth();

  OscProperties properties = new OscProperties();
  properties.setListeningPort(47120); // osc receive port (from sc)
  oscP5 = new OscP5(this, properties);
}

void oscEvent(OscMessage msg) {
  if (msg.checkAddrPattern("/sc3p5")) {
    x = msg.get(0).floatValue(); // receive floats from sc
  }
}

void draw() {
  background(x, x, x);
  println("POST: ", x);
  // draw rect
  stroke(256-x/2, 256-x*abs(sin(x)), 256-x/4);
  strokeWeight(4);
  fill(256-x/2, 256-x, 256-x*abs(sin(x)));
  translate(width/2, height/2);
  rotate(x%64);
  rect(x%64, x%64, x*abs(sin(x))%128, x*abs(sin(x))%128, 6);
}

The output of these two programs looks like the video below.

How to output your sketch to video format

In order to produce a video recording of my sketch in Processing I used the function saveFrame(). Instead of using Processing's Movie Maker I used the command line ffmpeg video converter. I had troubles to match the lengths between the audio recording from SC3 and the video. For that purpose, I counted the image files that were produced using saveFrame() function and I counted the samples of the audio recording in SC3 using Buffer class and numFrames method. The final formula was like this: (NUM_OF_SAMPLES / SAMPLING_RATE) / NUM_OF_FRAMES. Instead of 24fps that I used, my script in Processing produced frames approximately at 16fps.

In order to make an mp4 from png frames and embed the audio recording I used the following commands from ffmpeg wiki:

Produce mp4 from png
# -framerate is the processing's output rate, -r 24 is the fps of the out.mp4
ffmpeg -framerate 16 -i img%03d.png -c:v libx264 -r 24 -pix_fmt yuv420p video.mp4
Convert aiff to mp3
ffmpeg -i audio.aif -b:a 320000 audio.mp3 # 320000 is the bit rate of the mp3 (320k)
Merge audio (mp3) and video (mp4) files
ffmpeg -i video.mp4 -i audio.mp3 -map 0:0 -map 1:0 -vcodec copy -acodec copy output.mp4

Latest posts