Sound sprites in Processing.js with Howler.js

30 Dec, 2013 Blog Tutorials AlcwynParker

Download Project Files:

Tutorial Required Skills:

Basic level of Java/Processing.
Basic audio editing skills (Adobe Auditions).
A tiny bit of JavaScript.

Introduction:

Sound spriting is a technique very similar to css spriting for images. The process involves compiling all the sounds you intend to use in your processing sketch into one file and then using code to play different sections of that file when necessary. We will be using the JavaScript library Howler.js in this tutorial, which is capable of sound spriting out of the box so the process is very, very simple. As this library is written in JavaScript, you will have to install and activate JavaScript mode in the Processing IDE for this tutorial to work.

The Stages Of This Tutorial:

Prepare the individual audio files;
Consolidate the separate files into one;
Make a note of the start time and duration of each separate audio sample;
Export the file to OGG and MP3;
Import the Howler.js code into processing;
Write some JavaScript to control the playback of the sounds.

Let’s Begin…

In this tutorial we will be using Adobe Auditions to create the consolidated sound files. Obviously, any good audio editing software will do but Auditions has some handy features to fast track the process. I will presume that you have already prepped the individual sound files: all mastered and ready to consolidate.

Open up Auditions and select “Open Append” from the File menu.

Select all of the audio files you want to consolidate. In this example there are only two samples but you can select as many as you need. Click “Open” and Auditions will automatically append the files together into one track and then create markers with all the information we need to implement the code later.

Next, locate the Markers window and make a note of the name of the track, start time and duration of each marker. This information is vital when we come to write the code for Howler.js. (Usually, I just screen shot the markers panel as a permanent record).

Finally, export the file out for use in our sketch. Due to the nature of web audio, we will need to export the file out twice: once as a .OGG file and then an .MP3.

Make sure that you know where you saved the file in your file system so that you can find it later to drag into your Processing sketch.

The Code:

Make sure that you have the Processing IDE in JavaScript mode. The sketch for this example consists of 4 tabs: the main sketch tab, a player object tab, the howler.js tab and a script.js tab.

Here’s how it should look (don’t worry about the code for the tabs yet!):

Let’s take a look at the example code without any sound first. Most of the code is explained within the comments of the code except for where the steps involve actions that are beyond the scope of the code, such as adding files or importing howler.js.

The main sketch tab:

The main sketch sets the stage and defines an object of the player class.

// define an object of the Player class
Player player1;

// set the stage
void setup(){
     size(800,600);
     background(0);

     // instantiate the player1 object.
     player1 = new Player();
}

void draw(){
  background(0); 

  // animate the player1 object
  player1.step();
}

void keyPressed(){
  // alert the player object to the user input
  player1.userInput();
}

// define an object of the Player class

Player player1;

// set the stage

void setup(){

size(800,600);

background(0);

// instantiate the player1 object.

player1 = new Player();

}

void draw(){

background(0);

// animate the player1 object

player1.step();

}

void keyPressed(){

// alert the player object to the user input

player1.userInput();

}

Player Class Tab:

Sets up a basic character that is capable of moving around the stage but if it exceeds the boundaries, it will bounce off the side. At the moment, the Player class does not use any sound but later on this class is where we will trigger the sound from.

class Player {

  // INSTANCE VARIABLES
  PVector position;
  PVector velocity;
  PVector acceleration;
  PImage robotnik;

  // CONSTRUCTOR
  Player() {

    // instantiate the instance variables
    position = new PVector (width /2, height /2);
    velocity = new PVector ( 0, 0);
    acceleration = new PVector (0, 0);
    robotnik = loadImage("robotnik.png" );
  }

  // calls the methods through one method to save code in the main sketch
  void step(){
    update();
    checkBoundaries();
    display();
  }

  // draw player to the screen
  void display() {
    image( robotnik, position.x, position.y, 60, 60 );
  }

  // calculate the vectors to animate the player
  void update() {
    velocity.add(acceleration);  // the usual chain of motion - add acceleration to velocity, add velocity to position.
    position.add(velocity);
    velocity.limit(5);           // stop the velocity from getting out of control

    acceleration.mult(0);        // turn off the acceleration
  }

  // make sure the player never leaves the stage
  void checkBoundaries() {
    if (position.x > width - 60) velocity.x = velocity.x * -1;
    if (position.x < 0) velocity.x = velocity.x * -1;
    if (position.y > height -60) velocity.y = velocity.y * -1;
    if (position.y < 0) velocity.y = velocity.y * -1;
  }

  // handle user input
  void userInput() {
    if (keyCode == UP ) {
      acceleration.y = acceleration.y + -.5;
    }
    if (keyCode == DOWN ) {
      acceleration.y = acceleration.y + .5;
    }
    if (keyCode == LEFT ) {
      acceleration.x= acceleration.x + -.5;
    }
    if (keyCode == RIGHT) {
      acceleration.x = acceleration.x + .5;
    }
  }
}

class Player {

// INSTANCE VARIABLES

PVector position;

PVector velocity;

PVector acceleration;

PImage robotnik;

// CONSTRUCTOR

Player() {

// instantiate the instance variables

position = new PVector (width /2, height /2);

velocity = new PVector ( 0, 0);

acceleration = new PVector (0, 0);

robotnik = loadImage("robotnik.png" );

}

// calls the methods through one method to save code in the main sketch

void step(){

update();

checkBoundaries();

display();

}

// draw player to the screen

void display() {

image( robotnik, position.x, position.y, 60, 60 );

}

// calculate the vectors to animate the player

void update() {

velocity.add(acceleration); // the usual chain of motion - add acceleration to velocity, add velocity to position.

position.add(velocity);

velocity.limit(5); // stop the velocity from getting out of control

acceleration.mult(0); // turn off the acceleration

}

// make sure the player never leaves the stage

void checkBoundaries() {

if (position.x > width - 60) velocity.x = velocity.x * -1;

if (position.x < 0) velocity.x = velocity.x * -1;

if (position.y > height -60) velocity.y = velocity.y * -1;

if (position.y < 0) velocity.y = velocity.y * -1;

}

// handle user input

void userInput() {

if (keyCode == UP ) {

acceleration.y = acceleration.y + -.5;

}

if (keyCode == DOWN ) {

acceleration.y = acceleration.y + .5;

}

if (keyCode == LEFT ) {

acceleration.x= acceleration.x + -.5;

}

if (keyCode == RIGHT) {

acceleration.x = acceleration.x + .5;

}

Adding Sound:

Before we go any further, let’s add the two sound files that we exported earlier into our sketch. The best way to add any file to your sketch by dragging and dropping it straight on top of the Processing IDE window.

OK, so that’s the foundation work done but now we want to add the functionality to play sound. For this example, we are going to trigger the sound in two places: first when the user applies acceleration to the player and secondly when the player bounces off the sides. At this stage, it is vital that we acknowledge the tensions of using JavaScript mode in Processing; we are inevitably coding in two separate languages. Luckily, with the help of the processing.js library, the duplex of syntax plays fairly well together.

Whenever you add JavaScript to your Processing sketch you must create tab names that end in “.js”, that way the IDE will know how to handle them. You can include pretty much any JavaScript library you want and there are a lot out there.

Including the howler.js library into the Processing IDE:

If you have not already, create a new tab called howler.js and then head over to the howler.js Github raw code page and copy the code:

Howler.js rawcode

Then paste that code into the howler.js tab.

Utilising howler.js Library:

Now that we have added the JavaScript library, we need to write some JavaScript to utilise the library. For more information on writing Javascript for howler.js visit here.

If you have not already, create a new tab called script.js. This is where we will write any custom JavaScript we need to add to our sketch. Copy and paste the code below into this tab. Hopefully the code is commented enough that it should make sense.

// define an object of Howl class with all the properties of the expoerted audio
var sound = new Howl({
  // location of the files
  urls: ['samples.mp3', 'samples.ogg'],
  // information for each sample within the consolidated sound files 
  sprite: {
    // values of each sample
    // syntax:
    // name: [start time, duration], 
      laser: [0 , 374],
     rocket: [374, 1399]
    //sound3: [2804, 2321]   <--- add more sounds like this
  }
});

// a javascript function that plays any of the sounds defined above
function playSound(name){

  sound.play(name);

}

// define an object of Howl class with all the properties of the expoerted audio

var sound = new Howl({

// location of the files

urls: ['samples.mp3', 'samples.ogg'],

// information for each sample within the consolidated sound files

sprite: {

// values of each sample

// syntax:

// name: [start time, duration],

laser: [0 , 374],

rocket: [374, 1399]

//sound3: [2804, 2321] <--- add more sounds like this

}

});

// a javascript function that plays any of the sounds defined above

function playSound(name){

sound.play(name);

}

Finally, all that’s left to do is call the JavaScript function from within the Player class tab. Wherever you want to trigger the sound, you just have to write one line of code:

playSound("[name of sounds]");

1	playSound("[name of sounds]");

These means in order the complete the Play class, we need to change the checkBoundaries method to play a sound every time the player hits the side like this:

// make sure the player never leaves the stage
  void checkBoundaries() {
    if (position.x > width - 60){
      velocity.x = velocity.x * -1;
      playSound("laser");
    }
    if (position.x < 0) {
      velocity.x = velocity.x * -1;
      playSound("laser");
    }
    if (position.y > height -60) {
      velocity.y = velocity.y * -1;
      playSound("laser");
    }
    if (position.y < 0) {
      velocity.y = velocity.y * -1;
      playSound("laser");
    }
  }

// make sure the player never leaves the stage

void checkBoundaries() {

if (position.x > width - 60){

velocity.x = velocity.x * -1;

playSound("laser");

}

if (position.x < 0) {

velocity.x = velocity.x * -1;

playSound("laser");

}

if (position.y > height -60) {

velocity.y = velocity.y * -1;

playSound("laser");

}

if (position.y < 0) {

velocity.y = velocity.y * -1;

playSound("laser");

}

We could also change the player class so that when there is user input we play the rocket sound:

// handle user input
  void userInput() {
    if (keyCode == UP ) {
      acceleration.y = acceleration.y + -.5;
      playSound("rocket");
    }
    if (keyCode == DOWN ) {
      acceleration.y = acceleration.y + .5;
      playSound("rocket");
    }
    if (keyCode == LEFT ) {
      acceleration.x= acceleration.x + -.5;
      playSound("rocket");
    }
    if (keyCode == RIGHT) {
      acceleration.x = acceleration.x + .5;
      playSound("rocket");
    }
  }

// handle user input

void userInput() {

if (keyCode == UP ) {

acceleration.y = acceleration.y + -.5;

playSound("rocket");

}

if (keyCode == DOWN ) {

acceleration.y = acceleration.y + .5;

playSound("rocket");

}

if (keyCode == LEFT ) {

acceleration.x= acceleration.x + -.5;

playSound("rocket");

}

if (keyCode == RIGHT) {

acceleration.x = acceleration.x + .5;

playSound("rocket");

}

And that’s it!

Now you made it through the tutorial you deserve a present… here’s a video:

Blog

Sound sprites in Processing.js with Howler.js

Recent Posts