Final Assignment Requirements for Hand-In

Please prepare the following things to present and hand in on the final day of class.

  • Your processing code folder is to be clearly labelled with your name and date and zipped up, so it should be something like In this folder there is to be all files necessary to run your project. At the top of the .pde file, in comments, you are to list the links to any example code you used. Referencing here is very important. Your code is also to be clearly commented throughout, with comments saying what each block does. It is also to be properly formatted and laid out (remember formatting short-cuts are command-t or cntrl-t). Make sure this is emailed or submitted to me.

  • Prepare and publish a blog post describing (a) Your concept, questions, and aims for the project (b) Artists, artworks and research influencing this project (c) Process documentation eg. sketches, screenshots, photos etc. (d) Details of the technical workflow of your project (e) Screenshot or image of final finished product. (f) A list of references used.

This post will be presented in class with your work.


Helpful resources for your assignment development

Of course you can also not use a library and use the examples in the book and in the examples section of the processing menu. Either way is fine.

Thanksgiving homework

  • Listen to this podcast for homework too its about talking with machines:
  • Please prepare the pseudo code for your projects. I will be checking the you have done this at the start class. This should be small steps describing what happens at each stage in your project. Start small sketches to begin working out each of the steps. (You will combine them later). Also review relevant sections of the book and any libraries you will need. You should work on this this week and use the tutorial time as trouble shooting opportunities.

Text #2

Now that you have the hang of using strings, chars and arrays to manage text – what else can you do with these tools?

How can you make the computer speak?

If possible use a mac, as it is slightly easier. But for those who are using PCs you can use the Speech to Text library here. This uses the Google Speech API – you have to register for keys.

For mac users, you can use the great text to speech program on your computer. You saw this last week when we opened the terminal. To revise, open up the terminal and run through this guide on the mac text to speech engine:

We can send terminal commands from Processing to the terminal to make use of engines like these. Lucky for us, there is an easy to use class that Erica Mika has written. Details here. So take this code for a text to speech class and add it either below draw or in a new tab.

// the text to speech class
static class TextToSpeech extends Object {
  // Store the voices, makes for nice auto-complete in Eclipse
  // male voices
  static final String ALEX = "Alex";
  static final String BRUCE = "Bruce";
  static final String FRED = "Fred";
  static final String JUNIOR = "Junior";
  static final String RALPH = "Ralph";
  // female voices
  static final String AGNES = "Agnes";
  static final String KATHY = "Kathy";
  static final String PRINCESS = "Princess";
  static final String VICKI = "Vicki";
  static final String VICTORIA = "Victoria";
  // novelty voices
  static final String ALBERT = "Albert";
  static final String BAD_NEWS = "Bad News";
  static final String BAHH = "Bahh";
  static final String BELLS = "Bells";
  static final String BOING = "Boing";
  static final String BUBBLES = "Bubbles";
  static final String CELLOS = "Cellos";
  static final String DERANGED = "Deranged";
  static final String GOOD_NEWS = "Good News";
  static final String HYSTERICAL = "Hysterical";
  static final String PIPE_ORGAN = "Pipe Organ";
  static final String TRINOIDS = "Trinoids";
  static final String WHISPER = "Whisper";
  static final String ZARVOX = "Zarvox";
  // throw them in an array so we can iterate over them / pick at random
  static String[] voices = {
  // this sends the "say" command to the terminal with the appropriate args
  static void say(String script, String voice, int speed) {
    try {
      Runtime.getRuntime().exec(new String[] {"say", "-v", voice, "[[rate " + speed + "]]" + script});
    catch (IOException e) {
  // Overload the say method so we can call it with fewer arguments and basic defaults
  static void say(String script) {
    // 200 seems like a resonable default speed
    say(script, ALEX, 200);

This class basically sets up our sketch so we can use the function

  TextToSpeech.say(script, TextToSpeech.voices[voiceIndex], voiceSpeed);

The arguments are as follows: ‘script’ is the string that you want the computer to say, the voice index is a number that ranges from (0-23) and calls the different voices in the computer and the voiceSpeed sets voice speed.
She has a simple example available here.

To introduce voice to speech to the last text example, set up variables for voiceIndex and voiceSpeed. And add this function that will speak your answer (the string phrase) when you click.

void mousePressed() {
 TextToSpeech.say(phrase, TextToSpeech.voices[voiceIndex], voiceSpeed);

How can you vary the voices and the voiceSpeed with mouse position?
How can you get the computer to say when you are right or wrong?

Importing text from an external file
For this sketch to work you will need to create a text file in textEdit and put it into your sketch folder, inside a data folder.

When you import text files you import them into an array of Strings with each line of text as a new entry in the array.

String [] textIn; 
void setup(){
  size(200, 200);
  textIn = loadStrings("textFile.txt");

void draw(){
  text(textIn [0], 20, 100); 

This sketch would allow someone who has no programming ability at all write content for your program.

String [] textIn;
String phrase; 
int phraseNum; 
int fade = 255; 
PFont font; 
void setup() {
  size(800, 160);
  textIn = loadStrings("textFile.txt");
  font = createFont("Handwriting-Dakota", 60); 
  phrase = textIn[0];
void draw() {
  if (fade==0) {
    fade = 255; 
    phraseNum ++; 
    if (phraseNum == textIn.length) {
      phraseNum = 0;
  phrase = textIn[phraseNum];
  fill(200, fade); 
  if (phraseNum==textIn.length-1){
    fill(255, 100, 20, fade);
  else {

Here is an example with text to speech included too!

Exercise: Write a program that allows you to rap with your computer over some triggered beats – choose one of the files in this zip. You will need to use the minim library to play these files, like in the PlayFile example included in Processing.

Text #1 – Tutorial

A String (notice the capital “S”) is another variable type that holds type.
String theText = “hello Tega”;
would put “hello Tega” into the variable theText.

PFont font;
String theText;
void setup(){
size(400, 200);
theText = "cheese is good";
font = createFont("Helvetica", 24);
void draw(){
text(theText, 20, 90);

A char is a single character, and you can think of a String as an array of chars. You use the term charAt() to get a char from a string. The line
char letter = theText.charAt(2);
in the code above this line would set the variable letter to “e” (don’t forget arrays start at 0!)
You can add to a string with the + symbol, but you cannot compare strings with the = sign, you have to use “equals”

Strings can also go into arrays.

PFont font; 
String [] textList = new String [5]; 
int wordNum = 0; 
int wordX = 800; 
void setup() { 
  size(800, 100); 
  textList [0] = "stuff"; 
  textList [1] = "things"; 
  textList [2] = "objects"; 
  textList [3] = "junk"; 
  textList [4] = "detritus"; 

  font = createFont("Helvetica", 24);
void draw() {
  wordX -= 10; 
  if (wordX<-20) {
    wordX = width+20;
    fill(random(255), random(255), random(255));
  if (wordNum == 5) {
    wordNum = 0;
  text(textList[wordNum], wordX, height/2);

This sketch has a string called phrase that is catching keystrokes. Notice in this code we actualy set the phrase to “” which is an empty string. For some reason it won’t work if we query a string that has nothing in it.

PFont font; 
String [] textList = new String [5]; 
String phrase = ""; 
int wordNum = 0; 
int wordX = 800; 
boolean checkPhrase; 

void setup() { 
  size(500, 200); 
  textList [0] = "stuff"; 
  textList [1] = "things"; 
  textList [2] = "objects"; 
  textList [3] = "junk"; 
  textList [4] = "detritus"; 

  font = createFont("Helvetica", 24);
void draw() {
  text("what is in the drawer?", 30, 30); 
  text(phrase, 30, 60); 
  checkPhrase = false; 
  for (int i = 0; i < textList.length; i++) {
    if (phrase.equals(textList[i]) == true) { //equals() compares two string variables to see if they are the same
 // see here
      checkPhrase = true;
  if (checkPhrase == true) {
    text("you're right!", 200, 140);
  else {
    text("you're not even close!", 200, 140);

void keyReleased() {
  phrase += key;

This works well, but we can’t delete? Start by printing out the keyCode in the keyReleased area – this will get the code for return too. You’d think that deleting the string by one character would be easy, but it’s not. Here’s how you do it.

if (keyCode == 8 && phrase.length() > 0) { 
phrase = phrase.substring(0, (phrase.length() -1));

The first line says: “if the keyCode for the delete key is pressed and the String isn’t at 0 already, then …”
The second line says: “take your string and make another string that happens to be one character shorter than the current string, oh and name the new string the same thing as the old string.”

Get a zip file of these examples here.

Using additional hardware (like cameras, game controllers or kinects)

You may wish to use some extra hardware for your final project. This would typically require you to use an external library and/or install a driver. Here are a couple of low cost options:

Great for if you want to locate your camera somewhere strange – away from the computer.

A cheap robust webcam – playstation eye –

A USB game controller –

To use a game controller, I recommend using this library: 

You could also experiment with a kinect if you own one. See some kinect libraries here:

P5.js – Processing in your browser

Welcome to processing in the browser with…  p5.js

– Watch the introduction here:

– Download the p5.js master –

– Move this folder to your processing folder or to your documents folder. But make sure you know where it is…

– Review the tutorial for starting out here:

– There are a few key differences between straight processing and  using p5.js – take a look at the  transition guide here.

Project Proposal (Due Week 11 – 10% of homework mark)

Decide what inputs your system will use and how your system will respond to them. In your proposal presentation (due in class week 11) you will address the following:

  • Identify two interactive or performance artworks that have influenced you and describe why. Provide references to these works. What is the artist exploring in these examples? What strategies have they used to create the interaction? How does the work script the participants behaviour? How have these examples influenced your thinking, why have you chosen them?

  • In your proposed project, what inputs are required by your system? How will you collect these? (using a mouse, a camera, a microphone, the keyboard, a joystick?). Are you going to purchase any additional hardware to use in your installation? See links to hardware – joysticks and cameras on the blog.

  • How might your system respond to the inputs you choose? Show this by drawing a series of sketches.

  • How could your project to be installed? Is it a screen work? A floor projection? To be viewed in a browser online? Confined to a box? On a small phone? A large wall projection? Using a hidden camera or microphone embedded in an object?

  • Think about how the housing and presentation of this work will impact how the participant encounters it.

  • Identify any libraries you might need.

Your proposal presentation must be 3-5 minutes long and consist of at least 3 slides outlining the above.

Final project must be ready for presentation at 4pm in class in Week 14.

November 18th Examples

Download zip file below and see examples.

  1. Color Tracking
    1) Exercise 16-5: Take any Processing sketch you previously created that involves mouse
    interaction and replace the mouse with color tracking. Create an environment for the
    camera that is simple and high contrast. For example, point the camera at a black tabletop
    with a small white object. Control your sketch with the object’s, location. The picture shown
    illustrates the example that controls the “ snake ” (Example 9-9) with a tracked bottlecap.

  2. Frame Differencing
    2) Use frame differencing to create an interactive game, drawing tool or instrument.


This week’s examples