9b – Working with video

Key concepts: working with video – Processing movie class!

For a summary of the movie class in the Processing video library.. here is Shiffman:

Here are some video files:
lights2
frisbee golf pro

Processing can load and manipulate video like it can with PImages. To do so you need to use either the video or the capture class. The references for these classes are here:  https://processing.org/reference/libraries/video/

import processing.video.*;
Movie movie;
 
void setup() {
  size(640, 360);
  background(0);
  movie = new Movie(this, "mom.mov");
  movie.play(); // play until the end
  // movie.loop(); // does the same thing as play but loops the video
}
 
void draw() {
  if (movie.available() == true) {
    movie.read();
  }
  image(movie, 0, 0, width, height);
}

This next example pauses the video when you click on the screen.

import processing.video.*;
Movie movie;
boolean playPause = true; 
void setup() {
  size(640, 520);
  background(0);
  movie = new Movie(this, "lights2.mov");
}
 
void draw() {
  if (playPause == true) {
    movie.loop(); // play until the end
  }
  else {
    movie.pause(); // play until the end
  }
  if (movie.available() == true) {
    movie.read();
  }
  image(movie, 0, 0, movie.width, movie.height);
}
 
void mouseReleased() {
  playPause = !playPause;
}

Exercise:
Create a video player with 4 buttons,

  • pause
  • jump to a point
  • double speed
  • swap video

You can do this by looking at video examples, as well as the video library API.

When we draw the video on to the screen we’re actually using image(). Yup – same code that we use for PImages! This means that each frame of video is being drawn one at a time, which means we can mess with the pixel array, just like we did with images.

import processing.video.*;
Movie movie;
boolean flip;
PImage destination; 
void setup() {
  size(640, 360);
  background(0);
  movie = new Movie(this, "lights2.mov");
  destination = createImage(640, 360, RGB);
  movie.loop();
}
 
void draw() {
  if (movie.available() == true) {
    movie.read();
  }
  movie.loadPixels(); //load the frame into an array
  destination.loadPixels(); // load destination into an array
  for (int i = 0; i < movie.width; i++) { // For each pixel in the video frame...
    for (int j = 0; j < movie.height; j++) { // For each pixel in the video frame...
      color currColor = movie.get(i, j); // get current color
      color newcolor = color (green(currColor), blue(currColor), red(currColor));  // swap colors
      if (flip == true) { 
        destination.set(movie.width - i, j, newcolor); // put the new color into destination
      }
      else {
        destination.set(i, j, currColor); // put the old colors into destination
      }
    }
  }
  destination.updatePixels(); 
  image(destination, 0, 0); //put destination array on the screen
}
void mouseReleased() {
  flip = !flip;
}

You can “glitch” video as well.

import processing.video.*;
Movie movie;
PImage destination; 
int glitchStart, glitchEnd, glitchShift; 
void setup() {
  // size( 1920,  1080); 
  size(640, 360);
  background(0);
  movie = new Movie(this, "mom.mov");
  movie.loop(); // does the same thing as play but loops the video 
  destination = createImage(640, 360, RGB);
}
 
void draw() {
  if (movie.available() == true) {
    movie.read();
  }
  movie.loadPixels(); //load the frame into an array
  if (random(100) > 70) {
    glitchStart = int(random(0, movie.height - 100)); 
    glitchEnd = glitchStart + int(random(20, 300)); 
    glitchShift = int(random(-50, 50));
  }
  for (int i = 0; i < movie.width; i++) { // For each pixel in the video frame...
    for (int j = 0; j < movie.height; j++) { // For each pixel in the video frame...
      color currColor = movie.get(i, j); // get current color
      color newcolor = color (green(currColor + glitchShift), blue(currColor- glitchShift), red(currColor + glitchShift));  // swap colors
      if (j > glitchStart &&  j < glitchEnd) {
        destination.set(i+ glitchShift, j, newcolor); // put the new color into destination
      }
      else {
        destination.set(i, j, currColor); // put the old colors into destination
      }
    }
  }
  if (mousePressed == true) {
    for (int i = 0; i < movie.width; i++) { // For each pixel in the video frame...
      for (int j = 0; j < movie.height; j++) { // For each pixel in the video frame...
        color newcolor = movie.get(mouseX, j); // get current color
        destination.set(i, j, newcolor); // put the new color into destination
      }
    }
  }
  destination.updatePixels(); 
  image(destination, 0, 0); //put destination array on the screen
}

Live Video!

Live video uses the same library as above
Go under File – Examples – Video – Capture – GettingStartedCapture to find a good example.

You’ll notice that the program sends a list of the supported cameras to the message window. You may want to change the camera resolution by picking a new number in the line
cam = new Capture(this, cameras[0]);
for me, a camera[3] was a better fit, but it may be different on your computers. While you can use a camera to capture an image, what if we think of it as a “light sensor” instead?
This sketch compares the the pixels that have changed from the last time the screen refreshed. It compares the number of pixels that have changed on the right side vs the left side. One nice trick, this script uses dist to get the “distance” between the two colors. Awesome hack.

import processing.video.*;
Capture cam;
PImage destination; 
int ballX; 
 
void setup() {
  size(640, 360);
  String[] cameras = Capture.list();
  for (int i = 0; i < cameras.length; i++) {
    println(cameras[i]);
  }
  destination = createImage(640, 360, RGB);
  cam = new Capture(this, cameras[3]);
  cam.start();
  ballX = width /2;
}
 
void draw() {
  if (cam.available() == true) {
    cam.read();
  }
  cam.loadPixels();
  int left = 0; 
  int right = 0; 
  for (int i = 0; i < cam.width; i++) { 
    for (int j = 0; j < cam.height; j++) {
      color currColor = cam.get(i, j);
      color oldColor = destination.get(i, j);   
      if (dist(red(currColor), green(currColor), blue(currColor), red(oldColor), green(oldColor), blue(oldColor)) > 100) {
        // This line above adds a buffer so it's not so sensitive. 
        if (i < cam.width/2) {
          left ++;
        }
        else {
          right ++;
        }
      }
      destination.set(i, j, currColor); // put the old colors into destination
    }
  }
  destination.updatePixels(); 
  image(destination, 0, 0); //put destination array on the screen
  if (left > right) {
    ballX -= 10;
  }
  else if (left < right) {
    ballX +=  10;
  }
  ballX = constrain(ballX, 0, width); 
  color X = get(ballX, height/2); 
  fill(X); 
  ellipse( ballX, height/2, 80, 80);
}
'+
1
'+
2 - 3
4 - 5
6 - 7
8 - 9
10 - 11
12 - 13
13 - 14
[x]