7b Importing data

Importing Text Files

For this sketch to work you will need to create a text file in textEdit and put it into your sketch folder, inside a data folder.

When you import text files you import them into an array of Strings with each line of text as a new entry in the array.

String [] textIn; 
void setup(){
  size(200, 200);
  textIn = loadStrings("textFile.txt");
}
 
void draw(){
  background(0); 
  fill(200); 
  text(textIn [0], 20, 100); 
}

This sketch would allow someone who has no programming ability at all write content for your program.

String [] textIn;
String phrase; 
int phraseNum; 
int fade = 255; 
PFont font; 
void setup() {
  size(800, 160);
  textIn = loadStrings("textFile.txt");
  font = createFont("Handwriting-Dakota", 60); 
  textFont(font); 
  phrase = textIn[0];
  textAlign(CENTER);
}
void draw() {
  fade -= 2; 
  if (fade <= 0) {
    fade = 255; 
    phraseNum ++; 
    if (phraseNum == textIn.length) {
      phraseNum = 0;
    }
  }
  phrase = textIn[phraseNum];
  background(0); 
  fill(200, fade); 
  if (phraseNum == textIn.length-1) {
    fill(255, 100, 20, fade);
  }
  else {
    fill(200, fade);
  }
  text(phrase, width/2, 100);
}

Exporting files is much the same. In this code pressing return saves the file. Notice that we are not loading or saving in the draw – this keeps us from CONSTANTLY loading or saving the file.
This code also checks to see if the file loading works, and instantly save a sample file if there’s nothing there already.

String [] textIn;
String [] test = new String [1]; 
String phrase; 
 
void setup() {
  size(200, 200);
  test [0] = "joe is good"; 
  textIn = loadStrings("textFile.txt");
  if (textIn == null) { 
    saveStrings("textFile.txt", test);
  }
  phrase = textIn[0];
}
void draw() {
  background(0); 
  fill(200); 
  text(phrase, 20, 100);
}
void keyReleased() {
  if (keyCode == 8 && phrase.length() > 0) {
    phrase = phrase.substring(0, (phrase.length() -1));
  }
  else if (keyCode == 10 && phrase.length() > 0) {
    textIn [0] = phrase; 
    saveStrings("textFile.txt", textIn);
  }
  else {
    phrase += key;
  }
}

“Scraping” a website

Scraping mean pulling from the web – it often involves looking through the source file of a website to gra the information.
Any image on the web you can put into your sketch, however, there are three problems with this.

It’s called “hot linking” and now a very good etiquette
Someone could easily remove or change the image and it would screw up your sketch
Your sketch will crash it the computer is not on-line.

Awesome hotlinking reverse revenge story

PImage webImg;
void setup() {
  size(400, 400);
  webImg = loadImage("http://www.bampfa.berkeley.edu/images/art/fermata/MFA_mckay.jpg");
}
void draw() {
  image(webImg, 0, 0);
}
You can even grab the HTML itself using laodString();
String [] website; 
void setup() {
  size(400, 400);
  website  = loadStrings("http://faculty.purchase.edu/joseph.mckay");
  for (int i = 0; i < website.length; i ++) {
    println(i + "   " + website [i]);
  }
}
void draw() {
}

Although doing anything interesting with that will require some parsing, which can get messy, and it’s easy for someone to change the HTML and mess you up.
There are two ways to get live data from a website into Processing, XLM and Json. Both have their own difficulties and circumventing security issues is tricky on a number of sites. Often, files are saved as Comma Separated Value .csv files. You will have to download the data to your data folder, but it is relatively easy to work with once you do.

importing .csv files

link to csv file
At the heart of parsing a csv file is the command “split”. This lets us chop up a string based on the punctuation (in fact we could use any punctuation but commas are common)

void setup() {
size(400, 400);
String allAnimals = “dogs, cats, mice”;
String [] animaArray = split(allAnimals, “,”);
println(animaArray[1]);
}
void draw() {
}
Any spreadsheet application can export a .csv file, but you can also get them on the web. For large data sets, getting the data into the correct arrays is a little tricky, but it’s totally doable.
In the setup we import all the fields into the string array called allFields
Then we use split inside a for loop to filter out the first name, last name, and major into their own arrays.

In the draw we check the major array and change the color accordingly.

PFont font; 
String [] allFields; 
String [] firstName; 
String [] lastName;
String [] major; 
 
void setup() {
  size (500, 600); 
  font = loadFont("Dialog-48.vlw");
  textFont(font, 24);
 
  allFields = loadStrings("class.csv"); 
  firstName = new String [allFields.length] ; 
  lastName = new String  [allFields.length] ; 
  major = new String  [allFields.length] ; 
 
  for (int i=0; i < allFields.length; i++) {
    String [] person = split(allFields[i], ',');
    lastName [i] = person[0]; 
    firstName [i] = person[1]; 
    major[i] = person[2];
  }
}
 
void draw() {
  background(20); 
  for (int i = 1; i < allFields.length; i ++) {
    if (major[i].equals("NEW MEDIA") == true) {
      fill(255, 255, 0);
    }
    else if (major[i].equals("GRAPHIC DESIGN") == true) {
      fill(0, 255, 255);
    }
    else if (major[i].equals("SCULPTURE") == true) {
      fill(100, 100, 200);
    }
    else {
      fill(255, 255, 255);
    }
    text( firstName [i] +  " " + lastName [i], 30, i * 30 + 50);
  }
}

Some Processing libraries are created to interface with a web site’s API. This
yahoo weather library is fantastic.
joe’s weather example

Data sets

data.gov
usGov /
Data visualization of us knowledge