Use Data.beginSave() to initialize data string collection, then add data with Data.add(). To write to file, use endSave(filename). I’ve included code for auto-incrementing filenames, it’s used in the example code.

To load data, call Data.load(filename), then use readInt(), readFloat(), readString() etc. to get values from the strings read from the text file. Used properly this should give you most of the flexibility you need for simple data saving.

// saveData.pde
// Marius Watz - http://workshop.evolutionzone.com

// Example showing how to use the Data utility
// class to save and load data from text files. 

float a=2.5,b=3.144;
int x=10,y=20;
boolean bool=true;
String s="This is a string.";

Data data;

void setup() {
  size(400,400);

  data=new Data();

  // SAVING
  data.beginSave();
  data.add(s);
  data.add(a);
  data.add(b);
  data.add(x);
  data.add(y);
  data.add(bool);
  data.endSave(
    data.getIncrementalFilename(
      sketchPath("save"+
        java.io.File.separator+
        "data ####.txt")));

  // LOADING
  data.load(sketchPath("save"+
    java.io.File.separator+
    "data 0000.txt"));

  s=data.readString();
  a=data.readFloat();
  b=data.readFloat();
  x=data.readInt();
  y=data.readInt();
  bool=data.readBoolean();
}

///////////////////////////
// DATA CLASS

class Data {
  ArrayList datalist;
  String filename,data[];
  int datalineId;

  // begin data saving
  void beginSave() {
    datalist=new ArrayList();
  }

  void add(String s) {
    datalist.add(s);
  }

  void add(float val) {
    datalist.add(""+val);
  }

  void add(int val) {
    datalist.add(""+val);
  }

  void add(boolean val) {
    datalist.add(""+val);
  }

  void endSave(String _filename) {
    filename=_filename;

    data=new String[datalist.size()];
    data=(String [])datalist.toArray(data);

    saveStrings(filename, data);
    println("Saved data to '"+filename+
      "', "+data.length+" lines.");
  }

  void load(String _filename) {
    filename=_filename;

    datalineId=0;
    data=loadStrings(filename);
    println("Loaded data from '"+filename+
      "', "+data.length+" lines.");
  }

  float readFloat() {
    return float(data[datalineId++]);
  }

  int readInt() {
    return int(data[datalineId++]);
  }

  boolean readBoolean() {
    return boolean(data[datalineId++]);
  }

  String readString() {
    return data[datalineId++];
  }

  // Utility function to auto-increment filenames
  // based on filename templates like "name-###.txt" 

  public String getIncrementalFilename(String templ) {
    String s="",prefix,suffix,padstr,numstr;
    int index=0,first,last,count;
    File f;
    boolean ok;

    first=templ.indexOf('#');
    last=templ.lastIndexOf('#');
    count=last-first+1;

    if( (first!=-1)&& (last-first>0)) {
      prefix=templ.substring(0, first);
      suffix=templ.substring(last+1);

      // Comment out if you want to use absolute paths
      // or if you're not using this inside PApplet
      if(sketchPath!=null) prefix=savePath(prefix);

      index=0;
      ok=false;

      do {
        padstr="";
        numstr=""+index;
        for(int i=0; i< count-numstr.length(); i++) padstr+="0";
        s=prefix+padstr+numstr+suffix;

        f=new File(s);
        ok=!f.exists();
        index++;

        // Provide a panic button. If index > 10000 chances are it's an
        // invalid filename.
        if(index>10000) ok=true;

      }
      while(!ok);

      // Panic button - comment out if you know what you're doing
      if(index> 10000) {
        println("getIncrementalFilename thinks there is a problem - "+
          "Is there  more than 10000 files already in the sequence "+
          " or is the filename invalid?");
        println("Returning "+prefix+"ERR"+suffix);
        return prefix+"ERR"+suffix;
      }
    }

    return s;
  }

}

KBGAnimD01EB 0008, still from new sound visualization

Ever since doing Stockspace project it seems I am getting asked to do data-related work. This despite the fact that my personal interests diverge from such masters of insightful infographics as Martin Wattenberg, David McCandless or Jonathan Harris.

Suffice to say that I am more concerned with exploring data structures as spaces than I am with providing new understandings of the information contained within them. Manuel Lima’s Information Visualization Manifesto calls for a seriousness on the topic of data treatments, while my projects remain comfortably frivolous.

Recently I’ve been working on a project that has required researching data sources and adapting them to illustrate a bigger idea, which has led to much Googling in the absence of good data from the client. Sometimes you find the right thing immediately, but sometimes data is hard to find in a format that is freely available and easily parsable. Since I have found some good sources I thought I’d share them here…

• PVWATSS calculator from Renewable Resource Data Center (RReDC) provides theoretical calculations of yields (hour-by-hour, 365 days/yr) from photovoltaic solar panels for most international locations.
• World airport codes with geo name data, helpfully spidered by Wiebe Cazemier so you don’t have to.
• Unattributed FTP depository of geo-related CSV files, including postal and ZIP codes, world city info etc.
• Historical weather data from Weather Underground, providing year-by-year hourly information for most cities in the world. (See also this list of weather APIs.)
• Yahoo Finance offers downloads of historical stock price data (CSV) for just about any stock symbol out there. Sadly I have yet to find an open intraday stock data source, due to the proprietary nature of that kind of information.
• The Guardian’s Datablog is a recent venture, and I have yet to use any of the data sets they provide. But I like their approach, which is clearly aimed at democratizing data with a view to public agenda.

I would be interested in hearing tips about any great data sets out there, particularly interesting time series data.

Miska Knapek recently sent me a link to a source of weather sensor data from Helsinki, including measurements of wind direction at the top of Helsinki’s Olympic Tower in 5-minute intervals. He has already made some wind visualization videos and some fabricated wind data sculptures based on this data.

Jer Thorp: NY Times: 365/360

Processing visualization head Jer Thorp is putting his money where his mouth and publishing 7 pieces of code in 7 days, free to download and experiment with. Judging from the three that he’s released so far they’re not your standard 20-minute sketches either:

1. GoodMorning! is a Twitter vizualization, showing users around the world popping up on a globe as they utter the magic words “good morning”. With a little geocoding and spherical mapping thrown in, this is a sweet sketch
2. NY Times: 365/360 uses the New York Times open data API to retrieve news stories for an entire year and draw connections between them. The results combine complexity with elegance for that true infoporn look.
3. tree.growth revisits that old classic, the L-system tree. Thorp uses colors and abstract “leaves” to great effect.

With such a strong start, one certainly looks forward to seeing the next four sketches to come. It’s not so common to find sketches of this complexity freely available, so they’re a great study for users who are on the threshold of making more complex applications.

• added functions to FFT for doing forward transforms with an offset: forward(float[] samples, offset) and forward(AudioBuffer samples, offset)
• added a freqToIndex(float freq) method to FFT for finding out the index of the spectrum band that contains the passed in frequency.
• added a stop() method to AudioSample, so that playing samples can be immediately silenced.
• added setPanNoGlide(float pan) to Controller, which will snap the panning setting of a sound to the provided value.
• added setInputMixer(Mixer) and setOutputMixer(Mixer), which allow you to specify which Java Mixer object should be used when obtaining inputs (AudioInput) and outputs (AudioOuput, AudioPlayer, AudioSnippet, AudioSample).

Download from the Minim project page. Read more about the development process on the Compartmental blog.

The following hack demonstrates how to make a window that has no OS chrome, always stays on top of the UI and has an explicitly set screen position. You can even use the cursor keys to move the window around the screen.

Personally, what I like most about this hack is that it gets around Processing’s (or possibly Java’s) assumptions about a minimum window size of ~120×120. If you specify a size that’s less than 120 on one side, Processing will be pad that side with grey pixels to reach the minimum. But with this hack you can have tiny windows that you can micro-manage to your heart’s delight. I use it to make debug dispays, small control panels and other useful things.

Update: @CedricKiefer pointed out another Processing example that allows for transparent and even irregularly shaped windows. It’s Windows-only apparently, I bet this kind of thing is a major violation of Apple GUI laws anyhow.

Update 2: @ideoforms took my post literally and made a sketch with multiple bouncing windows. Nice.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95

// FloatingWindow.pde
// Marius Watz - http://workshop.evolutionzone.com
// 
// Provides a method for creating frame-less windows with smaller 
// than typical sizes. Also demonstrates how to set the precise
// location of the window on screen, as well as make it float over
// all other windows.
//
// Can be useful if you'd like to make a persistent "debug" app of
// some sort, or just because it's neat when things don't look like
// your everyday operating system.
//
// Works both with P2D and OPENGL.Uses a trick from this Processing hack:
// http://processing.org/hacks/hacks:undecoratedframe
 
PFont fnt;
int W,H,PX,PY;
 
void setup() {
  // W x H == desired size of frame. 
  // Normally, Processing will not create a frame smaller than ~120x120,
  // but will pad the window with blank pixels.
  W=100;
  H=40;
 
  // initial position of frame
  PX=100;
  PY=100;
 
  size(W,H);
  fnt=createFont("Arial",12,false);
}
 
// overriding PApplet.init() to add a hack of our own
void init() {
 
  // trick to make it possible to change the frame properties
  frame.removeNotify(); 
 
  // comment this out to turn OS chrome back on
  frame.setUndecorated(true); 
 
  // comment this out to not have the window "float"
  frame.setAlwaysOnTop(true); 
 
  frame.setResizable(true);  
  frame.addNotify(); 
 
  // making sure to call PApplet.init() so that things 
  // get  properly set up.
  super.init();
}
 
void draw() {
  background(0);
 
  // resize and set initial location a few frames after sketch start. 
  // our window will now be tiny and located at position PX,PY.
 
  if(frameCount==5) {
    frame.resize(W,H);
    frame.setLocation(PX,PY);
  }
 
  // draw window outline
  noStroke();
  fill(255,100,0);
  rect(0,0, width,height);
  fill(0);
  rect(3,3, width-6,height-6);
 
  textFont(fnt);
  fill(255,255,0);
  text(frameCount/10+" | "+
    frame.getLocation().x+","+
    frame.getLocation().y,16,24);
}
 
// the cursor keys may be used to move the window around
void keyPressed() {
  if(key==CODED) {
    if(keyCode==LEFT) frame.setLocation(
      frame.getLocation().x-5,
      frame.getLocation().y);
    if(keyCode==RIGHT) frame.setLocation(
      frame.getLocation().x+5,
      frame.getLocation().y);
    if(keyCode==UP) frame.setLocation(
      frame.getLocation().x,
      frame.getLocation().y-5);
    if(keyCode==DOWN) frame.setLocation(
      frame.getLocation().x,
      frame.getLocation().y+5);
  }  
}

However, I tried the original TileSaver code posted back in 2006 and miraculously that still works. This would imply a larger issue with the unlekkerLib library and Processing 1.0, which I’ll have to address when I have the time.

For now, here is a link to a working Processing sketch using the old TileSaver code: TileSaverTest.zip

In particular, STL export should work again. However I haven’t had the chance to test the code very much, so please let me know if you find any bugs.

• unlekkerLib0003c

Multiple JDIC browsers integrated into Processing sketch

A while back I posted a simple hack to open a web browser from Processing by using JDesktop Integration Components (JDIC). A recent discussion on the Processing forums asked about how to use it to open multiple browsers inside the actual Processing sketch window.

My original hack used an instance of org.jdesktop.jdic.browser.WebBrowser integrated into a java.awt.Panel instance and laid out in a java.awt.Frame. That meant that the browser would open in an external window. The discussion on the Processing forum asked specifically about opening multiple browsers in the main sketch window, so I made the following quick hack.

I’m posting the actual code below, but to run the example you should download the following ZIP file: JDIC_multiple.zip. It includes the JDIC binaries required to run it on Windows.

// JDIC_multiple.pde
// Marius Watz - http://workshop.evolutionzone.com

import java.awt.*;
import java.awt.event.*;
import javax.swing.*;

import java.net.URL;
import java.net.MalformedURLException;
import org.jdesktop.jdic.browser.*;

aBrowser browser[];
long last;

void setup() {
  size(800,600);
  frame.setLayout(new GridLayout(2,2));

  browser=new aBrowser[4];
  for(int i=0; i<4; i++) {
    browser[i]=new aBrowser();
    browser[i].initPanel(400,300);

    if(i==0) browser[i].setURL("http://www.google.com/");
    if(i==1) browser[i].setURL("http://processing.org/");
    if(i==2) browser[i].setURL("http://ffffound.com/");
    if(i==3) browser[i].setURL("http://sojamo.tumblr.com/"); 

    frame.add(browser[i].panel);
  }
}

void draw() {

}

public class aBrowser {
  Panel panel;
  WebBrowser webBrowser;

  public aBrowser() {
    // Set engine to IE
    BrowserEngineManager mng=BrowserEngineManager.instance();
    mng.setActiveEngine(BrowserEngineManager.IE);
    webBrowser = new WebBrowser();
  }

  public void initPanel(int w,int h) {
    panel = new Panel();
    panel.setLayout(new BorderLayout());
    panel.setSize(w,h);
    webBrowser.setSize(w, h);
    panel.add(webBrowser, BorderLayout.CENTER);
  }

  public void setContent(String htmlContent) {
    webBrowser.setContent(htmlContent);
  }

  public void setURL(String url) {
    try {
      webBrowser.setURL(new URL(url));
      webBrowser.setDebug(false);
    }
    catch (MalformedURLException e) {
      System.out.println(e.getMessage());
      return;
    }
  }
}

You can download unlekkerLib-0003 from the Code & Form Google Code repository. I’ve added the missing classes, but otherwise the library is the same. If you still experience problems please report them here.

MeshLibDemo.pde – Demo of Lee Byron’s Mesh library

Lee Byron has written a neat little Processing library called Mesh which allows for easy calculation and display of Voronoi, Delaunay and Convex Hull diagrams.

Given a set of points, these diagrams calculate the minimal regions around the points (Voronoi), an optimal triangulation of the points (Delaunay) or the polygon shape that contains all the points (Convex Hull). So far the library only supports the 2D versions of the diagrams, but it is in part based on the QuickHull3D java library which also handles 3D hulls.

Byron didn’t include any code examples in the current release, so I hacked up a quick demo.

To run this example, download MeshLibDemo.zip and unzip it inside your Processing sketches folder. The Mesh library is included in a “libraries” subfolder, but you’ll have to restart Processing for the library to be recognized.

I’m posting the full code below for easy reference.

// MeshLibDemo.pde - Demo of Lee Byron's Mesh library for
// calculating and drawing Voronoi, Delaunay and Convex Hull
// diagrams. 

// Uses a set of random points to calculate all
// three diagrams, with one point responding to the mouse
// position. Press space to reset points, press '1', '2' and
// '3' to toggle display of the different diagrams.
//
// The library is included in this sketch, see the "libraries"
// subfolder See http://leebyron.com/else/mesh/ for more
// information about the library.
//
// Marius Watz - http://workshop.evolutionzone.com/

import megamu.mesh.*;

Voronoi myVoronoi;
Delaunay myDelaunay;
Hull myHull;

float[][] points;
float[][] myEdges;
MPolygon myRegions[],myHullRegion;
int col[];

float startX,startY,endX,endY;
float[][] regionCoordinates;

boolean showVoronoi=true;
boolean showDelaunay=false;
boolean showHull=false;

void setup() {
  size(500,250);

  // initialize points and calculate diagrams
  initMesh();
  smooth();
}

void draw() {
  background(200);
  if(myRegions==null) return;

  // draw Voronoi
  if(showVoronoi) {
    strokeWeight(1);
    stroke(0);
    for(int i=0; i< myRegions.length; i++) {
      fill(col[i]); // use random color for each region
      regionCoordinates = myRegions[i].getCoords();
      myRegions[i].draw(this); // draw this shape
    }
  }

  // draw Voronoi as lines
  if(showDelaunay) {
    strokeWeight(2);
    stroke(255,0,0);
    for(int i=0; i< myEdges.length; i++) {
      startX = myEdges[i][0];
      startY = myEdges[i][1];
      endX = myEdges[i][2];
      endY = myEdges[i][3];
      line(startX, startY, endX, endY);
    }
  }

  // draw Hull in semi-transparent yellow
  if(showHull) {
    strokeWeight(1);
    stroke(0);
    fill(255,255,0, 150);
    myHullRegion.draw(this);
  }
}

void initMesh() {
  // is points array is null then initialize it
  if(points==null) initPoints();

  // save the current number of regions, so that
  // we can check if it's the same after the Voronoi
  // has been recalculated.
  int oldlength=0;
  if(myRegions!=null) oldlength=myRegions.length;

  myVoronoi = new Voronoi( points );
  myHull = new Hull( points );
  myDelaunay = new Delaunay( points );

  myRegions = myVoronoi.getRegions();
  myHullRegion = myHull.getRegion();
  myEdges = myDelaunay.getEdges();

  // if the number of regions is different than
  // before then recalculate the random colors
  if(oldlength!=myRegions.length) {
    col=new int[myRegions.length];
    for(int i=0; i< myRegions.length; i++) {
      float prob=random(100);
      if(prob>60) col[i]=color(random(30,100));
      else col[i]=color(random(200,255));
    }
    col[0]=color(255,0,0);
  }
}

void initPoints() {
    points = new float[(int)random(5,30)][2];

  for(int i=0; i< points.length; i++) {
    points[i][0] = random(width); // first point, x
    points[i][1] = random(height); // first point, y
  }

}

void keyPressed() {
  // reset points and mesh when spacebar is pressed
  if(key==' ') {
    initPoints();
    initMesh();
  }

  // use keys '1'-'3' to toggle display
  if(key=='1') showVoronoi=!showVoronoi;
  if(key=='2') showDelaunay=!showDelaunay;
  if(key=='3') showHull=!showHull;
}

void mouseMoved() {
  // if myRegions is null then mesh is not ready
  if(myRegions==null) return;

  // set first point to mouse position and recalculate
  points[0][0]=mouseX;
  points[0][1]=mouseY;
  initMesh();
}