Code: UMovesDownloader.pde

Aug 30, 2014 | Code, Geo / locative, Open source, Processing / Java

GitHub Gist: UMovesDownloader.pde is a simple tool to batch download all your Moves App data as JSON, from a provided starting date until the current date.

Parts of this code are taken from the MovesMapper vizualization by Nicholas Felton, including his web interface and instructions for getting a valid Moves API access token. MovesMapper is a good example of a locative viz that avoids the typical “here’s a map with some lines and points on it” model.

For a full-featured visualization of Moves data, I recommend Move-O-Scope by Half-Tone.

Continued…

, , , , , , , , , ,

ModelbuilderMk2 news, Aug 2014: unlekker.data and Tile Rendering

Aug 4, 2014 | Code, Libraries, Open source, Processing / Java

ModelbuilderMk2: UTileRenderer demo

High-res output from UTileRendering.pde demo

I haven’t had much news about ModelbuilderMk2 for a while, but I just pushed some new code to the GitHub repo that may be of interest. (Spoiler: Tile rendering for super-highres OpenGL output is back and I added some data stuff.)

  • New: unlekker.data is a new extension to ModelbuilderMk2 for dealing with typical real-world data scenarios, especially parsing, converting and preparing data from various sources (APIs, CSV, SQLite) for visualization. Similarly to how ModelbuilderMk2 relies on UVertex and UVertexList to provide a workflow for creating mesh geometry, UDataPoint and UDataList are intended to represent and manipulate data respectively in its atomic and aggregated forms.

    UDataPoint represents a single data point and stores data fields as a map of key-value pairs. This makes UDataPoint into a universal storage unit for an arbitrary amount of any type of data, whether regular primitives like strings and numbers or object instances. Correspondingly, the UDataList class is a mechanism for collecting and manipulate UDataPoint as a dynamic list. UDataList also automates certain tedious tasks like calculating min/max/median/average bounds on time series data, extracting a list of all values for a given field or filtering the data set by a time range.

    As an example, if you know that all UDataPoints in a UDataList contains a PVector object stored under the key “vec”, the following will produce an ArrayList instance containing all the individual PVector instances:

    UDataList l;
    [..]
    ArrayList pvl=l.getObjectList("vec",PVector.class);

    Continued…

, , , , , ,

You Are Big Data: CIID Summer School

Jul 15, 2013 | Open source, Processing / Java, Theory, Workshops

ManicTime

Screenshot: Manic Time, a particularly obsessive time tracking app

The following is a summary of tools and resources for my two week “You Are Big Data” workshop for CIID Summer School in Copenhagen, in which we’ll be dealing with Quantified Self and data sculpture. This is in part a repost of a previous list.

Andy Polaine wrote a post that referenced my previous summary, in which he made some good critical points and provided a link to a tool I was unaware of: Slogger by Brett Terpstra (sadly, I don’t have a MacOS / Linux setup for testing these kinds of apps myself.)

Continued…

, , , , , , , ,

Thoughts on PVector and data exchange between Processing geometry libraries

May 19, 2013 | Code, Open source, Processing / Java

Update: I just closed the Github thread, apologies for wasting anyone’s time. Given the mixed reactions on Github I have decided to leave the issue for greater minds to ponder.

I still think that standardization of low-level geometry data would be a good thing, promoting interoperability between libraries for that much more awesome. More importantly, it would provide a familiar class framework for users, lessening the confusion slash tedium of learning new data structures for every new library (not to mention writing boilerplate code to convert data to pass between libraries.)

Let’s face it, computational geometry is hard. But it’s also an essential element of generative systems and computational design. Anything that helps users ease into the world of vectors, vertices and meshes is surely a good thing.

———————–

I just posted a thread for discussion on the Processing Github repo:
Thoughts on PVector and data exchange between Processing geometry libraries.

For a while now I’ve been frustrated by the lack of universal and portable data structures for vector and geometry data in Processing. Using PVector sounds nice in theory, but the reality is that geometry libraries like Hemesh, Toxiclibs, Geomerative and Modelbuilder all rely on their own custom representations of vector and mesh data. The result is incompatible data structures with few if any methods for universal data exchange. Worse, PVector is largely ignored and almost never seen “in the wild”

There are many good reasons for this proliferation of incompatible code, most significantly developer preference and the overall focus and internal logic of any given library. But having had the occasional pleasure of integrating both Toxiclibs and Hemesh with my own Modelbuilder, it strikes me that a standard interface would benefit both developers and users. Translating UVec3 objects to Toxiclibs’ Vec3D isn’t all that difficult, but it is tedious. And going beyond simple passing of vector data to more ambitious structures like meshes with per-vertex shading is a headache.

My proposal would be a minimal Java interface (called PVertex, perhaps) representing vertex data (x,y,z + ARGB color + UV texture coordinates). Custom vector classes would implement this interface, guaranteeing interopability but leaving developers free to choose any further implementation details. An interace should be minimally intrusive, but would be very helpful in encouraging geometry data exhcange. (In an ideal world I’d also love to see a minimal mesh container interface (PMesh?) represented by a PVertex array plus an integer array of vertex ID triplets. But, hey, I’m a dreamer.)

If any of this piques your interest, I suggest you go follow (and maybe participate in) the GitHub thread. Karsten Schmidt and Frederik Vanhoutte have already given some valuable feedback. Ultimately, this isn’t so much a discussion about the Processing core as it is an attempt to get library coders to agree on some minimal conventions.

, , , , ,

Code: Catenary Madness

Apr 25, 2013 | Code, Flickr, Open source, Processing / Java, Watz work

OpenProcessing - catenary_mwTweak03 4-up 02

Marius Watz: Catenary Madness (Flickr sketches / Code on OpenProcessing.org)

In case you noticed the Flickr image flood last week of a series of fairly wild-looking sketches titled “Catenary Madness”, I just got around to posting that code on OpenProcessing.org, it even has a few bells and whistles on it: catenary_mwTweak03.

The origin of this series was very random, the original impetus being a very nice catenary curve sketch by Dominik Strzelec. Dominik used Toxiclibs to model a surface of interconnected springs (a piece of billowing fabric, essentially) with gravity and relaxation constraints making things interesting. The way it was set up the simulation reproduced the famous catenary curve effect (although with an actual surface rather than just chains).

The catenary curve is a geometrical gem famously simulated and used as a form-generating process by the very pre-digital architect Gaudi:

Wikipedia: Catenary Curve In physics and geometry, the catenary is the curve that an idealized hanging chain or cable assumes under its own weight when supported only at its ends.

The curve has a U-like shape, superficially similar in appearance to a parabola (though mathematically quite different). It also appears in the design of certain types of arches and as a cross section of the catenoid—the shape assumed by a soap film bounded by two parallel circular rings.

I rarely if ever use “proper” simulation of processes physics, chemical processes or artifical intelligence in my work, but I was intrigued by the organic mesh structure produced by the simulation. Downloading Dominik’s code and predictably scaling up the complexity of the mesh and forces involved quickly made me realize that I have been missing out (although I do remain skeptical of algorithms that can be easily reproduced.) The tipping point came when I applied per-vertex color shading haxxoring, the graphic craziness that ensued had me lost for hours…

The posted code is just one of many sketches exploring the system. I did end up converting up the mesh and color logic to Modelbuilder since I’m less familiar with Toxiclibs, but I chose to post a version using Toxiclibs since it’s closer to the original code. The posted example has very random colors, producing palettes that I would consider unacceptable 80% of the time. The principles of mapping a 1D color list to a 2D color mesh was the focus of interest, rather than the actual colors themselves.

For a more complete index of the Catenary Madness sketches, see Flickr:
http://flickr.com/photos/watz/tags/catenarymadness/

Catenary Madness Index-1500x1500

, , , , , ,

Modelbuilder-0020 and UMovieMaker

Mar 18, 2013 | Code, Libraries, Open source, Processing / Java

New minor version of Modelbuilder: Modelbuilder-0020, consisting mostly of minor fixes but with one piece of good news: Prompted by the disappearance of Shiffman’s MovieMaker library from Processing 2.0b I decided to look into alternative media export libraries. The result is UMovieMaker, a MovieMaker replacement for Quicktime export.

UMovieMaker uses the Monte Media Library by Werner Randelshofer, and is packaged as a separately downloadable sub-lib. Look at the included examples for syntax, it only replicates a subset of MovieMaker’s functionality but so far it has gotten the job done in my own projects. Let me know if you encounter any bugs.

Since I still use 1.5.1 for most projects I have also separated the Modelbuilder code base into 1.5.1-compatible, 2.0b-compatible and universal source folders. I’m not providing a 1.5.1-compatible build for now, but you can download the GitHub repo to get the code. If I get enough requests for a 1.5.1 version I can easily provide one.

Modelbuilder update March 16, 2013
  • Minor code fixes, additions
  • Separated code specific to Processing 1.5.1 and Processing 2.0b series to separate directories for convenience. “src-universal” contains code that works with both code bases. I still use 1.5.1 for production and will keep maintaining a 1.5.1-compatible code base until 2.0 is stable.
  • New sub-library: UMovieMaker, a hack replacement for Shiffman’s MovieMaker that will work with Processing 2.0b. The library uses uses the Monte Media Library (http://www.randelshofer.ch/monte/) by Werner Randelshofer and can write Quicktime without being dependent on QTJava.zip. See included examples.

Modelbuilder-0020 has been exported based on this code and briefly tested with Processing 2.0b8.

Downloads: Modelbuilder-0020 / UMovieMaker-001

, , , , ,

Modelbuilder update

Feb 25, 2013 | Code, Libraries, Open source, Processing / Java

I’ve updated the README for Modelbuilder on Github. It clarifies some issues regarding Processing 2.0 beta and the fact that unlekkerLib has been discontinued. Here is the full update:

Modelbuilder is a computational geometry library for Processing designed to help with parametric and generative modeling, while eliminitating complex and tedious tasks whenever possible.

Modelbuilder provides a set of object-oriented data structures and workflow metaphors (UVertexList define edges, UGeometry is used to generate and modify polygon meshes). The logic used is the familiar beginShape() / endShape() mechanism, optimized and abstracted to eliminate the need for tedious iteration through lists of vertices and faces etc.

Modelbuilder was first released in 2011 as part of my artist-in-residence project with Makerbot Industries, and the library is thus somewhat biased towards digital fabrication (3D printing, laser cutting etc.) My design priority has always been ease of coding rather than maximum realtime performance, but the library still has plenty of useful tools for realtime applications.

Users of my old library unlekkerLib should note that Modelbuilder replaces that library. Having both libraries installed will produce unresolvable conflicts. Modelbuilder does offers most of the same functionality, although with significant API changes. If there was something in unlekkerLib you need I suggest you open an issue on Modelbuilder and I will try to accommodate you.

Update for Processing 2.0 beta:

The code base in src is Processing 2.0b7 compatible, the compiled library is modelbuilder v0019 (not tested on Processing 2.0b8).

I’m still finding some issues using 2.0 for my own projects, so I will maintain code for 1.5.1 for a while longer. See see src-0151-compatible.

Modelbuilder as Swiss Army Knife

Somewhat limited in scope at its inception (i.e. 3D printing,) the library has since proven valuable as a teaching tool, especially for the shorter independent workshops I teach in New York. Consequently Modelbuilder has expanded in scope and become the code equivalent of a Swiss Army Knife, containing tools I find useful but hopefully avoiding outright feature bloat.

Some examples of secondary functions:

  • Color palette generation (UColorTool)
  • File saving and parsing (UIO, UDataText, UFileNode and ULogUtil)
  • FFT code with automatic damping and peak following for use in animation and sound-responsive applications.

With the final jump to Processing 2.0 I plan to do some much needed house-cleaning, making the code a little more consistent and changing some of the underlying design. This will likely come at the cost of some loss of backwards compatibility. Eggs and omelettes, etc.

Marius Watz, Feb 2013

, , , , ,

Self-Ethnography at AHO, aka Big Data OCD

Jan 21, 2013 | AHO, Links, News, Open source, Theory

ManicTime

Screenshot: Manic Time, time tracking app

Possible resources of interest in connection with the upcoming Self-Ethnography and Data Visualization module at AHO, focusing on tools and inspiration. Please also refer to the lecture note and the articles I put on Dropbox.

Articles / videos

Tools and software

  • Two options for auto-tracking everything you do on your computer: Rescue Time and Manic Time.
  • QuantifiedSelf.com has a decent overview of available self-tracking tools: Guide / iPhone products
  • More obscure but Open Source and quite powerful if the documentation can be trusted: Selfspy, Python-based and runs on MacOS, outputting to a local database.
  • Popular social services: FourSquare, Last.fm
  • Self-tracking goes well with an Open Source, information-wants-to-be-free attitude: See Open You or search GitHub for quantified self for all kinds of related projects.
  • Fitness enthusiasts, runners and bikers are avid data trackers. They drive the personal data tracking market while often remaining blissfully unaware of the Quantified Self utopia: MapMyRide, Nike Fuelband, Fitbit, Snore Lab
  • For periodic on-the-run logging of any type of information: your.flowingdata and Daytum offer convenient logging with charts and data export, clearly aimed at data heads. Compare Daily Tracker, which is more aligned with the narrative of self-improvement.
  • Evernote is not strictly about data tracking, but it is a very convenient tool for it. Focused on cloud-based note-taking that bridges across mobile devices, your web-based services and even normal office software, it’s a blank slate for any number of uses.
  • Services built around calendars and to-do lists: Clocked In, Wunderlist, Basecamp. Not obviously related to data logging, but often relying the same mentality of self-discipline.
  • Life Hacker is a respected blog discussing strategies to help you stay productive and in charge of your own life / career / inbox. Data tracking is one such strategy, popular since it promises to provide quantifiable analysis.
  • Somewhat related: Getting Things Done (or GTD) is a bestseller book about time-managing, but it also has a large online following that is pretty much a subculture in its own right.
  • On a more personal and emotional note, mood tracking was an early hit. It is now increasingly being presented as a possible self-help tool against mood disorders: Emoo.me, Moodpanda.
  • Also: Mydrinkaware, Recovery etc. focus on battling addiction issues.

Projects / inspiration: So much to pick from, so little time. Google is your friend.

, , , , ,

Modelbuilder getting some play

Dec 24, 2012 | Code, Libraries, Open source, Processing / Java

This made my week: Amanda Ghassaei posted an awesome Instructable showing how to use my Modelbuilder library to 3D print phonorecords (aka vinyl aka discs encoding sound signals – weird, I know.) Her post has gotten a lot of well-deserved press all over (from Boing Boing to The Verge), and you can find some sample STL files on Pirate Bay although currently there are no seeders for thosee torrents.

Ghassaei provides the code to try it for yourself, although without access to an Objet or a similar high-level 3D printer your mileage might vary. I suspect the process could be reproduced on a lo-fi printer like RepRap or Makerbot given some experimentation, although likely with a further loss of sound quality. If you want guaranteed success this Instructable detailing how to make discs for the Fisher Price toy record player might be of interest. The instructions describe using a CNC mill but the STL files should be easy enough to print on a Makerbot Replicator.

Modelbuilder is currently being upgraded to Processing 2.0 in time for the final release of Processing 2.0. My aim is to eliminate inconsistencies in the library in the process, so please email me (marius at mariuswatz dot com) with requests or bug reports.

, ,

OpenProcessing: UPhysicsBox2D03

Jun 24, 2012 | Code, Libraries, Open source, Processing / Java

UPhysicsBox3D03

Posted on OpenProcessing: UPhysicsBox3D03 (trails mode shown)

While preparing for yesterday’s Advanced Topics workshop I indulged in playing around with Mr.Shiffman’s PBox2D (GitHub) Processing wrapper of JBox2D. Above is a snapshot of the quick-and-dirty result: UPhysicsBox2D03 features a grid of sloping boundaries that you can drop balls through Pachinko-style.

Turn on the “DrawTrails” toggle to switch from normal drawing to a trails mode where you can see the trail of balls bouncing and falling through the grid. There’s also a “timeout” feature that deletes balls that don’t move for a given amount of time. Right now they just blink out of existence, I could think of much better ways but will need to have a closer look at the API first.

, , , , ,