Code: UMovesDownloader.pde

Aug 30, 2014 | Code, Geo / locative, Open source, Processing / Java

GitHub Gist: UMovesDownloader.pde is a simple tool to batch download all your Moves App data as JSON, from a provided starting date until the current date.

Parts of this code are taken from the MovesMapper vizualization by Nicholas Felton, including his web interface and instructions for getting a valid Moves API access token. MovesMapper is a good example of a locative viz that avoids the typical “here’s a map with some lines and points on it” model.

For a full-featured visualization of Moves data, I recommend Move-O-Scope by Half-Tone.

Continued…

, , , , , , , , , ,

ModelbuilderMk2 news, Aug 2014: unlekker.data and Tile Rendering

Aug 4, 2014 | Code, Libraries, Open source, Processing / Java

ModelbuilderMk2: UTileRenderer demo

High-res output from UTileRendering.pde demo

I haven’t had much news about ModelbuilderMk2 for a while, but I just pushed some new code to the GitHub repo that may be of interest. (Spoiler: Tile rendering for super-highres OpenGL output is back and I added some data stuff.)

  • New: unlekker.data is a new extension to ModelbuilderMk2 for dealing with typical real-world data scenarios, especially parsing, converting and preparing data from various sources (APIs, CSV, SQLite) for visualization. Similarly to how ModelbuilderMk2 relies on UVertex and UVertexList to provide a workflow for creating mesh geometry, UDataPoint and UDataList are intended to represent and manipulate data respectively in its atomic and aggregated forms.

    UDataPoint represents a single data point and stores data fields as a map of key-value pairs. This makes UDataPoint into a universal storage unit for an arbitrary amount of any type of data, whether regular primitives like strings and numbers or object instances. Correspondingly, the UDataList class is a mechanism for collecting and manipulate UDataPoint as a dynamic list. UDataList also automates certain tedious tasks like calculating min/max/median/average bounds on time series data, extracting a list of all values for a given field or filtering the data set by a time range.

    As an example, if you know that all UDataPoints in a UDataList contains a PVector object stored under the key “vec”, the following will produce an ArrayList instance containing all the individual PVector instances:

    UDataList l;
    [..]
    ArrayList pvl=l.getObjectList("vec",PVector.class);

    Continued…

, , , , , ,

Workshop: Intro to Generative Art with Processing (July 6, NYC)

Jun 23, 2014 | Code, Processing / Java, Workshops

Marius Watz: KBG (sound viz)

Workshop: Intro to Generative Art with Processing
Date: Sunday, July 6, Williamsburg, NYC

Participants will be introduced to the principles of Generative Art using Processing, a Java-based coding tool designed specifically for creative applications.

A generative system can take many forms, but essentially the term describes a set of aesthetic rules for creating layout, motion and geometry. Translated to code, these systems are capable of producing an endless variety of forms and visuals. We will look at how to create a generative visual system, from sketching in code to creating professional quality output for print or video.

Examples and topics will include: Useful building blocks and tools-of-the-trade applied to real-world problems, from basic composition and color in code, to more complex issues like geometry and animation. By adding GUI controls and output options for print + video to our sketches, we end up with the makings of a full-fledged generative system.

Continued…

, , , , , ,

Code: iOStoEpochTime.java

May 30, 2014 | Code, Processing / Java, Workshops

GitHub Gist: iOStoEpochTime.java Converts iOS Objective-C timestamps to their Java equivalent. Useful when parsing exported data from iOS backup SQLite files. I’ll post a demo parsing iMessage data shortly, written for my upcoming Quantified Self workshop.

Technical details: Objective-C timestamps (as encountered in iOS SQLite data) are given as seconds since Jan 1, 2001, while Java timestamps represent milliseconds since Jan 1, 1970.

To convert between them we first multiply the Objective-C timestamp by 1000, then we add a pre-calculated millisecond offset representing the difference between Jan 1, 1970 and Jan 1, 2001. This difference is calculated using the java.util.Calendar class, see code below for how this is done.

Continued…

, , , , , ,

Workshop: Intro to Generative Art with Processing (Jun 1, NYC)

May 27, 2014 | Code, Processing / Java, Workshops

Marius Watz: KBG (sound viz)

Workshop: Intro to Generative Art with Processing
Date: Sunday, Jun 1, Williamsburg, NYC

Participants will be introduced to the principles of Generative Art using the Processing programming tool. A generative system can take many forms, but is essentially a set of rules that when translated to code executed is capable of producing an endless variety of forms or visuals. We will look at how a generative visual can be created, from sketching in code to creating professional quality output for print or video.

The examples we will go through include useful some essential building blocks and tools-of-trade, applied to real-world problems. Topics include how to do basic composition and color in code, to more complex issues like creating geometry and animation. By adding GUI controls and print / video output to our sketches we have the makings of a full-fledged generative system.

Continued…

, , , , , ,

Code: PrimeList.pde

May 22, 2014 | Code, Processing / Java

GitHub Gist: PrimeList.pde Parses prime number lists as provided by http://primes.utm.edu/, storing primes as ArrayList.

Requires PDE source and a sample input file to run, use download link to get both as a ZIP.
Continued…

, , , , ,

Workshop: Quantified Self + Viz with Processing (Apr 20, NYC)

Apr 13, 2014 | Code, Processing / Java, Workshops

2013-Seamless-history-mariuswatz

Viz: Seamless.com takeout order history (timestamp demo)

Workshop: Quantified Self and data visualization with Processing
Date: Saturday, April 12, Williamsburg, NYC
Rescheduled: Sunday, April 20, Williamsburg, NYC

Update: I had to cancel Saturday’s workshop due to a brain-scrambling flu. I’ve rescheduled the workshop for Sunday, April 20th, and there are two spots still open.


This workshop will introduce participants to Quantified Self and personal data tracking, with the aim of creating custom code-driven visualizations.

We will use Processing to parse, analyze and visualize data (CSV, JSON) generated by popular tracking tools, establishing basic principles and useful workflows that can be applied to common QS scenarios.

Continued…

, , , , , , , , ,

Code: ULoremIpsum.java (text anonymizer)

Apr 8, 2014 | Code, Processing / Java, Workshops

New GitHub Gist: ULoremIpsum.java Simple Lorem Ipsum text replacer for Java/Processing. It is useful for anonymizing text content in data sets (email, SMS, direct messages etc.) Upper/lower case is preserved as best as Java String supports (Locale twiddling might be needed in some cases) and will leave all non-letters intact.

The class uses two built-in dictionaries: A list of replacement words and a “whitelist” of words that should be kept as is. For brevity, these are set as inline preset strings here. They can easily be changed in the code or changed to be customizable by adding a mechanism for setting the dictionaries.

Continued…

, , , , , , ,

A Cornucopia of Quantification: QS apps + tools, Pt.1

Mar 28, 2014 | Code, Geo / locative, Links, Software, Workshops

20140325-Quantified-Self-workshop

Outline for my March 29 Quantified Self workshop (now sold out, the next date will April 12.) Diagrammed with XMind.

For anyone who has been paying attention it will be clear that 2013 was arguably the year that Quantified Self exploded. It could also be argued that the focus on pedometers and personal fitness augmentations represents a sort of “QS Lite”, limiting itself to ideas that can be conveniently explained and marketed in the form of soundbites. Good for business, visionary not so much.

The sheer number of new tracking services and apps that have emerged in the last year is both a blessing and a curse. It’s exciting to see new approaches being explored, even though the vast majority are simply re-hashing the same basic ideas. How many workout apps can the market possibly support? QS might be on the brink of becoming a cash cow, but for now it’s mostly a bubble.

Some newcomers (Moves, Tictrac, Reporter ++) do feel like a real evolution, both in terms of user experience and their underlying design concepts. Data hackers and coders should be happy to note that some developers understand their needs and value their participation. A decent export mechanism (cloud-based or not) and maybe even a GitHub repo with sample code is a good start.

Continued…

, , , , , , , , , , , , , , ,

Workshop, NYC: Quantified Self and Data Visualization with Processing

Mar 19, 2014 | Code, Processing / Java, Workshops

Visualization: Last.fm history

Code: ULastFM_Simple, parses and displays Last.fm CSV data.

Workshop: Quantified Self and data visualization with Processing
Date: Saturday, March 29, Williamsburg, NYC

This workshop is now sold out. I will do another one in just a few weeks – watch this space.

This workshop will introduce participants to Quantified Self and personal data tracking, with the aim of creating custom code-driven visualizations.

We will use Processing to parse, analyze and visualize data (CSV, JSON) generated by popular tracking tools, establishing basic principles and useful workflows that can be applied to common QS scenarios.

Topics

  • Parsing and plotting typical QS data
  • Data structures for personal data
  • Mapping of locative and time-based data
  • Correlating multiple data sources to discover patterns of behavior
  • Useful tracking tools that are both open and code-friendly

Tools

Suitable for: Anyone with a basic knowledge of Processing or common programming languages. Familiarity with common data formats will be helpful, but not required. Ideally, participants should install and research the tracking tools mentioned above before attending.

Previous QS teaching: http://workshop.evolutionzone.com/tag/quantified-self/

Full disclosure: I am currently enjoying one year of complementary Rescue Time Premium access in connection with my teaching efforts. I also just created a Rescue Time affiliate account. RT is not the only time-tracking tool out there, but I’ve used the service for years because it provides open data access combined with just the right level of detail for QS purposes.

If you’re looking for true OCD by-the-second granularity activity tracking, have a look at tools like Manic Time, Selfspy or Slogger.

Continued…

, , , , , , ,