Workshop: Quantified Self + Viz with Processing (Apr 20, NYC)

Apr 13, 2014 | Code, Processing / Java, Workshops

2013-Seamless-history-mariuswatz

Viz: Seamless.com takeout order history (timestamp demo)

Workshop: Quantified Self and data visualization with Processing
Date: Saturday, April 12, Williamsburg, NYC
Rescheduled: Sunday, April 20, Williamsburg, NYC

Update: I had to cancel Saturday’s workshop due to a brain-scrambling flu. I’ve rescheduled the workshop for Sunday, April 20th, and there are two spots still open.


This workshop will introduce participants to Quantified Self and personal data tracking, with the aim of creating custom code-driven visualizations.

We will use Processing to parse, analyze and visualize data (CSV, JSON) generated by popular tracking tools, establishing basic principles and useful workflows that can be applied to common QS scenarios.

Continued…

, , , , , , , , ,

Code: ULoremIpsum.java (text anonymizer)

Apr 8, 2014 | Code, Processing / Java, Workshops

New GitHub Gist: ULoremIpsum.java Simple Lorem Ipsum text replacer for Java/Processing. It is useful for anonymizing text content in data sets (email, SMS, direct messages etc.) Upper/lower case is preserved as best as Java String supports (Locale twiddling might be needed in some cases) and will leave all non-letters intact.

The class uses two built-in dictionaries: A list of replacement words and a “whitelist” of words that should be kept as is. For brevity, these are set as inline preset strings here. They can easily be changed in the code or changed to be customizable by adding a mechanism for setting the dictionaries.

Continued…

, , , , , , ,

A Cornucopia of Quantification: QS apps + tools, Pt.1

Mar 28, 2014 | Code, Geo / locative, Links, Software, Workshops

20140325-Quantified-Self-workshop

Outline for my March 29 Quantified Self workshop (now sold out, the next date will April 12.) Diagrammed with XMind.

For anyone who has been paying attention it will be clear that 2013 was arguably the year that Quantified Self exploded. It could also be argued that the focus on pedometers and personal fitness augmentations represents a sort of “QS Lite”, limiting itself to ideas that can be conveniently explained and marketed in the form of soundbites. Good for business, visionary not so much.

The sheer number of new tracking services and apps that have emerged in the last year is both a blessing and a curse. It’s exciting to see new approaches being explored, even though the vast majority are simply re-hashing the same basic ideas. How many workout apps can the market possibly support? QS might be on the brink of becoming a cash cow, but for now it’s mostly a bubble.

Some newcomers (Moves, Tictrac, Reporter ++) do feel like a real evolution, both in terms of user experience and their underlying design concepts. Data hackers and coders should be happy to note that some developers understand their needs and value their participation. A decent export mechanism (cloud-based or not) and maybe even a GitHub repo with sample code is a good start.

Continued…

, , , , , , , , , , , , , , ,

Workshop, NYC: Quantified Self and Data Visualization with Processing

Mar 19, 2014 | Code, Processing / Java, Workshops

Visualization: Last.fm history

Code: ULastFM_Simple, parses and displays Last.fm CSV data.

Workshop: Quantified Self and data visualization with Processing
Date: Saturday, March 29, Williamsburg, NYC

This workshop is now sold out. I will do another one in just a few weeks – watch this space.

This workshop will introduce participants to Quantified Self and personal data tracking, with the aim of creating custom code-driven visualizations.

We will use Processing to parse, analyze and visualize data (CSV, JSON) generated by popular tracking tools, establishing basic principles and useful workflows that can be applied to common QS scenarios.

Topics

  • Parsing and plotting typical QS data
  • Data structures for personal data
  • Mapping of locative and time-based data
  • Correlating multiple data sources to discover patterns of behavior
  • Useful tracking tools that are both open and code-friendly

Tools

Suitable for: Anyone with a basic knowledge of Processing or common programming languages. Familiarity with common data formats will be helpful, but not required. Ideally, participants should install and research the tracking tools mentioned above before attending.

Previous QS teaching: http://workshop.evolutionzone.com/tag/quantified-self/

Full disclosure: I am currently enjoying one year of complementary Rescue Time Premium access in connection with my teaching efforts. I also just created a Rescue Time affiliate account. RT is not the only time-tracking tool out there, but I’ve used the service for years because it provides open data access combined with just the right level of detail for QS purposes.

If you’re looking for true OCD by-the-second granularity activity tracking, have a look at tools like Manic Time, Selfspy or Slogger.

Continued…

, , , , , , ,

ITP: Sound As Data workshop (code)

Nov 12, 2013 | Code, ITP-Parametric, Processing / Java, Workshops

20131111-ITP-Sound-As-Data

Nov 11, 2013 | Marius Watz: Sound as Data Workshop, ITP NYU
Download: See GitHub

As part of my on-going ITP residency I am doing a few workshops on specialized code topics. This weekend I revisited my previous work on using sound as an input for visuals or geometry, using Processing and Minim. It was only a 3 hour workshop, but we covered the basics:

  • Opening line in or sound file inputs
  • Accessing sound buffer
  • Normalizing volume
  • Using damping to control FFT behavior

All code has been posted on GitHub, original description below. Continued…

, , , , , ,

Code: ProcessingData library (2.0 Data API for 1.5.1)

Sep 5, 2013 | Code, Libraries, Processing / Java

Code: ProcessingData library Source, Exported library (ZIP)

During my CIID workshop I did a quick hack to make the Processing 2.0 Data API available for 1.5.1 users. The processing.data code from the 2.0 core needed only a few minor adaptations. The data methods (like loadTable()) that are native to PApplet in 2.0 are provided through a simple helper class (unlekker.data.ProcessingData.)

The data classes are one of the best features of 2.0, cleanly written and consistent as they are. But if you’re sticking with 1.5.1 out of preference or necessity this might be of use. All glory is owed to the Processing team, I simply repackaged the code. As proof of the quality of the 2.0 code, it took about 30 minutes to extract and refactor the data code. The biggest hurdle was exporting the JAR file and figuring out it had to be compiled to be Java 1.5 compatible

The code can be found on my Teaching repo on GitHub, the exported library can be downloaded as a ZIP file. See the included LoadSaveTable.pde example from Shiffman for a demo, it shows how to use the ProcessingData class to call loadTable() etc.

Disclaimer: Minimal testing was performed, any issues should be reported on GitHub.

, , , ,

You Are Big Data: CIID Summer School

Jul 15, 2013 | Open source, Processing / Java, Theory, Workshops

ManicTime

Screenshot: Manic Time, a particularly obsessive time tracking app

The following is a summary of tools and resources for my two week “You Are Big Data” workshop for CIID Summer School in Copenhagen, in which we’ll be dealing with Quantified Self and data sculpture. This is in part a repost of a previous list.

Andy Polaine wrote a post that referenced my previous summary, in which he made some good critical points and provided a link to a tool I was unaware of: Slogger by Brett Terpstra (sadly, I don’t have a MacOS / Linux setup for testing these kinds of apps myself.)

Continued…

, , , , , , , ,

New Generative Art workshops: March 9+16

Mar 1, 2013 | Code, Processing / Java, Watz work, Workshops

GCirc01E-025

Update: In my eagerness to announce these workshops I made a scheduling error, incorrectly thinking the dates would be March 15+16 rather than 16+17. As a result I need to move one of the workshops to the weekend before, and since the Intro workshop should happen before the Advanced the new dates will be:

  • Saturday March 9: Introduction to Processing and Generative Art
  • Saturday March 16: Generative Art, Advanced Topics

Sorry for the confusion! On the plus side the Intro workshop might now be a smaller group which should make it nice and intimate.

I haven’t done any workshops in New York since November, so I have decided to offer my Intro and Advanced Generative Art workshops back-to-back the weekend of March 16+17 on consecutive weekends, Saturday March 9 and Saturday March 17.

The venue will be my apartment in comfortable Park Slope, Brooklyn. As usual I have 8 spots available for each workshop, they do tend to reach capacity so get in touch sooner rather than later. Reservation is by email and your spot is confirmed once I receive payment via PayPal.

The workshops will be taught using the most recent Processing 2.0 beta version (2.0b8 as of this moment), and as usual I will be using my own Modelbuilder library as a toolkit for solving the tasks we look. Familiarizing yourself with Processing 2.0 and Modelbuilder would be good preparation.

Make sure to download Modelbuilder-0019 and Control-P5 2.0.4, then run through the provided examples. Check OpenProcessing.org for more Modelbuilder examples.

Note about dataviz: I know there is a lot of interest in data vizualization and I do get asked about that frequently in workshops. I can’t promise to cover data in detail since it’s a pretty big topic.

If you’re specifically looking for data techniques I would recommend looking at the excellent workshops series taught by my friend Jer Thorp. He currently offers two such workshops, titled “Processing and Data Visualization” and “Archive, Text, & Character(s)”.

, , , , , ,

Self-Ethnography at AHO, aka Big Data OCD

Jan 21, 2013 | AHO, Links, News, Open source, Theory

ManicTime

Screenshot: Manic Time, time tracking app

Possible resources of interest in connection with the upcoming Self-Ethnography and Data Visualization module at AHO, focusing on tools and inspiration. Please also refer to the lecture note and the articles I put on Dropbox.

Articles / videos

Tools and software

  • Two options for auto-tracking everything you do on your computer: Rescue Time and Manic Time.
  • QuantifiedSelf.com has a decent overview of available self-tracking tools: Guide / iPhone products
  • More obscure but Open Source and quite powerful if the documentation can be trusted: Selfspy, Python-based and runs on MacOS, outputting to a local database.
  • Popular social services: FourSquare, Last.fm
  • Self-tracking goes well with an Open Source, information-wants-to-be-free attitude: See Open You or search GitHub for quantified self for all kinds of related projects.
  • Fitness enthusiasts, runners and bikers are avid data trackers. They drive the personal data tracking market while often remaining blissfully unaware of the Quantified Self utopia: MapMyRide, Nike Fuelband, Fitbit, Snore Lab
  • For periodic on-the-run logging of any type of information: your.flowingdata and Daytum offer convenient logging with charts and data export, clearly aimed at data heads. Compare Daily Tracker, which is more aligned with the narrative of self-improvement.
  • Evernote is not strictly about data tracking, but it is a very convenient tool for it. Focused on cloud-based note-taking that bridges across mobile devices, your web-based services and even normal office software, it’s a blank slate for any number of uses.
  • Services built around calendars and to-do lists: Clocked In, Wunderlist, Basecamp. Not obviously related to data logging, but often relying the same mentality of self-discipline.
  • Life Hacker is a respected blog discussing strategies to help you stay productive and in charge of your own life / career / inbox. Data tracking is one such strategy, popular since it promises to provide quantifiable analysis.
  • Somewhat related: Getting Things Done (or GTD) is a bestseller book about time-managing, but it also has a large online following that is pretty much a subculture in its own right.
  • On a more personal and emotional note, mood tracking was an early hit. It is now increasingly being presented as a possible self-help tool against mood disorders: Emoo.me, Moodpanda.
  • Also: Mydrinkaware, Recovery etc. focus on battling addiction issues.

Projects / inspiration: So much to pick from, so little time. Google is your friend.

, , , , ,

Workshops Sept 8+9: Sound-responsive visuals / 3D Printing and Parametric modeling

Aug 30, 2012 | News, Processing / Java, Sound, Workshops

USound04Cylinder example from sound-responsive visuals workshop last weekend, see code below

I’ve just announced two more NYC workshops for the weekend of September 8+9:

It’s currently looking like my busy Fall schedule will mean that I’ll be doing less of these workshops over the next few months, so if you have any interest in taking one of them this might be a good time.

The first sound-responsive visuals workshop happened last weekend and was a lot of fun. Here are some of the key elements we looked at:

  • As it turns out, Minim’s FFT.logAverages() method (which divides the FFT into logarithmic averages) give a far more useful result than the raw spectrum data on their own. Using that as our starting point we built a FFT helper class to act as our source material.
  • Since our focus in looking at the sound data is to turn it into a useful parameter for driving visuals, I demonstrated a series of data modulating strategies that give us greater control over the sound input.
  • Adding temporal damping of the FFT data (interpolating between old and new values) allows us to control the rate of signal change, which is crucial to make the sound-driven animation match the perceived tempo of the sound space.
  • Finally, we used a simple envelope shaper (a 1D Bezier interpolation) to de-emphasize the lower part of the spectrum. Bass tends to be over-represented in the FFT data, so to get a better distribution we can simply tone down the low end of the FFT by multiplying each data point with a modifier dictated by the shaper function. Initially we also scaled up the top end, but that produced noise and artificially high values, so in the end we kept the shaper function within the [0..1] range.

See the attached Processing sketch for an example that we went through in detail. The libraries in the “libraries” folder must be copied to your Processing libraries folder before running.

Sample code: USound04Cylinder.pde

Download: USound04Cylinder.zip (includes Modelbuilder and ControlP5 0.5.4, requires Minim)

, , , , , , , ,