I’m currently writing a monster application doing indexing of feeds from blogs, del.icio.us, Flickr, Digg etc. It’s been a bit of a learning curve, but using the excellent and comprehensive Rome library has made it a great deal easier than it might otherwise have been. Rome handles Atom feeds as well as the various confusing RSS variants.

Making sense of feeds is often tricky, as different publishers can use various tags in different ways, or even add their own by introducing custom namespaces. The use of namespaces (which Rome supports through a plugin system) is a promising way of adding custom information, but not without its problems.

Having completed indexing of Flickr, del.icio.us and plain vanilla feeds, I turned to Digg feeds. Imagine my surprise to find out that not only is their advertised namespace (http://digg.com/docs/diggrss/) in fact offline, but they don’t even use it properly in their main RSS feeds.

http://digg.com/rss/containerscience.xml correctly specifies xmlns:digg=”http://digg.com/docs/diggrss/”, but http://digg.com/rss/index.xml specifies xmlns:digg=”docs/diggrss/”. And that just happens to be their main RSS feed. I know it seems niggling to complain, but it messes with the Module I’ve written to handle their tags and seems a tad careless. Oh well, guess I’ll just skip that feed.

More information about Rome, Digg etc:

• Why does Digg not follow standard RSS? (blog post)
• Rome Wiki
• ROME in a Day: Parse and Publish Feeds in Java, excellent article about Rome and introductory techniques over on XML.com
• del.icio.us/garuda/flowofthings is a collection of links I’ve collected, with plenty of feed and blog programming stuff for Java.

Stefan Kühn, a cartographer at the University Trier, Germany, has extracted all the geo coordinates embedded in articles on Wikipedia. The WikiProject Geographical coordinates is a Wikipedia project for ensuring standardized geocoding of locations in its articles.

Google Earth fans bent on instant gratification can simply download a KMZ file and start surfing. But more importantly, coders and infoviz geeks can get a comma-separated text file (CSV) with coordinates, titles and Wikipedia categories for all points.

Link: Geocoordinates from Wikipedia for Google Earth

Websites as graphs: Code & Form HTML structure

A while back I blogged Websites as Graphs on Generator.x. It’s a nice visualization of the structure of HTML documents. Since a well-formed HTML document has a logical hierarchy of tag containers, it is possible to visualize it as a strict graph. The results are both informative and beautiful, revealing the strategies used for structuring the document’s content. It will also reveal whether the document holds up to that most essential of post-CSS web principles: A tableless design.

The original post by Sala shows some examples, and also provides a live applet that you can try out on your own site. Be sure to have a look at all the pictures tagged 'websitesasgraphs' on Flickr.

Sala has generously provided the Processing source code for the application. It requires the HTMLParser, Traer.Animation and Traer Physics libraries to run. HTMLParser is a standard Java library and hence does not come with instructions for Processing. According to the project home page, all you should need to do is download the latest version of the library, and then copy the file htmlparser.jar from the /lib folder to your sketch’s “code” folder. I haven’t tested this, so if you find otherwise let me know.

Geotagged Flickr photos in Google Earth via Yuan.cc (maps.yuan.cc/kml.php)

Last year I picked up a cheap GPS unit just for fun, which I have since used to collect GPS traces that I have yet to use for anything useful. I find the whole idea of geotagging quite fascinating, but have never built any applications using it.

Currently I’m in Vienna as artist-in-residence at the MuseumsQuartier, and so I’ve been taking quite a few pictures and putting them on Flickr as documentation. By accident I have met the street artist Space Invader, who is currently here “invading” Vienna. He puts up space invader mosaics in public places, and then documents them in the form of a map. This activity mixes a lot of interesting topics: Urban space, street art, locative media, psychogeography etc. So of course I thought this was a perfect chance to put geotagging to the test.

So far my activities have been those of an end-user rather than a developer. I put pictures on Flickr and geotag them with this excellent bookmarklet. Almost all my Vienna pictures are geotagged, for examples see Invader #1 and Invader #2.

Once tagged, it would be useful to be able to browse these images in a geographical intergace. Yuan.cc is a home-brewed site that allows you to sign in to your Flickr account and see your geotagged pictures on Google Maps. If you “sync” your pictures they get added to the Yuan.cc database, in which case you can browse them with Google Earth. You will also be able to see all other pictures in the database at a given location, with thumbnails and links back to Flickr.

During my residency I hope to be able to document all the Invaders that get put up in Vienna, complete with geotags.

Relevant links:

• Flickr Geotagging discussion group
• Localize this bookmarklet for Flickr
• Yuan.cc
• Wikipedia: Geotagging

Daniel Dihardja has just released a new Processing library called jm-Etude. It is a wrapper for the Jmusic Java library, which provides a framework for musical composition. In the Processing tradition, jm-Etude simplifies this further, setting up a structure where “scores”, “parts” and “phrases” together make up compositions in the form of “Etudes”. The library uses notes as the molecular unit, and MIDI is used for sound playback.

jm-Etude should be useful for a classically structured approach to music, but also to create realtime generative compositions. It nicely fills gaps left by Ess and Sonia, which focus on playing audio files or simpler waveform synthesis. Hopefully Dihardja will add examples showing how to use other sounds than the built-in MIDI instruments, as they sound notoriously cheesy.

See the Jmusic javadoc for an impression of what the underlying library can do.

Update: Daniel has explained a little about future plans for custom instruments:

[...] jm-Etude is not heavy MIDI based, it is just that for now the only playback function which is implemented is the jMusic midi playback. In jMusic the music data can be played and exported as MIDI or AUDIO and there are also libraries to build and use custom instruments, but I haven’t implemented them in the wrapper yet, because I’m still working on the MIDI export which will be a feature in the next update (is still a bit buggy now).

Robert Hodgins just posted on the Processing forums asking about techniques for mapping colors in an image to a pre-specified palette. This process is known as color quantization, and is what any image software does when it saves a GIF. It consists in mapping a potentially very large number of unique colors in a RGB image to a limited number of colors (for GIF, typically some multiple of 2, i.e. 64, 128, 256 etc.)

I had to find a way to do quantization it for a project where I needed to output GIFs several years ago. At the time I tried some code for NeuQuant neural net quantization, which didn’t work for me. However, Mario Klingemann posted a NeuQuant Processing example (with code) a while back which does it successfully.

The solution I ended up using was the Java Imaging Utilities (JIU) library, an Open Source library written by Marco Schmidt. It has a number of quantization implementations, such as ArbitraryPaletteQuantizer, OctreeColorQuantizer and PopularityQuantizer. The only drawback is that JIU uses an internal custom RGBImage class, rather than the standard java.awt.BufferedImage. For use with Processing, one will need to convert the image before quantizing it.

Nodebox tool

I just posted an entry about NodeBox over on Generator.x. NodeBox is a Python-based coding environment that specializes in producing vector graphics, with very high quality results. Like Processing’s use of Java, NodeBox uses a simplified syntax but has access to the full power of Python for advanced programmers. It also has some interesting libraries, like the Pixie handwriting library or the PhotoBot image processing library.

Unfortunately, NodeBox is only for Mac OS, but the source is available under a MIT license. It looks like a fun alternative to Processing with a slightly different bias on what kind of work it is useful for. Have a look at the Gallery for some examples.