Creating Data Visualizations for the CBC
I recently started using Processing (processing.org) at the CBC to visualize the dependencies of the content areas on projects being built by Media Ops & Technology (MO&T). Roughly speaking, MO&T builds out platform related projects and the content areas leverage the functionality of those projects to build out their sites. The previous post here was a first sketch in chalk of how a tool to visualize these dependencies might end up looking. Turns out I wasn’t too far off.
These screenshots below show the hightlighted and un-highlighted views. What isn’t shown is the interactive component that enables users to pick which content areas they want to see and then the ability to select individual projects and highlight their dependencies.
The next visualization focused on displaying the content area’s over all dependency on each Media Ops project and then normalizing the relative impact based on audience or revenue. Below is the baseline view before audience and revenue adjustments are applied. One of the benefits of this view is that it makes it easy to see where the content areas aligned in their dependence on different Media Ops projects.
There are more projects underway to help inform discussions about where and how we expend our resources.
A big thanks goes out to everyone involved in the Processing project!
tnoc_Attractor_Duotone_vM500 from sspboyd on Vimeo.
Coloured Boxes Image Sampling 10 from sspboyd on Vimeo.
This goes with the image from the previous post.
Video from a Processing.org sketch that reads the colours in an image and represents them as rectangles. The width is based on the brightness levels.
Choosing Descriptive Variable Names: Code !is English
When starting a new project I often find one of the hardest parts is choosing accurate, descriptive variable names for the objects and variables I’m working with. I’m sure this isn’t a new observation but it is striking me now as I try to sort through the best way to structure a Class in Java that generates a histogram of some colour data in an array (PImage ints in Processing).
This has nothing really to do with programming but with finding the right words in english to describe what I think I’m trying to do as I work through the code. My sense is that the trouble stems from the fact I’m trying to use one structured communication mode (english) to describe what’s going on in another structured communication mode, Processing/Java code. Not surprisingly, there isn’t a 1:1 relationship there.
I’m no where near competent enough to understand 99% of what Donald Knuth writes, but this quote in “Literate Programming” seems to speak to this difficulty:
The practitioner of literate programming can be regarded as an essayist, whose main concern is with exposition and excellence of style. Such an author, with thesaurus in hand, chooses the names of variables carefully and explains what each variable means.
Edit: Thanks to Derek Watson for sending me the the exact quote I was looking for:
“There are only two hard things in Computer Science: cache invalidation and naming things” - Phil Karlton
Data Visualizations with Processing, TSU prints
This image,Tsu-20041226-005853UTC, was created shortly after the Christmas 2004 tsunami in the Indian Ocean. It was made using satellite images and population density information for the affected countries in the Indian Ocean. I wrote a program in Processing to read this data and generate the black and white representation below.
Lines representing people who died from the Tsunami are drawn from the epicentre to the earthquake to the shorelines where the tsunami landed. At the point where the inital line reaches the shoreline, a semi-circle is drawn upwards. You can make out the coastlines of Thailand and Indonesia are visible around the middle right area of the image (especially if you look at the larger size).
While this print is created by visualizing data, it is not a data vizualization in the sense that the term is currently used. It is meant to be aesthetically pleasing while still being thought provoking around a very tragic event. The print was created for a charity auction and I’m happy to report it sold very well.
This image and the Tsu Ean print (below) were created using Processing v74.
musLnsClrBox-30-167926-18588
18147 tweets about the Toronto Mayoral election
Toronto Mayoral Election: @RobFordTeam vs @G_Smitherman vs @JPantalone
- Tweets with @RobFordTeam are in red.
- Tweets with @G_Smitherman are in purple.
- Tweets with @JPantalone are in teal
- Tweets mentioning two or more are multicoloured.
- Tweets mentioning none of them are transparent.
Tweets were gathered from searching via the api for:
#voteTO OR #RobFord OR #Smitherman OR #Pantalone OR “Rob Ford” OR “George Smitherman” OR “Joe Pantalone” OR @g_smitherman OR @robfordteam OR @jpantalone OR from:robfordteam OR from:g_smitherman OR from:jpantalone OR to:robfordteam OR to:g_smitherman OR to:jpantalone since:2010-10-24 until:2010-10-25
First version. Just highlighted Ford and Smitherman in red and blue.
Toronto Mayoral Election: Rob Ford vs George Smitherman
- Tweets with @RobFordTeam are in red.
- Tweets with @G_Smitherman are in blue.
- Tweets mentioning both are in black.
Processing integration with Eclipse
Great news about a Processing plugin for Eclipse. While I love the minimalism and simplicity of the Processing IDE, larger projects start to stretch its capabilities. Quoted below from Ben Fry’s blog, ”Processing + Eclipse”.
“Exciting news! The short story is that there’s a new Processing Plug-in for Eclipse, and you can learn about it here.
…we settled on the Eclipse RCP as the platform, with the task of first making a plug-in that works in the Eclipse environment (everything in Eclipse is a plug-in), which could then eventually become its own standalone editor to replace the current PDE.”
Studio Sketchpad : an open studio for creating beautiful code
“Studio SketchPad was created by Ari Bader-Natal to bring the power of Etherpad to the Processing community.
Studio SketchPad aims to be a open studio for individuals learning to sketch beautiful code on the web canvas.
That’s pretty vague. How about something more concrete…
Studio SketchPad was named after the two web technologies on which the application is built: the Sketch fromProcessing and the Pad from EtherPad. (The Studio metaphor used throughout the site was inspired by the writings of John Seely Brown, which I’ll save for another post.) EtherPad is used as a lightweight development environment for authoring (or co-authoring) small visual programs written in the Processing language.”
City Lights
“A visualization of Thom Yorke’s song Harrowdown Hill using an image of New Yorks skyline. The Processing sketch analyses the song and distributes frequency bands to separate buildings in the photo. When a beat is detected in that frequency range, the lights of the building will turn on. If no beat is detected, the building will stay dark. Every time I ride my bike across Williamsburg bridge in the evening, the view reminds me of Thom Yorke’s album Eraser. This sketch was my way to illustrate this atmosphere.” Rune Madsen
Tweets to TSV
I created a nice Twitter to TSV (tab separated values) program this weekend using Processing and Twitter4J. The challenge of working within the Twitter API rate limiting and parsing out the data turned out to be more interesting than the original idea I wanted to work on.
What’s great is that I now have a simple way to be able to create a large dataset of tweets for any search criteria I want to work on. My last test grabbed over 48000 posts tagged with #ukvote.
Currently it outputs to a tsv file but that could be easily changed to talk to an SQL database as well if that made sense.
TreeMap Progress
The following two images show the changes to the TreeMapping project as I refine the code.
The grid of smaller images below shows the different stages of the image as I add colour coding for each team’s players (red and blue) and green/black for hockey specific terms.
The hockey specific terms are somewhat subjective, for example, I specified ‘period’ as being a term that has specific mean when used in a hockey story. I have used 183 terms which are listed at the bottom of this post. I’m sure there’s room to add a few terms and some that could be removed, however the point of this project is to play with TreeMaps, not hockey terminology (but hockey makes it more fun!).
Click to see the image on Flickr
Click to see the image on Flickr
The next steps for this project might be to create a version with roll-over states.
The following is the list of hockey terminology:
- penalty
- shot
- skate
- goal
- goalie
- assist
- plus
- minus
- coach
- player
- players
- line
- roster
- play
- puck
- points
- point
- advantage
- backcheck
- backchecking
- blocker
- boarding
- body
- check
- breakaway
- butterfly
- cage
- centre
- cherry
- coast
- coincidental
- penalties
- net
- cross-checking
- dangle
- defence
- offense
- first
- second
- third
- grinder
- fourth
- deke
- delay
- game
- offside
- dirty
- player
- pass
- dump
- chase
- faceoff
- face
- wash
- fan
- five
- four
- three
- five-hole
- freeze
- freezing
- garbage
- gate
- goon
- grinder
- hash
- hat-trick
- stick
- advantage
- hooking
- hip
- icing
- left
- wing
- winger
- right
- rush
- timer
- beard
- box
- poke
- whistle
- ref
- referee
- post
- pipe
- roughing
- block
- blocked
- saucer
- screen
- screened
- shorthand
- shorthanded
- shutout
- shutdown
- slapshot
- wrist
- slashing
- slewfoot
- slot
- snap
- snapper
- pads
- stickhandling
- stoned
- trap
- zone
- zamboni
- kill
- killing
- period
- Game
- score
- scored
- regular
- season
- playoff
- playoffs
- overtime
- death
- one
- two
- three
- four
- five
- six
- seven
- tie
- save
- lead
- victory
- goals
- assists
- series
- turnovers
- defenseman
- Stanley
- Cup
- pad
- scrum
- team
- trailing
- trail
- shots
- crowd
- video
- opener
- killers
- replay
- crease
- NHL
- hockey
- linemate
- scorers
- misconduct
- NHL’s
- rebound
- contact
- pond
- fans
- supporters
- netter
- career
- games
- sweater
- scoring
- teammates
- horn
- Arena
- arena
- jersey
- boards
- defenceman
- ice
- bench
- dive
- minor
- major
- assistant
- captian
- hat
- trick
I created a TreeMap from the words in CBC Sports stories written for each Habs / Caps game in Round 1. The TreeMap graph shows the most commonly used words. I removed the 15 most commonly used non-‘hockey’ specific words such as “the”, “it”, “a” etc…
There’s a few player names that pop up, Halak and Ovechkin for obvious reasons. This is a function of their popularity and the length of their names’ relative to the size of the box.
Here’s the page with links to the stories:
http://www.cbc.ca/sports/hockey/stanleycup/round1/e1vs8.html
Created with Processing and the Wattenberg/HCIL TreeMap lib for Java
