Showing posts with label Virginia. Show all posts
Showing posts with label Virginia. Show all posts

Sunday, June 27, 2010

Transparency and Visualization of City Data

Last year my city was considering closing one or two schools (HamptonRoads.com, WAVY.com). One of the justifications cited was decline in student enrollment. Though I can no longer find it anywhere online, one of the enrollment artifacts put forth was this image of a column chart from a (presumably) Excel spreadsheet.


From Flow

To look at this picture (of data) one would assume there was a HUGE drop in enrollment, something in the vicinity of 50-60%.

But a closer look at the values reveals quite a different quantitative result. In fact, the actual decline from the peak in 1994-95 to the valley in 2008-09 appears to be 5,600 students. Keep in mind, I'm doing my best to derive the actual number based on the Y-axis scale on the picture. The data weren't provided. To make matters more difficult, the column chart is shown in 3D. This undermines the utility of the visualization by making it hard to determine an actual value on the chart.

The actual difference is 6,824 students, from the high in 1994-95 to the projected low in 2012-13.

6,824 students is 18.1% of the peak (6,824/37,707). While significant, 18.1% is a far cry from what appears to be a 50-60% decline.

Fortunately, when I requested the data they were provided to me.

This is valuable for two reasons. First and foremost, I get the data, not an interpretation of the
data. Second, and more the point of this post, I can show what this chart should look like and what else the data have to say.

Notice the slope of the decline and the differences between yearly observations. Neither are so exaggerated as the first image.

The differences in view are the aspect ratio of the graph and the value of the Y-axis origin. In the first chart the aspect ratio of the picture is nearly square, or 1:1. The starting value of the Y-axis is 28000 (an arbitrary value not found anywhere in the data). In the second chart the Y-axis begins at zero (0) and the aspect ratio is closer to 3:1 (which is probably too wide).

So then, which image and which set of chart variables is correct?

The answer depends on the basis for the query...or what you are trying to say. Actually the answer has nothing to do with "what you are trying to say" since you/we shouldn't be trying to say anything except to represent quantitative values in visual form. The right thing is to understand the nature of the analysis and the decision trying to be made.

In this case, we can assume with confidence that the Y-axis has no business starting at any value other than zero. In this case what is being claimed, via what is shown, is a long term (since 1990) decline, not a year-to-year fluctuation. Over the 14 year span the decline is obviously not in the range of 50-60%, therefore the chart should no imply so. Otherwise, if year-to-year deltas are of interest it probably does not make sense to show a 14 year span.

By the way, the data for recent yearly decline are -216, -200, +30, -17 between the years 2006-2010. 200 students city wide is hardly justification for closing a school. (Our school has over 600 students.)

In the end, my city chose not to close a school. I think that was a wise choice. But I have to wonder what was going on in the minds of the decision makers. What analyses were they using? Did they, too, realize the truth in these numbers or was it some other (political) factor. Did we just get lucky?

When cities make claims based on data they should publish the data. When cities, or anyone, shows you a picture of data you should question the picture and demand politely request a reference to the data. At minimum, know what you are seeing.

Sunday, October 25, 2009

Mashing up Virginia Beach Police Data with Google Docs


Last Sunday, 10/18/2009, the editor of the neighborhood section of my local paper, The Virginian-Pilot, said that he was looking for ways to improve how he reports crime data. At the same time, I know that neighborhood crime is a hot topic in our community league meetings, casual get-togethers, and email lists.

Witness: hyperlocal supply and demand in Gov 2.0.

In fact, there's plenty my local paper can do to improve the reporting of [crime] data in our city. In the printed edition of the neighborhood section they only print the police "blotter" from the previous week. Online they use Google maps to report crimes by location, but interfaces are constrained and inconsistent.

What my paper should do to improve crime reporting is go to the source.

The Virginia Beach Police Department publishes crime data on a city website. Here's the URL:


Gnarly.

You can query for all crimes going back to January 1, 2006 (why only that far back?) up to the present day. But you can look at exactly 15 results at a time.





So what my local paper should do is appeal to the city to make [crime] data more accessible to everyone; where everyone includes The Virginian-Pilot. Short of that, the Pilot should take the energy it invests in supporting its Web presence and use it to scrape and publish city data. If only there were a "Machine Friendly" link next to the "Printer Friendly" link. (Both are designed to help people.)

Shortcomings notwithstanding, here's what Pilot reporters can do right now for free:

Compile and publish Google spreadsheets with charts:

http://spreadsheets.google.com/ccc?key=0AtBw9czGJjSTdGZGVzh3MzdzWEQ1SzJ6NTJDbl9taHc

http://spreadsheets.google.com/ccc?key=0AtBw9czGJjSTdFdlaEEwT2Y1RGotdGNWU2w4WDh3Nnc







Wow, who knew that 72% of all crime in Virginia Beach> occurs at the Oceanfront?

Update: HUGE, HUGE ERROR in my reporting of this stat. The pie chart above does not show all crime in Virginia Beach. It shows only the top 5 neighborhoods.

This chart, below, is the correct chart. It shows that, while still a standout piece of the pie, 4% of all crime happens at the Oceanfront.

I'm feeling really dense right now, but in a way I'm glad this happened. This is a perfect illustration of a fundamental in data viz (and reporting).

Saturday, October 10, 2009

Ten Things My City Can Do to Improve Our Website

  1. Publish events in iCal format
  2. Publish electronic police reports as XML
  3. Publish 311 data (as XML)
  4. Geocode public works projects
  5. Implement short, guessable URLs
  6. Use a free mapping service
  7. Update the Transportation Data Management System
  8. Add a "Web-Friendly" link next to "Printer-Friendly"
  9. Create a data catalog
  10. Get a .gov address
Details to be added here...