Where New Yorkers Don’t Vote
Where are people not voting in New York City municipal elections?
In other words, of the people who could vote (citizens age 18 and above), what’s the rate of those who actually cast a ballot? We combined demographic data from the American Community Survey and voter history data from the New York City Board of Elections to color in census tracts by their level of voter participation.
Here’s how we did it:
Who Voted in 2009?
For this map, we looked at voters in the 2009 municipal election (primary, runoff and/or general election). Although turnout in 2009 was relatively low, it remains the most up-to-date look at where municipal election voters were living in New York City. The Board of Elections has publicly available voter roll and voter history data which includes the address of each voter and which election they voted in, going back to the 1980s. To calculate the number of people who voted in each census tract, we geocoded and mapped the address of all voters in the 2009 election. After mapping the 2009 voters, we sorted them into census tracts by using the “points in polygon” function of an open-source mapping program called Quantum GIS.
Who is Eligible to Vote?
Using the most recent census tract-level demographic data (2011 American Community Survey 5-year estimates), we found the total number of citizens age 18 or older for each census tract in New York City.
Because of the nature of the ACS data, we are unable to account for people that are in prison or on parole for a felony conviction, or adjudged mentally incompetent by a court, who do not have the right to vote. In addition to this, our map does not capture the moving that some residents have done since the 2009 election or the 2011 ACS survey. Despite these limitations, we feel that the map gives a clear overview of voter engagement in municipal elections.
How to Register
With the August 16 voter registration deadline quickly approaching, this map is a reminder that there are still many eligible voters in our city that are not yet voting. Here’s our guide for registering to vote in New York and looking up your polling location.
— Jenny Ye
4 Million Trips
That’s what’s under the hood for the Transit Time NYC map we launched today — showing subway travel times from any spot in NYC to the rest of the city.
The map was built with a lot of hard, smart work by Data News developer Steve Melendez to go with a story by Jim O’Grady, In Bloomberg’s New York, The Poor Move Further Out, Lengthening Commutes.
We were inspired by Triptrop NYC, which I admired even before meeting — and admiring and working with — its creator, Jonathan Soma. Triptrop is a few years old now, and both web mapping and the subway system have changed. So, with Soma’s blessing, we set out to make a new version.
We started with an overlay of New York City made up of 2,930 hexagons (1).
Melendez then installed a trip-planning program called OpenTripPlan onto four virtual computers in the Amazon cloud. Using the MTA’s schedule data, he had the four copies of OpenTripPlan calculate the subway and walking time from the center of each hexagon to the center of every other — which comes to 4,290,985 trips (2).
It took the four computers almost a week.
When they were done, Melendez had a tiny data file for each hexagon (like this) containing the times to every other hexagon. When you click on the map, or enter an address, the program quickly downloads the corresponding hexagon’s file and uses that info to color all of the tiles. (Here’s the 33MB zip file with every hexagon’s data as json).
Have thoughts or questions? Tweet us @datanews and we’ll reply asap.
— John Keefe
(2) 2,930 times 2,929 divided by two. We decided to halve our computing time by assuming the travel time between two points was roughly the same in one direction as it is in the other. Travel times were calculated for 9 a.m. on a weekday.
Predicting Questions, Building Answers
With hurricane Sandy churning far off the Florida coast, we began anticipating questions people would have around the storm. And then we tried to code answers to those questions as fast as we could.
In order, those questions turned out to be:
Where’s the storm forecast to go? For this, we dusted off our Hurricane Tracker, built for hurricane Irene, and fed it with the National Weather Service data for Sandy. As with almost all of our work, we made it free and easy to embed, and many news outlets did.
What zone am I in? Again, we dusted off something made for Irene — our NYC Evacuation Zone Map. We updated it with better colors and areas newly designated as Zone A. We also published a project I’d been working on since Irene: a Storm Surge Map of the entire New York and New Jersey coastline.
Where’s the storm now? As the storm approached land, we switched layers in the hurricane tracker from the forecast track to a real-time radar image.
What systems are closing, and when? We knew there was a good chance the subways and other transit services would be shut down ahead of the storm. We also knew that there was no single place where all of that information resided. So days before the storm, we built our Transit Tracker. Thanks to Steve Melendez coding it over the weekend, we had it running when officials announced the transit shutdown plans. As an added bonus, the tracker is fed by a Google spreadsheet, so multiple producers can update it simultaneously.
Where’s the water rising? With everything shuttered, we wanted to help people watch the storm’s effects in real time. The National Weather Service maintains a network of flood-level monitors on the coast and on inland rivers. We took a feed off that system and modified it slightly to show pop-up charts for our Flood Gauge Watch — which we monitored through the storm.
As the storm hit, Melendez and I worked at WNYC under backup power making minor fixes and trying to catch up on our election-night mapping project.
After the storm, the team tackled two more questions:
What’s the traffic like? We heard traffic was gridlocked as people returned to work Wednesday, so we resurfaced our Traffic Map to show the trouble spots — which included most of Manhattan.
Which subway lines are open? Anticipating the subway system would open in stages, we wanted give people a map as the restoration progressed. I guessed that the MTA would provide a list of partially-open lines before they had maps, so Louise Ma built a beautiful base map from files left over from our Lost Subways project and prepared to update it. Melendez found a clever way to let people to pan and zoom it like a Google map. But I was wrong: The MTA issued clear PDF maps right away, so we scrapped Ma’s map and fed the official version into Melendez’s Changing Trains app.
There’s a lot more we wished we could have done — and could have done better. That may be the subject of another post. But we hope we what we made provided answers when they were needed.
- John Keefe
Mapping NYC’s Republicans
As the Republican National Convention kicked off in August, we put up this map locating the concentrations of Republicans in NYC, a notoriously Democratic stronghold:
At the time, we promised to post the data we used to make the map. Then we got swamped!
But here’s the shapefile with the data embedded. As the map above says, it’s based on voter rolls and district lines as of April 2011, and filters for people who’ve voted since 2004. That’s because an “active” voter is generally considered someone who has voted over the course of the last two presidential elections.
- By John Keefe
Sharing Our NYPD Precinct Data
By John Keefe
No sense keeping good data to yourself.
The map below went with these excellent WNYC stories about low-level marijuana arrests in New York City. After building it, I ended up with some data files that could be useful to others crunching population data by NYPD precinct. So we’re sharing them here.
The trick to doing this analysis was to determine the populations of each precinct. But the US Census Bureau doesn’t break down numbers that way. So I took the smallest Census unit — the block — and determined which blocks were in which precincts.
Data For You
Each of the following files are kept on Google Fusion Tables. You can use them there, or download them to your computer using File -> Export.
• precinct_block_key.csv is the Rosetta Stone for this project. It has two columns: each block’s identifier, which the Census calls “GEOID10,” and the precinct in which that block sits. Note that some blocks aren’t in any precinct, usually because they’re actually in the water.
• NYC_Blocks_2010CensusData_Plus_Precincts contains base-level 2010 Census data for each block, married to the precinct for that block. A nice Fusion Tables trick is to pick View -> Aggregate, check “Sum” for the columns you want and then, lower down, chose to aggregate by precinct. Then you get totals for each precinct. For descriptions of the population columns, get this rather large PDF from the Census Bureau and jump to page 6-21 (Updated: Or, go to the page online with DocumentCloud.)
• NYC_Police_Precinct_Shapes_4326 is the official police precinct map converted into a Google Map-friendly projection. I’ve used the fantastic tool shpescape.com to upload my transformed shape file to Fusion Tables, where it’s easy to play with.
I’ve done my best to be accurate in computing the intersection of blocks and precincts, even generating precinct maps and inspecting them visually. But errors may exist.
In fact, they do exist. While Census blocks generally fall nicely within precinct outlines, they don’t always. In particular, three blocks significantly straddle two precincts. If you’re doing very precise analysis, you’ll want to account for them:
• Block 360470071002003: An area near the north end of the Gowanus Canal in Brooklyn. About half is in Precinct 76 and half in Precinct78. Total people: 51
• Block 360050096002000: Mainly industrial. Half in Precinct 76, half in Precinct 78. Total people: 5.
• Block 360610265003001: This block consists of five similar-sized apartment buildings near the GW Bridge. The northern three buildings are in Precinct 34, and the southern two are in Precinct 33. Looks like roughly a 60/40 split of the 687 living there.
If you find this information useful, drop me a note or post a comment below. We’d love to know about it.
Making the WNYC Census Map
By John Keefe
When the New York census numbers arrived this week, we were ready. WNYC quickly published an interactive, sharable map so New Yorkers — and our reporters — could explore the new data and see the stories.
We built the map with free tools and timely help from some smart, kind people.
The long story is meandering and full of wrong turns. But here are the highlights, should anyone need a little navigation. Don’t hesitate to contact me for more help and insight; I’m due to pay some forward.
Getting in Shape
First up: Shapes of the census tracts plotted on Earth. I downloaded New York’s tracts from the U.S. Census Bureau’s TIGER/Line Shapefile page. They also have counties, blocks, zip codes, and more.
Then I uploaded this “shapefile” — actually collection of related files zipped together — to Fusion Tables with a free, online tool called Shpescape. (Thanks to Google’s Rebecca Shapley for sharing this key to my puzzle.)
Census data is publicly available, but can be a hassle to handle. In fact, on the day each state’s info was released, the files were available in a set that apparently requires one of two pricey programs — SAS or Microsoft Access — to assemble.
So I got clean, assembled, comma-delimited files — complete with 2000-to-2010 comparisons — from the USA Today census team, which provided them as a courtesy to members of Investigative Reporters and Editors. Huge props to Anthony DeBarros and Paul Overberg, who crunched the New York numbers in a blazing 30 minutes.
By the way, IRE membership is $60 for professionals and $25 for students. Well worth it, and cheaper than either of those programs. If you’re digging into census numbers and qualify, I recommend this route.
That said, every state’s 2010 data is now available free from the Census Bureau’s American Fact Finder. Navigating the site is a little tricky, and worth a separate post, but the bureau provides some tutorials, and there’s very detailed PDF about each data field.
With data in hand, I uploaded it to Google Fusion Tables in another table.
Next, I merged the shapes table and population table, using the unique tract ID to marry the data (the shape file calls it GEOID10. the IRE data calls it FULLTRACT). Note that the GEOID10 is formatted differently depending on whether you’re using tracts, blocks, counties, etc., so be sure you’ve got the right match in both files.
Clicking Visualize -> Map shows a map. It’ll be all default-red until you click on Configure Styles -> Fill Colors -> Gradient (or ->Buckets) and make different colors appear depending on values in the column of your choice.
Using the Share button makes the map viewable by others, and “Get embeddable link” does just that.
I used the Fusion Tables “Configure info window” option to make custom pop-up bubbles on our maps. This actually required some nicer-looking data, such as a columns with rounded percentages and + or – signs. I added these using the free R statistical program, which I learned how to do from The New York Times’ Amanda Cox at the 2011 Computer Assisted Reporting conference.)
Census tracts officially extend to the state lines, which made it look like a lot of people live in the Hudson River. So I had trimmed those tracts to the shorelines with a free mapping program called QGIS, using water shapefiles as a reference (those are here, in the drop-down menu).
Clearly, not all of this could happen in a couple of hours on Data Day. I’d been tinkering, testing and tweaking for a few weeks using New Jersey’s data, which came out much earlier.
I also wrote down, edited and revised every step I took to make the maps. So when the adrenaline was running I had a script to follow.
The WNYC Web Team also set up a slick, fresh project server, at project.wnyc.org, to host the html pages and track the traffic.
Using Google Fusion Tables made it super easy to manage, map and serve up a lot of data. And the FT feedback team was fantastic about responding to questions and glitches I encountered along the way.
I did run into a couple of hiccups: slow load times and pop-up bubbles that failed to pop up. The first was a product of displaying so much data — and I knew I was pushing things. The second was a Google glitch that their engineers managed to fix within a few hours, but was still spotty at times afterward.
Also, the Google Map engine starts dropping shapes when there are too many to show. So I funneled different counties’ data into almost a dozen different layers, a workaround the Google folks showed me ahead of time.
That said, I had time to code and tweak lots of neat things because I didn’t have to focus on building or running a database engine. Google’s free services took care of that.
What Could Be Better?
Probably a lot. I wanted to let people to add comments, right on the map, but didn’t have the chops or time to pull that off.
Another good thing would be a “Loading …” indicator displayed while the map data is pulled into your browser, which I may yet add.
But what couldn’t have been better was everything I learned, the help I got from other data folks and the support from my WNYC colleagues. Plus we gave New Yorkers a pretty nifty service and several great stories.
Need more details? Feel free to ask questions in the comments. Or drop us a line. We’ll try to help, too.