De Blasio Administration Releases Annual Update to Open Data Plan

July 15, 2016

Following last year’s Open Data for All – aimed at increasing community partnership and engagement with City data – MODA and DoITT release the annual Open Data Plan Update

NEW YORK—The Mayor’s Office of Data Analytics (MODA) and the Department of Information Technology and Telecommunications (DoITT) today released the annual update to the NYC Open Data Plan. Last year, the City rolled out Open Data for All – a more inclusive vision for Open Data aimed at increasing community engagement and gathering more feedback from New Yorkers on making open data more accessible and user-friendly.

In the last year, the Open Data Portal was accessed over 5 million times. There are now more than 1,500 data sets on the portal, including NYPD’s major felonies data set and NYC budget data. MODA and DoITT met with faces both familiar and new to the open data conversation, gathering feedback to help set priorities and make the City’s data more useful to more New Yorkers.

“Our administration’s work on open data is guided by complimentary notions that every New Yorker can benefit from open data, and that open data can benefit from the input of every New Yorker,” said Mayor Bill de Blasio. “We are determined to make the Open Data Portal exactly that – an incredible tool and resource that is both accessible and useful for all New Yorkers.”

“Open Data is increasingly the metric by which a city’s transparency is measured,” said DoITT Commissioner Anne Roest. “New York City’s Open Data Law is the most comprehensive legislation of its kind in the country. As we continue to mature and expand open data efforts, we welcome continued public engagement and constructive feedback on how to improve our efforts collectively as a City. We are committed to using open data to better inform citizens, improve service delivery and increase data sharing between City agencies.”

“Open Data lies at the nexus of digital services, government accountability, and cutting-edge civic analytics – and we are constantly aiming higher,” said Dr. Amen Ra Mashariki, Chief Analytics Officer and Chief Open Platform Officer. “Since the release of Open Data for All last year, we’ve talked to New Yorkers across the city and tested new ideas to improve the platform for Open Data veterans and newcomers alike.”

“Open Data is a critical part of our commitment to government transparency and high quality performance management,” said Mindy Tarlow, Director of the Mayor’s Office of Operations. “Open Data for All affirms our continued focus on operational excellence and delivering services equitably to all New Yorkers.”

The 2016 NYC Open Data Plan, including the full list of data sets scheduled for release, can be accessed here or through the interactive dashboard on the NYC Open Data Portal. The full Open Data for All 2016 Progress Report can be accessed here.

Below are some of the accomplishments and progress the de Blasio administration has made on open data initiatives across City agencies.

PORTAL USAGE

New Yorkers interact with Open Data on a variety of channels – dashboards on agency websites, articles written by data journalists, apps that use our APIs – but at the core of the City’s Open Data ecosystem is the Open Data Portal. Since our last update, traffic on the Open Data Portal has more than doubled. In the last year, the Portal had:

• Over 5 million hits. The Department of Building’s Job Applications Filings was the most viewed data set.
• More than 2,000 user-created views based on City data sets.
• More than 150 new data sets. This brings the total to over 1,500 data sets.

NOTABLE DATA RELEASES

NYPD Seven Major Felonies data: In 2015, the New York Police Department published incident-level data on the major felonies committed in the city that year. Due to significant public interest, NYPD has since expanded the data to include records dating back to 2006. The data set features an interactive map that makes incidents searchable by category, location, timeframe, and other variables, and has been used by NYPD to ease the processes of initiating external research partnerships and sharing data across City agencies.

311 Call Center Inquiry data: 311 is often the first point of contact the public has with City services. Though the phone is just one of many ways New Yorkers interact with 311, call center agents answered over 7 million calls this year. Building on lessons learned from the automation of 311 service request data, 311 officials joined BetaNYC at Civic Hall this spring to announce the public release of all data on agent-answered calls dating back to 2011.

TLC Trip Record data: Last August, the Taxi and Limousine Commission (TLC) published records on every ride taken in green and yellows taxis between January 2009 and June 2015. Previously, the Commission would respond to over 75 annual requests for information on green and yellow taxi trips by laboriously loading millions of data records onto physical hard drives. Putting this data on the Portal has cut down on the time and resources it takes to fill these one-off requests, while also spurring overall usage. In addition, TLC’s datasets of active drivers and active vehicles – which include data on all For Hire Vehicles, including livery, limousine, and app-based services like Uber, in addition to taxis – are now among the most popular on the Portal.

Searchable Open Budget: In May 2016, New York City’s $82 billion budget was made available on the Open Data Portal. What previously existed only in print or as lengthy PDFs is now machine readable and searchable. Now taxpayers can easily see the breakdown of how their money is being spent.

City Record Online: Maintained by the Department of Citywide Administrative Services (DCAS), the City Record is the official newspaper for all NYC municipal business. Last August, DCAS unveiled the newly expanded City Record Online (CROL), which makes all notices in the newspaper fully searchable not only as an Open Data dataset, but also through a new online navigation tool that makes it easier than ever to access information about public hearings, personnel changes, and opportunities to contract with the City.

NYCHA Energy Data: The New York City Housing Authority (NYCHA) is voluntarily joining the Open Data movement and leveraging the Open Data Portal for its first energy data public release. As part of its NextGeneration NYCHA Sustainability Agenda published in April 2016, NYCHA has committed to releasing energy consumption and cost data on all of its buildings.

Shelter Repair Scorecard:  As part of the effort to improve conditions in homeless shelters, Mayor de Blasio created the Shelter Repair Scorecard to report publicly on the conditions of homeless shelter facilities and track progress made by the expanded repair program to address sub-standard conditions. Data shows that increased inspections have been finding more violations than ever before, and that City and shelter providers have cleared more than 28,000 violations over the last two years. This data is shared both on the Mayor’s Office of Operations website and on the Open Data Portal.

EVENTS

Citywide Engagement Tour: MODA met with New Yorkers throughout the city to spread the word to those unfamiliar with Open Data. Their thoughts and questions provided valuable feedback; this fall, MODA staff will begin holding in-person office hours to continue the conversation.

NYC School of Data: On March 5, 2016, as part of International Open Data Day and Code for America’s Code Across America, elected officials and civic tech groups convened at Civic Hall to celebrate and discuss the future of Open Data. MODA gave a presentation on the way Open Data powers analytics projects and supports information sharing across the city. In addition, Manhattan Borough President Gale Brewer announced the formation of a task force with a mandate to explore digital literacy and data science for high school students across the city.

NYC Parks TreesCount! 2015: In 2015, NYC Parks launched a participatory mapping campaign to better understand our urban forest, collecting a citywide inventory of street trees in New York City. Parks worked with more than 2,300 volunteers and more than 60 local neighborhood and community-based organizations to conduct the TreesCount! street tree census, enabling everyday New Yorkers to create Open Data for the city. On June 4, 2016, Parks teamed with MODA, the Mayor’s Office of Technology and Innovation (MOTI), and BetaNYC to host the “TreesCount! Data Jam” where the community groups that collected data joined Parks to present five of their most pressing challenges to civic hackers. The event also included a workshop for Parks’ volunteers and Open Data newcomers to develop skills to analyze TreesCount! data themselves – linking data stewardship with Parks’ long history of tree stewardship.

PARTNERSHIPS

NYU CUSP Capstone: MODA is partnering with the academic community to examine where Open Data is reaching New Yorkers… and where it’s not. A capstone at the NYU Center for Urban Science + Progress (CUSP) is measuring “data poverty” across the city. Data poverty refers to a condition in which communities or people lack access to, use of, or representation within data that is nevertheless used to inform decisions that may affect them. This analysis, the first of its kind for any city, will be released publicly later this summer.

Columbia SIPA Capstone: MODA also worked with the Columbia University School of International & Public Affairs (SIPA) to assess existing and future opportunities for Open Data to advance the goals of the city’s many Community-Based Organizations (CBOs). Although municipal data is in high demand, CBOs pointed to several pain points in accessing and operationalizing Open Data. These results are informing conversations on ways to improve user experience on the Portal. Going forward, more data sets will include data dictionaries defining the attributes (and clarifying the usability) of their contents.

neighborhoods.nyc: Last fall, MOTI partnered with the local startup Vizalytics to unveil neighborhoods.nyc. With nearly 400 neighborhood-specific hubs, this platform centralizes some of the most in-demand information on Open Data – including alternate side parking information and 311 requests – in a web template available for licensing by local community groups.
 
OPEN GOVERNMENT LEGISLATION

Over the last year, Mayor de Blasio signed a package of City Council bills that bolster Local Law 11 of 2012 – the Open Data Law – with requirements to make it easier for New Yorkers to access City data online. These laws, which include stronger requirements on data dictionaries and data retention, response timelines for public requests, and timely updates for certain datasets help to anchor the City’s many transparency initiatives around Open Data.

"I'm pleased we're making progress on fully implementing the Open Data Law," said Manhattan Borough President Gale A. Brewer, the author of the New York City Open Data Law. "Every public dataset published creates new opportunities for this data to be analyzed and put to use for the public interest."

“Open Data for All supplies raw material for crowdsourcing, the assembly line of the 21st century,” said Brooklyn Borough President Eric L. Adams. “These datasets allow us to solve civic challenges that once seemed impossible to overcome, as when I was a police captain implementing CompStat to reduce crime in New York City. As the City continues to build on its open data program, I am hopeful that other agencies will follow the approach we have introduced at Brooklyn Borough Hall, incorporating smart city technology into the building itself to generate useful data that can improve our operations.”

“Open Data has been an amazing asset since its inception and is becoming better with every addition,” said Council Member James Vacca, Chair of the New York City Committee on Technology.  “Last year I passed a seven-bill Open Data package through my New York City Council Technology Committee, designed to make the nation’s best Open Data Portal even stronger. I will continue to look at legislative avenues to expand and enhance the Open Data Portal.  I’m glad that the de Blasio administration has made Open Data a priority and look forward to a continued partnership.”

"With the NYC Open Data Plan, the De Blasio Administration has shown leadership in making city government transparent for residents and anyone who requires public information," said Council Member Ben Kallos. "I look forward to working with the administration to put even more information into Open Data, and finding ways to make that data easier to use, in the coming years."

"New York City continues to make admirable progress opening up data that illuminates issues that are important to the public. NYC is a leader in automatically updating data, and the City has rolled up its sleeves to take on challenging problems with data quality. Truly opening government data is a revolutionary idea, and doing it isn't easy. But we are glad to see that New York City is forging ahead,” said John Kaehny, co-chair of the NYC Transparency Working Group.

"The past year has been a watershed moment for NYC's open data," said Noel Hidalgo, Executive Director of BetaNYC. "Since the launch of Open Data for All initiative, NYC civic technology and open data community has been honored to help our neighbors and the City demystify, explore, and empower people with data. We are excited to co-develop the next steps and build 21st century municipal data policies and programs for all."

About the NYC Open Data Plan

As required by Local Law 11 of 2012, each City entity must identify and ultimately publish all of its digital public data for citywide aggregation and publication by 2018. Every year on July 15, the NYC Open Data Plan provides an update of the City’s progress by listing pertinent City-managed public data sets yet to be published along with their anticipated publication dates.