SUMMER PROJECT: DISTANCE-RELATED VARIABLES AT BLOCK LEVEL IN NYC

PRESENTER: TIANYUAN LIU

INSTRUCTOR: MIN ZHU

0 8 / 2 0 1 5

CONTENTS

• Project Description

• Methodology

• Takeaway

PROJECT DESCRIPTION

APPROACH

• Exogeneity

• The attributes may be price-independent.

• Isolate the area-wide factors from property-dependent factors.

• Hedonic

• Distances to certain facilities increase/decrease the value as the level convenience of living increases/decreases

• Distances as attributes

• Distances to certain facilities contribute to the value of a block, a lot, or a single property.

PLACE OF INTEREST (POI)

• Facilities that impact on surrounding area.

• POIs (in ArcGIS) present as points, lines, polygons, or raster.

• We select some facilities as POIs to test if the impact of each POI is significant.

• We also summarize non-spatial factors as the zonal density of noise as a POIs.

PROXIMITY(DISTANCE) • Proximity:

• Attributes of each block • Test the sensitivity of block-level scale.

• Measured as network distances • Accessibility of facilities-dependent of road

network, such as walking distances

• Measured as Euclidean distances • Externality of the facilities-independent from

road network

• Proximity to certain facilities may positively/negatively impact on property values.

• Impacts diminish at certain rates as distances increase.

• The diminishing rates may be non-linear.

https://en.wikibooks.org/wiki/Transportation_Geography_and_Network_Science/Circuity#/media/File:TGNS_NetworkDistance.png

https://en.wikibooks.org/wiki/Transportation_Geography_and_Network_Science/Circuity#/media/File:TGNS_EuclideanDistance.png

http://resources.arcgis.com/en/help/main/10.1/index.html#/Near/00080000001q000000/

METHODOLOGY

•Block shapefile of each borough

•Use block suffix to identify block of the same block

•POI shapefile

Input

•Network Analysis

•Find closest facilities

•Calculate Network Distance

•Generate Near Table

•Calculate Euclidean Distance

•Rasterize non-spatial attributes

•Calculate the number of facilities within certain distance of a block

Interim •Distance Table

•Distance-Dummy Table

•Zonal Attribute Table

Output

PROCESS

INPUT-POI PREPARATION Name Selection Standard and Action Source Feature

SubwayStation Copy and Paste DOITT points

Copy and Paste DOITT points

SelectedPark_5a Acreage>=217800 (5 acres) DOITT polygons

Rail_grd ROW_TYPE=Elevated, Surface, Open Cut Depression, Embankment,Viaduct DOITT polylines

Bridge_Tunnel RW_TYPE=Bridges (across shoreline), dissolve, DOITT polylines

PublicAccessibleWaterfront Merge PAWS.shp and NYC_Waterfront_Parks.shp BYTE of BIGAPPLE polygons

WasteManagement Copy and Paste BYTE of BIGAPPLE points

College_3K SubGroup Type=13, Capacity>=3000 BYTE of BIGAPPLE points

College_10K SubGroup Type=13, Capacity>=10000 BYTE of BIGAPPLE points

CulturalFacilities_Others FacType=1601, Capacity>0 BYTE of BIGAPPLE points

Library_300K FacType=1401 and 1402, Capacity>300000 BYTE of BIGAPPLE points

RailStation Copy and Paste DOITT points

Hospital FacType=3102,Capacity>0 BYTE of BIGAPPLE points

HistoricDistrict Status=Designated NYC OPEN DATA polygons

Noise_311 Complaint_Type Contains Noise,Display XY data NYC OPEN DATA points

Noise_Den_25 Point Density, cell size=25, mask=nybb NA raster

Pharmacy Selected by Location (nybb), Amenity=Pharmacy/Name=CVS, Duane Reade, WALGREENS, Rite Aid OpenStreetMap points

Shelter FacType=4401,4402,4411,4412,4414,Capacity>0 BYTE of BIGAPPLE points

INPUT-BLOCK PREPARATION

Identify each Block

• Newbase table containing bbl and block suffix

• Select index lot from each physical block

• Sort by Boro, Block, Block Suffix, Lot

• Exclude lot of:

• Pid <0

• Land size=0

• BC=T*, U*, R*

Select block

• Digital Tax Map containing tax lot features

• Table containing bbl and block suffix

• Join by lot BBL

• Lot Feature containing Boro, Block, and Block Suffix.

Blocks with blksuf

• Digital Tax Map tax block feature

• Spatial Join the lot feature with block feature (get attributes)

• Dissolve to combine the small block with same block and suffix number

• Generate centroid for each block

PROCESS METHODS

•The accessibility of POI relies on road network

•Active Access

•walking

•Driving

Network Analyst

•The accessibility of POI doesn’t rely on road network

•Externality of noise/pollution

•Passive Access

Nearest Distance

•Summarize the non-spatial variables

•Create spatial distribution surfaces

Point Density

Subway Station

Rail Stations

Universities

Museum

Hospital

Shelter

Library

Pharmacy Publicly

Accessible Waterfront

Railroad on the

ground

Park

Bridge and

Tunnel Waste

Management

Brownfield

Historic District

Noise

METHOD LOGIC If the POI should be

actively accessed from each block…

Network Analyst

(5 nearest POIs)

Distance Table:

1st Nearest Distance

2nd Nearest Distance

3rd Nearest Distance

4th Nearest Distance

5th Nearest Distance

ArcGIS shapefile

If the POI should be passively accessed from each block…

Make Near Table

Nearest Distance Table

ArcGIS shapefile

If the non-spatial attributes can be

presented geographically…

Point Density/Raster/

Zonal Table

Zonal Table:

Non-spatial attributes

If the number of POIs were to be

summarized at block level…

Multiple Buffers/Spatial Join

Count Table:

Numbers of POIs of each block at distance_1

Numbers of POIs of each block at distance_2

ArcGIS shapefile

INTERIM-NETWORK ANALYST

Incidents-Blocks

•Block centroid shapefile (OID)

•By boro

•Generate IncidentID

•Reasonable Check

Facilities -POIs

•POI (Point features only

•Generate FacilityID

•From incidents to facilities

Use Network

•Road

Network

• Generated

from CSCL

Centerline

(topology)

Solve

• Use incidents, facilities, and network feature layers

• Find the Closest Facility

• Number of POIs to find=5

• Use trip length as impedance

Save results

•Save route feature class

•Save the 5 distance values to table

•Transpose by incident

Join Distance back to Block

•Distance table with IncidentID

•Blocks with IncidentsID

•Blocks with OID

• Input

• Tax block

• POI • Subway

Station

Distance to the 1st nearest Subway Station

Distance to the 2nd nearest Subway Station

Distance to the 3rd nearest Subway Station

Distance to the 4th nearest Subway Station

Distance to the 5th nearest Subway Station

Mean Distance

INTERIM-GENERATE NEAR TABLE

Input feature

-block

•Block centroid shapefile

•Add OID to identify each block

•By boro

Near feature

-POIs

•Polylines

•Polygons

•Points

•Euclidean distance

Generate Near Table

Join Distance back to Block

•Distance Table for each block

• Input

• Tax block

• POI • Park

• Larger than 5 acres

Nearest Distance

INTERIM- CAPTURE SPATIAL RELATED VARIABLES

Input feature

-block

•Block centroid shapefile

•Add OID to identify each block

•By boro

Create Raster

-POIs

•Polylines

•Polygons

•Points

•Attributes: density

Create zonal table to summarize the raster attributes into each block

• Sum

• Area

• Sum/Area

Join zonal table back to Block

•Spatial attributes for each block

Noise Complaint Density

INTERIM- GIS PROCESS-GENERATE DUMMY VARS

Buffer

•Block feature

•Generate OID for each block

•Generate Multiple Buffers for each block

•0.3-mile buffer

•0.5-mile buffer

Calculate numbers of facilities within buffers of each block

•Spatial Join with the point POI feature

•Field summarize the number of facilities

•Save the table

Generate Dummy Variables

• If none of the facilities fall in 0.3-mile buffer, then dist_030_var0=1, else=0

• If 1 facility falls in 0.3-mile buffer, then dist_030_var1=1, else=0

• Input

• Tax block

• POI • Subway

Station

Number of Subway Stations within 0.3-mile radius of each block

Number of Subway Stations within 0.5-mile radius of each block

Distance=0.3 mile

#=0 #=1 #=2 #=3 #=4 #>=5

Distance=0.5 mile

#=0 #=1 #=2 #=3 #=4 #>=5

TAKEAWAY

PROJECT DESCRIPTION

• Takeaway

• We create a pool of distance attributes for all blocks, and distances will be classified into different groups based on future modeling.

• The data can be collected at block/lot/property level.

• Reusable Python script tools enables distance calculation for point/polyline/polygon POI feature classes.

• The next step may be creating an index based on areal attributes, such as distance-value index system.

• The raw output as well as the index system can be input variables for future models.

FILE SYSTEM- ORIGINAL DATA

RawInput

DCP DOITT OPENDATA OpenStreet Collected workflow_documentati

on

NYC_PubliclyAccessibleWaterFront_2014

NYC_SelectedFacilities_2015

TANK Borough_Boundaries

cscl_pub.gdb NYC_Planimetrics_2010

Noise_311_07012014_0

7012015

TANK remedsiteborders

new-york_new-york.osm-point.shp

Potential Materials

nyc_paws_2014shp

nyc_waterfrontparks_2014shp

nyc_facilities2015_shp Potential Materials

nybb_15b CSCL SubwayStation.shp

NYC_DOITT_Planimetric_Seamless_2

010.gdb

Potential Materials

Remediation_site_bord

ers

PAWS.shp NYC_Waterfront_Parks.shp

Facilities - 01 - Schools.lyr

nybb.shp Centerline.shp

RailStation.shp

NYCPlanimetric

Remediation_site_border

s.shp

Facilities - 02 - Recreational & Cultural

Facilities.lyr

Rail.shp PARK.shp

Facilities - 04 - Nursing Homes, Hospitals,

Hospices and Ambulatory Services.lyr

Subway.shp

Facilities - 10 - Food Programs & Residential Facilities for Adults and

Families.lyr

Facilities - 12 - Waste Management Facilities.lyr

Table File

Shapefile or Layer File

Tools and Documentation

Folder or Geodatabase

FILE SYSTEM- NETWORK ANALYSIS/NEAR ANALYSIS

NetworkAnalysis

POI_input POI_output Output_dist Tools_Python Tools_SAS

boroBD.gdb dtmblock.gdb POI.gdb RoadNetwork.gdb BridgeTunnel.txt POI* dist_mean_input dist_mean_output

NA_block_meandist

1_BlkSuf.py POI_MakeNearTable

POI_NetworkAnalyst

boro*_BD.shp boro*_blk.shp POI*.shp RoadNetwork College_3K.txt boro*_POI*_CreationDate*.gdb

POI* POI* POI* 2_blksuf_cent_to_poigdb.py

POI* (create near table)

POI*(for network analysis)

nybb.shp boro*_blkcent.shp

POI*=BridgeTunnel, Brownfield, College_3K,

College_10K, CulturalFacilities_Others

, HistoricDistrict, Hospital, Library_300K,

Noise, Pharmacy, PublicAccessibleWaterfr

ont, Rail_grd, RailStation,

SelectedPark_5a, Shelter, SubwayStation,

WasteManagement

RoadNetwork_ND College_10K.txt boro*_POI*_CreationDate*

(table)

blkcent_boro*_POI*_CreationDat

e*.dbf

boro*_POI*_CreationDate*_mea

ndist.dbf

boro*_POI*_blk.dbf (for raster)

3_NA_NF.py boro_macro.sas

boro_macro.sas

boro*=MH, BX, BK, QN, SI

CulturalFacilities_Other.txt

blkcent-boro*_POI*_CreationDate*.shp

blkcent_boro*_POI*_CreationDat

e*.dbf

4_Blkcent_dist_join.py

macrocall.sas macrocall.sas

Library_300K.txt 5_MakingNearTable.py

Pharmacy.txt 6_near_Blkcent_dist_join.py

PublicAccessibleWaterfront.txt

9_raster_blk_join.py

Rail_grd.txt

SelectedPark_5A.txt

Table File

Shapefile or Layer File

Tools and Documentation

Folder or Geodatabase

FILE SYSTEM- CREATE DUMMY VARIABLE (BETA)

DistanceAnalysis

POI_buffer_input

POI_buffer_ouput table_input_Python table_interim_S

AS table_output_SAS table_tablejoin Tools_Python Tools_SAS

SubwayStation SubwayStation SubwayStation condosuff_SubwayStation_count

.dbf SubwayStation

condosuff_SubwayStation_count

.dbf

blk_boro*_SubwayStation_72015_dummy.dbf

7_number_count.py

buffer_count.sas

cdsuff_xy.csv boro*_SubwayStation_72015.gd

b scratch.gdb

boro*_SubwayStation_72015_b

fct.dbf

boro*_SubwayStation_bfct.dbf

8_count_join.py

boro*_SubwayStation_72015.shp

boro*_SubwayStation_72015_bf

ct.shp

blk_boro*_SubwayStation_72015

_dummy.shp

Table File

Shapefile or Layer File

Tools and Documentation

Folder or Geodatabase

*FUTURE ACTIONS- ADD POI

• Download original shapefiles in RawInput Folder

• Sort by the source of the files (DCP, DOITT, OPENDATA, OpenStreetMap, or SelfCollection…)

• Put POI shapefiles in POI_input\POI.gdb

• Select the Python Tools and SAS Tools to process

• Need to change POIs manually in each script

*FUTURE ACTIONS- TOOLS AND RESULT TABLES…

• Network Analysis-

• Input • POI_input\POI.gdb\

• POI_input\dtmblock.gdb\blk(cent)

• Point Features only

• Tool_Python\3_NA_NF.py • dist_mean_input\POI*\dbf

• Tool_SAS\POI_NetworkAnalyst\boro_macro • dist_mean_output\POI*\dbf

• Tool_Python\4_blkcent_dist_join • NA_block_meandist\POI*\dbf

• Generate Near Table-

• Input • POI_input\POI.gdb\

• POI_input\dtmblock.gdb\blk(cent)

• Point/Polyline/Polygon features

• Tool_Python\5_make_near_table.py • dist_mean_input\POI*\dbf

• Tool_SAS\POI_MakeNearTable \boro_macro • dist_mean_output\POI*\dbf

• Tool_Python\ 6_near_Blkcent_dist_join.py • NA_block_meandist\POI*\dbf

*FUTURE ACTIONS- SUMMARIZE THE RESULT

• Summarize the result in the master table of each boro

• Output_dist\Descriptive\boro*.xlsx • Sort the result based on the method of distance calculation

• Near • Sorted by ORIG_FID

• Network Analyst • Sorted by ORIG_FID • Mark the missing value with IncidentID

• Raster (Beta) • Sorted by OID_12 • Mark the missing value with IncidentID