This tutorial will show you how to use arcpy to do basic geoprocessing tasks.
I have a feature class of private properties with attributes such as property’s name, address, latitude, longitude and completion year, i.e the year it was completed development.
Unfortunately in some of the records, the completion year value is missing. Now, I will show you how to estimate the missing data by using geoprocessing with arcpy.
Following illustration shows the process flow I used to achieve this task.
Extract Records with missing completion year
First I need to extract those records with missing year from the feature class. The records are stored in “C:\\workspace\default.gdb” as “PRIVATEHOUSES” feature class.
import arcpy arcpy.env.workspace ='C:\\workspace\\default.gdb' input_feature = 'PRIVATEHOUSES' # First make a layer from feature class. Because we are going to use # arcpy.SelectLayerByAttribute_management tool and it doesn’t allow # feature classas input arcpy.MakeFeatureLayer_management(input_feature, 'input_lyr') # This will select the records from input_lyr with # Comletion_Date value is Null mising_records = arcpy.SelectLayerByAttribute_management('input_lyr', 'NEW_SELECTION', 'Completion_Date IS Null')
This code snapshot shows you how to use SelectLayerByAttribute_management tool to select records based on attribute value of feature class or layer. After running this code, I have all the records with missing year value in “missing_records” variable.
Search for other records within specified range
Next, I need to loop through all missing_records and get estimation for each of them.
# Create a search cursor to loop through records rows = arcpy.SearchCursor(missing_records) for row in rows: # Estimation code goes here # … # …
arcpy.SearchCursor() creates a read-only access cursor to records of a feature class or table. For each record, I try to find the surrounding data points within specified range __ in this case, I use 100 meters.
# Get object id from the search cursor objectid = row.getValue("OBJECTID") query = "OBJECTID=%d" % objectid # Selecting the feature with the current OBJECTID temp_selected = arcpy.SelectLayerByAttribute_management('ph_lyr', "NEW_SELECTION", query ) # Selecting surrounding records within 100 meters selected_within_100m = arcpy.SelectLayerByLocation_management('ph_lyr', 'WITHIN_A_DISTANCE', temp_selected, 100, "NEW_SELECTION")
I extract the object id of the current record by using row.getValue() method. After that, the query is prepare for selecting current feature. Then I use arcpy.SelectLayerByAttribute_management(…) to select the current feature and assign it into variable temp_selected. This variable is passed to arcpy.SelectLayerByLocation(…) function to get the records in specified range.
Get a median completion year
Next step is to get estimation of completion year from the retrieved records within range. For estimation I just calculate and return median value of the other records in range.
# Make an estimation of year of completion based on surrounding records def getEstimateFromSurroundings(surrounding_rows): surrounding_years =[] # Ignore records with null values for row in surrounding_rows: val = row.getValue("Completion_Date") if val != None: surrounding_years.append(val) # if there is no values in the list, return None if len(surrounding_years) == 0: return None # I use numpy library to calculate median median_year = numpy.median(surrounding_years) return median_year
I write a function called getEstimateFromSurroundings() function to do the estimation work. In the function, first, I check and remove the records with Null value. The rest are put into a list and median value is extracted. I use numpy to calculate median value from the list, you can implement your own function to get it but I am too lazy do it myself 🙂
After I got the estimation, I update the current record completion year with the estimated value. Since I don’t want to modify the input feature layer, I make a copy of the input feature layer for output using CopyFeatures_management and update the estimated value to it.
# Copy the input feture arcpy.CopyFeatures_management(input_feature, output_feature)
Update the missing completion year with estimated one
Next I implement a function to update estimated value to current record.
def updateCompletionYear(objectid, year): update_cursor = arcpy.UpdateCursor(output_feature, "OBJECTID=%d" % objectid) if(update_cursor): update_row = update_cursor.next() update_row.Completion_Date = year update_cursor.updateRow(update_row) # Release pointer to update record del update_row # Release pointer to update cursor del update_cursor
The function above do the updating of record to output feature layer for the given OBJECTID and year value. I use arcpy.UpdateCursor(…) method to update the Completion_Year data field.
Putting all together
# To estimate completion year from surrounding units import arcpy import numpy from datetime import datetime arcpy.env.workspace = "C:\\workspace\\default.gdb" arcpy.env.overwriteOutput = True input_feature = "PRIVATEHOUSES" output_feature = "OUTPUT_PRIVATEHOUSES" # Make an estimation of year of completion based on surrounding houese def getEstimateFromSurroundings(surrounding_rows): surrounding_years =[] for row in surrounding_rows: val = row.getValue("Completion_Date") if val != None: surrounding_years.append(val) if len(surrounding_years) == 0: return None median_year = numpy.median(surrounding_years) #print surrounding_years #print "Estimated age: ", median_year return median_year def updateCompletionYear(objectid, year): update_cursor = arcpy.UpdateCursor(output_feature, "OBJECTID=%d" % objectid) if(update_cursor): update_row = update_cursor.next() update_row.Completion_Date = year update_cursor.updateRow(update_row) del update_row del update_cursor def main(): # Timing starttime = datetime.now() # First make a layer from feature class # Because select layer by attribute doesn't work with feature class arcpy.MakeFeatureLayer_management(input_feature, 'ph_lyr') arcpy.CopyFeatures_management(input_feature, output_feature) uncomplete_records = arcpy.SelectLayerByAttribute_management('ph_lyr', "NEW_SELECTION", "Completion_Date IS Null") rows = arcpy.SearchCursor(uncomplete_records) for row in rows: objectid = row.getValue("OBJECTID") query = "OBJECTID=%d" % objectid temp_selected = arcpy.SelectLayerByAttribute_management('ph_lyr', "NEW_SELECTION", query ) selected_within_40m = arcpy.SelectLayerByLocation_management('ph_lyr', 'WITHIN_A_DISTANCE', temp_selected, 1200, "NEW_SELECTION") surrounding_rows = arcpy.SearchCursor(selected_within_40m) estimated_year = getEstimateFromSurroundings(surrounding_rows) updateCompletionYear(objectid, estimated_year) if estimated_year == None: estimated_year = -1 print "Objectid %d, year %d" % (objectid, estimated_year) print (datetime.now() - starttime) if __name__ == '__main__': main()
This process takes place for all the records in the for loop. After finished, all the records in my output feature layer has completion year.