Design and Implementation of a Pilot Program for Body Worn Cameras

(Excerpted from the Monitor’s First Report, pages 67-78)

The court noted the potential benefits of outfitting NYPD officers with body-worn cameras (BWCs). These potential benefits include creating objective records of stop encounters, encouraging lawful and respectful police-citizen interactions when both parties know exchanges are recorded, alleviating mistrust between the NYPD and the public, and offering a way to substantiate whether officers have been wrongly or rightly accused of misconduct.

As described below, the monitor team is working with the NYPD to plan a BWC pilot program that will be a randomized control trial so that the NYPD and others will be able to evaluate the impact cameras have on the behavior of officers and civilians. The pilot is expected to involve approximately 1,000 cameras, with about fifty cameras in twenty precincts. The activities of officers wearing BWCs will be compared to those of officers with similar assignments in twenty control precincts.

Milestones

The court orders require the NYPD to conduct a one-year pilot BWC program. The purpose of the pilot is to assess whether the benefits of the cameras outweigh their financial, administrative and other costs, and whether the program should be expanded or terminated. The monitor will work with the parties to evaluate the pilot.

The court ordered that the monitor establish procedures for reviewing stop recordings by supervisors and senior managers, preserving stop recordings, and measuring the effectiveness of BWCs in reducing unconstitutional stops and frisks. The court provided that BWCs must be worn for a one-year period by officers on patrol in one precinct per borough—specifically, the precinct with the highest number of stops in that borough during 2012. These NYPD commands were identified as the 23rd, 40th, 75th, 103rd , and 120th Precincts.

Status

The NYPD is currently conducting its own small-scale BWC pilot program in the five commands identified in the remedial order and in one housing PSA (PSA 2) to test different BWC equipment, understand the IT infrastructure necessary to support BWCs, and gain insight on matters of policy and practical implementation. Nine officers in each command volunteered to wear BWCs and have now been trained and equipped (fifty- four in all). In collaboration with the NYPD, the Marron Institute of Urban Management at New York University is conducting an evaluation of the NYPD small-scale pilot program. This effort is separate from the larger one-year pilot BWC program and monitor evaluation required by the court orders and the parties’ agreement.

With respect to the pilot program required by the court orders, the monitor team has begun to work with several units within the NYPD to plan the design and implementation of the court-mandated pilot program. We have met with representatives from the NYPD’s Risk Management Bureau, Information Technology Bureau, the Office of the Chief of Department, and the Office of Management Analysis and Planning.

The monitor team appreciates that there are many important issues that need to be discussed, such as training, policy, technology, outreach to other stakeholders, privacy concerns and possible legal issues such as those raised by the state’s Freedom of Information Law; and we have had very preliminary conversations on these subjects. However, most immediately, the monitor team has been focusing on a design for a rigorous evaluation of the court-mandated pilot program.

The a priori selection of the five NYPD precincts for the BWC program is problematic for several reasons. First, some of the precincts with the highest counts of stop reports in 2012 were no longer the precincts with the highest number of reported stops in 2014. For example, in 2012 the 75th Precinct had the highest number of reported stops in the city; it was down to sixteenth in 2014. Second, the selection in advance prevents the use of a rigorous randomized experimental design, since it would be impossible to have a random selection of commands matching those that had cameras with those that did not. The advantage of a randomized experiment is the relatively high degree of certainty that any observed differences between the precincts with cameras and those without would be attributable to the cameras, not to chance or other causal factors. Third, requiring that every officer in a precinct wear a BWC creates significant IT and logistical challenges for the NYPD.

For all these reasons, the monitor will be recommending to the court that it modify its orders. We describe below the outlines of a randomized controlled trial (RCT), but emphasize that we are still in the process of designing the program and that further changes may be made after consultation with the NYPD, the plaintiffs and others.

1. Design of a Randomized Controlled Trial
Randomized experimental designs allow researchers to assume that the only systematic difference between the “control” group (here, those without cameras) and the “treatment group” (those with cameras) is the presence of the intervention (i.e., use of cameras). This design permits a clearer assessment of causes and effects.[2] We plan to use a well-recognized variant of the classic randomized controlled trial called “cluster” randomization. In these trials, clusters (groups) of subjects, rather than individual subjects, are randomly allocated to treatment and control conditions.[3] In the proposed cluster randomized controlled trial, NYPD officers will be randomly allocated by precinct to the BWC treatment group or non-BWC comparison group.The cluster randomized experimental design will allow better control of “contamination” across individual subjects. As suggested by the study on BWCs done in Rialto, California, officers with BWCs could influence the behavior of officers without BWCs if they simultaneously worked in the same area and interacted with the same people.[4] This contamination undermines the ability of analyses to detect the effects of using cameras because both treatment and control officers might be modifying their behaviors due to the presence of BWCs. Randomly allocating groups of officers who work in distinct precincts to treatment and control conditions limits this problem of contamination.In the proposed trial, eligible NYPD precincts will be ranked according to the 2012-2014 mean yearly counts of complaints handled by the CCRB in New York City. We picked this measure because the number of CCRB complaints is highly correlated to the highest rates of reported stop activity and because the number of stops reported in each precinct in 2012 no longer reflects the stops now being reported. Using the number of CCRB complaints as the measure, the top forty eligible precincts will be matched into pairs based on those counts as well as other measures—specifically, the crime rate, arrests, calls for service, use of force, number of police officers in the precinct and neighborhood characteristics, such as racial composition and poverty level. Precincts in the twenty matched pairs will then be randomly allocated to the BWC treatment group and non-BWC comparison group.We are also discussing which officers within the treatment precincts will be required to wear BWCs. Identifying a well-defined group of officers ensures an “apples to apples” comparison of officers in the treatment and control groups. One possibility is to select all officers assigned to a specific shift. For instance, BWCs could be provided to all patrol officers working the third platoon (4:00 p.m.–midnight shift) in the treatment group (there will be approximately fifty patrol officers in each platoon). The comparison group will consist of patrol officers working the third platoon in the non- BWC precincts. Thus, there will be approximately 1,000 patrol officers in the treatment group (twenty precincts and fifty officers per precinct) and 1,000 patrol officers in the control group. This is roughly the same number of cameras as would have been required under the terms of the court orders.[5]

2. Exclusions
Ten NYPD precincts will be excluded from the randomized experiment. The NYPD is currently piloting the BWC camera technology in five precincts—the 23rd, 40th, 75th, 103rd, and 120th Precincts. If one of these precincts were to be in a pair and randomly selected as the control, this would compromise the experiment because the use of BWCs in that precinct might already have affected behavior. The NYPD is also piloting a “neighborhood-based policing” program in the 33rd, 34th, 100th, and 101st Precincts. These precincts will be excluded because it would be difficult to distinguish the effects of the BWC technology on key outcome measures from the effects of the community engagement reforms implemented as part of the neighborhood-based policing program. Finally, the 22nd Precinct serving Central Park will be excluded because it has relatively low levels of NYPD activity and an almost non-existent residential population.All five boroughs have at least one precinct eligible for inclusion in the randomized field experiment (see Table 1 below)

20150910-graphic1

3. Ranking, Matching and Randomization
Sixty-seven precincts were ranked according to 2013–2014 CCRB mean yearly counts. The top forty precincts are distributed throughout the city in all five boroughs (see Table 2 below).
20150910-graphic2

The matching process should yield twenty similar pairs of NYPD precincts that can then be randomized to treatment and control conditions. A member of the monitor team will flip a coin to determine randomly which of the precincts within the pair will receive the BWCs. The precincts not selected from each of the pairs will serve as controls. The monitor team will then assess the randomization process by determining whether balanced treatment and control groups were created. The monitor team will first compare treatment and control precincts on selected police, crime and neighborhood characteristics. Balanced clusters help to ensure that the treatment and control officers will be working in similar neighborhood, crime and policing contexts. The monitor team will then analyze data on treatment and control officers to determine whether the units of analysis (e.g., NYPD officers working the third platoon) systematically differ in the treatment and control groups. The officer comparison will include important information such as age, sex, race, rank and years on the job.

Assessment
There are many issues to be addressed in the development of a pilot program for BWCs. The monitor will be consulting with the NYPD, the plaintiffs and others on these issues.

The monitor team needs to devise an evaluation plan for the implementation of BWCs on NYPD officers working in PSAs. The characteristics of PSAs are too different from precincts for them to be included in the design described above. Also, there are too few PSAs to do a cluster randomized design. Therefore, we are currently considering a quasi-experimental research design comparing PSA 2 (treatment area) to the other PSAs (control areas).

In addition, the monitor team needs to develop outcome measures based on official data sources such as officer use of force incidents,officer injury and resisting arrest information, and CCRB complaints. We are also considering the practicalities and cost of using officer surveys, community surveys, surveys of enforcement action subjects (e.g., arrested individuals, stop and frisk subjects) and videos of police-citizen interactions.

The monitor team is working with the NYPD to develop a plan to implement the BWC technology that will consider the time and resources required to acquire the cameras, wire the precincts to transfer videos via secure internet connections, train officers and supervisors on policy and procedures, and undertake other important steps. The NYPD has published a Request for Proposal. The NYPD anticipates a requirements contract that would allow for the purchase and installation of up to 5,000 BWCs. The monitor has begun to identify, but not yet fully discuss or resolve, the legal and policy issues mentioned earlier.

Several important steps are needed before a BWC pilot can begin. These include moving forward on the NYPD procurement process, ensuring sufficient technology infrastructure in the pilot treatment precincts, and developing protocols for camera use and data collection, review and evaluation. The NYPD anticipates that because it needs to decide which technology suits its purposes best and to have an infrastructure plan in place to meet the requirements of a pilot of this magnitude, it could be as long as twelve months before the pilot is started. Once the cameras are in place, there would be a one-year trial period.

 


[2] William R. Shadish, Thomas D. Cook, and Donald T. Campbell. Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin. 2002.

[3] Frederick Mosteller and Robert F. Boruch, eds. Evidence matters: Randomized trials in education research. Washington, DC: Brookings Institution Press, 2002; David M. Murray. Design and analysis of group-randomized trials. New York: Oxford University Press. 1998.

[4] Barak Ariel, William Farrar, and Alex Sutherland. 2014. The effect of police body- worn cameras on use of force and citizens’ complaints against the police: A randomized controlled trial. Journal of Quantitative Criminology, http://link.springer.com/article/10.1007%2Fs10940-014-9236-3 (last visited May 30, 2015).

[5] For the more technically minded reader: since outcomes for individuals within clusters may be correlated, standard sample sizes need to be inflated for cluster randomized controlled trials. Using the “Optimal Design” software available from the University of Michigan (http://sitemaker.umich.edu/group-based/optimal_design_software), a total sample size of 2,000 (40 clusters of 50 subjects each) will provide statistical power at the .78 level to detect a standardized effect size of .20 and statistical power at the .99 level to detect a standardized effect size of .40, depending on assumptions about the intra-class correlations in the outcome measure. Stephen W. Raudenbush, et al. (2011). Optimal design software for multi-level and longitudinal research (Version 3.01) [Software]. Available from www.wtgrantfoundation.org.