This report provides an analytical framework for evaluating the San Francisco Urban Partnership Agreement (UPA) under the United States Department of Transportation (U.S. DOT) UPA program. It identifies the hypothesis and questions to be tested and answered in the evaluation, the evaluation analyses and measures of effectiveness, and the data needed to conduct the analysis.
In 2006, the U.S. DOT, in partnership with metropolitan areas, initiated a program to explore reducing congestion through the implementation of pricing activities combined with necessary supporting elements. This program was instituted through the UPAs and the Congestion Reduction Demonstrations (CRDs). Within each program, multiple sites around the U.S., including San Francisco, were selected through a competitive process. The selected sites were awarded funding for implementation of congestion reduction strategies. The applicants' proposals for congestion reduction were based on four complementary strategies known as the 4Ts: Tolling, Transit, Telecommuting, which includes additional travel demand management (TDM) strategies, and Technology.
The evaluation of the UPA/CRD national evaluation is sponsored by the U.S. DOT. The Research and Innovative Technology Administration (RITA) Intelligent Transportation Systems Joint Program Office (ITS JPO) is responsible for the overall conduct of the national evaluation. Representatives from the modal agencies are actively involved in the national evaluation. The Battelle team was selected by the U.S. DOT to conduct the national evaluation through a competitive procurement process.
The purpose of the national evaluation is to assess the impacts of the UPA/CRD projects in a comprehensive and systematic manner across all sites. The national evaluation will generate information and produce technology transfer materials to support deployment of the strategies in other metropolitan areas. The national evaluation will also generate findings for use in future federal policy and program development related to mobility, congestion, and facility pricing. The Battelle team developed a National Evaluation Framework (NEF) to provide a foundation for evaluation of the UPA/CRD sites. The NEF is based on the 4Ts congestion reduction strategies and the questions that the U.S. DOT seeks to answer through the evaluation.
The San Francisco UPA
The San Francisco UPA partners are the San Francisco County Transportation Agency (SFCTA), The San Francisco Transportation Authority (SFMTA), and the Metropolitan Transportation Commission (MTC). Other Bay Area partners identified in the agreement with U.S. DOT include Alameda-Contra Costa Transit District, Bay Area Toll Authority, Caltrans, and Golden Gate Bridge Highway and Transportation Authority, but they are not part of the projects that are the subject of the national evaluation.
The San Francisco UPA projects that will be the focus of the national evaluation are those related to variable pricing of parking, which include the following:
- Variable Parking Pricing in San Francisco. SFpark is the name given to the parking pricing system for on-street and off-street parking to be implemented by SFMTA in the City of San Francisco as illustrated in Figure ES-1. SFMTA will also disseminate information on parking availability and price on dynamic message signs, the SFMTA website and through text messaging.
- 511 Upgrades. The 511 phone and website in San Francisco Bay Area, operated by MTC, will be enhanced to provide parking space availability and pricing information for municipal parking garages in downtown San Francisco on 511 phone and web, MY 511 and in the 511 traffic internet service provider feed.
- TransLink® Parking Payment. The smartcard electronic payment system being deployed for transit providers in the Bay Area by MTC will be expanded to include parking payment in SFMTA-operated garages.
- Expansion of San Francisco Telecommuting and Alternate Commute Programs. This effort by SFCTA will support the SFpark and 511 enhancements through additional outreach and possible co-location of a bike-sharing station at SFMTA garages.
Other UPA funded projects unrelated to variable parking pricing that will not be included in the national evaluation are:
- Reconstruction of Doyle Drive. SFCTA received UPA funds to reconstruct Doyle Drive.
- SFCTA area-wide/HOT network pricing. SFCTA received UPA funds to further advance work conducted under the Area-Wide Value Pricing project and work with Bay Area agencies to coordinate informational initiatives that support HOV to HOT implementation in the region.
- Improvements to regional ferry service. The Golden Gate Bridge and Highway Transportation District received UPA funds to improve regional ferry boat service.
- Improvements to travel forecasting in Oakland. AC Transit received funds to develop a “simplified travel forecasting approach” for a Very Small Starts project in Grand/MacArthur BRT corridor in Oakland.
- VII1 test bed. MTC will create an open architecture vehicle infrastructure integration test bed in support of a HOT lane tolling application.
- Additional 511 Upgrades. Additional 511 enhancements receiving UPA funds that are not part of the national evaluation include a multimodal trip planner and real-time transit information.
Figure ES-1. SFpark Pilot and Control Zones
The SFpark variable pricing will be rolled out by zone starting in April 2010. Other parking-related projects will become operational between April and December 2010.
Evaluation Analyses and Test Plans
The national evaluation of the San Francisco UPA projects focuses on 10 of the 12 analysis areas outlined in the NEF. Two of the analysis areas—transit and safety—were determined to be unnecessary for San Francisco. Since none of the transit projects using UPA funds support the parking pricing project—the focus of the national evaluation—a transit analysis is not included in the evaluation. A safety analysis is not deemed necessary, because none of the projects being evaluated is thought to have safety concerns. Plans for collecting and analyzing the data to support the 10 analyses are described in 10 test plans. Table ES-1 presents the relationship among the analysis areas and the test plans. The pricing analysis area and the data test plans supporting the pricing analysis are summarized below in Table ES-2 to provide an example of the approach used in the San Francisco UPA National Evaluation Plan.
SFpark variable pricing is expected to have many positive outcomes on travel in downtown San Francisco. When parking supply is priced to meet demand, it is believed that travelers seeking parking spaces will locate parking more readily, traffic on the streets will be reduced by less double parking and fewer drivers circling to look for parking, and traffic will flow more freely as a result. Transit is expected to become a more attractive alternative to driving and parking, as transit travel time improves due to less traffic congestion and transit cost is more competitive when the true cost of parking is more accurately reflected in total travel costs. Table ES-2 presents, as an example of the hypothesis-driven approach used throughout the evaluation, a couple of the hypotheses that capture the expected impact of parking pricing that will be tested in the evaluation.
The first hypothesis deals with the expectation that the time a driver spends searching for parking will be reduced in zones where SFpark is implemented. The measures of effectiveness (MOE s) include the change in the search time and the change in the variability of search time. The data for these MOE s will come from a parking search time survey to be conducted by SFMTA to measure the time it takes to locate the first available on-street parking space along a specific search route.
In the second hypothesis variable pricing is expected to cause a reduction in the average amount of time that customers park in response to higher prices. The measures of effectiveness are the change in the number of parking sessions in a zone and the average duration of the parking sessions. The data for these MOE s will come from the parking system technology. On-street parking sensors and electronic parking meters will record the data which will be stored in the SFpark data warehouse for analysis.
Plans for collecting and analyzing data pertaining to these two hypotheses and all other evaluation hypotheses will be detailed in a series of test plan documents. Responsibility for collecting the data will reside with the San Francisco UPA partners. The national evaluation team will provide guidance to the partners on data collection and will be responsible for analyzing all the data and reporting the results.
|San Francisco UPA Test Plans||Congestion Analysis||Pricing Analysis||Telecommuting/ TDM Analysis||Technology Analysis||Equity Analysis||Environmental Analysis||Goods Movement Analysis||Business Impact Analysis||Non-Technical Success Factors Analysis||Cost Benefit Analysis|
|Traffic System Data Test Plan|
|Parking Data Test Plan|
|Transit System Data Test Plan|
|Telecommuting/TDM Data Test Plan|
|Traveler Information Data Test Plan|
|Surveys and Interviews Test Plan|
|Environmental Data Test Plan|
|Content Analysis Test Plan|
|Cost Benefit Analysis Test Plan|
|Exogenous Factors Test Plan|
|Hypotheses/Questions||Measures of Effectiveness||Data|
The next steps in the San Francisco UPA National Evaluation include developing the detailed test plans and initiating data collection and analysis activities. The detailed test plans will be developed based on this final San Francisco UPA National Evaluation Plan. It is anticipated that the test plans will be completed by February 2010. The results of the San Francisco UPA national evaluation are expected in late-2011.
1 VII (Vehicle-Infrastructure Integration) was the term current at the time of San Francisco's UPA award. Since then U.S. DOT has introduced the term IntelliDriveSM to replace VII.