BUYER PAST PERFORMANCE CYCLE

At a high level, the buyer uses past performance information to assess performance: How has this party performed in the past? These assessments are most commonly part of source selection during a competitive procurement. A clear understanding of the buyer’s process for obtaining, evaluating, and using past performance information is essential to sellers as they develop responses to solicitations, because they seek to showcase their past performance in all potential sources the buyer might turn to for information about their past performance. Buyers also should, naturally, be aware of their own agency’s processes. The buyer past performance cycle is shown in Figure 2-1. (Not shown directly is the storage or retention of past performance information.)

Figure 2-1: Buyer Past Performance Cycle

Contract award and contract performance are shown as output of and input to, respectively, the buyer past performance cycle. The term contract award means that the government has given notification that it is entering a contract with a private party. This may have been an award decision where one or more bidders learned they did not receive contract awards, or it may have been an award decision where there was only one bidder. For sellers, a contract award is the desired outcome from the buyer’s use of past performance information in source selection (the process of determining which offeror to choose to make a contract award to among a group of offerors or other alternatives).

Bidding on and subsequently being awarded a contract means that contract performance information now starts accumulating experience that can be converted to past performance information. Thus begins the cycle: collection, evaluation, and use. Bidding on and subsequently not being awarded a contract means that no contract performance experience that can be used in the future as past performance information will be accumulated.

Collection

Sellers that do business with the federal government are usually familiar with its three main collection methods: past performance write-ups submitted as part of proposals, past performance questionnaires (PPQs) collected as part of a solicitation response, and contractor performance assessment reports (CPARs) collected during and after contract performance. Past performance information is not solely provided by the bidder and may come from multiple sources.

These sources can have a large impact on an evaluation outcome. Government agencies that frequently buy commercial items and place a significant number of orders under Federal Supply Schedules, governmentwide acquisition contracts, or other task order contracts may conclude that their quest to award contracts to good performers over good proposal writers is better served by evaluations that rely only on past performance and price, and not on extensive technical or management proposals. For a firm competing for work under this scenario, any individual source of past performance information can rise to a critical level when the buyer is deliberating on which firm will receive a contract. Table 2-1 summarizes all the collection methods used by government agencies and when/how past performance data are commonly recorded, used, and retained for later use.

Persistent problems regarding past performance can be categorized as stemming primarily from inputs and from organization. Persistent input problems arise from agencies not completing past performance reports, evaluators avoiding writing narratives, evaluators giving high scores rather than using evaluations to provide meaningful feedback, and written narratives that don’t match or support assigned grades. Additional input problems come from the continued and pervasive use of burdensome collection methods, namely ad hoc, short-fused, and unpredictable (to the party completing them) paper-based past performance questionnaires.

Persistent organization problems regarding past performance come from continued use of a burdensome and disjointed collection, retention, and retrieval system for past performance information. Information is difficult to access, access is limited to a small population of users out of a potentially broader user base, and access is parsed and stovepiped so that the user experience is poor and utility derived from the system is low.

Table 2-1: Buyer Past Performance Information Collection Methods

These persistent issues contribute to a longstanding critique that the past performance information collected by an agency or activity is not made available to others (either within the agency or outside it) and therefore is not used by other agencies or activities in making procurement decisions. This criticism holds true across collection methods despite many years of attempts to address this fundamental problem. In 2006, the General Services Administration observed the following concerning past performance and its nuances:

It has long been a vulnerability that Government agencies would award to a vendor who owes another part of the Government money or services, or is in the process of being debarred. This was due to the fact that information about performance was maintained at the local contracting office level. (GSA, 2006)

A brief discussion of each method and its impact on data usage and retention/retrieval follows.

Proposal

Bidders submit past performance information in proposals, typically in a separate section or volume from the proposal itself, and otherwise as directed in the solicitation. The information collected is used for proposal evaluation. A proposal consists of an offer or promise of work, price or cost information, and information about the bidder (Figure 2-2). Past performance information is part of the information about the bidder (proposer capability information). This information can be provided (as directed in the solicitation) in a variety of formats, most commonly written information and oral presentations.

Figure 2-2: Components of a Proposal

Past performance information collected in proposals typically goes no further than the preaward contract file, whether paper or electronic. These files generally are not searchable or electronically archived in any manner conducive to access or use for any other purpose.

Past Performance Questionnaire

PPQs are typically collected as part of a proposal response and used during proposal evaluation. Solicitations provide guidance on how they are to be completed and returned to the contracting officer or other government official. The norm is to include a specific PPQ form in the solicitation. This form is not filled out by the bidder but sent by the bidder to several client references of their choice, who are asked to return the completed form directly to the contracting officer or designee. PPQs collected during preaward typically go no further than preaward contract files, which generally are not searchable or electronically archived in any manner conducive to access or use for any other purpose.

Dun & Bradstreet Open Ratings Report

For a fee, Open Ratings, Inc. (a Dun & Bradstreet company), conducts an independent audit of customer references and calculates a rating based on statistical analysis of various performance data and survey responses. Some agencies, notably the General Services Administration (GSA) for Federal Supply Schedule proposals, use Open Ratings for collecting past performance information in proposals and for evaluating past performance during proposal evaluation.

The performance data collected in the Open Ratings survey is provided in an Open Ratings evaluation report and includes customer ratings on a contractor’s overall performance, reliability, cost, order accuracy, delivery/timeliness, quality, business relations, personnel, customer support, and responsiveness. The solicitation directs sellers to file with Open Ratings and show evidence of this filing (to include, in some cases, a copy of the evaluation report).

Aside from firm identification data, the primary information supplied to Open Ratings by a seller is contact information for 20 client references. Open Ratings considers four responses from these references to be the minimum needed to complete a rating and declares its past performance evaluation reports to be valid for one year. The soliciting agency will most likely retain Open Ratings reports along with proposals in the preaward contract file. They are also kept on file and are accessible (for a fee) directly from Open Ratings during the one-year validity period.

Phone Interview

Phone interviews are occasionally conducted by contracting officers and designees as a part of past performance evaluation during a proposal evaluation for source selection purposes. Phone interviews may be conducted using a standard set of questions—in essence, a PPQ form.

In an evaluation of past performance, there is no legal requirement that a government agency must contact all the references submitted by a particular bidder. But the agency must act reasonably when it decides which references to contact or not contact: Government agencies must make a reasonable effort to make contact, and if that reasonable effort does not result in contact with the reference, the agency can complete its evaluation without the reference.

When instructed to provide contact information for its past performance references in a proposal, a prudent bidder will include references familiar with it that will respond to queries in a timely fashion and provide a good reference. Many agencies ask for both a technical and a contracts reference for each cited past contract performance. In some cases, only contact information is requested (i.e., no past performance write-up).

Phone interviews may be recorded as call notes or on a PPQ (or similar) form. The notes are most likely to be retained in the preaward contract file, with the same level of access as for PPQ questionnaires.

Email Response

Similar to phone interviews, email responses may be used as part of proposal evaluation and may be solicited directly from the past performance references provided by bidders using a set of questions or a PPQ template. Email responses are recorded in the email body or as an attachment and may be retained in preaward contract files.

Award or Incentive Plan Report

Award fee plans (used in award fee contracts) provide guidance on the collection of performance information for specified contract periods. This past performance information is typically collected in a standard response format (as specified by each plan) that, ideally, contains elements tying important contract objectives to the incentives or awards in the contract. This information is used to determine award fee amounts during and following contract performance; it is kept in the contract files, which are not typically archived or stored electronically in a manner conducive to later retrieval.

FAR 42.1503 was revised in September 2013 to specifically state that incentive and award fee performance information is to be included in past performance ratings and evaluations entered into CPARS. This FAR language does not state that CPARS evaluation reports are to be produced coincident with award and incentive fee plan evaluations of contractor performance and does not apply to procedures used by agencies in determining fees under award or incentive fee contracts. When past performance of an award or incentive fee contract is evaluated, this element of contract performance is to be rated as part of any CPARs covering the same period, using the adjectival rating scale provided by the same FAR section (see “Evaluation”). It also sets the expectation that the fee amount paid to contractors under award or incentive fee arrangements should reflect the contractor’s performance and the past performance evaluation should closely parallel and be consistent with the fee determinations.

Close-at-Hand Observation

When agency or government staff directly observe contractor performance, this is referred to as close-at-hand observation. Although this type of past performance information is not generally captured or recorded, it is an established precedent in case law that close-at-hand observation of contractor performance cannot be ignored during source selection evaluations (see for example GAO B-280511.2, GTS Duratek, Inc., 1998, and GAO B-401679.4 et al., Shaw-Parsons Infrastructure Recovery Consultants, LLC and Vanguard Recovery Assistance, Joint Venture, 2010) and, if it exists, must be used during proposal evaluations. GAO review precedent is that this information should be considered the same as any other past performance information presented by the seller or found by the buyer.

Some agencies and activities purposefully and systematically collect close-at-hand contract performance information, but the majority do not. The likely means of collection and retention of this type of past performance information, if it is documented, is in any evaluation working papers or decision documents retained in the preaward contract files.

CPARS Reports

CPARS reports are collected periodically as described in Chapter 4. CPARS forms are sometimes reviewed during proposal evaluation or as a reference when completing performance evaluations during contract performance.

The standardization, retention, and retrieval provided by CPARS is intended to make records of past performance information accessible across agencies for the purpose of evaluating contractor past performance. Currently, these records are accessible via CPARS and the Past Performance Information Retrieval System (PPIRS). A planned update to the System for Award Management (SAM) will include the information in PPIRS in SAM. Past performance information in CPARS reports is retained for three years.

Program Review

It is a best practice for government programs to periodically conduct some form of a program review. There is no specified form or format for such review that is universally applicable, and this type of past performance information is not likely to be recorded or retained. These reviews may generate past performance information regarding contractor performance during a contract period or periods.

Use of this information is similar to that described for close-at-hand observation. It is unlikely that this information would be retained in any way that is searchable or accessible outside the office that generated the review.

Evaluation

Evaluation refers to a buyer’s evaluation of a bidder’s past performance against stated evaluation criteria.

In the solicitation, the agency is required to describe the evaluation approach it will follow and provide an opportunity for bidders to identify past or current contracts that they performed on and to detail any problems encountered and corrective actions taken. The evaluation of past performance must consider the currency and relevance of the information, the source and context of the information, and general trends in contractor performance. The government must consider information from the offeror and other sources and should consider information about predecessor companies, key personnel, and subcontractors.

As described in Chapter 1, three standard rules in the government’s evaluation of past performance are that (1) If no record of relevant past performance exists, the offeror may not be evaluated either favorably or unfavorably on past performance; (2) the offeror must be provided the opportunity to address adverse past performance information that it has not previously had an opportunity to comment on; and (3) the government cannot reveal the names of individuals providing reference information about an offeror’s past performance.

In addition to these three standards, what is to be evaluated and how it is to be evaluated must be stated in the solicitation, and when conducting the evaluation this “what and how” must be followed. (This information is in Section M of the solicitation.) Typically, the stated evaluation scheme will specify the evaluation criteria for the past performance itself, along with how the relevance of the past performance will be evaluated. Evaluation criteria for past performance may include things such as the management performance of the specified relevant past performance, the quality of the products or services delivered, how well schedule requirements were adhered to, and the cost control performance of the specified relevant contracts cited as past performance. Relevance of past performance information can take into consideration such things as the nature of the business areas involved, the required levels of technology, contract types, similarity of materials and products, location of work to be performed, and similarity in size, scope, and complexity.

The government’s basic objective regarding past performance evaluation is to measure the level of confidence in an offeror’s ability to successfully perform based on previous and current contract performance data. This level of confidence is expressed in a performance risk assessment, in which evaluators assign an adjectival rating based on their evaluation of the offeror’s past performance. A typical adjectival rating scheme for assessing performance risk based on past performance is shown in Table 2-2. A thorough past performance evaluation, by including reviewing information from sources in addition to those submitted in a proposal, ensures that awards are made to good performers over good proposal writers.

Table 2-2: Adjectival Rating Scheme for Assessing Performance Risk Based on Past Performance

An important and often misunderstood differentiation between past performance and experience can come into play during evaluation. As stated above, government agencies are not allowed to unfavorably evaluate bidders with no past performance. This is sometimes referred to as the neutral rule, because a bidder with no past performance is to be evaluated neutrally (not negatively) on past performance. However, agencies can negatively evaluate a bidder’s experience if it has no past performance, if the solicitation makes this distinction.

GAO protest decisions are based in part on how the solicitation is written. If no distinction is made between experience and past performance in the solicitation, the decision normally is that the two elements cannot be evaluated separately. If a clear distinction is made in the solicitation between past performance and experience, then a bidder can be evaluated separately on these two elements, and a bidder with no past performance and no relevant experience could be evaluated neutrally on past performance and negatively on experience. The majority of solicitations and evaluation schemes are not this clear, however, and the terms past performance and experience are often conflated or used interchangeably in solicitations.

Past performance evaluation as a part of source selection for award of a contract can take place as a prequalification, as in a two-step or other phased source selection process. For example, a first round of past performance and capability information submission could result in a down-select decision by the buying agency that limits the final or subsequent participation in this acquisition to a subset of the interested respondents.

Evaluation of a company’s past performance also takes place outside source selection evaluations: during program reviews, during the completion of CPARS reports, during award or incentive fee determinations, and at other less formal times during or after contract performance. These are shown in the expansion of the “evaluation” area of the buyer past performance cycle shown in Figure 2-3.

Figure 2-3: Evaluation, Buyer Past Performance Cycle

Historically, a higher level of scrutiny is applied to—and a higher degree of compliance is achieved in—the form, format, and procedures that are used during source selections, when an agency determines to which offeror among a group of offerors it will award a contract. At other times, however, such as evaluating contractor performance during contract performance or assessing bidder past performance in a responsibility determinations, less oversight and attention has been paid. It is in part because of this that several issues with past performance evaluation persist across government agencies (GAO, 2009). First, past performance evaluations (or contractor evaluations during contract performance) are not always completed. Second, even when completed, evaluations either are cursory or fail to provide direct, actionable, or constructive critiques on performance. Third, opportunities to collect and retain past performance information and information on contractor performance are missed, and past performance information repositories are thus rendered ineffective at best. Weaknesses in the collection, retention, and use of past performance information have been repeatedly decried by oversight and other organizations such as GAO, inspectors general, and industry groups. Some government agencies have dramatically improved their past performance information collection efforts, particularly via CPARS. (It is worth noting that CPARS is for unclassified information only, and intelligence community agencies largely do not use CPARS.)

Additionally, FAR 42.15 on contractor performance information was revised in September 2013 to detail when, where, what, how, and by whom contractor performance is to be evaluated. The when is annually and when contract performance is completed. The where is electronically in CPARS, from which data are automatically migrated to PPIRS after 14 days. The who includes the technical office, contracting office, program management office, and, where appropriate, quality assurance and end users of the product or service who provide inputs to evaluations. If a specific person is not designated as being responsible for completing the evaluation (e.g., contracting officer, contracting officer’s representative, project manager, program manager) the contracting officer is the party responsible for the evaluation.

The what is first established in FAR 42.15 by defining past performance information as relevant information, for future source selection purposes, regarding a contractor’s actions under previously awarded contracts or orders and includes the contractor’s record of

1. Conforming to requirements and standards of good workmanship

2. Forecasting and controlling costs

3. Adherence to schedules, including the administrative aspects of performance

4. Reasonable and cooperative behavior and commitment to customer satisfaction

5. Reporting into databases

6. Integrity and business ethics

7. Businesslike concern for the interest of the customer.

The how is by using standardized adjectival rating schemes and specified elements of past performance information to be reviewed and evaluated. Required evaluation factors include (at a minimum):

1. Technical quality of product or service

2. Cost control (not applicable for firm fixed-price or fixed-price with economic price adjustment arrangements)

3. Schedule/timeliness

4. Management or business relations

5. Small business subcontracting (as applicable)

6. Other (as applicable—e.g., late or nonpayment to subcontractors, trafficking violations, tax delinquency, failure to report in accordance with contract terms and conditions, defective cost or pricing data, terminations, suspensions, debarments).

These factors can include subfactors, and each factor and subfactor is required to both be evaluated and have a supporting narrative. The evaluation rating scale and definitions are shown in Figures 2-4 and 2-5, reproduced directly from FAR Tables 42-1 (for prime contractor performance) and 42-2 (for small business and subcontracting performance).

Figure 2-4: Past Performance Evaluation Rating and Definitions from FAR Table 42-1

NOTE 1: Plus or minus signs may be used to indicate an improving (+) or worsening (-) trend insufficient to change the evaluation status.

NOTE 2: N/A (not applicable) should be used if the ratings are not going to be applied to a particular area for evaluation.

Figure 2-5: Small Business Subcontracting Past Performance Evaluation Rating and Definitions from FAR Table 42-2

NOTE 1: Plus or minus signs may be used to indicate an improving (+) or worsening (-) trend insufficient to change evaluation status.

NOTE 2: Generally, zero percent is not a goal unless the contracting officer determined when negotiating the subcontracting plan that no subcontracting opportunities exist in a particular socio-economic category. In such cases, the contractor shall be considered to have met the goal for any socioeconomic category where the goal negotiated in the plan was zero.

Evaluation of past performance information typically precedes its storage and retention—that is, it is not stored or retained unless it has been evaluated. The implication of this statement is that past performance information exists in two forms: information that has been evaluated and information that has not been evaluated. An example of the former is the formal documentation (including the evaluation) of a contractor’s performance in a CPARS report. This evaluated past performance information has a high likelihood of being captured and retained in a past performance repository.

An example of the latter is information generated during a program review conducted in an agency. This illustrative program review likely was conducted using a locally devised framework and may or may not include judgmental evaluations of contractor performance. Therefore, it could include past performance information that has not been evaluated. The likelihood of this unevaluated past performance information being captured and retained in a past performance information repository of any sort is extremely low. Contractors should be aware of this occurrence and apply the same management principles recommended for evaluated past performance information. Government agencies should likewise take advantage of this past performance information.

One could argue that unevaluated past performance information regarding contract awards, dollar amounts, and so on, such as can be found in sites or systems like the Federal Funding Accountability and Transparency Act site, on the USAspending.gov site, in the System for Award Management (SAM), or in the Federal Procurement Data System, are past performance information repositories. While this is a true statement, the extremely abbreviated and encoded nature of these repositories makes them unsuitable for past performance evaluations.

Storage and Retention

The primary repository of past performance information used by the government is CPARS. All data in CPARS are to be accessible (per FAR Part 42) in PPIRS to agencies.

Government agencies are required by regulation to document contractor performance. A more recent addition to the requirements to store and retain past performance information is the requirement to document terminations for cause or default, determination of contractor fault by DoD, determinations of non-responsibility, and defective cost or pricing data in the Federal Awardee Performance and Integrity Information System (FAPIIS). Grant issuers and administrators are also charged with documenting “recipient not qualified” determinations and terminations for material failure to comply in FAPIIS. Storage of past performance information is covered in more detail in Chapter 4.

Use

At a high level, the buyer uses past performance information to assess performance: How has this firm performed in the past? These assessments are most frequently used and thought of as part of source selection during a competitive procurement (as described in Evaluation).

The purpose of using past performance information, from the buyer’s perspective, is twofold: First, to make sound business decisions during contractor selection and during contract performance. Second, to provide feedback to contractors to improve overall contract performance. These purposes are fulfilled during the “use” section of the buyer’s past performance cycle (Figure 2-6). A discussion of the three uses—responsibility determination, source selection, and award or incentive determination—follows.

Figure 2-6: Use, Buyer Past Performance Cycle

Responsibility Determination

The FAR uses the term responsible to describe an offeror that has the capability, tenacity, and perseverance to perform a particular contract. Every contract award must have a determination of responsibility. A contracting officer signing an awarded contract signifies the affirmative determination that the contractor is responsible.

Before awarding a contract, a contracting officer conducts a responsibility determination. An affirmative responsibility determination is when a government agency (i.e., a contracting officer) concludes that an offeror is in fact an acceptable contract awardee. At this point, multiple offerors may be responsible.

A responsibility determination is different from a source selection evaluation, which is the process of determining which offeror to choose to make a contract award to among a group of offerors or other alternatives. Award of any contract, often done without a formal source selection process (e.g., a sole source contract award or a small purchase), always requires a positive responsibility determination.

The Small Business Administration may get involved (likely at the request of a small business offeror that was determined to be non-responsible by a contracting officer) and may issue a certificate of responsibility called a certificate of competency. In fact, this certificate can override a non-responsibility finding by the awarding agency.

The past performance of a prospective contractor is one element considered during this process. A responsibility determination is a subjective determination made based on seven factors:

1. Have or can get adequate financial resources to perform

2. Have or can get capacity to perform

3. Have a satisfactory performance record (i.e., past performance)

4. Have a satisfactory record of ethics and integrity (i.e., past performance)

5. Have or can get needed business and performance skills to perform

6. Have or can get needed facilities to perform

7. Be qualified and eligible under applicable laws and regulations.

The best evidence for satisfying that a business “has or can get” these elements is that it has successfully used them on other government contracts similar in size, scope, and complexity—in effect, that it has satisfactory past performance in all seven areas.

Source Selection

Source selection is the most recognizable use of past performance information by the government, being the use most often considered in discussing the use of past performance by government agencies. This use is covered in detail in Chapter 5 on past performance evaluations.

Award Fee, Award Term, and Incentive Determinations

Award and incentive fee arrangements used in award and incentive type contracts are intended to:

• Provide for assessments of contractor performance levels

• Focus the contractor on areas of greatest importance

• Communicate assessment procedures and thus government priorities

• Provide for effective, two-way communication between the contractor and evaluators

• Provide for an equitable and timely assessment process

• Establish an effective organizational structure to administer award fee provisions.

Retrospective performance (i.e., recent past performance or performance during the contract award or incentive period just completed) is the primary factor in an award or incentive determination. Award fee, award term, and incentive refer to the specific contract types requiring these determinations. The contract will specify (or should specify) the performance assessment criteria, the periodicity, and other aspects of the administration of the award or incentive. These matters are covered in an award or incentive fee determination plan included as an addendum to the contract.

Incentives and awards can be either subjective or objective. For objective plans, typically associated with incentive-fee arrangements, there is less likely to be an evaluation of past performance as part of the administration of the plan. In subjective plans, typically associated with an award fee or award term plan, there is a high likelihood that subjective measures will be defined in the contract and contractor performance will be assessed against these criteria to determine the amount of the award.

These contract types are less frequently used but do generate past performance information that must be documented and stored. As of the September 2013 rewrite of FAR 42.1500, these performance assessments are required to be captured in CPARS, will thus be retained in PPIRS, and will be available for either six years (for construction and architect-engineer contracts) or three years (for all other contracts) for agencies to review as part of past performance evaluations during source selection.

Award fee determinations are unilateral decisions by the government, even though a best practice in award fee determination plans is to have the contractor perform a self-assessment and provide input during the review cycle. Award fees are typically paid in three-to six-month increments, so an award or incentive fee contract can generate a number of award fee determinations as well as past performance evaluation content in CPARS over the life of the contract.