Return
to Milestone Home PageThe DRS Milestone Payment System has received a $20,000 Ford Foundation
grant for being selected as one of the 25 Finalists in the 1997 Innovations
in American Government Program. The Milestone System was selected
from among
1540 Government innovations, the program is sponsored by the Ford Foundation
in conjunction with the Harvard Kennedy School of Government and the Council
on Excellence in Government. Following are the questions asked and responses
made in the process of achieving the grant.
Bookmarks of Grant Questions (Click on
each question to go to the response for that question.)
1. Describe your innovation; include the specific
problem it addresses, and how it has changed previous practice.
2. What is the single most important achievement of your program or policy initiative to date?
3. What are the three most important measures you use to evaluate your program’s success?
4. Please describe the target population served by your program or policy initiative.
5. What would you characterize as the program's most significant remaining shortcoming?
15. Has the program or policy initiative received any awards or other honors?
16. Has the program received any press or other
media coverage to date?
PROGRAM SUMMARY
1. Describe your innovation; include the specific problem it addresses, and how it has changed previous practice.
No publicly funded service agency has adequate dollars to fund every worthy program or to meet the needs of every customer potentially eligible for its services. Agencies make tough choices about the quality, scope and extent of services for the customers it serves. Faced with these tough decisions, agencies seek methods to improve outcomes while containing costs to stretch funds to more customers. One of the methods of cost containment at the disposal of agencies is their choice of vendor reimbursement systems. Unfortunately, publicly funded social programs all too often create reimbursements systems for vendors that work against the very goals they strive to accomplish.
Like many other public agencies, the Oklahoma Department of Rehabilitation Services (DRS) for many years used typical fee-for-service vendor payment systems, primarily monthly or hourly rates for services. While paying for the process of "doing good," the system often did not promote successful outcomes for the customers of the service. Hourly based funding created perverse incentives that worked against the customer choice i.e., competitive employment with the least amount of professional support. Hourly billing amounts had an inverse relationship to customer independence. The more independent the customer, the lower the billing, thus creating a financial incentive to delay customer independence in employment.
The problem for DRS was to create a reimbursement system which created incentives to achieve the desired outcome for the customer in the most cost effective, efficient way. The Milestone (i.e. outcome based) payment systems instituted by the Oklahoma Department of Rehabilitation Services (DRS) to purchase employment services for individuals with disabilities successfully solves many of the problems inherent in the reimbursements systems used by public agencies. The Milestone system creates financial incentives through multiple levels of payment based on severity of the individual's disability through paying for increments of pre-defined outcomes (milestones) at rates which are negotiated with the vendor. This system differs from the fee-for-service (hourly rate) funding commonly used in that it reimburses the vendor for the average cost of providing the outcome of the service rather than the cost of staff time spent on the process.
A key element in the success of the Oklahoma experiment with milestones is the commitment to creating a payment system which meets customer needs while addressing the financial realities of both the state agency and its service vendors. A novel aspect of the innovation is the extensive collaboration and negotiation between the state agency and its service vendors.
One early problem in the process is developing a realistic operational definition of the outcome of the service which can be sub-divided into incremental outcomes or "milestones" to which payment can be tied. DRS staff devoted significant effort in the year prior to the initial pilot and during each revision to achieving consensus with vendors on the milestone definitions. We believe this collaboration was a critical step in reducing both unanticipated consequences and resistance to change from the service vendors. The negotiations with vendors centered on development of definitions of service increments (milestones) which represent realistic implementation of the best practices in the field. The outcomes thus defined serve as practice and payment standards as well as quality assurance markers.
An example of outcomes defined for the Milestones project is a six milestone payment structure: (1) Determination of Consumer Needs - 10% of bid; (2) Vocational Preparation Completion - 10% of bid; (3) Job Placement - 10% of bid; (4) 4 weeks job retention - 20% of bid; (5) Job Stabilization - 20% of bid; and (6) Consumer Rehabilitated (Stabilization+90 days) - 30% of bid. Each milestone definition includes quality outcome indicators which must be accomplished for payment to be made.
A structured process guides vendors in making a milestone bid. The bid
process allows the vendor to include in average cost calculations the projected
number of "drop outs" at each milestone as well as the cost of providing
the pre-defined quality outcomes. To reduce the disincentives to serve
the most difficult customer, the milestone system requires multiple bids
based on the anticipated level of need/difficulty. The payment system reinforces
outcomes by increasing the percentage of the total payment as the consumer
moves toward the final desired outcome: a stable job which is satisfactory
to the consumer of the service and to his/her employer.
(Return to Questions summary)
DETAILED PROGRAM DESCRIPTION: RESULTS AND POLICY IMPACT
2. What is the single most important achievement of your program or policy initiative to date?
The primary achievement of the Milestone system is creation of a self-regulating,
customer focused, outcome driven service delivery system that resembles
a business transaction. This involves several significant aspects. First,
it shifts payment from
process to outcomes. Second, it creates a consensus on desired outcomes
between customer, agency and vendors which incorporates the latest research
and best practices data. And finally, it creates a system that is self-correcting,
creating a greater tendency to do right rather than wrong.
Milestone based funding naturally stresses outcomes. Vendors report that staff training takes less time, staff are more focused on achieving outcomes for customers, and accounting departments are supportive of employment outcomes. First line staff report that paperwork is greatly reduced, leading to over 10% of their total work time being re-channeled to direct customer services. In times of fiscal restraint, this system can reduce inefficiency and accomplish more work with the same resources.
The definition of commonly agreed upon outcomes with their subordinate milestones is a revolutionary step in the provision of services. The hourly system created a dual system of implicit and explicit goals and outcomes. The system made formal obeisance to the explicit goal of career placement in a job of the customers choice. The implicit goal understood by the most powerful driving force in the vendor agency, the accounting department, was more hours of service. The milestone system inextricably weaves together the customers and funding agency's goals with the service vendor's survival goal of adequate income to continue providing the service.
The final achievement is the ability of government funded services to
be self correcting. Traditional approaches to funding require intensive
regulation and monitoring to avoid waste and abuse. The milestone system
attempts to create a payment system which naturally reinforces quality
services by weaving the regulatory responsibilities of the governmental
unit into the fabric of the payment mechanism. This change allows the oversight
responsibility to be delegated to lower levels of the bureaucracy and creates
hundreds of micro-audits, which lead to constant self-correction. This
causes the vendor's accounting department to work with the governmental
unit and for the customer instead of at cross purposes. Agency resources
formerly devoted to problem-focused program monitoring can be shifted to
quality-focused technical assistance. State office assistance is now focused
on improving the quality of program management.
(Return to Questions summary)
3. What are the three most important measures you use to evaluate your program’s success?
In qualitative or quantitative terms for each measure, please provide the outcomes of the last full year of program operation and, if possible, at least one prior year.
The three most important measures of success are customer satisfaction (both employers and consumers of the services), reduced need for regulation and oversight by the state agency, and increased achievement of the core outcome, successful competitive employment.
Opportunities for customer satisfaction checks come into play at several points in the Milestone process. For instance, if a vendor places an individual on a job which he or she does not like or for some other reason does not want, the vendor is not approved for payment for that milestone. This motivates the vendor to please two major customers, the consumer of the service and his/her Rehabilitation Counselor, so that payment may be received. Since job satisfaction is a major component of success and the vendor cannot be paid without achieving it, this has resulted in a major shift of vendor emphasis toward quality as defined by the customer.
Formal customer satisfaction surveys are required for payment of the final milestone. Rehabilitation Counselors are instructed not to pay for the final, largest milestone unless there is evidence the consumer of services and employer are satisfied with the service. This has resulted in vendors returning to the job site to correct problems which vendors could have ignored under previous payment structures.
The initial three year pilot program in the Oklahoma Department of Rehabilitation Services (DRS) has achieved excellent results for DRS, vendors and consumers of the service. Waiting lists are reduced because consumers are now more efficiently and effectively served. Consumers who previously had been perpetually "assessed" were now moved through the milestone process to employment and independence. Vendors report as much as a 100% increase in productivity in terms of consumers placed and stabilized in competitive employment.
Vendor efficiency is improved by reducing the paperwork load required of the direct service vendor. When hourly rates are the basis for payment, the process of documenting increments and categories of services provided must be meticulously tracked. Our service vendors had as many as 10,000 data entry screens to complete each month to insure payment for every increment of service time. It was then necessary for the counselor approving the vendor's claim to spend an inordinate amount of time reading each bit of documentation to determine whether the service was "billable" under our system. Now, time formerly devoted to detailed documentation is redirected toward accomplishing the outcome for the consumer. The only information which must be documented by the vendor and approved by the counselor is the proof that the outcome was achieved. This reduced the data entry screens to be completed from as many as 10,000 to less than 100 per month. This results in over 10% of vendor staff time being re-channeled from record keeping to direct customer services.
Effectiveness is improved by linking payment for the final milestone
to the achievement of outcomes which have real meaning for the customer.
For example, goal planning is reinforced by the requirement that the job
achieved matches the customer's career goal. One of the best measures of
effectiveness is whether the employer and the consumer of services are
satisfied with the job placement and service. Before the final milestone
payment can be made, the employer and consumer of services must be satisfied.
(Return to Questions summary)
4. Please describe the target population served by your program or policy initiative.
How does the program or policy initiative identify and select its clients or consumers? How many clients does your program or policy initiative currently serve? What percentage of the potential clientele does this represent?
The target population for Milestones are individuals with any type of severe disability who have been determined "eligible" for services according to the following federally defined criteria (Section 361.42 of the regulations implementing the Rehabilitation Act Amendments of 1992):
(i) A determination that the applicant has a physical or mental impairment.
(ii) A determination that the applicant's physical or mental impairment constitutes or results in a substantial impediment to employment for the applicant.
(iii) A presumption that the applicant can benefit in terms of an employment outcome from the provision of vocational rehabilitation services.
(iv) A determination that the applicant requires vocational rehabilitation services to prepare for, enter into, engage in, or retain gainful employment consistent with the applicant's strengths, resources, priorities, concerns, abilities, capabilities, and informed choice.
Individual with a severe disability (Section 361.5) means an individual with a disability (i) Who has a severe physical or mental impairment that seriously limits one or more functional capacities (such as mobility, communication, self-care, self-direction, interpersonal skills, work tolerance, or work skills) in terms of an employment outcome;
(ii) Whose vocational rehabilitation can be expected to require multiple vocational rehabilitation services over an extended period of time; and
(iii) Whose has one or more physical or mental disabilities resulting from amputation, arthritis, autism, blindness, burn injury, cancer, cerebral palsy, cystic fibrosis, deafness, head injury, heart disease, hemiplegia, hemophilia, respiratory or pulmonary dysfunction, mental retardation, mental illness, multiple sclerosis, muscular dystrophy, musculo-skeletal disorders, neurological disorders (including stroke and epilepsy), spinal cord conditions (including paraplegia and quadriplegia), sickle cell anemia, specific learning disability, end-stage renal disease, or other disability or combination of disabilities determined on the basis of assessment for determining eligibility and vocational rehabilitation needs to cause comparable substantial functional limitation.
Individuals are selected for Milestone employment programs by their counselors based on the individual's need for the assistance of a "job coach" in finding, getting and/or keeping a job. Although the majority of individuals served in Milestone Programs presently are those with primary disabilities of mental retardation or mental illness, there are many other disabilities represented among those served (e.g. traumatic brain injury, cerebral palsy, deaf/blind, etc.).
There are a total of approximately 800 individuals served annually who are scattered across the state, served by various vendor agencies. This number will increase to approximately 1000 individuals when the final existing fee-for-service programs are converted to milestones beginning July 1. The Department of Rehabilitation Services also plans to increase the number of new vendors delivering Milestone based services in the next fiscal year by $780,000 to a total of $4.8 million, expanding the client base by another 150 to 200 consumers.
The total potential client base is difficult to estimate. The Department
of Rehabilitation Services serves a total of approximately 30,000 consumers
each year. Those served in Milestone programs represent only those individuals
of this total who have severe disabilities and for whom direct placement
in a job with or without supports is the desired service.
(Return to Questions summary)
5. What would you characterize as the program's most significant remaining shortcoming?
Government programs may have difficulty implementing a business partnership due to traditional constraints. Thinking and acting like a business may be difficult for government organizations with entrenched traditions. The required level of collaboration may be difficult to actualize in hierarchial systems, which may have a history of adversarial relations with vendors and other stakeholders. Vendors of services become business partners in the Milestone system. Vendors become essential to the operation of the core business of the agency rather than tertiary providers of services.
This partnership requires that government entities operate like a business.
Vendor networks which are established to provide services must be able
to depend on timely authorization, timely payment and reliable annual budgeting.
The disruption of any of these critical items can destroy a vendor network
which takes years to develop good levels of efficiency. This inter-dependent
relationship involves a sharing of control and mutual obligation that may
be new for government entities.
(Return to Questions summary)
ADOPTION AND IMPLEMENTATION PROCESS
6. When and how was the program or policy initiative originally conceived in your jurisdiction? What individuals or groups are considered the primary initiators of your program? Please substantiate the claim that one or more government institutions played a formative role in the program's development.
The program was conceived in 1991 as a method of payment for supported employment contracts between the Oklahoma Department of Rehabilitation Services and community based vocational service vendor agencies. The program was developed by the Department of Rehabilitation Services (DRS) Community Rehabilitation staff Dan O’Brien and Becky Cook, the primary initiators of the program. Mr. O'Brien had several years of management experience as a vendor of services and knew the negative pressures of the old payment system from the service delivery perspective. Ms. Cook had several years of experience managing various vendor-based services and related payment systems for the state agency. Their mutual desire to build a fair, streamlined reimbursement system which rewarded vendors for achieving desired outcomes for consumers lead to development of the milestone system.
In 1991, Mr. O'Brien and Ms. Cook contacted the national technical assistance center on supported employment at the Rehabilitation Research and Training Center at Virginia Commonwealth University to learn if other states had implemented "outcomes based payment systems". Although a couple of states had conceived systems, upon further investigation, DRS learned neither had been in place long enough to produce adequate information and neither contained all of the desired elements.
After the new system was piloted with two vendors, a group of vendors
of supported employment services were convened. The design of the first
milestone payment system conversion evolved as a result of several meetings
between the state agency and the vendors who would be affected by the change
in payment system. The group of vendors who played a part in designing
the system were the first to convert.
(Return to Questions summary)
7. Please identify the key milestones in program or policy development and implementation and when they occurred (e.g., pilot program authorization enacted by state legislature in June 1986; pilot program accepted first clients, September 1986; expanded program approved by legislature in July 1987). How has the implementation strategy of your program or policy initiative evolved over time?
Milestones have been implemented in a staged process of pilot programs. The first stage of the pilot program involved establishment of two new employment programs using a Milestone funding model. The model was developed and the initial Invitation to Bid (ITB) was issued on 8/25/92 to two vendors to serve individuals with mental illness. Two pilot programs were funded on Oct. 1, 1992.
The second stage of the pilot involved conversion of existing Supported Employment programs to milestones from hourly contracts. The ITB was issued for conversion of the first mental health vendor contract on Oct. 11, 1993, and was funded at Red Rock Mental Health Center starting Jan. 1, 1994. The remaining Mental Health vocational programs were converted to Milestones on July 1, 1995.
Another Invitation to Bid was developed and issued on Aug. 8, 1994. The new ITB built upon the previous two experiences and stretched the system by providing opportunities for vendor bids on a variety of employment options serving any disability group as a target population. After receiving over 25 bids, contracts were issued to eight programs providing vocational services on Oct. 1, 1994.
A fourth ITB was issued in October 1994. This milestone program had a different design than the previous three ITBs issued, focusing on a Milestones model describing service patterns specific to individuals with developmental disabilities. A separate group of vendors serving this population of individuals with disabilities had been drawn together for several meetings to design this system. As a result of this ITB, one program was funded on Jan.1, 1995. Based on this vendor's experience with the system, the ITB was revised and re-issued to convert all remaining programs providing Supported Employment to persons with developmental disabilities in the fall of 1996. All programs delivering supported employment services to individuals with developmental disabilities were converted by Jan. 1, 1997.
Some work has been done by state agency staff on setting rates for employment services delivered by community mental health centers based upon their three years of experience under a Milestone system. Vendors participated in focus groups addressing development of this rate structure.
Current plans are to convert the remaining Community Rehabilitation
Services contracts to Milestones by July 1, 1997. A total of 35 existing
community rehabilitation agencies will be under milestone contracts by
July 1997, and 5-10 new program contracts are expected to be initiated
July 1, 1997.
(Return to Questions summary)
8. Please describe the most significant obstacle(s)
encountered thus far by your program. How have they been dealt with? Which
ones remain?
Obstacles include changing attitudes from process thinking to outcome
thinking. Systems change may be resisted by those profiting from the status
quo. Others will fear the change because of the performance demand it places
on the systems which heretofore have only been required to "do the best
you can." It requires government agencies and vendors to think like a business.
Conversion from fee-for-service to Milestones can best be achieved by collaboration
of the staff and vendors affected by the change. This requirement for collaboration
will be difficult for state agency staff who may take a more adversarial
approach to service vendors.
The Oklahoma Department of Rehabilitation Services' staff provided extensive
training on the conversion process prior to the Milestone implementation.
Training included practical "how to" processes (e.g., how to claim for
Milestones, how to use the new computer-based client data system, case
management, predicting individual milestone achievements, etc.), but also
how the change would affect the philosophical orientation of the agency
changing from process to outcomes. Despite the many hours of training provided
the vendors prior to converting to Milestones, there was still some panic
among them at the time the change occurred, especially over the bid process.
State agencies or others implementing a Milestone system need to anticipate
the problems that occur with any major conversion process, especially one
which shifts philosophy as well as funds. Training in advance of the conversion
is a requirement for success.
(Return to Questions summary)
9. What other individuals or organizations have been the most significant in a) program development and b) on-going implementation and operation? What roles have they played? What individuals or organizations are the strongest supporters of the program or policy initiative and why? What individuals or organizations are the strongest critics of the program or policy initiative and why? What is the nature of their criticism?
a) The service vendors and DRS field staff who participated in the planning groups which assisted in crafting the designs of the now several types of milestone contracts have been the most significant contributors to program development.
b) The state agency's Technical Assistants play a significant role in on-going implementation. They provide training and technical assistance to managers and staff of vendor agencies in conversion of payment systems, on-going management and implementation and quality service provision.
The vendor organizations' program managers, DRS field counselors and consumers of services play the most significant role in program operation. The DRS counselors provide on-going oversight and approval of payment for all milestones. They review milestones claim documentation for compliance with the outcomes defined by the contract. This step serves as a "mini-audit" of outcomes and service quality. Each vocational program includes six milestones or steps which have been defined as indicating progress on the way to the ultimate outcome. This serves as a review and correction step which insures the on-going quality of program services.
Program managers get feedback numerous times each month and are more
aware of the quality and quantity of outcomes achieved by staff under
their supervision. This provides the information the manager needs to improve
service quality and staff
productivity. The customer provides direction to the process through
participation in the goal setting process, customer goals must be achieved
or payment will not be made. In addition, the customer and employer must
complete satisfaction questionnaires and payment is contingent on their
ratings.
The strongest supporters of the program are the DRS field staff, vendors, consumers of the service and the agency finance personnel.
The critics of the program have been limited to vendors who have been
cut off from new referrals or who have not received timely authorization
or payment. The milestone system leaves them more vulnerable to these service
disruptions than under hourly. Their feedback instructs the agency to be
efficient in its budgeting and payment systems.
(Return to Questions summary)
10. If your innovation is an adaptation or replication of another innovation, please identify the program or policy initiative and jurisdiction originating the innovation. In what ways has your program or policy initiative adapted or improved on the original innovation?
The Milestone Payment System is not an adaptation or replication of
another innovation.
(Return to Questions summary)
EVALUATION AND REPLICATION
11. If your program or policy initiative has been formally evaluated or audited by an independent organization or group, please provide the name, address, and telephone number of a contact person from whom the materials are available. Please summarize the principal findings of the independent evaluator(s) and/or auditor(s).
Although the Milestone payment system has not been formally audited, it has received the attention and scrutiny of entities interested in implementing programs modeled after Milestones. For instance, in 1996, after study of Oklahoma's Milestone payment system, in a report entitled "Disturbing the Peace," a Texas Performance Review, the Texas State Comptroller recommended that Oklahoma's Milestone method be a model for the Texas Rehabilitation Commission in development of a new payment system for their vendors. In 1995 the Australia Department of Disability Services Programs sent a study team to the U.S. to research outcome based payment systems. After extensive review they chose to use the Oklahoma Milestone System as the foundation of their system reorganization.
The Oklahoma DRS Milestone funding system has received national attention from the Rehabilitation Research and Training Center on Supported Employment at Virginia Commonwealth University. They stated that the Oklahoma Milestone model is "the fairest combination of cost control and individualized cost reimbursement ...to date nationally". A recently completed study of funding mechanisms by VCU concluded that outcome based payments with negotiated rates appeared to be the most effective method of program funding for supported employment. A follow-up study is being planned that would look specifically at Milestone type programs.
Numerous government organizations and service vendors have requested information. Information has been requested by rehabilitation and mental health agencies in Alabama, Arizona, Georgia, Massachusetts, Nevada, New Hampshire, Missouri, Texas, and Washington. Several states and localities are adopting the Oklahoma Milestone model. The City of St. Louis is modeling its Transitional Employment funding on the Oklahoma Milestone model. The Alabama Department of Rehabilitation services is currently studying the DRS Milestone funding system, including an on-site visit to Oklahoma, for conversion of its Supported Employment programs.
Oklahoma DRS is participating in an academic study answering the question,
"Is the method of funding supported employment related to program cost
efficiency from the tax payer and societal perspectives?" by Robert Cimera,
Ph.D., at Southern Missouri State University. An additional study including
"Milestones" has been proposed to the Rehabilitation Research and Training
Center on Supported Employment at Virginia Commonwealth University.
(Return to Questions summary)
12. To what extent do you believe your program or policy initiative is potentially replicable within other jurisdictions and why? To your knowledge, have any other jurisdictions or organizations established programs or implemented policies modeled specifically on your own?
The Milestone model of funding is adaptable to any public service with a definable outcome where service provision is contracted out to vendors. The State-Federal Vocational Rehabilitation system, of which DRS is a part, has the advantage of having a clear outcome: competitive employment of people with disabilities. This system could certainly be used by the 80 state rehabilitation agencies (blind and general rehabilitation agencies). It has been recommended to the Texas Rehabilitation Commission by the Texas Comptroller, and the Alabama Rehabilitation agency will soon visit Oklahoma to further investigate adoption of the system.
Many other federal and state programs desire the same employment outcomes for their customers. For instance, state agencies implementing Welfare-to-Work programs would be a key audience for this type of payment system. The Department of Rehabilitation Services has provided a copy of the Milestone payment system to the Oklahoma Department of Human Services, which is presently using a straight grant type payment system for their new work programs.
The model can also be used as the foundation of a voucher payment system,
which was a much discussed portion of the proposed Manpower Consolidation
legislation. The milestone steps (incremental payments) could each be vouchered
separately, allowing the consumer to move their voucher to a new vendor
at the end of any step. A voucher system built on a milestone foundation
would have the advantage of a clearly defined product with integrated quality
standards. The development of a market for these services would empower
the customer and increase service quality through the operation of competitve
forces.
(Return to Questions summary)
BUDGET
13. What is the program's current operating budget? What are the program's funding sources (e.g., local, state, federal, private)? What percentage of annual income is derived from each? Please provide any other pertinent budget information. Federal, state, local, or tribal government institutions must currently provide at least 50 percent of ongoing funding.
For delivery of consumer services via contracts with local vendors of services (e.g. Vocational Technical Schools, private non-profit sheltered workshops, community service agencies, etc.) the Department of Rehabilitation Services uses primarily Title I and Title VI-C funds administered by the Rehabilitation Services Administration of the U. S. Department of Education under the Rehabilitation Act of 1973, as amended in 1992. The split of federal/state funds is approximately 78%/22% for Title I funds and 100% federal funds for Title VI-C.
The operating budget for the Rehabilitation and Visual Services Division
of the Department of Rehabilitation Services is $45 million. The budget
for the Milestone programs is $4.1 million.
(Return to Questions summary)
ORGANIZATION CHART
14. Please attach an organization chart after your answer to question 17 to show the current number, responsibilities, and reporting relationships of key program employees or staff.
PROGRAM AWARDS AND RECOGNITION
15. Has the program or policy initiative received
any awards or other honors?
If yes, please list and describe the awards or honors and the
sponsoring organizations. Yes __. No X.
16. Has the program received any press or other
media coverage to date?
If yes, please list the sources and briefly describe relevant
coverage. Yes __. No X.
Return
to Milestone Home Page