The quality of physiotherapy care: the development and application of quality indicators using scientific evidence and routinely collected data embedded in the process of clinical reasoning

Quality improvement has become a central tenet of physiotherapy care. Quality indicators (QIs) as measurable elements of care have been used over the past 25 years to analyze and evaluate the quality of physiotherapy care. The aim of this article is to describe the state of the art regarding the development and application of QIs in physiotherapy primary care when embedded in a clinical reasoning process. In contrast to international clinical practice guidelines, Dutch physiotherapy clinical practice guidelines are generally based on the clinical reasoning process in combination with best available evidence. Information required to develop QIs is preferably derived by combining available systematic review-based scientific evidence, guideline-based recommendations, and routinely collected data with clinical evidence, professional expertise and standards, and patient perspectives. A set of QIs (n=28) in patients with whiplash-associated disorders was developed and embedded per step of the clinical reasoning process in physiotherapy care: (a) administration (n=2); (b) history taking (n=7); (c) objectives of examination (n=1); (d) clinical examination (n=4); (e) analysis and conclusion (n=2); (f) treatment plan (n=3); (g) treatment (n=2); (h) evaluation (n=5); and (i) discharge (n=2). The use of QIs represents a useful tool for measuring the (improvement of) quality of physiotherapy primary care, as many evidentiary gaps still exist in terms of diagnostics, prognostics, and treatment, and concerning patient-related outcome measurements in different patient groups such as patients with musculoskeletal pain. The recommended set of QIs embedded in the clinical reasoning process for patients with whiplash-associated disorders can be used as a starting point for the development of a general set of QIs that measure the (improvement of) quality of primary care physiotherapy.


Introduction
Quality improvement has become a central tenet of physiotherapy care and a statutory obligation in many countries [1]. There are numerous reasons why it is important to improve the quality of physiotherapy care, and these include enhancing the clinical reasoning process and making best use of clinical practice guidelines (CPG) and scientific evidence, improving patient-related outcomes and safety, and aligning care to what patients want in addition to what they need. These factors have prompted many new initiatives to develop and apply quality measurements, that is, quality indicators (QIs), over the past decades [2].
The purpose of this article is to describe the state of the art regarding the development and application of QIs in primary care physiotherapy when embedded in a clinical reasoning process.

Defining quality indicators and clinical reasoning
QIs have been defined as 'measurable elements of practice performance for which there is evidence or consensus that they can be used to assess the quality of the care provided' [3]. QIs may relate to structures (such as staff, equipment, and appointment systems), processes (such as clinical reasoning), or outcomes of care (such as a patient's functioning, disability, and participation) [4]. QIs have been used over the past 25 years to analyze and evaluate the quality of physiotherapy care [5,6]. Rational development of QIs is preferably based on systematic reviews and CPGs, supplemented by expert clinical experience and patient perspectives and values. Process and outcome indicators are also often based on patient registration systems that encompass operationalized, guideline-based recommendations as measurable elements of clinical practice [3,4,7]. It is generally accepted that the quality of patient registration systems is a reliable indicator of the quality of care [8].
In contrast to international CPGs, for example, for whiplash-associated disorders (WAD) [9], low back pain [10], and neck pain [11], Dutch physiotherapy CPGs are generally based on the clinical reasoning process in combination with best available evidence [12]. Most Dutch physiotherapy CPGs and evidence statements have been translated into the English language under the auspices of the Royal Dutch Association for Physiotherapy (KNGF), including those on low back pain [13] and neck pain [14].
Clinical reasoning has been defined 'as a process in which the physiotherapist, interacting with the patient and significant others, structures meaning, goals and health management strategies based on scientific evidence, clinical data, client choices and professional judgment and knowledge' [15,16]. Clinical decisions based on this clinical reasoning process are not only related specifically to guidelinebased recommendations but are also consistent with a more comprehensive approach to managing patients with, for instance, musculoskeletal pain, such as nonspecific low back pain or neck pain.
The clinical reasoning and decision cycle is an internationally accepted concept that facilitates problem solving and decision making in daily practice. The transparency of the clinical reasoning process, as provided by measurable elements such as QIs, is considered a cornerstone of the quality of care [7]. Physiotherapists tend to focus on the quality of implementation and evaluation of CPGs, including process and outcome indicators.

Development and application of quality indicators
The most commonly used method for development of QIs in the Netherlands is an iterated consensus rating procedure (similar to that used internationally) [17]. A number of Dutch studies of different patient groups have generated a set of guideline-based QIs, expressed as percentages ranging from 0 to 100%, with the number of times a QI was met as the numerator and the number of patients assessed as the denominator [18][19][20][21][22][23][24]. We give a few examples from our study of patients with WADs [23]. The numerator score for the number of patients subjected to a methodically performed history taking and recording of sociodemographic characteristics (noted as yes) was 365; the extent to which examination objectives were in agreement with patient's history taking (noted as yes) was 319, and the extent to which treatment goals were in agreement with prognostic health profile and time phase since accident (noted as yes) was 411. The denominator was the number of patients who participated in the study (N=457). The QIs were 79.9, 69.8, and 89.9%, respectively. To allow for interpretation as performance targets, percentage scores of QIs were categorized as negligible (0-20%), weak (21-30%), very inadequate (31-40%), inadequate (41-55%), sufficient (56-65%), substantial (66-75%), good (76-85%), very good (86-95%), and excellent (96-100%). The achieved QIs in the given examples in our study were respectively good, substantial, and very good.
As mentioned previously, a desired performance target can be determined in consultation with different stakeholders. The Dutch Royal Association for Physical Therapy (KNGF), in consultation with physiotherapists working in primary care, has set the target standard for QIs concerning the steps of the clinical reasoning process to a minimum of 'substantial' (66-75%). This minimum has been chosen to prevent ceiling effects. Ceiling effects in this context refer to the percentage of physiotherapists that have the highest score possible, thus making it difficult to measure relevant changes in the quality of physiotherapy care over time.

Method of development and evidence supporting quality indicators
The preferred method of QI development consists of five steps: (a) extraction of recommendations from CPGs, patient-related outcome measurements, and literature, particularly systematic reviews; (b) transformation of recommendations into QIs by phrasing them as the average degree (in %) to which patients were subjected to a methodically performed clinical reasoning process, including the level of evidence supporting the formulated QIs graded from levels I-IV, based on a national consensus document [25]; (c) appraisal by an expert and user panel, including scoring of the set of QIs on a five-point Likert scale (1=not at all to 5=completely) based on acceptability, feasibility, clarity, and relevancy to the physiotherapy care process; (d) classification of process indicators into the nine steps of the clinical reasoning process; and (e) classification of outcome indicators in accordance with the International Classification of Functioning, Disability and Health (ICF) [26] such as body functions, activity and participation, and personal and environmental factors. For a detailed description of the methodology involved in the development of QIs for general practice, see Campbell et al. [7], and for physiotherapy, see Oostendorp et al. [22], Oostendorp et al. [23], and Scholte [24].
The methods used for indicator development in physiotherapy are now briefly explained by means of a recently published example on the quality of physiotherapy care in patients with WAD [23]. Recommendations (n=96) in relation to the physiotherapy clinical reasoning process were independently extracted by two specialized physiotherapists from the Dutch CPG Physiotherapy Management and WADs [27,28], and the Quebec Task Force on WAD [29]. These recommendations were then transformed into a set of 28 QIs by phrasing them as the average degree (in %) to which patients were subjected to a methodically performed clinical reasoning process, for example, the average degree (in %) to which patients underwent a methodically performed history taking, the average degree (in %) to which accident-related information was noted, the average degree (in %) to which treatment goals were determined and recorded in agreement with individual prognostic health profiles and the time phase since an accident, and the average degree (in %) to which physiotherapy modalities agreed with treatment goals and with time phases since an accident.

From quality indicators to the process of clinical reasoning and evidence support
In the aforementioned example, the set of QIs was classified per step of the clinical reasoning process in physiotherapy care, including the number of QIs and the level of evidence per step: (a) administration (n=2); (b) history taking (n=7); (c) objectives of examination (n=1); (d) clinical examination (n=4); (e) analysis and conclusion (n=2); (f) treatment plan (n=3); (g) treatment (n=2); (h) evaluation (n=5); and (i) discharge (n=2). A complete overview of these 28 indicators was recently published as a supplement to our study [23]. Table 1 presents the complete set of QIs (n=28) for the physiotherapy care process of patients with WADs [23].
Only two indicators were supported by level I evidence (psychometric quality of the outcome measures), whereas five indicators were partly supported by level II evidence (evidence combined with consensus). Twenty-one indicators were supported by level IV evidence (expert opinion and professional consensus or standards). Most QIs were therefore based on level IV evidence. Table 2 presents the levels of evidence per step of the process of clinical reasoning.
Internationally, a number of clinical reasoning models in physiotherapy have been described, such as deductive reasoning versus narrative reasoning [15,16]. In our study [23], the clinical reasoning and decision-making process was based on a combination of deductive and narrative reasoning, which can be compared to internationally accepted general instruments such as the Hypothesis-Oriented Algorithm for Clinicians (HOAC II) [30,31]. Recently, a critical review described the utility of the ICF model in facilitating clinical decision making for physiotherapists and structuring the documentation of assessments and interventions [32]. The ICF model facilitates the process of clinical reasoning and decision making but is not a measurement tool for the quality of physiotherapy care. The ICF facilitates 'what to measure' but not 'how to measure the quality of care.' Despite a general focus on clinical reasoning over the past decades in medicine and physiotherapy, a method of evaluating the clinical reasoning process that is both objective and comprehensive has limited ability to evaluate this process [33]. The Script Concordance Test is one of the evolving tests which are considered to be valid and reliable tools for assessing clinical reasoning and judgment [34,35]. To the best of our knowledge, the use of various clinical reasoning and decision models in physiotherapy is not linked to a set of QIs. This means, in effect, that (improvement of) the quality of the clinical reasoning process in physiotherapy care is barely measurable.

Incompleteness of clinical reasoning in randomized clinical trials
Randomized clinical trials (RCTs) are commonly conducted to estimate the effectiveness of physiotherapy interventions. However, there are many areas of physiotherapy care for which the level of evidence is limited or entirely lacking, especially within the setting of the primary care physiotherapy practice.
In this context, a study by Maissan et al. [36] is instructive as it provides insight into the completeness of the clinical reasoning process in Reason for discharge, written report to physician in copy to patient

A final report is written and noted IV
If necessary, arrangement of aftercare 28. Aftercare is arranged IV a Levels of evidence: I=systematic review or more than 2 high-quality controlled trials or high-quality diagnostic studies or high-quality psychometric studies; II=two high-quality controlled trials or high-quality diagnostic studies or high-quality psychometric studies; III: high-quality noncontrolled trials or low-quality diagnostic studies or low-quality psychometric studies; IV: experts opinion and professional consensus or standard.
RCTs. In most RCTs (n=122) involving patients with nonspecific neck pain, the clinical reasoning process was reportedly incomplete, specifically in the diagnostic aspect of the process, with only 6% of the RCTs including a complete diagnostic process [36]. Similar findings were reported in the study of Smith and Bolton [37] who found that RCTs (n=30) included in the systematic review did not report diagnostic strategies and criteria for spinal manipulative therapy in patients with neck pain. These are important findings because it reveals how often the effectiveness of physiotherapy and manual therapy interventions is examined without a prior adequate diagnostic and decision-making process.
These findings suggest that it is necessary to use other methods to develop, implement, and evaluate the process of clinical reasoning, particularly the diagnostic steps of this process. The measurement of (the improvement of) the quality of physiotherapy care should be based on professional consensus and a complete process of clinical reasoning and decision making using a set of QIs from the perspectives of physiotherapists as stakeholder.
Routinely collected data as supplement The use of routinely collected data (RCD) is one of the preferred methods to measure the (improvement of) quality of physiotherapy care. RCD are collected in practices for reasons unrelated to research or prior research questions and are increasingly used in retrospective research. Nevertheless, RCD are not a substitute for RCTs [38] but they are a necessary counterpart, allowing measurement of the quality of the clinical reasoning process using QIs. Data from daily practice are readily available (although accuracy and completeness may vary) and represent a potentially rich source of information on large numbers of patients with diverse conditions. Use of existing data is less demanding and has fewer ethical constraints than planning, funding, and executing long-term pragmatic or experimental studies. RCD are diverse, available worldwide in both hospitals and general practice, and include clinical information from electronic health records, disease registries, and epidemiologic surveillance studies. Examples of RCD in primary physiotherapy care are, nevertheless, scarce [23,39].
QIs derived from RCD may cover the steps of the clinical reasoning and decision-making process (e.g. from the objectives of an examination to a clinical examination or from treatment goals to physiotherapy modalities that agree with treatment goals) or outcome measures (e.g. pretreatment and posttreatment of pain and functioning). However, proper use of RCD may require certain challenges to be overcome [40]. Accordingly, to improve the quality of reporting of studies that use RCD in physiotherapy, a checklist of items − guidelines for REporting of studies Conducted using Observational Routinely collected Data (RECORD) − has been developed [41] and adopted by journal editors (including the Journal of Orthopedic & Sports Physical Therapy) [42].
Despite the limitations of RCD studies, we expect that the results of studies using RCD could plausibly act as preliminary evidence regarding the completeness of the physiotherapy clinical reasoning and decision-making process and could be used to improve the design of future RCTs. In summary, although improvement of RCT quality is an important goal, broadening our focus to include the improved, accurate documentation of patient records is also a worthwhile goal.

Concluding remarks
Quality improvement has become a central tenet of health care, primarily in hospitals but increasingly also in primary care physiotherapy. A variety of methods can be used in processes of quality measurement and improvement. One of the most commonly used Table 2 The number of quality indicators (n=28) per step of the clinical reasoning and decision process a and per level of evidence b Step of clinical reasoning a Level of evidence b  I  II  III  IV  V  VI  VII  VIII  XI  Total   I  1  1 (n=4); v analysis and conclusion (n=2); VI treatment plan (n=3); VII treatment (n=2); VIII evaluation (n=5); and IX discharge (n=2). b Levels of evidence: level I=systematic review or more than 2 high-quality controlled trials or high-quality diagnostic studies or high-quality clinimetric studies; level II=two high-quality controlled trials or high-quality diagnostic studies or high-quality clinimetric studies; level III=high-quality noncontrolled trials or low-quality diagnostic studies or low-quality clinimetric studies; and level IV=expert opinion and professional consensus or standard. c () variables of composite indicators on two levels of evidence (see supplement: dx.doi.org/10.17504/protocols.io.uthewj6). For instance: Step IV Clinical examination: indicator 11 'The results of clinical evaluation of cervical musculoskeletal function testing are noted': measurement range of motion level of evidence II, and palpation level of evidence IV.
methods is the development and application of QIs as measurable elements of care. Information required to develop QIs is preferably derived, using systematic methods, by combining available systematic reviewbased scientific evidence and CPG recommendations with clinical evidence, professional expertise and standards, and patient perspectives.
The use of QIs derived from RCD represents a useful tool for understanding the quality of physiotherapy care, as many evidentiary gaps still exist in terms of diagnostics, prognostics, and treatment, and concerning patientrelated outcome measurements in patient groups such as those with low back pain or neck pain.
The combination of different sources of evidence regarding physiotherapy management in patients with diverse conditions in primary care may provide a broader view of the clinical reasoning process, and a more comprehensive and realistic view of the (improvement of) quality of routine practice compared with data gathered exclusively during an RCT.
The recommended set of QIs embedded in the clinical reasoning process for patients with WAD can be used as a starting point for the development of a general set of QIs that measure the quality of primary care physiotherapy.
International consensus on a set of QIs embedded in the physiotherapy clinical reasoning process, and on performance targets and scoring procedures, would improve the comparability of studies of the quality of physiotherapy care.