PDF Asesoramiento

8/3/2019 PDF Asesoramiento

1/7

1994;105;710-714Chest

S L Gray, A C Nance, D M Williams and C C Pulliamtechnique.in the evaluation of metered dose inhalerAssessment of interrater and intrarater reliability

http://chestjournal.chestpubs.org/content/105/3/710

can be found online on the World Wide Web at:The online version of this article, along with updated information and services

) ISSN:0012-3692http://chestjournal.chestpubs.org/site/misc/reprints.xhtml(without the prior written permission of the copyright holder.reserved. No part of this article or PDF may be reproduced or distributedChest Physicians, 3300 Dundee Road, Northbrook, IL 60062. All rights

ofbeen published monthly since 1935. Copyright1994by the American Collegeis the official journal of the American College of Chest Physicians. It hasChest

1994 American College of Chest Physiciansby guest on October 21, 2011chestjournal.chestpubs.orgDownloaded from
http://chestjournal.chestpubs.org/content/105/3/710http://chestjournal.chestpubs.org/content/105/3/710http://chestjournal.chestpubs.org/content/105/3/710http://chestjournal.chestpubs.org/site/misc/reprints.xhtmlhttp://chestjournal.chestpubs.org/site/misc/reprints.xhtmlhttp://chestjournal.chestpubs.org/http://chestjournal.chestpubs.org/http://chestjournal.chestpubs.org/site/misc/reprints.xhtmlhttp://chestjournal.chestpubs.org/content/105/3/710


2/7

*From the U niv ers ity o f North Carolina , School of Ph arm acy,Chap el H il l, N C .

M anuscrip t receiv ed F eb ru ary 9, 1 993 ; revision accepted June 30 .

71 0 Evaluat ion of M D I T ec hn iq ue (G ra y et a !)

A ssessm en t o f In te rra te r and In tra ra te rR e liab ility in the E va lu a tion o f M e te redDose Inha le r T echn ique *S he lly L . G ra y, P ha rm .D .; A lisa C . N an ce , B .S .;D enn is M . W illiam s, P ha rm .D .; an d C harle s C . P u lliam , M S .

Study ob jec t ive : To d eterm ine if a tra in ing sess ionusing v ideo taped m etered d ose in ha ler (M D I) p er for -man c e s ca n resu lt in h igh in terra ter an d in trara terrel iab i l i ty o f five eva lu a tors a sse s sin g M D I tech n iq ue .Des ign : F ive ev a lua tors ( three ph arm a cis ts, tw o pu l-monary fellow s) w ere tra in ed to eva lua te M D I tech -n iq ue d ur in g a 2 -h tra in in g sess io n . T he tra in ingses sion con s is ted o f verba l in s truc tion a nd p rac tica lexpe rien ce in eva lua ting M D I tech n iq ue us ing v ideo -taped M D I per form ances o f si x n on stu dy su b jec ts .Af t e r th e tra in in g sess ion , th e ev a lua tors in dep en -den tly o bse rv ed the sam e v id eo tap ed M DI d em on -stratio ns o f 14 sub jec ts on tw o occas ion s sepa rated b ya 7 - to 10-clay in te rv al. In terrate r and in tra ra te rre l iab i l i ty w as d ete rm ined fo r ind iv idua l s tep s byca lcu la t ing p ercen t ag reem en t an d in trac la ss co rre la -lion (ICC ) coe ffic ien t.Resu l ts : In terra ter . T he in terra ter re lia b ility for m d i-v idua l steps ran ged from 29 to 86 percen t ( IC Ccoe ffic ien t=O .13 to 0 .81 ). S tep s in w h ich ev a lua torsw ere In ag reem en t for les s th an 9 o f the 14 su b jec tsw ere sh ak in g th e inh a ler b e fore in h a la tion , ex ha lin g ,

con tin u ing to in ha le slow ly , and adeq ua te b rea thho ld . In trara ter : T he overa ll p ercen t agreem en t b ystep ra nged from 74 to 97 p ercen t. E xha lin g tofu nc tiona l res idua l vo lum e (7 6 p ercen t) an d con tinu -in g to inhale slow ly an d deep ly (7 4 percen t) h ad th elow es t ov era ll agreem en t b e tw een the f ir s t an d sec -on d observa tion da y Th e cons isten cy o f eva lua ting astep b e tw een th e tw o observat ion day s var ied con sid -erab ly d epen d in g o n the step an d eva lu a tor .Conc l us i ons : H igh in terra ter and in trara ter re liab ilityin M DI eva lua tion is d ifficu lt to ob ta in . C lin ic ian s andresearchers in vo lved in M D I eva lua tion an d ed uca -lio n sh ou ld be tra in ed to ach ieve co ns isten cy . A s in g let ra in ing se ss ion u s in g v id eo tap ed M D I dem onstra -lio n s w as n o t ad equ a te in ach iev in g con sisten cy am on geva lu a tors. T o im p rove a ccu racy o f research resu lts ,r e searchers sh ou ld inc lud e a t lea s tw o eva lu a tors toassess M D I tech n iq ue o r ta ke o th er m easu res to showand rep or t re lia b ility . (C hes t 19 94; 105 : 7 10-14 )

IC C = in trac la ss corre la tion ; MDI = m etered d ose in -ha ler

I nh a latio n the rap y is the p rim ary treatm en t op tio nfo r pa tien ts w ith pu lm ona ry d isease . T he b en -

e fits tha t inh ala tion trea tm en t o ffers o ve r o ra l treat-m en t in clu de a rap id on se t o f ac tion , a low erin cid en ce of sys tem ic ad ve rse e ffec ts an d d irec tadm in istra tion to the re sp ira to ry trac t. T hese ad -v an tages o f inh ala tio n the rap y , how eve r , can beoffs et if p atien ts do n o t u se p rope r in ha ler tech -n iq ue . Inad eq ua te inh ale r tech n iq ue has been re -p o rted to b e as h igh as 62 to 69 p ercen t.2 In co rrecttech n iqu e substan tia lly dec reases th e am oun t o fd rug reach in g th e fun ctio na l airw ay and , the re fo re ,red uces the e ff icacy o f th e in ha led ag en t.3 4

S eve ra l fac to rs m ay accoun t fo r th is h igh ra te o finco rrec t u se . A m ajo r p rob lem is tha t hea lthcarep ro fe ssion als o ften d o no t spend su ffic ien t tim eeduca ting pa tien ts rega rd ing prop er use. V erb alin struc tion is m o re e ffec tiv e th an w ritten instruc -tion in teach in g pa tien ts p rop er m e tered do se in -ha le r (M D I) tech n iqu e .5 6 S eve ral s tud ies have show ntha t m any physic ians an d pha rm ac ists are no t fam il-ia r w ith p rop er M D I techn iq ue .7 9 T hus, hea lthca re

p ro fes siona ls m ay g ive in ad eq ua te and inco rrec tin struc tions to pa tien ts w hen they do co unsel. M anypa tien ts m ay need repe titive instruc tion to m asterp rope r inha le r tech n iqu e.1# {176} S teps th at po se spec ia ld iff icu lty in clu de the co ord ina tion of ac tu atio n andin ha la tion , inh a ling slow ly and deep ly , an d an ad-eq ua te b rea th h o ld . F urthe rm ore, pa tien ts m ayhave physica l o r m en tal cond itions im ped ing prop erus e of an M D I.2

P eriod ic eva lua tion o f M D I techn ique is e ssen tia lto m a in ta in p rope r u se and m ax im ize th e b en efitsfrom th is ty pe of d rug de live ry sy stem .#{176}3 In add i-tio n to these p ractice re la ted issues, the im po rtanceo f op t im a l in h a la tion techn ique shou ld be em pha-sized in c lin ica l re sea rch . The m easu rem en t o fm any eff icacy p a ram e te rs fo r c lin ica l tr ia ls in theev alu atio n of new agen ts a re d ep en den t on g oodin ha le r techn ique . Thus, s tudy sub jec ts m u st hav ead eq ua te inh ale r tech n iq ue to de te rm ine th e e ffec-tiv eness o f th e agen t u nde r in vestiga tion . A com -m on m eth od o f a sse ssing techn iq ue is fo r th e c lin i-cian o r in vestiga to r to observe the p atien t using thein ha le r. M any hea lthca re p ro fe ssion als m ay be in -vo lv ed in th e eva lua tio n an d ed uca tio n of M D Itechn ique , so consis ten t and rep ro duc ib le ev a lua -

http://chestjournal.chestpubs.org/http://chestjournal.chestpubs.org/


3/7

CHEST I 10 5 I 3 I MARCH , 1994 7 1 1

Tab l e 1-L i sts of Stepsf or E valuati n g Cor r ectM etered Dose Inhaler Techn ique

Step Description

1 Shake the in ha ler th orough ly2 Ib Id the in haler upr igh t, neck hyp erextend ed3 Exhale normally (to functional residual volume)4 P lace m ou thp iece in m outh , l ips closed around

mouthp i e ce5 A ctiva te can ister in beg in n in g o f slow inh ala tion6 Continue to inh ale slow ly and deep ly (ov er 4 s )7 H old b rea th at fu ll in sp iration for at least 5 to 10 s8 Shake inhaler thoroughly between inhalations9 Wait at least 1 ziOn between inhalations (this should

be tim ed)

tion of M D I technique among evaluators over timeand among multiple evaluators is important.

The evaluation of M D I technique is difficultbecause of the subjective nature of many of thesteps. W e used a training program including video-taped M D I demonstrations to assist health profes-sionals to become effective in consistently evaluat-ing M D I technique. By engaging the evaluators in apractice session, we attempted to show throughactual M D I demonstrations what consititutes cor-rect technique. T he objective of this study was todetermine if a brief training program could result inhigh interrater and intrarater reliability ofthe evalu-ation of M D I technique.

M ETHODSFourteen patients (9 w om en and 5 m en) w ho w ere curren tly

using an oral M D I were included in the study. T he patients wererecruited from the Pulmonary M edicine Clinic at the Universityof N orth Carolina H osp ita ls ov er a 4-w eek p eriod . A fter in -formed consent wa s ob tain ed , the p atien ts w ere asked to dem -ons tra te their techn iqu e w ithou t prom p tin g from the inv estiga-tor. Each patient was instructed to take two inhalations duringwhich time they were videotaped. V ideotaped MDI p erfor-i ances w ere used so th at the sam e perfo rm ance cou ld beview ed a t a second period to de term in e in tra rater re liab ility . A llpatients were given standard verbal instructions and were placedin a well-lit room at about the same angle to the video recorder.A fter their techn iqu e w as recorded , p atien ts w ere in stru cted onp rop er inh ale r techn iq ue if any prob lem s w ere n oted .

Th e list of s tep s used to d efine correct inha ler te chn iq ue is inTab le 1 . Th e sele ction of th ese steps w as based on a litera tu rereview, product package inserts, and input from 10 pulmonaryphysicians.3#{ 176} The N ational A sthma Education Program Ex-per t P ane l recogn izes three accep tab le m ethod s of M D I d eliv -ery. A lthou gh the op en m outh techn iq ue is cons idered op ti-m al, the closed m ou th techn iqu e is acceptab le fo r p atien tshaving difficulty. Closed mouth technique was the standard ofpractice at this institution and, therefore, was considered theonly accep tab le m e thod .

Three pharmacists and two pulmonary fellows were selectedto evaluate M D I technique. Selection of the evaluators wasbased on their fam ilia rity w ith and understand in g o f p rop erMD I use. T he f ive eva lua to rs p art icip ated in a 2 -hour train in gsession before study initiation. The training session consisted ofverbal instruction and a practice session using videotapedMDIperformances. Each step of inhaler technique was discussed in

detail , focu sin g on the m in im um criteria tha t shou ld b e m et tocon sid er that the step w as perform ed correct ly . Fo r steps thatconsist of two par ts, eva lua tor s w ere ins tru cted th at bo th pa rtsneeded to b e p erform ed co rrectly to score the step as correct .A fter v erb al in struc tion , the eva lua tor s gain ed experience ineva lua tin g techn iqu e b y observing v ideotaped M DI dem onstra-tions of six n on stu dy pa tien ts . T he v id eotape wa s s topped a ftereach pat ien t dem on stration , and th e eva lua tor s scored th eperformance independently using a standardized score sheetcom po sed of s tep s lis ted in T able 1 . T he scor ing of each p atien tdem on stration w as then d iscussed a s a group . If there w asd isagreem en t am ong eva lua tor s for a given step , the d em onstra -tion w as rep lay ed until a consensus wa s rea ched regard in g thesco rin g o f the s tep . Th e ag reem ent am ong evalua tor s w as no tform ally asses sed at the conc lus ion of th e train in g session .

T he vid eo tap e of the M D I p er form ance s of th e 14 stu dysub jec ts w as th en m ade availab le to each evaluator. Th e evalu -ato rs w ere in stru cted to clo sely observ e the fir st inha la tion o fthe tw o- inh ala tion sequence to b ecom e acqua in ted w ith thesub jec ts te chn iq ue. T he second inh ala tion wa s evalua ted andsco red u sin g a s tandard ized sco re sh eet . To evaluate steps 1 and8, th e evalu ato rs w ere also instruc ted to eva lu ate the shakebefore the f irs t in ha lation and reco rd th e tim e b etw een in hala-t ion s. Th e eva lu ato r sco red the sam e M DI p erform an ces on asecond occasion 7 to 10 day s late r. Th e evalu ato rs w ere allow edto view the video tap es on ly once on each s tud y day . To p reven tb ias , ea ch evalu ato r sco red th e p er form ance s in dep enden tly andwa s b lind ed to the o ther eva lu ato rs sco res .

Sta tistic a l Ana lysisIn te rra ter reliab ility is the m easu re of assoc iat ion o f the fiv e

eva lua tor s sco res wh en evaluat ing a sub jec ts M DI techn iq ue atone tim e p eriod . F or in terrate r re liab il ity , p ercen t agreem en twas calcu lated by countin g the number of tim esll fiv e e va lu a-tors w ere in agreem en t fo r a specific step d iv ided b y the num berof subjects and multiplied by00. Overall agreement wascalcu la ted as th e num ber o f t im es the evalu ato rs w ere inagreement divided by the p rodu ct of n um ber of subjects andnum ber of steps and m ultip lied by 100 .

In trara ter re liab il ity is the m ea sure o f var iat ion that occur sw ith in an eva lua tor wh en scor ing th e sam e M DI techn iq ued em onstra tion on m ore than one occasion . S im ila rly , fo r in tra raterreliab il ity , p ercent ag reem ent w as calcu la ted b y coun tin g thenum ber o f tim es an evalu ato r sco red a sp ecific s tep the sam e onth e f irs t and second observation days d iv id ed by th e num ber o fsu bjects and m ultip lied by00. Overall agreement was calcu-la ted a s the num ber o f tim es the eva lua to r w as in agreem ent ond ay 1 and day 2 d iv ided b y the produ ct of num ber of su bjectsand num ber of steps and then mu ltip lied b y 100.

Th e percent agreem ent is in f luenced by the d istrib ution o f correc t and in co rrect sco re s. If there is a h igh p rop or tion ofe ither correct or in co rrect scores, th en th e p ercen t agreem en tw ou ld be h igh p otentia lly b y chan ce agreem en t alone . Toa ccoun t for chance ag reem ent, in tra- and in terreliab ility wa sa lso de term in ed by using analysis of v ariance (ANOVA ) tech -n iqu es to ca lcu late the in tra cla ss correla tion (ICC ) co eff icien t(SAS In stitu te In c., C a ry, N C ). T he ICC coefficien tyieldsid en tica l re su lts a s the k appa coefficien t .7 Th e ICC coeffi-c ien t can ran ge from 0 to 1 .0 . An ICC coefficien t of 0 ind ica testh e reliab ility is no be tter th an chance, w herea s an ICC coe ffi-c ien t o f 1 .0 in d icates excellen t (p erfect) rate r reliab ility .

Inte rra te r Re liab ilityRESULTS

The percen tage o f sub jec ts for w h ich th e fiv eeva luato rs w ere in tota l ag reem ent by step is listed



4/7

Tab le 3-I n tr ar ater Rel i abi l i ty: Per cen t A gr eemen t a nd I CC Coeff icien t By E va lua tor a n d Step

Step

Evaluator

IC C

1

%

Evaluator

IC C

2

%

Evaluator

IC C

3

%

Evaluator

IC C

4

%

Evaluator

IC C

5

% O verall, %

1 1.0 100 0 57 .77 93 1 .0 100 0 57 812 1 . 0 1 0 0 . 6 5 9 3 0 7 9 NE* 1 0 0 NE* 1 0 0 9 43 1 . 0 1 0 0 0 7 1 . 0 6 6 4 0 5 0 . 8 7 9 3 7 64 1.0 100 1.0 100 .65 93 .77 93 1.0 100 975 . 6 5 9 3 NE* 9 3 NE* 9 3 . 2 9 7 9 NE* 1 0 0 9 16 .87 93 .15 57 .70 86 .17 57 .48 79 747 .87 93 1.0 100 .73 86 .45 71 .82 93 898 .86 93 .59 79 1.0 100 .86 93 .87 93 919 . 4 6 8 6 . 6 5 9 3 . 6 5 9 3 . 8 2 9 3 . 6 5 9 3 9 1

O verall % (Evalu ato r) 95 83 87 82 90

* NE = ICC coeff icien t cou ld n ot be ca lcu lated b ecause th ere w as no t suff icien t va ria tion in the scores.

7 1 2 Evaluat ion of M D I T ec hn iq ue (Gr ay e ta !)

Tab l e 2-In ter rater Rel iabi l i ty: Per cen t A greemen ta nd I CC for 5 E valuator s by Obser vati on Day

O bservation D ay I O bservation Day 2

S teps ICC % ICC

1 .26 57 .51 642 .14 86 .3 8 863 .13 36 0 214 .50 71 .6 7 865 .20 71 0 716 .28 29 .19 217 .58 57 .5 1 438 .81 79 .89 869 .78 86 .7 0 86

O verall 63 63

in T ab le 2 a long w ith th e ICC coeffic ien ts. T hepercen t agreem en t var ied con sid erab ly accord ing tothe step, ranging from 29 to 86 percent on the firstobservation day. T here were differences in thep e rc e n t a g re e m e nt b e tw e e n the first and secondobservation days for some steps with no consistenttrend for improved or reduced percent agreementon th e second day . T he h ighest p ercen t agreem en twas achieved for steps 2 (86 percent-holding inhalerupright), 8 (79 percent-shaking inhaler betweeninhalations) and 9 (86 percent-waiting at least 1 mmb e t w e e n inhalations). The ICC coefficient for thesesteps was quite variable. Steps 8 and 9 had an ICCcoefficient of 0.81 and 0.78 on the first observationday, respectively, whereas step 2 only had an ICCcoefficient of 0.14. A n ICC coefficient of less than0.3 was calculated for five of the nine steps on thefirst observation day, suggesting poor interraterreliability for these steps.In tr a r a te r Re lia b ility

The percen t ag reem en t and ICC coefficien ts forintrarater reliability are presented in Table 3. I ngeneral, the percent agreement achieved for mdi-v idua l steps was higher w ithin an evaluator thanb e t w e e n evaluators (T able 2). The overall percent

agreement by step ranged from 74 to 97 percent.Out o f a ll step s in wh ich an ICC coefficien t cou ld becalculated (40 out of45 steps), an ICC coefficient of0 .6 or above ind icating good reliab ility b etw een th efirst and second observation day was achieved for 68percent of the steps by individual evaluators. A nICC coeffic ien t of le ss than 0 .3 w as ca lcu la ted for23 percent of the steps.

The five eva lua to rs d iffered in th e ir ab ility toconsistently evaluate individual steps of M D I tech-nique. For example, evaluator one had excellentreliability w ith a 95 percent overall agreement be-tween the first and second observation days. A ll ofth e ICC coeffic ien ts w ere greater than 0 .6 w ith th eexcep tion o f one step . E va luator one a lso had per-fect reliability as indicated by an ICC coefficient of1 .0 fo r four o f th e n in e step s. In con tra st, eva luato rfour had an 82 percent agreement overall betweenth e tw o ob servation days and an ICC coefficien tgreater than 0.6 for only four of the steps.

S t e p s 3 (exhale normally) and 6 (continue toinhale slowly and deeply) had the lowest overallpercen t ag reem ent, 76 percen t and4 percent,respectively. N ot all of the evaluators, however, haddifficulty in consistently evaluating these steps.Three of th e five eva lua tors had agreem en t nobetter than chance betw een th e tw o days fo r step 3(IC C coeffic ien t = 0 , 0 , 0 .0 6), w hereas th e rem ain -ing two evaluators achieved an ICC coefficient of1.0 and 0.87.

DISCUSS IONA chieving high interrater and intrarater reliabil-

ity in the assessment of M D I technique is difficult.This study incorporated videotaped M D I perfor-mances into a -h training session in an attempt totra in five h ea lth professiona ls to con sistently eva lu-ate M D I technique. V ideotaped M D I performanceswere used to give the evaluators practice in ad-equately evaluating M D I technique in an interac-tive session to enhance assessment skills and ulti-



5/7

CHEST I 10 5 I 3 I MARCH , 1994 7 1 3

mately result in high rater reliability. D espite thistraining session, individual evaluators still varied inthe assessment of M D I technique. These findingscorroborate the results of a previous study con-ducted by OConnell etal. 8 These authors reportedthat six trials were required before a high interraterreliability was achieved between multiple evalua-tors. 5 Results from these studies carry importantimplications for clinicians and researchers involvedin the assessment of M D I technique.

T he direct benefit of using videotaped M D Idemonstrations to train healthcare professionals toconsistently evaluate M D I technique can not bedetermined from this study, because we did not usea control group. The interrater reliability observedin this study is similar to that reported by OConnellet al.8 A lthough different definitions of correctinhaler technique were used in these two studies,four steps were identif ied that were defined simi-larly and thus the interrater reliability could becompared. In general, our ICC coefficients weresimilar to the kappa coefficients achieved in the firstdemonstration trial (labeled T rial 2) ofthe OConnellstudy; kappa values of 1.0, 0.12, 0.54, and.5 5 wererepor ted for shaking, exhaling, close lips on inhaler(or ho ld 2 .5 to 5 cm from m ou th ), and brea thholding, respectively. 8 T he major discrepancy ininterrater reliability between these studies occurredfor the initial shaking of the inhaler, in which ourreliability was markedly lower. D espite the lowkappa coefficients after the first trial in the OConnellet al trial, a kappa score of 0.8 or greater wasachieved for all steps by the fifth trial (T rial 6),except for inhaling slow ly (ic 0 .6 ). T he au thorsconcluded that the revision of the checklist of stepsto be more concise, discussions before each trial,and practice resulted in high interrater reliabilityafter the fifth trial.

One lim ita tion of th is study is tha t w e d id no tdirectly assess interrater reliability at the end of thetraining session. A lthough a consensus was reachedfor each patient demonstration, this was accom-plished as a result of discussion and replaying thedemonstration if necessary. T he training did notcontinue until the evaluators were able to agree ona demonstration on the initial evaluation withoutdiscussion. T herefore, the evaluators may not haveachieved total agreement in assessing M D I tech-nique at the conclusion of the training session,which would explain the low interrater reliabilityobserved during the study. In any case, the lowinterrater reliability achieved after a single trainingsession illustrates the diff iculty in achieving consis-tency in M D I evaluation among multiple evalua-tors. In practice, most healthcare professionals in-volved in inhaler training and evaluation are unlikely

to undergo a tra in ing session a s exten sive as u sed inth is tr ia l. F urth erm ore , ag reem en t m ay have beenh igher than wha t w ou ld no rm a lly occur becau seeva luato rs w ere aw are that th ey w ere being eva lu -a ted a s pa rt of a research study .

Certain steps were more diff icult for evaluators toagree on . For exam p le, it w as o ften d ifficu lt tod eterm ine if a patien t exha led proper ly . Fo r som epa tien ts, th eir exhalation w as not a ccom pan ied bydetec tab le ch est m ovem en t o r aud ib le b reath sound s ;th ere fo re, exha lation was d ifficu lt to a ssess. T heo th er m a jor a rea s of d ifficu lty w ere in determ in ingmf th e patien t a ctiva ted in th e first pa rt of th einhala tion , con tinued to inha le slow ly and deep lyand held th e ir b reath fo r 5 to 10 s. F requen tly , itw as d ifficu lt to d ist in gu ish betw een when a patien tstopped inhaling and w here the breath hold began .R ecen tly , d ev ices hav e becom e availab le to ass ist inth e assessm ent of M D I techn ique (eg , a ero so linhala tion m on itors); how ever , it is not know nw heth er th ese a re u se fu l in addressing th e prob lem sme n t i o n e d.Lack of concen tration m ay exp la in the low reli-ability which was ob serv ed fo r shak ing the inhalerbefore th e fir st inha lation . T he in terra ter re liab ilitybetween the initial shake (57 percent agreement,IC C coefficient = 0 .26) w as con sid erab le low er thanth e shake betw een inhala tion s (79 percent ag ree -ment, IC C coefficient = 0 .81 ) . S in ce th e eva luatorsa chiev ed h igh re liab ility for th e second shake ,t isun lik e ly that th e low reliab ility o f th e firs t shake w asa resu lt o f d iff icu lty in a ssessing th is step . M oreprobab le is that th e eva luators w ere not pay inga tten tion in th e beg inn ing of th e dem onstration , orth ey fo rgot to record th e fir st shake.

T h is is th e fir st s tudy to eva luate in trara ter re li-ab ility . D efin ite d ifferen ces ex isted in th e ab ility o fth e five eva luators to con sis ten tly eva luate M D Itechn ique betw een th e first and second ob serva tiondays. T he reasons fo r low in trarater re liab ility forsome step s fo r som e eva luators is no t clear . T heeva luation days w ere sepa ra ted by on ly 7 to 10 days ;there fore , it was unlik e ly tha t th e eva luators fo rgotth e cr iter ia u sed to judge th e step s. Ano th er po ten -tia l fa ctor in flu encing in trarater reliab ility is tha tth e eva lua to r m ay have rem em bered a prob lem theyhad in a ssessing a sp ecif ic step on th e fir st ob serva-tion day , m ade a spec ia l po in t to crit ica lly assessthat step on th e second day , and perhap s changedth eir score . Add itiona lly , w ith exper ien ce th e eva lu -ators m ay have deve loped an in creased aw areness oftechn ique and sco red dem onstration s d ifferen tly onth e second ob servation day . P erhaps in trarater re -liability w ill improve as evaluators acquire moreexperience in evaluating M D I techn iqu e, as was d em -onstrated in a previous trialor intr ar ater reliability.8



6/7

7 1 4 Evaluat ion of M DI T ech niq ue (G ray et a! )

These results indicate the importance of report-ing the ICC coefficients (or kappa) along with thepercent agreement. The ICC coefficient allows aperSOn to evaluate the actual ability of the evalua-tors after taking into account tile amount of agree-ment due to chance. One limitation of using ICC orkappa is that it requires some variation in thescoring of subjects to calculate the coefficient. Inthis study, a high percent agreement (93 to 100percent) was calculated in f ive cases; however, theI C C coe ff icien t cou ld n o t b e ca lcu la t ed , b eca u sethere was no or limited variation in the scoring ofthe step for all patients. For example, in three cases,a n eva lu a to r h a d 100 percent agreement for a stepbetween the first and second observation day. I n allcases, no variation existed in the scores for thatspecif ic step, ie , the evaluator scored the step aseither correct or incorrect for all subjects for bothdays; therefore, an ICC coefficient couldot becalculated. I t is unlikely that the reliability of scor-ing the step on two different occasions was nobetter than chance, but rather the step was easier toscore, more patients performed the step correctly,o r t h e eva lu a to r w a s n o t v er y d isc r im in a t in g b e tw eensmall differences among the subjects. T o answer thisquestion definitively would require a larger samplesize with variability in scoring of the step.

I n summary, consistent assessment of M D I tech-nique is important in practice and research. T he useofvideotaped M D I performances in a single 2-hourtraining session was not successful in achieving highinterrater and intrarater reliability. The need fortraining and assessing evaluation skills of thoseinvolved with inhaler education is necessary. Therole that inconsistent evaluation and subsequentretraining plays in the high prevalence of subopti-mal inhaler technique is not known. Since consis-tent M D I eva lu a t ion is d ifficu lt t o a ch ieve , r e -searchers should consider incorporating at least twoevaluators to assess M DI t ech n iq u e t o m in im ize t h einfluence of observer error and thus improve accu-racy of results. Subsequent published reports in thisarea of practice and research should include somemeasure of interrater reliability.

R E F E R E N C E S1 H or s ley M C , B a il ie C R . R isk fa c t or s fo r in a d eq u a t e u se of

pressurized aerosol inhalers. J C lin P h a r m a co l T h er 1988;13:139-43

2 Dc B la q u ier e P , C h r ist en sen DB , C a r t e r W B , M a r t in 1 11 . U seand misuse of metered-dose inhalers by patients with chroniclung disease. Am RevRespir D is 1 989 ; 1 40: 910 -16

3 Newman SP, Pavia D , Clarke SW . I low should a pressurizedadrenergic bronchodilator benhaled? Eur J R esp ir D is 19 81 ;62:3-21

4 L in d gr en S . B a k e B , L a r sso n S . C lin ic a l c on seq u en ces o finadequate inhalation technique in asthma therapy. Eur JRespir D is 1987; 70:93-8

5 D c T u llio P L , C o r so n M E . E ffect o fp h a r m a cis t c ou n se lin g ona m b u la t o r y p a t ie n t s u se of a er o so lized b r o n ch od ila t o r s. AmH osp P h a r m 1987; 44 :18 02 -06

6 R ob er t s R J , R ob in son J D , D oe r in g P L , D a llm a nJ, S t e ev es R A .A com p a r ison of v a r iou s t y p es o f p a t ien t in st r u c t io n in t h ep r op e r a dm in ist r a t io n o f m e t er ed in h a le r s. D I C P 1982; 16 :5 3-9

7 M ickle T R , SeI fT il, F a r r C E , B ess DT , T siu S J , C a ld w ell F L .E va lu a t io n o f p h a r m a c ist s p r a ct ic e in patient education whend isp en sin g a m et e r ed -d ose in h a le r . D I C P An n P h a r m a cot h er19 90 ; 24 :92 7-3 0

8 Cuidiy CC, Brown WD , Stogner SW , George RB. Incorrectu se ofm et er ed d o se in h a le r s b y m ed ica l p er son n el . C h est 19 92;1 0 1: 3 1 - 3

9 K ellin g J S , S t r oh l K P , Sm it h R L , A lt o se M D . P h ys icia n k n ow l-ed ge in t h e u se o fca n is t er n eb u lize r s. C h es t 1 983 ; 83 :61 2-1 4

10 O B ey K A , J im L K , G ee J P , C ow en M E , Q u ig ley AE . A ned u ca t io n p r og r am th a t im p r ov es t h e p sych om o tor sk ills n eed edfor metaproterenol inhaler use. D ICP 1982; 16:945-48

1 1 A r m it a ge J M , W ill iam s SJ . I n h a ler t ech n iq u e in t h e e ld e r ly . A geAging 1988; 17:275-78

12 A llen SC, Prior A . W hat determines whethern elderly patientca n u se a m et e r ed d ose in h a le r cor r ect ly . B rD is Chest 1986;80:45-9

13 Sh im C , W ill iam s M I I . T h e a d eq u a cy of in h a la t ion of a er oso lf r om ca n ist er n eb u lizer s. Ani J M ed 1980; 69 : 891 -94

14 W h ela n AM , h a h n NW . O p t im iz in g d r u g d e liv er y f r ome-t er e d -d ose in h a ler s. D IC P A n n P h a r m a coth er 19 91; 2-5 : 6 38-45

15 N a t ion a l 1 -lea r t , L u n g , a n d B lo od I n st it u t eA sthma EducationProgram Expert P a n el R ep or t . G u id e lin e s for t h e d ia gn os is andniallageniellt of asthma. J A llergy Clin Immunol 1991; 88:460-72

16 Sh r ou t P E , F le iss J L . I n t r a cla ss co r r e la t ion s: u ses in a s ses sin gr a t e r r e lia b il it y . P sy ch o l B u ll 1 979 ; 8 6 :4 20-28

17 Streiner D L , Norman C R . I -le a lt h measurement scales: ap r a ct ica l g u id e t o t h e ir d eve lop m en t and use. Oxford: OxfordUniversity P r es s, 1 98 9; 7 9-9 5

18 OConnell M B, H ew itt JM , L ackner TE. Consistency of evalu-ators assessing inhaler technique. A nnlle r gy 1 991; 6 7 : 603 -08



7/7

1994;105; 710-714ChestS L Gray, A C Nance, D M Williams and C C Pulliam

dose inhaler technique.Assessment of interrater and intrarater reliability in the evaluation of metered

October 21, 2011This information is current as of

http://chestjournal.chestpubs.org/content/105/3/710Updated Information and services can be found at:

Updated Information & Services

http://chestjournal.chestpubs.org/content/105/3/710#related-urlsThis article has been cited by 6 HighWire-hosted articles:

Cited Bys

http://www.chestpubs.org/site/misc/reprints.xhtmlonline at:Information about reproducing this article in parts (figures, tables) or in its entirety can be foundPermissions & Licensing

http://www.chestpubs.org/site/misc/reprints.xhtml

Information about ordering reprints can be found online:Reprints

the right of the online article.Receive free e-mail alerts when new articles cite this article. To sign up, select the "Services" link to

Citation Alerts

slide format. See any online figure for directions.articles can be downloaded for teaching purposes in PowerPointCHESTFigures that appear in

Images in PowerPoint format

http://chestjournal.chestpubs.org/content/105/3/710http://chestjournal.chestpubs.org/content/105/3/710http://chestjournal.chestpubs.org/content/105/3/710http://chestjournal.chestpubs.org/content/105/3/710#related-urlshttp://chestjournal.chestpubs.org/content/105/3/710#related-urlshttp://www.chestpubs.org/site/misc/reprints.xhtmlhttp://www.chestpubs.org/site/misc/reprints.xhtmlhttp://www.chestpubs.org/site/misc/reprints.xhtmlhttp://www.chestpubs.org/site/misc/reprints.xhtmlhttp://chestjournal.chestpubs.org/http://chestjournal.chestpubs.org/http://www.chestpubs.org/site/misc/reprints.xhtmlhttp://www.chestpubs.org/site/misc/reprints.xhtmlhttp://chestjournal.chestpubs.org/content/105/3/710#related-urlshttp://chestjournal.chestpubs.org/content/105/3/710

PDF Asesoramiento

Documents

Transcript of PDF Asesoramiento