Letter from NCSBN written to student who failed NCLEX...

3,932 Posts

Specializes in OB, M/S, HH, Medical Imaging RN.

this letter is posted as a free public service. i would like to take this opportunity to thank the national council of state boards of nursing, inc. (analiese garrison)

676 north st. clair street suite 550

chicago, illinois 60611-2921

312-787-6555

fax 312-787-6898

dear ms. garrison:

i am writing in response to your letter concerning computerized adaptive testing for nclex. perhaps i can explain how computerized adaptive testing for the nclex works. the goal of computerized adaptive testing or cat, is to determine your competence, based on the difficulty of questions you can answer correctly, not how many questions you can answer correctly. this is a fundamentally different approach than is used on paper-and-pencil tests, where everyone receives the same questions. cat examinations are individualized.

we know the exact difficulty of each of the approximately 3,000 questions in the pool, because each has been taken as a tryout question by thousands of candidates and then statistically analyzed. picture the questions all lined up, from easiest to hardest. if we asked you the easiest questions, you'd get most of them right. if we asked you the hardest, you would probably get most wrong. as we move from easy to hard, there will come a point where you go from getting more right...to...getting more wrong. this is the point where you are answering 50% correctly. questions harder than that, you would probably answer incorrectly (you'd get some right, but more wrong); questions easier than that, you would probably answer correctly. that point is different for everyone. nursing experts could probably answer at least one-half of the hardest questions we could ask. whereas, we'd have to ask beginning nursing students the very easiest ones before they could answer even one-half correctly. you probably fall somewhere between those two points. the goal of cat is to find that point for you. your competence level is related to the difficulty level of the questions at the point where you can answer half of the questions correctly.

first, the computer asks a relatively easy question, and if you answer it correctly, it asks a somewhat harder question. as you continue answering correctly, the questions get harder and harder. when you start missing questions, they get easier until you start answering them right again, then they get a little harder. each time you answer one correctly, the next is harder. each time you answer one incorrectly, the next is easier. this process continues as it zig-zags, narrowing in on the point where you answer 50% correctly, e.g., one right, then one wrong. that point represents your competence level. this is why everyone ends up correctly answering 50% of the questions they are asked. after you have answered the minimum number of questions, the computer compares your competence level to the passing standard amd makes one of three decisions:

if you are clearly above the passing standard, you pass and the examination ends.

if you are clearly below the passing standard, then you fail and the examination ends.

if your competence level is close enought to the passing standard that it's still not clear whether you should pass or not, then the computer continues to ask you questions.

"clearly" passing or failing is defined as when the "gray zone" around your competence level falls entirely above or below the passing standard. the gray zone is the region within which your competence level might vary if you answer more questions. the gray zone shrinks a little after each question because your competence level is based on more information.

if you can answer the difficult questions correctly, there's no point in wasting your time giving you a lot of easy questions. or, if you can't answer the easy ones correctly, then you won't be able to answer the difficult ones. in fact, the computer often could make a decision after less than the minimum of 60 questions, but 60 is necessary to ensure coverage of the nclex test plan. it is improtant you get the opportunity to answer several questions in each of the nclex test plan content areas in case you have particular strengths or weaknesses.

after each question, your competence level and the gray zone are recomputed, adding your new response to all of your previous answers. when the gray zone in entirely on one side or the other of the passing standard, you've clearly passed or failed and the examination ends.

of course, some people's competence level is very close to the passing standard. for some of these people, all 3,000 questions in the item pool still might not be enought to make it "clear" whether they should pass or fail. these are the people who take the maximum number of questions. at this point, the computer disregards the gray zone and simply looks at whether the final competence level, based on every question answered, is above or below passing. if you are above it, you pass. if not, you do not pass.

therefore, a candidate's final competence estimate is not determined by the number of questions she/he can answer correctly. this is a fundamentally different approach than is used on paper-and-pencil tests, where everyone receives the same questions. cat examinations are individualized. the examination continues until the difficulty level is found where you are answering about half of the questions correctly. this corresponds to your competence level. if the level is above the passing standard, then you pass; if not, you do not pass.

because the examination continues until it finds the level where you are consistently answering about 50% of the questions right, in the end everyone gets 50% right. what differs is the difficulty of the questions they were able to answer correctly half of the time. the pass/fail decision is based on the competence level corresponding to that difficulty, not on a percentange correct.

each examination is designed to meet all requirements of the nclex test plan with a certain percentage of questions in each test plan area. it is not designed to administer a rephrased questions for questions you answered incorrectly. each question is selected randomly from the item pool and any similarity between items is a coincidence.

you are not allowed to skip questions or go back to review or change previous answers because the heart of the cat methodology, adaptive branching through the examination, makes skipping or revising earlier answers logically invalid. once a answer is recorded, all subsequent questions administered depend, to an extent, on that response. if that response had been different, you would have received different questions. skipping and returning to earlier questions my be appropriate stategies for taking a conventional paper-and pencil examination; they do not make sense for a cat examination. you are not being disavantaged by the inability to skip questions or to go back to change previous answers. if you are uncertain of an answer and make an incorrect guess, your competence level is calculated to be slightly lower than it was just before the last question was administered. the next question presented to you will be easier, making it more likely you will answer correctly. thus, you will not "dig yourself into a hole" from which you cannot return, since computerized adaptive testing has a built-in, self-correcting mechanism. test anxiety is indeed a problem for many people. in fact, it was part of the motivation for going to computerized testing, where a candidate may test on the time and day of their choice, in a more private and peaceful environment than a crowded gymnasium with hundreds of other worried candidates. the sample questions give you an opportunity to "settle in" to the testing situation and to practice with the necessary keyboard strokes. if the first 10 "real" questions did not count towards your score,...{which you have suggested}...they would not be "real" anymore, and we would simply have 13 sample questions. the tutorial and sample questions provoide extensive practice with the system. additional sample questions probably would not help.

all legal and psychometric studies, and field test of computerized adative testing (cat) methodology indicate it is valid, reliable, fair, and defensible. as series of studies were conducted on over 11,000 candidates before the decision was made to implement cat. the studies consistently showed that nclex using cat provided comparable candidate performance to paper-and-pencil nclex. in addition, pass rates from the first year of cat were practically identical to those from the last year of paper-and pencil.....

i hope this information will assist you....

sincerely, ellen julian, ph.d. psychometrician

http://caring4you.net/concept.html