Development of a Numeracy Assessment Instrument for Vocational High School (VHS) Students

Numeracy  skills  are  essential  components  that  must  be  mastered  by  Vocational  High  School (VHS)  graduates  to  enter  the  workforce.  Developing a  quality  instrument  based  on  the  Item Response Theory (IRT) approach is necessary to measure numeracy skills. Since the development process involved VHS students, the instrument is developed based on the characteristics of the users. This study aims to (1) develop a quality numeracy assessment instrument for VHS students and (2) describe the numeracy skill profile of VHS eleven graders in Sleman Regency based on the assessment result. The subjects of the wide-scale trial were 374 students from nine VHSs in the Sleman  Regency  of  Indonesia,  assessed  in  nine  study  areas.  Data  analysis  used  the  Item Response Theory (IRT) of the Quest program with the outputs: (1) An instrument consisting of 35  questions  involving multiple-choice,  complex  multiple-choice,  short answer,  and  essay.  All items are valid qualitatively based on expert judgment and quantitatively based on infit and outfit MNSQ value and unidimensionality. The test item reliability value is 0.96, including in the special category,  and  the  person  reliability estimateis  0.89,  categorized  as  good.  Based  on  Total Information Function (TIF) and Standard Error of Measurement (SEM), the instrument is reliable in estimating the numeracy skills of the students with the ability ranging from -3.2 to 3.3 logit or covering the ability categories of low, medium, and high. The difficulty level of all items is in the range  of -2.0  to  2.0,  falling  into  the  good  category.  (2)  Most  students,  covering  67%  of  VHS students in Sleman Regency, have numeracy skills at the Basic level.