Validation of the Objective Structured Assessment of Technical Skill in Brasil

SUMMARY BACKGROUND The aim of this study was to perform a cross-cultural adaptation of the Objective Structured Assessment of Technical Skill (OSATS) tool into Brazilian Portuguese and to determine its reproducibility and validity in Brasil. METHODS A Brazilian Portuguese version of OSATS was created through a process of translation, back-translation, expert panel evaluation, pilot testing, and then its validation. For the construct and the concurrent validities, twelve participants were divided into a group of six experts and six novices, who had to perform tasks on a simulation model using human placentas. Each participant was filmed, and two blinded raters would then evaluate their performance using the traditional subjective method and then the Brazilian Portuguese version of OSATS. RESULTS The Brazilian Portuguese version of OSATS had the face, content, construct, and concurrent validities achieved. The average experts’ score and standard deviations were 34 and 0.894, respectively, for Judge 1 and 34.33 and 0.816 for Judge 2. In the case of novices, it was 13.33 and 2.388 for Judge 1 and 13.33 and 3.204 for Judge 2. The concordance between the judges was evident, with the Correlation Coefficient (Pearson) of 0.9944 with CI 95% between 0.9797 and 0.9985, with p < 10-10, evidencing the excellent reproducibility of the instrument. CONCLUSION This preliminary study suggests that the Brazilian Portuguese version of OSATS can reliably and validly assess surgical skills in Brasil.

Saved in:
Bibliographic Details
Main Authors: Campos,Marcelo Esteves Chaves, Oliveira,Marcelo Magaldi Ribeiro de, Assis,Lilian Bambirra de, Reis,Augusto Barbosa, Gonçalves,Flávio Bambirra
Format: Digital revista
Language:English
Published: Associação Médica Brasileira 2020
Online Access:http://old.scielo.br/scielo.php?script=sci_arttext&pid=S0104-42302020000300328
Tags: Add Tag
No Tags, Be the first to tag this record!
id oai:scielo:S0104-42302020000300328
record_format ojs
spelling oai:scielo:S0104-423020200003003282020-05-29Validation of the Objective Structured Assessment of Technical Skill in BrasilCampos,Marcelo Esteves ChavesOliveira,Marcelo Magaldi Ribeiro deAssis,Lilian Bambirra deReis,Augusto BarbosaGonçalves,Flávio Bambirra Educational measurement Simulation Training Surgical procedures, operative/education Surveys and questionnaires SUMMARY BACKGROUND The aim of this study was to perform a cross-cultural adaptation of the Objective Structured Assessment of Technical Skill (OSATS) tool into Brazilian Portuguese and to determine its reproducibility and validity in Brasil. METHODS A Brazilian Portuguese version of OSATS was created through a process of translation, back-translation, expert panel evaluation, pilot testing, and then its validation. For the construct and the concurrent validities, twelve participants were divided into a group of six experts and six novices, who had to perform tasks on a simulation model using human placentas. Each participant was filmed, and two blinded raters would then evaluate their performance using the traditional subjective method and then the Brazilian Portuguese version of OSATS. RESULTS The Brazilian Portuguese version of OSATS had the face, content, construct, and concurrent validities achieved. The average experts’ score and standard deviations were 34 and 0.894, respectively, for Judge 1 and 34.33 and 0.816 for Judge 2. In the case of novices, it was 13.33 and 2.388 for Judge 1 and 13.33 and 3.204 for Judge 2. The concordance between the judges was evident, with the Correlation Coefficient (Pearson) of 0.9944 with CI 95% between 0.9797 and 0.9985, with p < 10-10, evidencing the excellent reproducibility of the instrument. CONCLUSION This preliminary study suggests that the Brazilian Portuguese version of OSATS can reliably and validly assess surgical skills in Brasil.info:eu-repo/semantics/openAccessAssociação Médica BrasileiraRevista da Associação Médica Brasileira v.66 n.3 20202020-03-01info:eu-repo/semantics/articletext/htmlhttp://old.scielo.br/scielo.php?script=sci_arttext&pid=S0104-42302020000300328en10.1590/1806-9282.66.3.328
institution SCIELO
collection OJS
country Brasil
countrycode BR
component Revista
access En linea
databasecode rev-scielo-br
tag revista
region America del Sur
libraryname SciELO
language English
format Digital
author Campos,Marcelo Esteves Chaves
Oliveira,Marcelo Magaldi Ribeiro de
Assis,Lilian Bambirra de
Reis,Augusto Barbosa
Gonçalves,Flávio Bambirra
spellingShingle Campos,Marcelo Esteves Chaves
Oliveira,Marcelo Magaldi Ribeiro de
Assis,Lilian Bambirra de
Reis,Augusto Barbosa
Gonçalves,Flávio Bambirra
Validation of the Objective Structured Assessment of Technical Skill in Brasil
author_facet Campos,Marcelo Esteves Chaves
Oliveira,Marcelo Magaldi Ribeiro de
Assis,Lilian Bambirra de
Reis,Augusto Barbosa
Gonçalves,Flávio Bambirra
author_sort Campos,Marcelo Esteves Chaves
title Validation of the Objective Structured Assessment of Technical Skill in Brasil
title_short Validation of the Objective Structured Assessment of Technical Skill in Brasil
title_full Validation of the Objective Structured Assessment of Technical Skill in Brasil
title_fullStr Validation of the Objective Structured Assessment of Technical Skill in Brasil
title_full_unstemmed Validation of the Objective Structured Assessment of Technical Skill in Brasil
title_sort validation of the objective structured assessment of technical skill in brasil
description SUMMARY BACKGROUND The aim of this study was to perform a cross-cultural adaptation of the Objective Structured Assessment of Technical Skill (OSATS) tool into Brazilian Portuguese and to determine its reproducibility and validity in Brasil. METHODS A Brazilian Portuguese version of OSATS was created through a process of translation, back-translation, expert panel evaluation, pilot testing, and then its validation. For the construct and the concurrent validities, twelve participants were divided into a group of six experts and six novices, who had to perform tasks on a simulation model using human placentas. Each participant was filmed, and two blinded raters would then evaluate their performance using the traditional subjective method and then the Brazilian Portuguese version of OSATS. RESULTS The Brazilian Portuguese version of OSATS had the face, content, construct, and concurrent validities achieved. The average experts’ score and standard deviations were 34 and 0.894, respectively, for Judge 1 and 34.33 and 0.816 for Judge 2. In the case of novices, it was 13.33 and 2.388 for Judge 1 and 13.33 and 3.204 for Judge 2. The concordance between the judges was evident, with the Correlation Coefficient (Pearson) of 0.9944 with CI 95% between 0.9797 and 0.9985, with p < 10-10, evidencing the excellent reproducibility of the instrument. CONCLUSION This preliminary study suggests that the Brazilian Portuguese version of OSATS can reliably and validly assess surgical skills in Brasil.
publisher Associação Médica Brasileira
publishDate 2020
url http://old.scielo.br/scielo.php?script=sci_arttext&pid=S0104-42302020000300328
work_keys_str_mv AT camposmarceloesteveschaves validationoftheobjectivestructuredassessmentoftechnicalskillinbrasil
AT oliveiramarcelomagaldiribeirode validationoftheobjectivestructuredassessmentoftechnicalskillinbrasil
AT assislilianbambirrade validationoftheobjectivestructuredassessmentoftechnicalskillinbrasil
AT reisaugustobarbosa validationoftheobjectivestructuredassessmentoftechnicalskillinbrasil
AT goncalvesflaviobambirra validationoftheobjectivestructuredassessmentoftechnicalskillinbrasil
_version_ 1756410054989316096