CAT Tools Usability Test with Eye-Tracking and Key-Logging: Where Translation Studies Meets Natural Language Processing
DOI:
https://doi.org/10.12797/MOaP.24.2018.41.03Keywords:
translation studies, usability research, CAT tools, natural language processing, eye-trackingAbstract
Computer assisted tools used to seem as though not made from the point of view of their targeted users [O’Brien, 2012:15]. However, their usability has been improving. In Translation Studies there exists a gap in research on process-oriented usability involving data triangulation. In our study based on the assumption that translation is a situated and complex activity [Risku, 2002, 2004], we aimed to address this gap with our experiment testing a new tool for translators, Concordia. This usability experiment with eye-tracking, keylogging, and screen recording directly involved translators (six translation trainees) in the development process through objectively collected data on effectiveness and efficiency of Concordia. Their satisfaction with Concordia was also a part of the usability test. We hypothesised that participants would be more efficient and effective when translating European Union texts with Concordia and that they will be satisfied with the tool. The results indicate that Concordia at its current state of development does not facilitate the process, but the participants were generally satisfied with the tool’s features.
Downloads
References
Bond, S. (2016), “Gunning Fog Index”, [online:] http://gunning-fog-index.com/ – 13.11.2016.
View in Google Scholar
Brooke, J. (1996), “SUS-A quick and dirty usability scale”, Usability Evaluation in Industry, 189(194), pp. 4-7.
View in Google Scholar
Campbell, S.G., Wayland, S.C., Goldman, A., Blok, S., Powell, A.L. (2013), “Speaking the user’s language: Evaluating translation memory software for a linguistically diverse workplace”, in: Proceedings of the Human Factors and Ergonomics Society Annual Meeting: Volume 57, SAGE Publications Sage CA, Los Angeles, CA, pp. 2042-2046, [online:] http://dx.doi.org/10.1177/1541931213571456.
View in Google Scholar
Christensen, T.P., and Schjoldager, A. (2011), “The Impact of Translation-Memory (TM) Technology”, in: Sharp, B., Zock, M. (eds.), Human-Machine Interaction in Translation: Proceedings of the 8th International NLPCS Workshop, Samfundslitteratur, pp. 119-130.
View in Google Scholar
Colominas, C. (2008), “Towards chunk-based translation memories”, Babel, 54(4), pp. 343-354, [online:] http://dx.doi.org/10.1075/babel.54.4.03col.
View in Google Scholar
Cooper, A. (2004), The Inmates Are Running the Asylum: Why High-tech Products Drive Us Crazy and How to Restore the Sanity, Sams Indianapolis, IN, USA.
View in Google Scholar
Dillon, S., Fraser, J. (2006), “Translators and TM: An investigation of translators’ perceptions of translation memory adoption”, Machine Translation, 20(2), pp. 67-79, [online:] http://dx.doi.org/0.1007/s10590-006-9004-8.
View in Google Scholar
Dragsted, B. (2004), Segmentation in translation and translation memory systems: An empirical investigation of cognitive segmentation and effects of integrating a TM system into the translation process, Ph.D. series, 5, Samfundslitteratur, København.
View in Google Scholar
Dragsted, B. (2006), “Computer-aided translation as a distributed cognitive task”, Pragmatics & Cognition, 14(2), pp. 443-464, [online:] http://dx.doi.org/10.1075/pc.14.2.17dra.
View in Google Scholar
Duchowski, A. (2007), Eye Tracking Methodology: Theory and Practice: Volume 373, Springer Science & Business Media, London.
View in Google Scholar
Ehrensberger-Dow, M., Massey, G. (2014), “Translators and machines: working together”, Man vs. Machine, 1, pp. 199-207.
View in Google Scholar
Ehrensberger-Dow, M., O’Brien, S. (2015), “Ergonomics of the Translation Workplace: Potential for Cognitive Friction”, Translation Spaces, 4(1), pp. 98-118.
View in Google Scholar
Flesch, R. (1948), “A new readability yardstick”, Journal of Applied Psychology, 32(3), pp. 221-233, [online:] http://dx.doi.org/10.1037/h0057532.
View in Google Scholar
Giammarresi, S. (2008), “Second Generation Translation Memory Systems and Formulaic Sequences”, Łódź Studies in Language, 17, pp. 419-430.
View in Google Scholar
Guillardeau, S. (2009), Freie Translation Memory Systeme für die Übersetzungspraxis, University of Vienna.
View in Google Scholar
Hutchins, E. (1995), “How a cockpit remembers its speeds”, Cognitive Science, 19(3), pp. 265-288.
View in Google Scholar
Hvelplund, K.T. (2011), Allocation of Cognitive Resources in Translation: An Eye-tracking and Key-logging Study, PhD Series, 10, Samfundslitteratur, Frederiksberg.
View in Google Scholar
Hvelplund, K.T. (2014), “Eye tracking and the translation process: reflections on the analysis and interpretation of eye-tracking data”, MonTI Special Issue – Minding Translation, 1(1), pp. 201-223.
View in Google Scholar
ISO 9241 (2011), “Ergonomics of Human-System Interaction”, Geneva: International Organization for Standardization.
View in Google Scholar
Jaworski, R. (2015), “Approximate sentence matching and its application in corpus-based research”, The Future of Information Sciences: E-Institutions, Openness, Accessibility and Preservation, 5, pp. 21-30.
View in Google Scholar
Krings, H.P. (2001), Repairing Texts: Empirical Investigations of Machine Translation Post-Editing Processes, Koby, G.S. (ed.), The Kent State University Press, Kent, Ohio.
View in Google Scholar
Krüger, R. (2016), “Contextualising Computer-Assisted Translation Tools and Modelling Their Usability”, Trans-Kom-Journal of Translation and Technical Communication Research, 9(1), pp. 114-148.
View in Google Scholar
Lagoudaki, E. (2006), “Translation memories survey 2006: Users’ perceptions around TM use”, in: Proceedings of the ASLIB International Conference Translating & the Computer: Volume 28, pp. 1-29.
View in Google Scholar
Lagoudaki, E. (2009), Expanding the Possibilities of Translation Memory Systems: From the Translator’s Wishlist to the Developer’s Design, Imperial College, London.
View in Google Scholar
Lemhöfer, K., Broersma, M. (2012), “Introducing LexTALE: A quick and valid Lexical Test for Advanced Learners of English”, Behavior Research Methods, 44(2), pp. 325-343, [online:] http://dx.doi.org/10.3758/s13428-011-0146-0.
View in Google Scholar
O’Brien, S. (2012), “Translation as human–computer interaction”, Translation Spaces, 1(1), pp. 101-122, [online:] http://dx.doi.org/10.1075/ts.1.05obr.
View in Google Scholar
Pavlović, N., Jensen, K. (2009), “Eye tracking translation directionality”, in: Pym, A., Perekrestenko, A., (eds.) Translation research projects 2, Intercultural Studies Group, Universitat Rovira i Virgili, Tarragona, pp. 93-109.
View in Google Scholar
Risku, H. (2002), “Situatedness in translation studies”, Cognitive Systems Research, 3(3), pp. 523-533, [online:] http://dx.doi.org/10.1016/S1389-0417(02)00055-4.
View in Google Scholar
Risku, H. (2004), Translationsmanagement: Interkulturelle Fachkommunikation im Informationszeitalter: Volume 1, Gunter Narr Verlag.
View in Google Scholar
Risku, H., Windhager, F. (2013), “Extended translation: A sociocognitive research agenda”, Target. International Journal of Translation Studies, 25(1), pp. 33-45, [online:] http://dx.doi.org/10.1075/target.25.1.04ris.
View in Google Scholar
Rudlof, C. (2006), “Handbuch Software-Ergonomie”, Usability Engineering.
View in Google Scholar
Smallwood, J., Schooler, J.W. (2006), “The restless mind”, Psychological Bulletin, 132(6), pp. 946-958, [online:] http://dx.doi.org/10.1037/0033-2909.132.6.946.
View in Google Scholar
Strohner, H. (1995), Kognitive Systeme: eine Einführung in die Kognitionswissenschaft, Springer-Verlag.
View in Google Scholar
Whyatt, B. (2012), Translation as a Human Skill: from Predisposition to Expertise = Przekład jako umiejętność człowieka: od predyspozycji do poziomu eksperta, Wydawnictwo Naukowe Uniwersytetu im. Adama Mickiewicza, Poznań.
View in Google Scholar
Zapata, J. (2016), “Investigating Translator-Information Interaction: A Case Study on the Use of the Prototype Biconcordancer Tool Integrated in CASMACAT”, in: Carl, M., Bangalore, S., Schaeffer, S. (eds.), New Directions in Empirical Translation Process Research: Exploring the CRITT TPR-DB, Springer, pp. 135-152, [online:] http://dx.doi.org/10.1007/978-3-319-20358-4_7.
View in Google Scholar
Downloads
Published
Issue
Section
License
Copyright (c) 2018 Olga Witczak, Rafał Jaworski
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
