Large Language Models as a Tool of Psycho-Anthropological Researches
Abstract
Abstract
Abstract. The modern development of artificial intelligence has led to the active use of large language models (LLM) in various fields of scientific and professional knowledge. LLM is based on the principles of deep machine learning using a large amount of data and parameters to solve natural language tasks. As part of our research on the use of large language models in scientific papers, we conducted a theoretical and analytical study based on the materials of leading scientific journals in psychology in articles published since 2023. Two trends in research were identified. In domestic works, the LLM model was used to develop expert systems for categorizing socio-psychological phenomena. In foreign works, the use of psychodiagnostic methods for assessing the “abilities” of large language models and constructing reasoning is actively presented. In LLM construction systems, they often rely on psychological theories focused on the construction of everyday knowledge, socially conditioned interactions, and systems of attributive processes. Currently, there is a significant expansion of work in the humanities using LLMs, which provides an opportunity for the development of new areas within cognitive sciences and expands the possibilities for implementation in robotics, but also presents many challenges, particularly in assessing the quality of reasoning and the reliability of the acquired knowledge.
Резюме. Современное развитие искусственного интеллекта привело к активному использованию больших языковых моделей (LLM) в различных областях научного и профессионального знания. В основе LLM лежат принципы глубокого машинного обучения с использованием большого объема данных и параметров, направленного на решение задач на естественном языке. В рамках исследования использования больших языковых моделей в научных работах мы провели теоретико-аналитическое исследование на базе материалов ведущих научных журналов по психологии в статьях c 2023 года. Были выделены две тенденции работ. В отечественных работах LLM модель использовали для развития экспертных систем категоризации социально-психологических феноменов. В зарубежных работах активно представляется использование психодиагностических методик для оценки «способностей» больших языковых моделей и построения рассуждений. В системах построения LLM они часто опираются на психологические теории, ориентированные на построение повседневных знаний, социально обусловленных взаимодействий, систем атрибутивных процессов. На данный момент происходит большое расширение работ в гуманитарных науках с использованием LLM, что дает возможность для развития новых направлений в рамках когнитивных наук, расширения возможностей для реализации в робототехнике, но несет и много трудностей, прежде всего в вопросе об оценке качества построения рассуждений и достоверности получаемых знаний.
Ключевые слова: большие языковые модели (LLM), искусственный интеллект, научная статья, социальная группа, рассуждение, психодиагностика
Introduction
The modern development of artificial intelligence has led to the active use of large language models (LLM) in various fields of scientific and professional knowledge. LLM is based on the principles of deep machine learning using a large amount of data and parameters to solve natural language tasks (Chuganskaya et al., 2024). Now, it is not just about generating plausible texts; modern models have gone further by combining the creation of text, human communication, and solving tasks related to intelligent information retrieval and expert work. All of this is becoming a tool for expanding and, in many ways, reshaping human work in a number of text-related fields (e.g., writing academic and news texts, evaluating motivation letters and resumes in HR departments) (Makhmutova, Kuzmina, 2019). In a broad sense, this is a change in the perception of human abilities, which are expanded by the use of such an “external” tool as LLM, as described by L.S. Vygotsky. The use of large language models can serve as an intermediate tool in the development of specific human abilities, which, as noted by V.N. Druzhinin, “are specifically human abilities that are latent and manifest themselves through specific abilities that are influenced by the environment” (Druzhinin, 1999, p. 103). The technologicalization of the environment and the human relationship with accelerating time act as facilitating factors for such development (Chuganskaya, 2022).
Representatives of MGIMO’s scientific schools, who studied axiological issues, intercultural communication, and the development of digitalization and artificial intelligence, also addressed certain aspects of the study of digitalization in the context of anthropological research. Modern society is in a global process of migration of people from different ethnic groups and the formation of culture-specific types of society. The mixing and mutual influence of cultures leads, in some situations, to integration, in which cultural dialogue contributes to changes in the ethnos and constructive interaction, while in others, it leads to confrontation that can escalate into military conflicts. It is becoming more difficult for people to adapt to a multicultural environment, form their own ideas about ethnic identity and cultural values, and cope with the challenges of cultural diversity. This is especially true for young people. One way to overcome language barriers is through the use of digital products and large language models. In Russia, MGIMO University is studying the significant issues and directions of modern global change, including the interdisciplinary foundations of technology development. The scientific works of A.V. Torkunov (Torkunov, 2022), M.V. Silantieva (Silanteva, 2013, 2019), P.I. Kasatkin (Kasatkin, 2018), O.V. Gaman-Golutvina (Gaman-Golutvina, Smorgunov, 2023), and others, taking into account the results of the analysis of global challenges, are among the most significant in building the foundations of scientific knowledge in new interdisciplinary fields related to the topic of “human-technology”.
The direction of digitalization of modern society and the formulation of an understanding of the use of trust-based artificial intelligence as a generator of new information and the systematization of knowledge, as well as the use of new technologies in education, are reflected in the works of Ulanova A.E. (Ulanova, 2022).
Method
As part of our research on the use of large language models in scientific papers, we conducted a theoretical and analytical study based on the materials of leading scientific journals in psychology in articles published since 2023 (with an indexing level of at least RSCI and included in the CyberLeninka database (https://cyberleninka.ru/) and a foreign archive of articles on the use of psychodiagnostic methods for training large language models (https://www.researchgate.net/) The selection of articles from this period was based on the active development of large language models. Our interest was focused on works that were categorically related to the field of psychology.
Results
Among the total volume of the data articles received, it should be noted their small number, less than 10. Two trends of works were highlighted. First, in domestic works, the LLM model was used to develop expert systems for categorization of socio-psychological phenomena. In the work of Vinokurov F., Panov K., Sadovskaya E. (Vinokurov et al., 2024), 900 messages about innovative technologies (for example, virtual assistants) were analyzed using gpt-4-turbo and some previous models (two models ChatGpt and Bert). With their help, content analysis was carried out based on the pre-training of models on a coordinated expert sample by categories: “Clarification”, “Status of action”, “Question”, “Expression of consent / approval”, etc. The results were obtained with a degree of consistency of 56% and 70%, which indicates the high potential for using such models for socio-psychological tasks.
Discussion
In foreign works, the use of psychodiagnostic methods for assessing the “abilities” of large language models and reasoning is actively presented. In LLM construction systems, they often rely on psychological theories focused on building everyday knowledge, socially conditioned interactions, and systems of attributive processes. This approach is experimentally presented in the works of Kosinski M. (Kosinski, 2023). The basis is based on techniques from the model of the theory of consciousness (theory of mind, ToM). The very definition of ToM as a system of logical reasoning used to predict the behavior of an individual by attributing certain mental states to him, emerged within the framework of the work of psychologists Wimmer H. and Premack D. G. with Woodruff G. (Chuganskaya et al., 2024), who studied the models of the emergence of human reason in the process of the development of the psyche. For Tom, it is important to consider only a few mental aspects of interactions, such as attributional processes and emotional understanding in the context of life situations, which are largely based on the construction and subsequent implementation of scenario information.
Methodologically, this understanding is mainly based on the concept of consciousness by D. K. Dennett, who believed that it consists of information streams that compete for access to the brain, which eliminates the difference between processes in natural human consciousness and the construction of robot programs. This understanding of the mind has been used in studies that not only evaluate the quality of tasks performed by new programs in human-computer interaction, but also use psychodiagnostic material as a basis for training. So, M. Kosinski used 40 tasks that were given as part of the study of thinking and attribution processes based on ToM (Kosinski, 2023). The result showed that it was in these tasks that the LLM models that formed the basis of GPT significantly improved their performance since 2020, reaching 95% correct answers for GPT-4 in March 2023.
Conclusions
It should be noted that there is currently a significant expansion of work in the humanities using LLM, which provides an opportunity for the development of new areas within cognitive sciences and the expansion of opportunities for implementation in robotics, but also presents many challenges, particularly in terms of evaluating the quality of reasoning and the reliability of the acquired knowledge.
References
- Chuganskaya, A. A. (2022) Modern trends in psychodiagnostics of large groups in social networks. In: R.V. Ershova (Ed.) Digital society as a cultural and historical context of human development: Collection of scientific articles and materials of the international conference, Kolomna, February 17, 2022. Kolomna: State Educational Institution of Higher Education of the Moscow Region “State Social and Humanitarian University”, pp. 281-284.
- Chuganskaya, A. A., Kovalev, A. K., & Panov, A. I. (2024) Sign-based image criteria for social interaction visual question answering Interest Group in Pure and Applied Logics. Logic Journal, 32 (4), 656-670. https://doi.org/10.1093/jigpal/jzae026
- Druzhinin, V. N. (1999) Psychology of general abilities.: Publishing House “Peter”
- Gaman-Golutvina, O. V. & Smorgunov, L. V. (2023) The Political in the Space of a Turbulent World Political Studies, 1, 7-10. DOI: https://doi.org/10.17976/jpps/2023.01.02
- Kasatkin, P. I. (2018). Value Axiomatics of the Educational Space: Specialty 09.00.03 “History of Philosophy“: Dissertation… Doctor of Philosophy Moscow
- Kosinski, M. (2023) Theory of Mind May Have Spontaneously Emerged in Large Language Models URL:https://arxiv.org/pdf/2302.02083v1 (accessed 14.04.2025).
- Makhmutova, E. N. & Kuzmina, A. A. (2019). Psychological Components of Success in the Economic Socialization of Young People in the Digital Age. Herzen Readings: Psychological Research in Education, 2, 447-450 https://doi.org/10.33910/herzenpsyconf-2019-2-55
- Silanteva, M. V. (2013) Metamorphoses of Social Organisms in the Light of the Transformation of Cultural Borders. In: A. V. Malgin, A. V. Shestopal, M. V. Silantieva (Eds.) Intercultural Communication: Modern Theory and Practice: Proceedings of the 7th RAMI Convention: Scientific Publication, Moscow, September 28–29, 2012 (pp. 103 – 111). Moscow: Aspect Press
- Silanteva, M. V. (2019) Identity as a Communicative Project: “New Mythologies” in the Space of Modern Culture. Scientific Research and Development. Modern Communication Studies, 8 (1),19-25. https://doi.org/10.12737/article_5c5a88d8040dd6.09767248
- Torkunov, A. V. (2022) Russia and the Political Order in a Changing World: Values, Institutions, and Prospects Political Studies,5, 7-22. https://doi.org/10.17976/jpps/2022.05.02
- Ulanova, A. E. (2022). The Role of Creativity in Human Adaptation to the Introduction of Artificial Intelligence Elements (on the Example of Journalism): Dissertation… Candidate of Philosophy: 09.00.13 Moscow
- Vinokurov, F., Panov, K., & Sadovskaya, E. (2024) The legacy of G.M. Andreeva: the role of technological innovations in social change National Psychological Journal, 3(19). https://doi.org/10.11621/npj.2024.0304
Comments (0)
Abstract. The modern development of artificial intelligence has led to the active use of large language models (LLM) in various fields of scientific and professional knowledge. LLM is based on the principles of deep machine learning using a large amount of data and parameters to solve natural language tasks. As part of our research on the use of large language models in scientific papers, we conducted a theoretical and analytical study based on the materials of leading scientific journals in psychology in articles published since 2023. Two trends in research were identified. In domestic works, the LLM model was used to develop expert systems for categorizing socio-psychological phenomena. In foreign works, the use of psychodiagnostic methods for assessing the “abilities” of large language models and constructing reasoning is actively presented. In LLM construction systems, they often rely on psychological theories focused on the construction of everyday knowledge, socially conditioned interactions, and systems of attributive processes. Currently, there is a significant expansion of work in the humanities using LLMs, which provides an opportunity for the development of new areas within cognitive sciences and expands the possibilities for implementation in robotics, but also presents many challenges, particularly in assessing the quality of reasoning and the reliability of the acquired knowledge.
Резюме. Современное развитие искусственного интеллекта привело к активному использованию больших языковых моделей (LLM) в различных областях научного и профессионального знания. В основе LLM лежат принципы глубокого машинного обучения с использованием большого объема данных и параметров, направленного на решение задач на естественном языке. В рамках исследования использования больших языковых моделей в научных работах мы провели теоретико-аналитическое исследование на базе материалов ведущих научных журналов по психологии в статьях c 2023 года. Были выделены две тенденции работ. В отечественных работах LLM модель использовали для развития экспертных систем категоризации социально-психологических феноменов. В зарубежных работах активно представляется использование психодиагностических методик для оценки «способностей» больших языковых моделей и построения рассуждений. В системах построения LLM они часто опираются на психологические теории, ориентированные на построение повседневных знаний, социально обусловленных взаимодействий, систем атрибутивных процессов. На данный момент происходит большое расширение работ в гуманитарных науках с использованием LLM, что дает возможность для развития новых направлений в рамках когнитивных наук, расширения возможностей для реализации в робототехнике, но несет и много трудностей, прежде всего в вопросе об оценке качества построения рассуждений и достоверности получаемых знаний.
Ключевые слова: большие языковые модели (LLM), искусственный интеллект, научная статья, социальная группа, рассуждение, психодиагностика
The modern development of artificial intelligence has led to the active use of large language models (LLM) in various fields of scientific and professional knowledge. LLM is based on the principles of deep machine learning using a large amount of data and parameters to solve natural language tasks (Chuganskaya et al., 2024). Now, it is not just about generating plausible texts; modern models have gone further by combining the creation of text, human communication, and solving tasks related to intelligent information retrieval and expert work. All of this is becoming a tool for expanding and, in many ways, reshaping human work in a number of text-related fields (e.g., writing academic and news texts, evaluating motivation letters and resumes in HR departments) (Makhmutova, Kuzmina, 2019). In a broad sense, this is a change in the perception of human abilities, which are expanded by the use of such an “external” tool as LLM, as described by L.S. Vygotsky. The use of large language models can serve as an intermediate tool in the development of specific human abilities, which, as noted by V.N. Druzhinin, “are specifically human abilities that are latent and manifest themselves through specific abilities that are influenced by the environment” (Druzhinin, 1999, p. 103). The technologicalization of the environment and the human relationship with accelerating time act as facilitating factors for such development (Chuganskaya, 2022).
Representatives of MGIMO’s scientific schools, who studied axiological issues, intercultural communication, and the development of digitalization and artificial intelligence, also addressed certain aspects of the study of digitalization in the context of anthropological research. Modern society is in a global process of migration of people from different ethnic groups and the formation of culture-specific types of society. The mixing and mutual influence of cultures leads, in some situations, to integration, in which cultural dialogue contributes to changes in the ethnos and constructive interaction, while in others, it leads to confrontation that can escalate into military conflicts. It is becoming more difficult for people to adapt to a multicultural environment, form their own ideas about ethnic identity and cultural values, and cope with the challenges of cultural diversity. This is especially true for young people. One way to overcome language barriers is through the use of digital products and large language models. In Russia, MGIMO University is studying the significant issues and directions of modern global change, including the interdisciplinary foundations of technology development. The scientific works of A.V. Torkunov (Torkunov, 2022), M.V. Silantieva (Silanteva, 2013, 2019), P.I. Kasatkin (Kasatkin, 2018), O.V. Gaman-Golutvina (Gaman-Golutvina, Smorgunov, 2023), and others, taking into account the results of the analysis of global challenges, are among the most significant in building the foundations of scientific knowledge in new interdisciplinary fields related to the topic of “human-technology”.
The direction of digitalization of modern society and the formulation of an understanding of the use of trust-based artificial intelligence as a generator of new information and the systematization of knowledge, as well as the use of new technologies in education, are reflected in the works of Ulanova A.E. (Ulanova, 2022).
As part of our research on the use of large language models in scientific papers, we conducted a theoretical and analytical study based on the materials of leading scientific journals in psychology in articles published since 2023 (with an indexing level of at least RSCI and included in the CyberLeninka database (https://cyberleninka.ru/) and a foreign archive of articles on the use of psychodiagnostic methods for training large language models (https://www.researchgate.net/) The selection of articles from this period was based on the active development of large language models. Our interest was focused on works that were categorically related to the field of psychology.
Among the total volume of the data articles received, it should be noted their small number, less than 10. Two trends of works were highlighted. First, in domestic works, the LLM model was used to develop expert systems for categorization of socio-psychological phenomena. In the work of Vinokurov F., Panov K., Sadovskaya E. (Vinokurov et al., 2024), 900 messages about innovative technologies (for example, virtual assistants) were analyzed using gpt-4-turbo and some previous models (two models ChatGpt and Bert). With their help, content analysis was carried out based on the pre-training of models on a coordinated expert sample by categories: “Clarification”, “Status of action”, “Question”, “Expression of consent / approval”, etc. The results were obtained with a degree of consistency of 56% and 70%, which indicates the high potential for using such models for socio-psychological tasks.
In foreign works, the use of psychodiagnostic methods for assessing the “abilities” of large language models and reasoning is actively presented. In LLM construction systems, they often rely on psychological theories focused on building everyday knowledge, socially conditioned interactions, and systems of attributive processes. This approach is experimentally presented in the works of Kosinski M. (Kosinski, 2023). The basis is based on techniques from the model of the theory of consciousness (theory of mind, ToM). The very definition of ToM as a system of logical reasoning used to predict the behavior of an individual by attributing certain mental states to him, emerged within the framework of the work of psychologists Wimmer H. and Premack D. G. with Woodruff G. (Chuganskaya et al., 2024), who studied the models of the emergence of human reason in the process of the development of the psyche. For Tom, it is important to consider only a few mental aspects of interactions, such as attributional processes and emotional understanding in the context of life situations, which are largely based on the construction and subsequent implementation of scenario information.
Methodologically, this understanding is mainly based on the concept of consciousness by D. K. Dennett, who believed that it consists of information streams that compete for access to the brain, which eliminates the difference between processes in natural human consciousness and the construction of robot programs. This understanding of the mind has been used in studies that not only evaluate the quality of tasks performed by new programs in human-computer interaction, but also use psychodiagnostic material as a basis for training. So, M. Kosinski used 40 tasks that were given as part of the study of thinking and attribution processes based on ToM (Kosinski, 2023). The result showed that it was in these tasks that the LLM models that formed the basis of GPT significantly improved their performance since 2020, reaching 95% correct answers for GPT-4 in March 2023.
It should be noted that there is currently a significant expansion of work in the humanities using LLM, which provides an opportunity for the development of new areas within cognitive sciences and the expansion of opportunities for implementation in robotics, but also presents many challenges, particularly in terms of evaluating the quality of reasoning and the reliability of the acquired knowledge.
- Chuganskaya, A. A. (2022) Modern trends in psychodiagnostics of large groups in social networks. In: R.V. Ershova (Ed.) Digital society as a cultural and historical context of human development: Collection of scientific articles and materials of the international conference, Kolomna, February 17, 2022. Kolomna: State Educational Institution of Higher Education of the Moscow Region “State Social and Humanitarian University”, pp. 281-284.
- Chuganskaya, A. A., Kovalev, A. K., & Panov, A. I. (2024) Sign-based image criteria for social interaction visual question answering Interest Group in Pure and Applied Logics. Logic Journal, 32 (4), 656-670. https://doi.org/10.1093/jigpal/jzae026
- Druzhinin, V. N. (1999) Psychology of general abilities.: Publishing House “Peter”
- Gaman-Golutvina, O. V. & Smorgunov, L. V. (2023) The Political in the Space of a Turbulent World Political Studies, 1, 7-10. DOI: https://doi.org/10.17976/jpps/2023.01.02
- Kasatkin, P. I. (2018). Value Axiomatics of the Educational Space: Specialty 09.00.03 “History of Philosophy“: Dissertation… Doctor of Philosophy Moscow
- Kosinski, M. (2023) Theory of Mind May Have Spontaneously Emerged in Large Language Models URL:https://arxiv.org/pdf/2302.02083v1 (accessed 14.04.2025).
- Makhmutova, E. N. & Kuzmina, A. A. (2019). Psychological Components of Success in the Economic Socialization of Young People in the Digital Age. Herzen Readings: Psychological Research in Education, 2, 447-450 https://doi.org/10.33910/herzenpsyconf-2019-2-55
- Silanteva, M. V. (2013) Metamorphoses of Social Organisms in the Light of the Transformation of Cultural Borders. In: A. V. Malgin, A. V. Shestopal, M. V. Silantieva (Eds.) Intercultural Communication: Modern Theory and Practice: Proceedings of the 7th RAMI Convention: Scientific Publication, Moscow, September 28–29, 2012 (pp. 103 – 111). Moscow: Aspect Press
- Silanteva, M. V. (2019) Identity as a Communicative Project: “New Mythologies” in the Space of Modern Culture. Scientific Research and Development. Modern Communication Studies, 8 (1),19-25. https://doi.org/10.12737/article_5c5a88d8040dd6.09767248
- Torkunov, A. V. (2022) Russia and the Political Order in a Changing World: Values, Institutions, and Prospects Political Studies,5, 7-22. https://doi.org/10.17976/jpps/2022.05.02
- Ulanova, A. E. (2022). The Role of Creativity in Human Adaptation to the Introduction of Artificial Intelligence Elements (on the Example of Journalism): Dissertation… Candidate of Philosophy: 09.00.13 Moscow
- Vinokurov, F., Panov, K., & Sadovskaya, E. (2024) The legacy of G.M. Andreeva: the role of technological innovations in social change National Psychological Journal, 3(19). https://doi.org/10.11621/npj.2024.0304