ISSN 0798 1015

logo

Vol. 39 (Number 29) Year 2018. Page 12

The implementation of the concept of internal quality assessing system of the preschool education based on the principles of state and public administration

La implementación del concepto de sistema interno de evaluación de la calidad de la educación preescolar basado en principios de Estado y administración pública

Marina Vladimirovna POTAPOVA 1; Artem Arkadievich SALAMATOV 2; Boris Alexandrovich ARTEMENKO 3; Irina Viktorovna KOLOSOVA 4; Oksana Gennadievna MISHANOVA 5; Irina Anatolievna SELIVERSTOVA 6

Received: 15/03/2018 • Approved: 01/04/2018


Contents

1. Research relevance

2. Materials and methods

3. Results and discussion

4. Conclusion

Acknowledgements

Bibliographic references


ABSTRACT:

The article deals with the design, verification, and validation of the concept of internal quality assessing system of the preschool education (hereinafter referred to as IQASPE) implemented based on the principles of state and public administration, which is consistent with the requirements of state education policy. This is scientifically sound tool, potentially distributed across the region, which allows receiving the comprehensive information and making democratic rational and timely management decisions to improve the quality of education in preschool educational organizations. The methodological framework of the concept development is based on systemic, activity, axiological, and qualimetric approaches (Kuz'mina, et. al. 2002). Experimental and practical work on the project includes the development and validation of a criterion-level scales, valid diagnostic materials, mathematical models, and procedures to implement state and public examination. The research results are designed to conduct systematic standardized observations of the educational process status in preschool educational organization and trace the dynamics in quality changes of offered educational services in accordance with the requirements of the Federal State Educational Standard of Preschool Education (FSES PE).
Keywords: Preschool education, quality of education, state and public administration, the concept of the internal quality assessing system of the preschool education

RESUMEN:

El artículo trata del diseño, verificación y validación del concepto de sistema interno de evaluación de la calidad de la educación preescolar (en adelante, IQASPE) implementado en base a los principios del estado y la administración pública, que es consistente con los requisitos de la educación estatal política. Esta es una herramienta científicamente sólida, potencialmente distribuida en toda la región, que permite recibir información integral y tomar decisiones de gestión democráticas, racionales y oportunas para mejorar la calidad de la educación en las organizaciones educativas preescolares. El marco metodológico del desarrollo del concepto se basa en enfoques sistémicos, de actividad, axiológicos y cualimétricos (Kuz'mina, et al., 2002). El trabajo experimental y práctico en el proyecto incluye el desarrollo y la validación de escalas a nivel de criterio, materiales de diagnóstico válidos, modelos matemáticos y procedimientos para implementar exámenes estatales y públicos. Los resultados de la investigación están diseñados para realizar observaciones estandarizadas sistemáticas del estado del proceso educativo en la organización educativa preescolar y rastrear la dinámica en los cambios de calidad de los servicios educativos ofrecidos de acuerdo con los requisitos del Estándar Educativo Estatal Federal de Educación Preescolar (FSES PE).
Palabras clave: educación preescolar, calidad de la educación, administración estatal y pública, el concepto del sistema interno de evaluación de la calidad de la educación preescolar

PDF version

1. Research relevance

The issue of assessing the quality of preschool education is substantiated by the state educational policy aimed at improving the availability and quality of preschool education. The article 28 of the Federal Law No. 273-FZ "On education in the Russian Federation" designates the responsibility of an educational organization to ensure the functioning of the internal quality assessing system of education. However, a unified conceptual and methodological understanding of the solution to the problem of assessing the quality of preschool education and approaches to its measurement is not developed so far (Sobranie Zakonodatel’stva Rossiiskoi Federatsii [Collection of Legislation of the RF] 31.12.2012, No. 53(1), Item 7598). In this regard, the development of the concept of the internal quality assessing system of the preschool education (hereinafter referred to as IQASPE) in preschool educational organization (PEO) becomes quite relevant.

The purpose of the present study consists in designing, verifying, and testing the concept of the IQASPE based on the principles of state and public administration.

Target group consists of executives of PEOs and parents of children studying at PEOs.

As noted by A.K. Belolutskaya et al. (2016), T.B. Yedakova (2012), (Edakova 2013) and many other experts in the field of  quality assessment of the education, the creation of the IQASPE Concept is caused by the increased need of all participants involved in the educational process in the systematization of measures  and procedures of quality assessment of preschool education. The concept defines methodological bases to improve the system of quality assessment, including the conceptual framework and principles, the objectives and tasks, as well as its objects and content. 

The methodological framework of the concept development is based on the systemic, activity, axiological, and qualimetric approaches (Kuz'mina, et al. 2002).

The system approach (I.V. Blauberg, (1973), B.S. Gershunsky (1998), Yu.A. Konarzhevsky (1990), and A. I. Ujomov (1978), et al.) has found its application when forming an aggregate of the IQASPE components.

When assessing the quality of education in PEO, the activity approach (P.Ya. Galperin (1999), V.V. Davydov (1998), A.N. Leont'ev (1975), et al.) is defined by interaction nature of educational process participants.

The axiological approach (L.V. Vershinina (2003), Z.I. Ravkin (1995), V.A. Slastenin, G.I. Chizhakova (2003), et al.) provides analysis of pedagogical values, which are the basis in determining the structure and content of the IQASPE.

The qualimetric approach (V.P. Bespal’ko (1997), I.Ya. Lerner (1981), E.V. Yakovlev (1999), et al.) combines monitoring of quality assessment, used to substantiate decisions on quality management of preschool education and related management.

Many researchers (A.K. Belolutskaya, T.N. Le-van. et al. (2016) V. A. Bolotov, I.A. Val’dman,  et al. (2012), O.A. Shiyan, E.V. Vorob'eva (2015), D.J. Cassidy, L.L. Hestenes, A. Hegde (2005), D.S. Harcourt (2008), et al.) single out a general criterion for assessing the quality of education as the compliance of the object with the established requirements. Comprehensive application of the described approaches makes the IQASPE crucially transparent and soluble, allowing involvement of parents as education agents, to the education quality assessment.

2. Materials and methods

Design, verification and testing of the IQASPE concept was based on the analysis of normative legal documents defining the implementation principles of the IQASPE, namely the Law of the Russian Federation No. 273-FZ of 29.12.2012 "On education in Russian Federation" (Sobranie Zakonodatel’stva Rossiiskoi Federatsii [Collection of Legislation of the RF] 31.12.2012, No. 53(1), Item 7598), as well as orders of the Russian Ministry of Education and Science No. 462 of 14.06.2013 "On approving the Procedure of self-assessment in educational organization" (Rossiiskaia Gazeta [Ros. Gaz.] 12.07.2013, No. 151), and No. 1155 of 17.10.2013 "On approval of Federal state educational standard of preschool education" (Rossiiskaia Gazeta [Ros. Gaz.] 25.11.2013, No. 265).  Validation of the concept was based also on the processing of the results of two questionnaire surveys conducted among executives of PEOs of the Chelyabinsk Region and parents of children studying in PEOs. Carrying out the questionnaire survey pursued two objectives: firstly, identifying the need to develop procedure for implementing IQASPE; and secondly, identifying the respondents' opinion about the objectivity and reliability of the developed system for assessing the quality of preschool education and its compliance with the legislative framework of the Russian Federation.

In total, the study involved 1228 PEOs of the Chelyabinsk Region that made up 84% of the total number of PEOs of the region.

The number of respondents from among executives of PEOs amounted to 1228 people. The number of parents, who agreed to participate in the survey, amounted to 122800 people (100 people from each PEO).

3. Results and discussion

The questionnaire survey showed that 92% of the 1228 respondents among the executives of PEOs were ready to implement the self-assessment procedure, while 5% of respondents expressed their doubt concerning the suitability of implementation of self-assessment in PEO, however they realized the need for this measure, referring to the requirements of the regulatory framework, that hardly proved their true readiness for carrying out self-assessment procedure. Only 3% of PEO executives noted negative attitude to the self-assessment procedure. It is probably due to the fact that executives, many of whom are well experienced in administrating process, are not ready for such procedures, because they may identify deficiencies in the functioning of entrusted PEO.

As for the results obtained from the questionnaire survey of parents, 98% of those underwent questionnaire noted the need for the implementation of the self-assessment procedure in the educational organization, while 2% reported that they were not interested in the results of such self-assessment, because they were not competent in the issues related to the organization of preschool education.   

Based on the answers of PEO executives and children’s parents to the questionnaire concerning the need to develop a self-assessing procedure, analyzing the criteria set forth in the " Procedure of conducting self-assessment in educational organization" (Rossiiskaia Gazeta [Ros. Gaz.] 25.11.2013, No. 265), and the experience of foreign researchers (E.D. Caffrey (2009), D. S. Harcourt (2008), M. Perlman, G.L. Zellman, Vi-Nhuan Le. (2004) et al.), we have developed an algorithm for conducting a self-assessment for a PEO, which is presented below.

 I. "Educational activity" - as the object of self-assessment.

  1. Satisfaction of parents with educational services provided by PEO: the percentage of parents satisfied with the basic education program quality of PEO; the percentage of parents satisfied with quality of educational services; the percentage of parents satisfied with the quality of implementation conditions of basic education program at PEO; the percentage of parents satisfied with the quality of additional educational services at PEO.
  2. The basic education program development status of PEO: the conformity of structure and content of each basic education program section with the requirements of the FSES PE; the availability of organizational and methodological support to implement the basic education program,  including that in terms of interaction with society; the admissibility of providing information concerning the basic education program to the family and all persons concerned involved in the educational process, as well as to the wide audience.
  3. Maturity of the curricular support of the basic education program: the degree of sufficiency of curricular support for the implementation of the target partition of the basic education program.
  4. Maturity of the basic education program, formed by participants of educational relations: the compliance of the basic education program part, formed by the participants of the educational relations, to the specifics of PEO and capabilities of the teaching staff.
  5. Elaboration of teachers’ working programs (the compliance of teachers’ working programs with the target and substantive sections of the basic education program, as well as teachers’ qualifications).
  6. The implementation of new forms of preschool education taking into account the special educational needs of certain categories of children, including those with disabilities.
  7. The efficiency of children’s health preservation measures: the availability of conditions for medical care of children in order to protect and promote their health; the creation and implementation of health status assessment system as well as mental and physical development of children.
  8. The creation and use of developing educational technologies: percentage of teachers, who have mastered the innovative technologies in the framework of the coursework training and methodical work.
  9. The effectiveness of psychological and pedagogical conditions for the implementation of the basic education program: creation and implementation of system of psychological and pedagogical assessment of children’s development, dynamics, including the measurement of personal educational outcomes of the child; consideration of psycho-educational evaluation of children’s development when planning the organization of educational activities; support of individuality, children’s initiative and independence in educational activities; consultative support to educators and parents on issues of children’s upbringing and education.
  10. The effectiveness of developing subject-spatial environment of PEO: the conformance of subject-spatial environment contents of PEO to the age features of children; the organization of educational space and a variety of materials, equipment, and inventory in accordance with the requirements of the Federal State Educational Standard (convertibility, multi-functionality, variability, availability, and security); the availability of conditions for communication and joint activity of pupils and adults in the whole group and small groups, physical activity of the pupils, as well as opportunities for privacy.

II. "Control system of PEO" - as the object of self-assessment.

  1. The effectiveness of PEO development Program: the implementation quality of the projects in the framework of PEO development program.
  2. The effectiveness of the planning and forecasting system in PEO: availability of annual and other applicable plans, their informational and analytical substantiation; succession with development program; timely adjustment of plans; ensuring publicity; and information openness of PEO.
  3. The effectiveness of the organizational conditions of PEO: legal and regulatory framework ensuring the functioning of PEO (license, charter, contracts); conformance of the documents regulating activities of PEO, to legislative regulations in the field of education; development of state and public administration; development of networking; compliance with the ethical standards in management activities; presence of business relations in the team (cohesion, initiative, openness, and self-criticism); a favorable psychological climate; the involvement of teachers in the management of PEO, delegation of authorities in the team; diligence of teachers in the educational process development (mastering new programs and technologies; diagnostics of teachers’ professionalism; coordination and cooperation of teachers’ efforts, various forms of interaction between teachers); creative self-actualization of teachers identifying the main difficulties in teacher’s activity and their causes; identifying priority areas; free choice of educational process content and technology.
  4. The effectiveness of innovation activity of PEO: the relevance and prospects of the elective programs and technologies selected by PEO; harmonization of PEO and the local programs in terms of the concept, target, and meaning; the security of the innovation process (material and technical conditions, information and methodological conditions); availability of a sound plan (program, model) to organize innovation in PEO. 
  5. The effectiveness in providing a safe environment in PEO: adherence to fire safety instructions; compliance with the labor protection standards.
  6. The effectiveness of nutrition in PEO: percentage of compliance with the standard for foodstuffs in comparison with citywide level, with own figures for the previous period, the implementation of the Municipal assignment; lack of complaints or comments on the part of parents and supervisory authorities with regard to the level of catering.

III. "Training quality of children in PEO" - as the object of self-assessment. 

  1. Satisfaction of parents with the quality of educational outcomes after graduation of children from PEO; the proportion of parents satisfied with the quality of educational outcomes.
  2. Achievements of children: percentage of children participating in regional, district, city, national, and other events.
  3. The health status of children: proportion of children with positive health dynamics.
  4. Readiness of children for school: percentage of children having a positive assessment of maturity for studying at school.

IV. "Organization of training (educational) process in PEO" - as the object of self-assessment.

  1. The effectiveness of the regulations of directly educational activities (DEA): the compliance with the sanitary rules and norms (SanPiN) when carrying out DEA; the conformance of NOD regulations to children’s age peculiarities.
  2. The effectiveness of the DEA organization: ensuring the developing nature of DEA; providing conditions for individual cognitive activity of children; the optimal alternation of different types of activities; optimal use of technical means as well as information and communication technologies in education.
  3. The efficiency of planning the educational process: the compliance of planning with today’s normative as well as conceptual and theoretical foundations of preschool education; the compliance of plans with the age opportunities of children; the assessment of individual development of children when planning educational work.
  4. The efficiency of conditions required to organize the educational work in everyday life: conformance of educational subject-spatial environment to the basic education program of PEO; conformance of educational subject-spatial environment to sanitary regulations and standards; conformance of educational subject-spatial environment to the FSES PE.
  5. The effectiveness of informational support of educational process: connection to Internet, providing Internet filtering; availability of the local PEO network; timely renewal of equipment.

V. "Material and technical base of PEO" - as the object of self-assessment.

  1. Infrastructure of sectional and functional premises: the compliance of the material and technical basis with the basic education program requirements; the use of material resources in the educational process; the extent to which technical means are used in the educational process; the compliance of the equipment, furniture, and teaching aids with the sanitary regulations and standards; the compliance of the technological equipment with the modern requirements; the conformance of equipment, furniture, and teaching aids to the requirements of labor protection and fire safety.
  2. The updating dynamics of material and technical basis: the timeliness of necessary repairs to buildings and equipment; timeliness of equipment replacement (water supply, sewage, ventilation, and lighting); the fullness of the provision of goods necessary for PEO activities by third parties.

VI. "Human resourcing of PEO" - as the object of self-assessment.

  1. The efficiency of human resourcing in PEO: staffing compliance with the staff schedule; compliance of the employees with the qualifying requirements of the occupied jobs; staffing with teaching personnel; the proportion of teachers taking part in the professional motion on a competitive basis; the proportion of teachers who have industry awards, titles, and academic degrees.
  2. The effectiveness of human resources management in PEO: the presence of effective mechanisms for staff development in PEO; the existence of system of material and moral incentives; the personification of advanced training of teachers.

VII. "Curricular, library, and information support" as the object of self-assessment.

  1. Provision of the educational process of PEO with teaching aids; the compliance of curricular support with the basic education program; optimality and validity of the choice of curricular support for solving educational tasks.
  2. The effectiveness of methodical work on creation of original learning and teaching support kit in the frameworks of the basic education program: the availability of the original resource materials of PEO teachers on educational work with children; the availability of the original resource materials of PEO teachers on work with parents.
  3. Provision of the educational process of PEO with reference materials and fine literature: optimality and validity of the literature selection for PEO library; the availability of files and directories systematizing the literature.
  4. The effectiveness of information support: the optimality and validity of the video materials selection; satisfaction of information requests of the educational process participants in the educational organization.

VIII. "Functioning of the internal system of education quality assessment in PEO" - as the object of self-assessment.

  1. Management effectiveness of the internal system of education quality assessment: the existence of organizational structures of PEO carrying out an assessment of the preschool education quality; the effectiveness of legal and regulatory framework when assessing the education quality in PEO.
  2. The effectiveness of the tooling background for internal quality assessing system of PEO: the availability of software for the collection, storage, and statistical processing of information on the status and the development dynamics of the education system in PEO; the availability of the methodologies to assess the education quality in PEO; the availability of measurement materials for assessing the education quality in PEO.

The assessment of the "Performance analysis of the organization, subject to self-assessment" as the self-assessment object is carried out in accordance with the indicators set forth by the Ministry of Education and Science of the Russian Federation No. 1324 of 10.12.2013. That’s why it is not included in the present algorithm.

This analysis algorithm was proposed to executives of PEOs as well as to parents to reveal their views about the objectivity and reliability of the developed IQASPE and its compliance with the legislative framework of the Russian Federation.

The following results were obtained in the course of questionnaire survey: 97% of respondents from among the executives noted that the proposed self-assessment algorithm met the requirements of the legislative framework and allowed obtaining unbiased and reliable information about quality of offered educational services; 3% of respondents noted that the developed system met the requirements of the self-assessment procedure, however, they perceived the practical implementation of the proposed system negatively.

The processing of respondents’ questionnaires from among parents has shown 100% satisfaction with the developed algorithm for self-evaluation of PEO. Parents believe that the implementation of this tool into the preschool education practice will allow creating openness and accessibility of the educational organization.

4. Conclusion

The implementation of the IQASPE into educational activities allows the executives to form a comprehensive assessment concerning the functioning of their PEO. Gathering complete and accurate information for analysis of each self-assessment object is carried out in the course of internal monitoring process as a special kind of management activity.

The application of tools presented in developed algorithm serves the basis for the formation of the databank concerning educational activities, the generalization and analysis of which will enable PEO to generate a report on self-assessment satisfying all applicable regulatory requirements.

Acknowledgements

The work was performed within the framework of the state assignment of the Ministry of Education and Science of the Russian Federation No. 27.9462.2017/NM "Design, verification and testing of the concept of the IQASPE in accordance with FSES PE based on the principles of state and public administration".

Bibliographic references

Belolutskaya, A.K., Le van, T.N., Krasheninnikov Khight, E.E., Smirnova, D.S.  and Shiyan, I.B. (2016). Zarubezhnye modeli nezavisimoj ocenki kachestva doshkol'nogo obrazovaniya: opyt SSHA i Singapura [Foreign models of independent assessment of the preschool education quality: The experience of the United States and Singapore]. Modern Preschool Education. Theory and Practice, 9, 60-67.

Bespal'ko, V.P. (1977). Osnovy teorii pedagogicheskih sistem: problemy i metody psihologo-pedagogicheskogo obespecheniya tekhnicheskih obuchayushchih sistem [Fundamentals of the theory of pedagogical systems: Problems and methods of psycho-pedagogical support of technical training systems]. Voronezh: Voronezh University, pp. 304.

Blauberg, I.V. and Yudin, E.G. (1973). Stanovlenie i sushchnost' sistemnogo podhoda [The formation and essence of system approach]. Moscow: Nauka, pp. 69.

Boguslavskaya, T.N. (2012). Formirovanie podhodov k ocenke kachestva doshkol'nogo obrazovaniya [Forming approaches to assessing the quality of preschool education]. Problems of Modern Education, 4. Retrieved from http://www.pmedu.ru/res/2012_4_4.pdf

Bolotov, V.A., Valdman, I.A. and Kovaleva, G.S. (2012). Rossijskaya sistema ocenki kachestva obrazovaniya: chemu my nauchilis' za 10 let? [Russian system of education quality assessment: What have we learned in 10 years?]. Proceedings of the  VIIIth International Research-to-Practice Conference “Trends in education development: Management problems and education quality evaluation”, Moscow: Universitetskaya Kniga, pp. 22-31.

Vershinina, L.V. (2003). Aksiologicheskoe prostranstvo obrazovaniya: cennostnoe soznanie uchitelya [Axiological educational space: The value consciousness of the teacher]. Samara: Publishing Office Samara State Pedagogical University, pp. 148.

Gal’perin, P.Ya. (1999). Vvedenie v psihologiyu [Introduction to psychology]. Moscow, Universitet, pp. 332.

Gershunsky, B.S. (1998). Filosofiya obrazovaniya dlya XXI veka [Education philosophy for the XXIst century]. Moscow: Sovershenstvo, pp. 432.

Davydov, V.V. (1998). Novyj podhod k ponimaniyu struktury i soderzhaniya deyatel'nosti [A new approach to understanding the activity structure and content]. In L.V. Bartfai and B.A. Zel’tserman (Eds.). Poslednie vystuplenija [Recent speaking engagements]. Riga: Pedagogical Center Experiment, pp. 88.

Edakova, I.B. (2013). K voprosu ob ocenke kachestva doshkol'nogo obrazovaniya v usloviyah vvedeniya FGOS [Revisiting preschool education quality assessment in the context of implementation of the FSES]. Elementary school plus BEFORE organization and AFTER, 12, 17-22.

Sobranie Zakonodatel’stva Rossiiskoi Federatsii [SZ RF] [Collection of Legislation of the RF] 31.12.2012, No. 53(1), Item 7598.

Konarzhevski, Yu.A. (1990). Sistemnyj podhod kak osnova perestrojki upravleniya v narodnom obrazovanii [System based approach as the basis for changing management in public education]. Democratization of public education management. Minsk, pp. 35-48.

Leont'ev, A.N. (1975). Deyatel'nost'. Soznanie. Lichnost' [Activity. Consciousness. Personality]. Moscow: Politizdat, pp. 304.

Lerner, I.Ya. (1981). Didakticheskie osnovy metodov obucheniya [Didactic bases of teaching methods]. Moscow: Pedagogika, pp. 98-100.

Kuz'mina, N.V., Grigor'eva, E.A. and Jakunin, V.A. (2002). Metody sistemnogo pedagogicheskogo issledovaniya [Methods of system based pedagogical research]. Moscow: Narodnoje Obrazovanie, pp. 208.

Rossiiskaia Gazeta [Ros. Gaz.] 12.07.2013, No. 151 (Russ.).

Rossiiskaia Gazeta [Ros. Gaz.] 25.11.2013, No. 265 (Russ.).

Ravkin, Z.I. (1995). Razvitie obrazovaniya v Rossii: novye cennostnye orientacii [The development of education in Russia: New value system (The research concept: the axiological aspect)]. Pedagogy, 5, 87-90.

Slastenin, V.A. and Chizhakova, G.I. (2003). Vvedenie v pedagogicheskuyu aksiologiyu [Introduction to pedagogical axiology]. Moscow: Academy, pp. 92.

Uyomov, A.I. (1978). Sistemnyj podhod i obshchaya teoriya sistem [System approach and general systems theory]. Moscow: Mysl, pp. 272.

Shiyan, O.A. and Vorob'eva, E.V. (2015). Novye vozmozhnosti ocenki kachestva obrazovaniya: shkaly ECERS-R aprobirovany v Rossii [New opportunities in education quality assessment: ECERS-R scales were tested in Russia]. Modern Preschool Education. Theory and Practice, 7, 38-49.

Yakovlev, E.V. (1999). Kvalimetricheskij podhod v pedagogicheskom issledovanii: novoe videnie [Qualimetric approach in pedagogical research: A new vision]. Pedagogy, 3, 49-54.

Caffrey, E.D. (2009). Assessment in elementary and secondary education: A primer. Congressional Research Service. Retrieved from https://fas.org/sgp/crs/misc/R40514.pdf

Cassidy, D.J., Hestenes, L.L., Hegde, A., Hestenes, S. and Mims, S. (2005). Measurement of quality in preschool child care classrooms: An exploratory and confirmatory factor analysis of the Early Childhood Environment Rating Scale-Revised. Early Childhood Research Quarterly, 20, 345-360.

Harcourt, D.S. (2008). Constructing ideas and theories about quality: The accounts of young children in two early childhood classrooms in Singapore. Retrieved from http://eprints.qut.edu.au/16658/1/Deborah_Sue_Harcourt_Thesis.pdf

Perlman, M., Zellman, G.L. and Le Vi-Nhuan (2004). Examining the psychometric properties of the Early Childhood Environment Rating Scale-Revised (ECERS-R). Elsevier, Early Childhood Research Quarterly, 19, 398-412.


1. South Ural State Humanitarian Pedagogical University, 69 Lenina Avenue, Chelyabinsk, 454080, Russia, E-mail: artemenkoba@cspu.ru

2. South Ural State Humanitarian Pedagogical University, 69 Lenina Avenue, Chelyabinsk, 454080, Russia

3. South Ural State Humanitarian Pedagogical University, 69 Lenina Avenue, Chelyabinsk, 454080, Russia

4. South Ural State Humanitarian Pedagogical University, 69 Lenina Avenue, Chelyabinsk, 454080, Russia

5. South Ural State Humanitarian Pedagogical University, 69 Lenina Avenue, Chelyabinsk, 454080, Russia

6. South Ural State Humanitarian Pedagogical University, 69 Lenina Avenue, Chelyabinsk, 454080, Russia


Revista ESPACIOS. ISSN 0798 1015
Vol. 39 (Nº 29) Year 2018

[Index]

[In case you find any errors on this site, please send e-mail to webmaster]

revistaESPACIOS.com