Bfq-Rb(Russian Brief): Short Big-Five Questionnaireto Measure Facets And Factors Of Personality

Abstract

Several questionnaires exist for brief estimation of the Big Five personality factors. The majority of the short forms of the Big Five instruments aim to estimate the Big Five factors but not the facets within each factor. Assessing facets can be beneficial because facets may explain external behavior better than factors do. This paper presents a short form of the Big Five Questionnaire (BFQ) designed to assess both factors and facets, validated on a sample of Russian adolescents (14-18 years old). We created a short version (BFQ – Russian Brief; BFQ-RB), using data from a sample of 1128 adolescents (14-18 years) and then confirmed the factor structure on another subsample of 1087 adolescents. The psychometric properties of the newly created instrument – the BFQ-RB (Russian Brief) – were evaluated via item-level confirmatory factor analysis. We estimated three main models. In the first model, the selected items represented the Big Five factors. In the second model, the selected items represented ten correlated latent factors (facets). The third model was the second-order factor model fitted the data well, suggesting that the BFQ-RB enabled the estimation of both facets and factors. Our final instrument consists of 43 items, with each facet represented by 3-4 items and each second-order factor consisting of two facets, including the Lie scale.

Keywords: Big Fivepersonalityfacetsshort formconfirmatory factor analysis

Introduction

The Big Five model is one of the common models of personality. This model implies that the five factors of Extraversion, Openness, Agreeableness, Neuroticism (or Emotion Stability as a positive pole) and Conscientiousness are the basic dimensions of personality (e.g., Costa & McCrae, 1992). It has been postulated that these factors can identify major sources of individual differences in personality (McCrae & Costa, 1991). The Big Five model of personality has a hierarchical structure, with low-level traits (or facets) merging into high-level five factors. This model has been confirmed in cross-cultural studies on adult and children samples using a variety of questionnaires (e.g., De Fruyt, McCrae, Szirmák, & Nagy, 2004; Klimstra et al., 2011).

The majority of the existing Big Five instruments include a large number of items, requiring considerable time to complete. On the one hand, the large number of items allows assessing factors with greater accuracy. On the other hand, long questionnaires cannot be used when time or resources for assessment are limited. It is particularly difficult to use long instruments with children or adolescents.

To overcome the limitations of long instruments, several short versions of Big Five questionnaires for children and adults were created (e.g., for children – the Five Factor Personality Inventory – Children (FFPI-C; McGhee, Ehrler, & Buckhalt, 2007); the Big Five Questionnaire – Children (BFQ-C; Barbaranelli et al., 2003); for adults – Big Five Inventory (BFI; John & Srivastava, 1999); Mini-IPIP (Donnellan, Oswald, Baird, & Lucas, 2006). The majority of the short Big Five instruments aim to provide a brief estimation of the five factors, without assessing separate facets within each factor.

The Big Five model and its full instruments have focused on a two-level hierarchy. Assessing individual facets within each factor has benefits when examining relations between personality and external criteria (e.g., Paunonen et al., 2003). Using facets instead of factors can increase the predictive accuracy for various behavioral characteristics. Paunonen and Ashton (2001) compared the predictive ability of the five factors and the facets that constitute those factors for various behavior characteristics (Paunonen & Ashton, 2001). They concluded that facets could explain a substantial part of the variance of behavior characteristics better than factors could. These conclusions were confirmed in several studies (Paunonen et al., 2003; Ekehammar & Akrami, 2007).

Problem Statement

The current study was developed to create a short version of the Big Five instrument to assess facets and factors. Saving facets as a valid level of measurement in the short version of Big Five instruments can be useful from methodological perspective. When the full form implies facet and factor levels but the short form preserves the factor level only, the validity of the instrument may be compromised (Smith, McCarthy, & Anderson, 2000). Thus, saving the two hierarchical levels of measurement in the short form can improve the psychometric properties of the instrument.

Research Questions

We used the Big Five Questionnaire (BFQ; Caprara et al., 1993) as a basis for the development of the short version (BFQ-RB (Russian Brief). We have focused on the within-network construct validity approach (Marsh et al., 2005). The focus of this approach is the estimation of specific features of a construct – its components, structure, and dimensionality. We aimed to test the factor structure of the facets and factors using item-level confirmatory factor analysis.

Purpose of the Study

The main aim of the current study is to develop a short version of the Big Five instrument suitable for administration with Russian-speaking populations of adolescents to assess facets and factors. To achieve our goal, we used the Big Five Questionnaire (BFQ; Caprara et al., 1993) as a basis for the development of the short version (BFQ-RB (Russian Brief)). We used data from a survey of 2215 Russian adolescents (14-18 years) and applied confirmatory factor analysis to select items for the short version and assess the psychometric properties of the new instrument.

Research Methods

Participants and procedure

The sample consists of 2215 participants (59% women), 14 -18 years of age (M= 16.31, SD = .69). The respondents came from seven different regions of Central Russia. To obtain more accurate estimations of the psychometric properties of the instrument, we used two randomly selected subsamples that were extracted from the whole sample. At the beginning, we developed a brief version of the questionnaire on the first subsample, which consisted of 1128 students (60% women), 14-18 years of age (M=16.29, SD =.69). Next, we estimated the obtained models on the second subsample, which consisted of 1087 (58% women), 14 - 18 of age (M=16.32, SD = .69).

All respondents filled in the questionnaire online in their schools: first, the demographic information (gender, age, education status, etc.), then the BFQ. They completed the tests in their classrooms, in small groups. The data were collected anonymously. The respondents gave their informed consent online before the start of the survey.

Instrument

The BFQ was developed by Caprara and co-authors in 1993 (Caprara et al., 1993). The BFQ consists of 132 items integrated into twelve facets and six factors (ten facets within the Big Five factors and two facets for the Lie scale). The factor Extraversion consists of Dynamism and Dominance facets. Agreeableness consists of Empathy and Politeness facets. Conscientiousness consists of Scrupulousness and Perseverance facets. The Emotion Stability factor comprises Emotion Control and Impulse Control facets. The factor of Openness consists of Openness to Culture and Openness to Experience. The respondents completed the items using a 5-choice scale that ranges from complete disagreement (1 = very false for me) to complete agreement (5 = very true for me).

The original validation of the BFQ was done on a sample of Italian adults (18 years and older). Confirmatory factor analysis with facets scores confirmed the five-factor structure and demonstrated that each facet belonged to its factor only and had low factor loadings on other factors. The construct validity of the BFQ scales was demonstrated by high correlations with the analogous scales of the NEO-PI for both the Italian and American samples (Barbaranelli, Caprara, & Maslach, 1997; Caprara et al., 1993).

The BFQ was also previously validated on a Russian sample (Osin et al., 2015). To confirm the factor structure of the instrument, Osin and co-authors used confirmatory factor analysis with parcels. They demonstrated that the model with a ten-factor solution where each facet was treated as separate factor had a better fit indices than the five-factor and second-order factors solution. They also tested the correlations between factor scores and the scores on MMPI scales. Based on these results, they concluded that the Russian version of the BFQ had good psychometric properties.

Although there exists a short version of the BFQ developed especially for children (BFQ-C), we choose the full version of the BFQ for adaptation for two reasons: (1) The BFQ-C aimed to assess the five factors only and did not assess facets within the factors, and (2) The BFQ-C was created to assess the Big Five factors in children from 9 to 13 years old. Although the psychometric properties of the BFQ-C were examined on samples of different ages (e.g., French sample – 8-14 years old (Olivier, &Herve, 2015); American sample – 11-14 years old (Gaio, 2012) and Russian sample – 8-15 years old (Malykh et al., 2015), there is a relative deficit of instruments that briefly estimate the Big Five factors and facets on high school student samples (15-18 years old).

Statistical approach

We have focused on the within-network construct validity approach (Marsh et al., 2005). The focus of this approach is the estimation of specific features of a construct – its components, structure, and dimensionality. The between-network approach aims to locate a construct in a broader conceptual space, estimating its relations with other constructs. Marsh and colleagues pointed out that the “resolution of within-construct issues should be a logical prerequisite to between-constructs research” (Marsh et al., 2005, p.84).

In contrast with Osin and colleagues’ study (2015), we aimed to test the factor structure of the facets and factors using item-level analysis instead of parcels analysis. As Little and colleagues (2002) noticed, parceling may be problematic when constructs are not unidimensional and when it is unclear what dimensions may underlie a construct (Little et al., 2002). We began with the existing 136-item Russian version of the BFQ and carried out preliminary item selection by experts’ assessment. Experts (teachers and parents) selected 80 items that were more informative to describe adolescents’ behavior. After evaluation, the 80 items were selected for the survey. On the first subsample, we ran confirmatory factor analysis with a maximum likelihood restricted (MLR) estimator and tested several models. Before analysis, scores for negatively directed items were reversed.

First, we tested the models without the Lie scale to get a clear estimation of the Big Five factors. In Model 1, 67 items from the questionnaire (80 items minus the 13 items of the Lie scale) merged into the Big Five factors. Then, we examined a model where 67 items merged into ten correlated facets (Model 2). Based on the results of Model 1, we summed modification indices for each item. Then, we deleted from each facet 3-4 items that showed a higher sum of modification indices and lower factor loadings to obtain a model where each facet was represented by 3-4 items. In total, 36 items were retained.

Then, we tested three models on the short version. Model 1 and Model 2 were the same as those for the full questionnaire. Model 3 was a model with a second-order factor solution. The second-order factor solution implied that 36 items merged into ten facets, which in turn merged into the Big Five factors. We suggested that the second-order factor model should be confirmed to show the suitability of the short instrument for the measurement of both facets and factors. Then, we tested three models adding seven items from the Lie scale: 1) six-factor solution, 2) twelve-factor solution and 3) 6 second-order factors solution. To compare models with the same number of items and different degrees of freedom, we used the chi-square difference test. As we used the maximum likelihood restricted estimator, we used the Sattora-Bentler scaled chi-square (Sattora & Bentler, 2010).

To examine the quality of the brief version of the questionnaire we obtained, we ran analysis on the second subsample. We examined the three previously described models with 36 selected items without the Lie scale. We hypothesized that the replicability of results would confirm the good quality of the instruments.

Findings

First, we tested two models (5-factor and 10-factor solutions) with 67 items. Both models had low goodness-of-fit (GOF) indices, although Model 2 fit the data better than did Model 1 (Table 01 ). The ten-factor model had some problems with the identification facet “Impulse Control” due to this facet having a high correlation with the facet “Emotion Control”.

Table 1 -
See Full Size >

After examining the factor loadings and modification indices, we excluded items that had higher modification indices and lower factor loadings. As a result, 36 items were retained. Then, we tested three models with the 36 selected items.

All models had satisfactory GOF indices (Table 02 ). Model 2 (36 items merged into ten latent facets) had better indices than did the other models. The model with the second-order factor solution fitted the data better than did the model with the five-factor solution. The model with the five-factor solution had the lowest GOF indices of the three models.

Table 2 -
See Full Size >

All standardized factor loadings in the model with the ten-factor solution were greater than 0.50 (Table 03 ), potentially confirming the good construct validity of the latent variables. In the model with the Big Five factor solution, the standardized factor loadings of some items for Extraversion and Openness factors were lower than 0.50.

Table 3 -
See Full Size >

In the model with the second-order factor solution, all factor loadings of the first-order factors were greater than 0.50 (Table 04 ). Moreover, the standardized factor loading of the facet “Emotion control” on the factor Emotion Stability was greater than one.

In the model with the ten-factor solution, the correlations between some facets that theoretically belong to one factor were lower than correlations between facets from different factors (Table 05 ). For example, “Dynamism”, which theoretically belongs to the Extraversion factor, had higher correlations with facets from other factors than it did with another facet from the Extraversion factor.

Table 4 -
See Full Size >
Table 5 -
See Full Size >

In the model with the Big Five factors solution, four factors (Agreeableness, Extraversion, Conscientiousness, and Openness) had high intercorrelations (0.68 - 0.76), while the factor Emotion Stability had low correlations with the other four factors (0.15 - 0.31) (Table 06 ).

Table 6 -
See Full Size >

In summary, our results demonstrated that the ten-factor solution and the second-order factors solution had better fit indices than did the Big Five factors solution.

Finally, we added seven items from the Lie scale (three items for the Egoistic subscale and four items for the Morality subscale) and tested three models: 1) the six-factor solution (Model 1); 2) the twelve-factor solution (Model 2); and 3) the second-order factor solution (Model 3). The fit indices for these models are presented in Table 07

Table 7 -
See Full Size >

According to these results, all the models with the Lie scale had satisfactory fit indices. The model with the six-factor solution had the worst fit indices of all the models. In summary, we can conclude that our short instrument (BFQ-RB) had satisfactory psychometric properties and was suitable to measure facets and factors.

To test the properties of the three models and replicate the results obtained on the first sample, we tested the three models without the Lie scale on the second subsample. All models had satisfactory fit indices. As in the first subsample, the model where each facet represented a separate latent construct had better fit indices than did the model with the Big Five factors solution. The five-factor model had the worst fit indices of the three models (Table 08 ).

Table 8 -
See Full Size >

All standardized factor loadings for the ten-factor model of the second subsample were greater than 0.50. In the model with the second-order factors solution, the standardized factor loadings of the first-order factors were greater than one for the Extraversion and Emotion Stability factors.

Summarized, we replicated the results obtained in the first sample and confirmed that the short version of the instrument fitted to measure both facets and factors.

Conclusion

The main aim of the study was to develop a short version of the Big Five instrument that could be useful for brief estimation of both the Big Five factors and their facets on a Russian language sample. To achieve this goal, we used the Big Five Questionnaire (BFQ; Caprara et al., 1993) as a basis for developing a short version. The BFQ was based on a model where each factor was represented by two facets, including the Lie scale.

We created a short version (BFQ – Russian Brief; BFQ-RB) using data from a sample of 1128 adolescents (14-18 years) and then confirmed the factor structure on another subsample of 1087 adolescents. We estimated three main models. In the first model, the selected items represented the Big Five factors. In the second model, the selected items represented ten correlated latent factors (facets). The third model was the second-order factor model. The third model fitted the data well, suggesting that the BFQ-RB enabled the estimation of both facets and factors.

Our final instrument consists of 43 items, with each facet represented by 3-4 items and each second-order factor consisting of two facets, including the Lie scale. All latent factors in the BFQ-RB had good indicators of internal consistency (Cronbach’s alpha) and overcame passed the threshold of .7, suggested as acceptable by Smith and colleagues (2000).

Previous researchers often used Cronbach’s alpha to confirm the good psychometric properties of facets. A high value of the alpha was supposed to demonstrate high reliability. However, a high value of Cronbach’s alpha does not necessarily indicate a high degree of internal consistency. First, Cronbach’s alpha does not confirm the unidimensionality of a scale. If a test has more than one concept or construct, reporting the alpha can be misleading (Tavakol &Dennick, 2011). Second, the length of the test affects the alpha. If the test length is too short, the value of the alpha is reduced (Tavakol&Dennick, 2011).

We used item-level confirmatory factor analysis, which confirmed the two-dimensionality of the factors and the unidimensionality of the facets. Although McCrae and co-authors (1996) pointed out that using CFA to investigate personality structure in the study of the Big Five had some restrictions (McCrae et al., 1996), their position was later criticized (e.g., Borsboom, 2006). Particularly, Borsboom pointed out that estimating personality traits and testing the hypothesis about the causal relations between latent traits and items required the specification of a reflective latent variable model similar to CFA (Borsboom, 2006).

All items had positive direction except the items of the Emotion Stability factor. Some authors recommended that scales include a roughly equal number of positively directed and negatively directed items to reduce the effect of potential acquiescence response bias (e.g., Furr, 2011; Wolfe, 1993). However, there is evidence that the negative items tend to form a different dimension (e.g., Wong, Rindfleisch & Burroughs, 2003). Consequently, some authors recommended using only positively directed items (e.g., Salazar, 2015). We decided not to include positively and negatively keyed items in the same factor or facet to improve the internal consistency of the scales.

The newly created instrument requires further validation and development. Although the model where each facet was considered as a separate latent construct fitted the data better than did the Big Five factors solution, a closer look at the results indicated that some factors were unidimensional rather than two-dimensional. Specifically, the facets “Impulse Control” and “Emotion Control” correlated highly in both subsamples (.88 and .86, respectively). This problem was also identified for facets within the Conscientiousness factor. In the second-order factors solution, the standardized factor loading of the “Emotion Control” facet was greater than one. Although standardized factor loadings are allowed to be greater than one, it may indicate on a high degree of multicollinearity in the data (Jöreskog, 1999). This problem may be solved by changing some items to make the two facets within the Emotion Stability and Conscientiousness factors more distinct.

Our analyses demonstrated that the Extraversion and Openness factors were two-dimensional. Moreover, some facets within these factors had greater correlations with facets from other factors. For example, in the first subsample, the facet “Dynamism” had a greater correlation with “Politeness” (.74), “Perseverance” (.72) and “Openness to Experience” (.77) than with the “Dominance” facet (.62). The same pattern was observed in the second subsample. Some revision of items from the “Dynamism” and “Openness to Experience” facets should be undertaken to achieve greater correlations of the facets within factors than between factors. It should be noted that a previous item-level study of the Big Five instrument also identified some problems with the Extraversion and Openness factors (Egan, Deary & Austin, 2000).

The “Dynamism” facet correlated highly with other facets, with the exception of facets from the Emotion Stability factor. It is possible that “Dynamism” not only is a subscale of Extraversion but also reflects a more general activity and energy of a person, with wider biological underpinnings than those of the Extraversion factor. Further validation studies will show whether it is necessary to revise the existing operationalization of the “Dynamism” facet as a subscale of the Extraversion construct.

The “Impulse Control” facet had low or non-significant correlations with other facets. Items representing facets of the Emotion Stability factor were reversed. There is evidence that reversed items have lower correlations with positively directed items (e.g., Dunbar et al., 2000). It is also possible that “Impulse Control” truly has low correlations with other facets. To disentangle these potential explanations, further studies should measure all facets in the same (positively keyed) format.

In this study, we focused on the evaluation of the internal structure of the BFQ-RB. All participants in the study completed a short 80-item questionnaire. To test whether the short instrument measures the same factors as the full questionnaire, it is necessary to compare this short instrument with the full BFQ. Finally, further research is needed to examine the criterion validity of the BFQ-RB by testing its correlations with other Big Five inventories and behavioral outcomes. This can be done in future studies.

Acknowledgments

This study was supported by the grant from the Russian Science Foundation [grant RSF №17-78-30028

References

  1. Barbaranelli, C., Caprara, G. V., & Maslach, C. (1997). Individuation and the Five Factor Model of personality traits. European Journal of Psychological Assessment, 13(2), 75.
  2. Barbaranelli, C., Caprara, G. V., Rabasca, A., & Pastorelli, C. (2003). A questionnaire for measuring the Big Five in late childhood. Personality and Individual Differences, 34(4), 645-664.
  3. Borsboom, D. (2006). The attack of the psychometricians. Psychometrika, 71(3), 425.
  4. Caprara, G. V., Barbaranelli, C., Borgogni, L., & Perugini, M. (1993). The “Big Five Questionnaire”: A new questionnaire to assess the five factor model. Personality and individual Differences, 15(3), 281-288.
  5. Costa, P. T., & McCrae, R. R. (1992). Four ways five factors are basic. Personality and individual differences, 13(6), 653-665.
  6. De Fruyt, F., McCrae, R. R., Szirmák, Z., & Nagy, J. (2004). The Five-Factor Personality Inventory as a Measure of the Five-Factor Model Belgian, American, and Hungarian Comparisons with the NEO-PI-R. Assessment, 11(3), 207-215.
  7. Donnellan, M. B., Oswald, F. L., Baird, B. M., & Lucas, R. E. (2006). The mini-IPIP scales: tiny-yet-effective measures of the Big Five factors of personality. Psychological assessment, 18(2), 192.
  8. Dunbar, M., Ford, G., Hunt, K., & Der, G. (2000). Question wording effects in the assessment of global self-esteem. European Journal of Psychological Assessment, 16(1), 13.
  9. Egan, V., Deary, I., &Austin, E. (2000). The NEO-FFI: Emerging British norms and an item-level analysis suggest N, A and C are more reliable than O and E. Personality and Individual differences, 29(5), 907-920.
  10. Ekehammar, B., & Akrami, N. (2007). Personality and prejudice: From Big Five personality factors to facets. Journal of personality, 75(5), 899-926.
  11. Furr, M. (2011). Scale construction and psychometrics for social and personality psychology. SAGE Publications Ltd.
  12. Gaio, V. M. (2012). Psychometric properties of the big five questionnaire-children (BFQ-C) in American adolescents (Master's thesis, Arizona State University).
  13. John, O. P., & Srivastava, S. (1999). The Big Five trait taxonomy: History, measurement, and theoretical perspectives. Handbook of personality: Theory and research, 2(1999), 102-138.
  14. Jöreskog, K. G. (1999). How large can a standardized coefficient be. Unpublished Technical Report. Retrieved from:http://www.ssicentral.com/lisrel/techdocs/HowLargeCanaStandardizedCoefficientbe. pdf.
  15. Klimstra, T. A., Crocetti, E., Hale III, W. W., Fermani, A., &Meeus, W. H. (2011). Big Five personality dimensions in Italian and Dutch adolescents: A cross-cultural comparison of mean-levels, sex differences, and associations with internalizing symptoms. Journal of Research in Personality, 45(3), 285-296.
  16. Little, T. D., Cunningham, W. A., Shahar, G., & Widaman, K. F. (2002). To parcel or not to parcel: Exploring the question, weighing the merits. Structural equation modeling, 9(2), 151-173.
  17. Malykh, S.B., Tikhomirova, T.N., Vasin, G.M. (2015). Adaptation of the Russian version of the «Big Five Questionnaire – Children (BFQ-C)». Theoretical and Experimental Psychology (Teoreticheskaya i eksperimentalnaya psikhologiya) , 8(4), 6-12.
  18. Marsh, H. W., Ellis, L. A., Parada, R. H., Richards, G., &Heubeck, B. G. (2005). A short version of the Self Description Questionnaire II: operationalizing criteria for short-form evaluation with new applications of confirmatory factor analyses. Psychological assessment, 17(1), 81.
  19. McCrae, R. R., & Costa, P. T. (1991). The NEO Personality Inventory: Using the Five‐Factor ModeI in Counseling. Journal of Counseling & Development, 69(4), 367-372.
  20. McCrae, R. R., Zonderman, A. B., Costa Jr, P. T., Bond, M. H., & Paunonen, S. V. (1996). Evaluating replicability of factors in the Revised NEO Personality Inventory: Confirmatory factor analysis versus Procrustes rotation. Journal of Personality and Social Psychology, 70(3), 552.
  21. McGhee, R.L., Ehrler, D.J., & Buckhalt, J.A. (2007). Five Factor Personality Inventory-Children. Austin, TX: Pro-Ed
  22. Olivier, M., & Herve, M. (2015). The Big Five Questionnaire for Children (BFQ-C): A French validation on 8-to 14-year-old children. Personality and Individual Differences, 87, 55-58.
  23. Osin ,E.N., Rasskazova, E.I.,.Neyaskina, Yu. Yu., Dorfman, L.Ya., Aleksandrova, L.A. (2015). Оperacionalizacion of five-factor model of personality lines on the Russian selection. Psychological diagnostics (Psihologicheskaya diagnostika), (3), 80-104.
  24. Paunonen, S. V., & Ashton, M. C. (2001). Big five factors and facets and the prediction of behavior. Journal of personality and social psychology, 81(3), 524.
  25. Paunonen, S. V., Haddock, G., Forsterling, F., & Keinonen, M. (2003). Broad versus narrow personality measures and the prediction of behaviour across cultures. European Journal of Personality, 17(6), 413-433.
  26. Salazar, M. S. (2015). The dilemma of combining positive and negative items in scales. Psicothema, 27(2), 192-200.
  27. Satorra, A., & Bentler, P. M. (2010). Ensuring positiveness of the scaled difference chi-square test statistic. Psychometrika, 75(2), 243-248.
  28. Smith, G. T., McCarthy, D. M., & Anderson, K. G. (2000). On the sins of short-form development. Psychological assessment, 12(1), 102.
  29. Tavakol, M., &Dennick, R. (2011). Making sense of Cronbach's alpha. International journal of medical education, 2, 53.
  30. Wolfe, R. N. (1993). A commonsense approach to personality measurement. In K. H. Craik, R. Hogan, & R. N. Wolfe (Eds.), Fifty years of personality psychology (pp. 269–290). New York: Plenum
  31. Wong, N., Rindfleisch, A., & Burroughs, J. E. (2003). Do reverse-worded items confound measures in cross-cultural consumer research? The case of the material values scale. Journal of consumer research, 30(1), 72-91.

Copyright information

This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

About this article

Cite this paper as:

Click here to view the available options for cite this article.

Publisher

Future Academy

First Online

18.12.2019

Doi

10.15405/epsbs.2018.11.02.37

Online ISSN

2357-1330