CRITERIA FOR THE DEFINITION OF INDICATORS IN ARCHITECTURAL LEARNING IN THE DESIGN STUDIO THROUGH THE USE OF THE E-PORTFOLIO

The objective of this research is to identify, within the context of teaching-learning through the Design Studio, factors and criteria which may support the construction of architectural indicators of learning in students. This may be achieved through the integrated analysis of the evidences of learning included in the e-portfolio, using the traditional learning process as a complement in the Design Studio. This research is of mixed type and includes an exploratory sequential design of longitudinal type that ran through a period of six consecutive semesters. The results obtained show the full potential of objectual and collaborative evidences contained in the e-portfolio, in order to reflect the presence of architectural learning elements, as well as the presence of factors and features specific to the context and dynamics of the learningteaching process, which may result in an approach to the construction of potential architectural indicators of learning.


Introduction
Within the context of architectural education, the Design Studio is where most of the learning-teaching process occurs (Pasin, 2017). Within this space, students experience the construction of knowledge within a collaborative and multi-sensory environment, based on the concept of learning by doing (Kurt, 2011). As a result of that, the teaching strategies to develop learning processes become relevant. However, the development of processes often occurs only in an indirect way, whether because the processes are not properly identified or because the practical-theoretical means to assess those processes are not available. The development of processes starts with tasks oriented to the solution of problems instead of tasks oriented to the development of cognitive (talents, skills and abilities) and effective (values and attitudes) processes that are specific to the person who learns (Román & Diéz, 2000).
According to (Webster, 2004) there is a need to present the development of a critical vision of the cognitive and practical aspects that happen within the teaching-learning process in the Design Studio and that show the importance of a reflexive dialogue that allows students to comprehend the quality of their actions (emerging practice), a consist-ency between what they say (presented theory) and what they do (practical theory). Promoting the integration of an instructed and a constructed education would ensure the development of the student's autonomy when it comes to the ability of thinking, judging, choosing and acting independently (Mabardi, 2012). According to this concept, the Design Studio commits to the construction of learning, where students create their own knowledge based on the experiences gathered at the workshop, be it their own or their classmates' , once the different stages of the process in a project are executed (Masdéu & Fuses, 2017), in order to carry out the integration of knowledge that the subject requires.

Elements of architectural learning
Architectural learning is the process of acquiring the necessary knowledge in order to understand and solve a specific type of problems or situations and to professionally execute such knowledge (Foqué, 2010). Architectural learning begins at an assumed point of lesser knowledge and ends at a new point of greater knowledge in terms of architectural skill (Carmel-Gilfilen & Portillo, 2010). This process necessarily implies a transformation in the epistemological field of the person doing the learning, and this evolution should manifest itself through both the development and specialization of already existing knowledge and the acquisition of new knowledge. The success of the learning process should be seen in the ability to understand the architectural phenomenon in all its scope and in the ability to act properly and flexibly when managing specific situations, as stated by (Alba Dorado, 2016).
Thus, the global process of learning architecture must allow, according to (Saldarriaga, 1996), the fulfillment of the following pedagogical objectives: a) the development of a reference field or architectural culture, based on the total knowledge that architecture encompasses, in its material, environmental, urban, social and cultural aspects; b) the development of the ability to represent space and architectural ideas; c) the development of strategies to identify and understand common situations and problems within the world of architecture, in their corresponding contexts; d) the development of knowledge and management of the elements and regulations of the architectural language, especially its forms and applications in the resolution of problems; e) the development of project strategies; f) the development of sustainment and assessment criteria of the architectural and urban proposals within their corresponding context.
Fulfilling these pedagogical objectives, developed within the context of curricular competences and when applied to the subject of the architectural project in the academic unit, forms an architectural mentality in the student, different from the mentality he had when he first began studying. Referents may be either quantitative or qualitative, where the correspondences of the expected competences support the way of acting, where education is a systematic process of mutual responsibility between the person and the social, cultural, economic and environmental context (Tobon, 2013). Considering that competence is shown through actual performance, the assessment of competences must be executed on the activity done by the student with his evidences as part of the meta-cognitive processes. Based on this, it is essential to be able to observe the existence of these evidences related to the pedagogical objectives. Some studies (Fuentealba et al., 2017) point to the presence of these evidences as relevant elements in the fulfillment of the pedagogical objectives within the Design Studio. These evidences can be observed in the learnings that students build and show through the different resources and media they utilize in order to evidence and communicate them.

The e-portfolio and the integration of evidences of learning
In the context of current education, with the use of new technologies and ICT used in professional training and based on the learning paradigm that centers on the students' performance (Sakar, 2012;Ben Youssef & Dahmani, 2014;Karamti, 2016), the e-portfolio appears to be the pedagogical instrument best suited for education based on learning by doing (Barberà et al., 2006;Gutiérrez & Roco, 2013). As a specific pedagogical strategy, the eportfolio opens opportunities to new selected evidences of learning, improving our understanding of applied and integrated learning (Barrett, 2010). The evidences of learning in the e-portfolio help to describe and interpret the ways of understanding, the levels of reflection and the assessment, evidencing the complexity and diversity of the learning-teaching processes (Gelabert et al., 2009). Since the e-portfolio melds technological and analogical dimensions of the learning construction, the relevant challenge in our society is the understanding that students exist both in a digital and a physical world, and they often see these two versions of themselves as separate and distant. The e-portfolio gives students the opportunity to analyze their digital identity in a critical way and to explore how this identity can connect with the values, beliefs and interests of their real identity (Kehoe & Goudzwaard, 2015). Therefore, the e-portfolio is an essential tool to verify the presence of evidences of learning built by the student, from the visualization and analysis of the demonstrative and interactive contents included in it.
With its unique process feature based on learning by doing in order to achieve the pedagogical goals set, implementing the e-portfolio in the teaching-learning process performed in the Design Studio generates the following research question: what type of evidences reflected in the e-portfolio could construct architectonic indicators of learning in the Design Studio? Thus, the objective of this research is to be able to identify factors and criteria that may support the construction of architectonic learning through the integrated analysis of the evidences held in the e-portfolio, in order to determine the nature of, the amount of and the quality of the possible architectonic indicators of learning that the e-portfolio is capable of reflecting.

Methodology
The focus of this research is of mixed type, with the purpose of counting with as broad an understanding of the study object as possible, according to (Pereira Pérez, 2011). It presents a sequential exploratory design, as explained by (Creswell & Plano Clark, 2011), and it is of longitudinal type, with data collected through 6 consecutive semesters between 2012 and 2014. A two-approach structure is offered: a theoretical one (qualitative) and a practical one (quantitative), which are interconnected in order to practically test the proposal of the theoretical construction, related to the presence and features of the architectural evidences of learning contained in the e-portfolio. The data collected through the research is mostly quantitative and it was analyzed in a descriptive and statistical way using Microsoft Excel 2013. The qualitative data refers to the construction of the features and category of the data and it relates to the observation and online follow-up of the evidences of learning reflected in the e-portfolios.
The sample corresponds to 6 academic semesters in the Design Studio subject for second-year students between 2012 and 2014. During the study period, the e-portfolio was used as a pedagogical strategy to complement the regular development of the Design Studio class; its objective was to provide additional support to the learning construction performed by each student and to provide evidence of this process. Each class had two professors in charge of the teaching process and an average of 24 students, for a total of 144 students. This research has some limitations regarding the singularity of the studied context, taking into consideration a specific sample of the teaching-learning process in the Design Studio with second-year students of the architecture major. Data gathering was performed within a limited time period adapted to the curricular dynamic of each semester, which restricted the amount of information collected and may limit the reach and general applicability of this research.

Results of the research
Results are shown in relation to the declarative-effective double structure orientation in the design of this research, which aims to verify the correspondence between the theoretical construction of indicators that support architectonic learning done by the students (from the emerging practice) and their effective observance in the evidences reflected in the e-portfolio (from the exposed theory and theory in practice).

From a declarative perspective
This orientation is based on the construction of a theoretical relation based on the idea of an emerging practice in the Design Studio, among architectonic learning elements and the types of evidences expressed in the e-portfolio and may be considered as possible indicators of learning in students.
These learning elements are related to the pedagogical objectives of the general architectonic learning process (Saldarriaga, 1996), and they are a normal part of the teaching-learning process performed in the Design Studio. These elements may be removed in components, which define specific aspects of the architectonic knowledge. These components may be observed through certain evidences, which may be pre-indicators for the pedagogical objectives of the teaching-learning process. Table 1 shows the possible theoretical relation frame between the elements of architectonic learning and the type of evidence that can be observed through the e-portfolio and that may be considered evidences of architectonic quality.
The six elements of architectonic learning (letters a, b, c, d, e and f) are related to 14 components associated to them, from which pre-indicators of learning can be observed in the pedagogical performance of the Design Studio: the 6 elements cover certain central criteria for the formation process done in the Design Studio. The 14 com po nents correspond to aspects that are specific to architectonic knowledge, associated to the different areas that involve each of the elements of architectonic learning. In this particular case, they relate to the expected learnings defined in the curricular design for the second-year subject of Design Studio. The pre-indicators are resources (that can be observed and measured) used to communicate the learnings from the perspective of the students. They are used to establish a connection between the theoretical construction of the expected pedagogical objectives (architectonic learning elements) and the hints that verify their fulfillment, which gives them the sense of evidence with architectonic quality. Table 1 shows the need to build a matrix to verify the connection between the components of architectonic learning and the possible evidences contained in the eportfolios, which would allow for an empirical verification of the presence of evidences with high architectonic quality potential. In order to structurize this matrix, the possible evidences contained in the e-portfolio were organized into types, groups and categories in order to achieve a higher level of precision when connecting the possible resource to the respective element and learning component. The types of evidences were objectual and interactive: Objectual, defined as the evidence that provides demonstrative knowledge, either declarative or procedural, and that the student uses to reflect knowledge; and Interactive, defined as the evidence that provides bidirectional communication and that the student uses to exchange comments and opinions (Feedback) with the participants of the formative process. Both types of evidences naturally stem from working with the e-portfolio (Barrett, 2010).
Four groups of evidences were structured which drive aspects considered relevant for the construction of learning: Structural, referring to the personal experience of the author and the curricular context of the learning process; Demonstrative-informative, referring to the graphic material that shows the learning process carried out by the student; Collaboration, referring to the interaction and communication (feedback) between the participants (students and professors); and Evaluation, referring to the reflections and results of the learning built by the students. The reflection levels scale was adapted from (Roberts, 2012): elementary (reports and responds), medium (tells) and expert (reasons and rebuilds).
The total number of possible evidences that could be observed through the e-portfolio was classified into 9 categories that relate to the nature of their content. Each category of evidences presents, in turn, different aspects associated to the communicative resource used which is defined according to the nature of each one of them: authoring, resource origin, process stage, achievement level, reach, etc. Table 2 shows the organization of the evidences.
As a result of the relation between the learning element and the evidences, Table 3 shows the relation between the different evidences categories (with their specific aspects) and the components of architectonic learning, indicating their presence or absence. These categories correspond to the resources normally used as evidence of learning in the  Design Studio and that can be reflected in the structure of the e-portfolio. The structural evidence group was not included in this matrix since the nature of the documentation category in the subject corresponds to the informative record of the tasks, instructions and indications for the development of the exercises performed during that corresponding semester while not being evidence created by the student in order to show the learning process. Based on the observation of the proposed matrix, it can be surmised that all the categories of evidences have the ability to reflect architectonic learning components to different degrees. At the same time, it becomes apparent that some aspects of specific categories do not possess the ability to reflect any learning component. The categories of evidences have specific abilities to show elements of architectonic learning and in that regard, the particular aspects of each category play a relevant role.
Notes: * instructions and the student's own; ** professor observations and the student's own.   Figure 1 shows the specific ability of the particular aspects in each category in order to make reference to the biggest number of architectonic learning components, represented in a percentage scale from 0% for no ability and 100% for a total ability. In order to establish a relation from the level at which each specific aspect of the evidence category connects to the presence of architectonic learning components, a scale value is shown in Table 4. This scale defines the level of quality for the evidence according to the presence percentage integrated in the different components of the architectonic learning elements 1 .
By applying this value scale on the specific aspects of the evidences to be observed in the e-portfolio, a table can be created to show the level of quality of the architectonic evidence with regard to its nature ant type of content, establishing an organization system of the different aspects to be considered. This can be seen in Table 5.
The previous table shows that the degrees of quality of the architectonic evidence form different groups of evidences with similar categories, but with different specific 1 The percentage structure is the result of dividing the total 100% into three equal parts, each one assigned to a level of quality. aspects which may be linked to different degrees of instructed knowledge (connected to the emergence practice) for the basic and medium levels and to the constructed knowledge (connected to the exposed theory and practical theory) for the advanced level. The evidence categories are grouped in each quality level, where the advanced level gathers the largest number (6 of the 8 categories), two of which only appear in this level (Video and Links). The basic and medium levels show a similar number and type of categories (4 out of 8). The categories of Online presentation software and Professor commentaries are only included in these levels. The Text and Images categories are the only ones included in all the proposed quality levels.
When the relation between the components and the evidence categories is synthesized, a matrix is created (Table 6) which shows that the type of demonstrative evidence (objectual) covers all the elements of architectonic learning appearing in 100% of 3 out of the 5 categories (images, video and links). The type of interactive evidence is also connected to all of the elements of learning, appearing in 100% of two of them (reflections and student commentaries).
From the perspective of the integration between the demonstrative-informative evidences and the interaction for each proposed learning element, this condition can be seen in 6 learning elements, although there are variations in their respective components. Likewise, it can be seen that all evidence categories may reflect different aspects of the architectonic learning components according to their nature and features. This allows an association with the quality levels of the evidence, in which the relevant categories of images, video, links, reflections and student  commentaries, as well as the secondary categories of text and online presentation software, hold the potential to become a possible and specific factor that may support architectonic learning and that may be reflected through the e-portfolio.

From the effective perspective
This orientation is based on the theory in practice of the students' output, in relation to the architectonic learning elements and their evidences through the quantification of the evidences expressed in the e-portfolio. Its results may coincide with the declarative perspective which would boost the presence of possible indicators in the students' architectonic learning. From the quantification of the quality levels in the architectonic evidence, in the different categories shown in the e-portfolio, we can surmise the real hierarchization that the evidences of learning assume. Based on the data compiled during the studied period between 2012 and 2014, Figure 2 shows the dynamic of the presence of the evidence categories and the architectonic learning elements in the analyzed e-portfolios.
The 4 evidence groups appear to be consistent during the studied time period, although there are variations among semesters. It is important to note the presence of the demonstrative-informative group above all other categories, along with the minimal presence of the structural group. The detailed quantification of the groups, categories and specific aspects of the evidences contained in the e-portfolios allows for an analysis on how the specific importance of each evidence is distributed, which can be seen in Table 7.
Broadly speaking, it is possible to see that in all the evidences categories there are records for the different semesters studied, both of the objectual and interactive types. However, the specific importance for each category show important variations: the greatest amount of    Notes: * instructions and the student's own; ** professor observations and the student's own. evidences recorded are found between specific aspects of the demonstrative-informative category and the evaluation, without an even distribution. The collaboration category shows a small presence with an average of less than 50 records for the 6-semester period of this study. The quantification of this distribution can be seen in Figure 3. Based on the average value of the accumulated records for each specific aspect in the categories of the evidences analyzed, it is possible to produce a summary of the specific aspects of the evidences with the highest presence and that may help to visualize the types of architectonic learning components present in the e-portfolios. In order to do this, a base value of 50 records for each aspect was considered during the studied period. The result can be seen in Table 8.    It is possible to observe that the evidences are linked only to 2 groups (Demonstrative-informative and evaluation) and 3 categories (text, images and reflections). Other groups and categories show values below 50 records, therefore they were not considered given their low representation. So, the potential levels of architectonic quality from the evidence shown in the e-portfolios is limited to three categories (text, images and reflections), which show specific aspects from the typology and nature of the used resource. The analysis does not show the significant inclusion of categories such as video, links, student commentaries and online presentation software which have a great potential to become an architectonic learning indicator. It also shows that the evidences present a high preference evidence of the objectual type, in all of its categories, and a partial preference towards the interactive type, in a specific category and aspect.
Thus, when comparing the percentual distribution of the evidences categories given by the analysis of the eportfolio use with the percentual distribution theoretically proposed for the integration between evidences and learning components, a great difference can be observed in the potential coverage of the architectonic learning elements for the learning-teaching process in the Design Studio (see Figure 4).
The comparison shows the preference, in practice, of some evidence categories (images, texts and reflections) in order to support the construction of learning through the e-portfolio, as well as the lack of balance in the management of other categories (video, links, online presentation software, student commentaries) that make it easier to integrate the elements of architectonic learning to be developed in the Design Studio.

Discussion and conclusions
Based on the results of this research, it is possible to appreciate the partial coincidence between the declarative and effective perspectives proposed by the study in the verification of the evidences that may point to the presence of possible architectonic indicators of learning in the Design Studio.
The generality of the teaching-learning process as seen through the evidences of the e-portfolio shows the absence of balance between the cognitive and practical work, when related to the demonstrative-informative and interactive types of evidences. A strong bias towards the demonstrative may be observed, along with a low incidence towards the interactive, which suggests a lack of reflection and articulation in the teaching-learning process structured and developed in the Design Studio, which supports the work of (Román & Diéz, 2000).
However, the relation between instructed and constructed learning can be seen in the evidences of learning present in the e-portfolio with specific categories (texts, images and reflections) that seem to be integrated into the resources used by the students in order to share their own reflections, judgments and decisions in their constructed learning, according to the work of (Mabardi, 2012). With this, the critical vision of the cognitive and practical aspects of the teaching-learning process in the Design Studio proposed by (Webster, 2004) appear to be partially complete, establishing in the set of evidences an integration from the theory in practice towards the exposed theory and the emergent practice.
On the other hand, the presence of evidences of learning in the e-portfolio, related to the different levels of architectonic quality, presents the idea that the architectonic learning elements can be reflected through this pedagogical strategy in order to achieve the learning objectives in the Design Studio. According to this perspective, the e-portfolio can be used as a measuring stick to gauge the construction of learning in the student, as proposed by (Blanch Gelabert et al., 2009). The evidences categories mainly shown (texts, images and reflections) fall into the advances quality level but only represent 50% of their universe. This condition may be explained in the demonstrative-informative preferred features of the evidences which try to communicate the different dimensions of the learning-teaching pro-  cess inside the Design Studio: while the most observed evidences (texts, images and reflections) hold a learning construct characteristic with a rather individual reach, the least observed evidences (online presentation software, video, links, professor and student commentaries) hold a learning construct characteristic with a more collective reach. The vast majority of the evidences is linked to the ability to relate physical elements typically found in the traditional learning-teaching process in the Design Studio (physical models, drawings, sketches, pre-designs, texts, infographic presentation) with virtual interaction elements based on digital resources and technological media (audiovisual stories, digital images, feedback, links, tutorials, digital illustration software). Although the e-portfolio acts as a pedagogical tool that reflects the learning-teaching process in the Design Studio, it is not able to reach its full potential as a pedagogical strategy to integrate the formative benefits that new technologies can contribute to the performed process. According to these results, it can be determined that there is a tendency towards the construction of reflected learning in the e-portfolio, from the descriptive to the reflexive, that allows for a partial observation of evidences that aim towards the integral recognition of the architectonic learning elements, as well as the didactic processes linked to the construction of new knowledge in the students. Thus, the relevant evidences, such as texts, may be linked to arguments used to explain ideas and purposes; images may be linked to a summary of the projectual processes and the effective use of the architectonic language, and reflections may be linked to explaining that which is learnt and to a criticism of the work performed. The more specific conditions and features of these resources need to be studied in order to reach a definition of an indicator of learning, as well as establishing the conditions for the development of these resources with a high learning indicator potential, even though they are not clearly reflected in the e-portfolio evidences (video, links, online presentation software, student commentaries).
The learning-teaching process in the Design Studio presents conditions of high level commitment and collaboration among its participants, both individually and collectively, representing a learning environment in which the construction of knowledge is experienced through learning by doing. It is important to identify the factors and criteria that support the process of architectonic learning construction, especially if the evidences gathered in the e-portfolio may lead to the determining of some features in potential architectonic indicators of learning.
Based on the integrated analysis of the results in this research, it is possible to determine the existence of some factors that may support the construction of architectonic learning and that are related to the integration of reflexive dialogue (Webster, 2004) in the Design Studio: -The correspondence between evidence and learning-teaching process, considering the need for the evidences to be a genuine reflection of the learning-teaching process constructed in the Design Studio and with which it is possible to notice the growth level in the training process. -The theoretical-practical integration as it relates to the idea of having evidences that possess both demonstrative-informative and integrated elements with which to present a more comprehensive vision of the reach and depth of the constructed knowledge, both in its individual and collective dimensions. -The constant presence of multiple learning-teaching evidences, seen as the need to reflect a learning process that is built with a constant revision and reflection on the work performed and the achievement of the expected pedagogical objectives, with a strong emphasis on meta-cognitive development. -The feedback articulation in the learning-teaching process, understood as the possibility of building a new collective among the members of the learningteaching process based on cooperation, interdependence and the construction of meaning. Regarding the criteria for the definition of possible indicators of learning, these fit the pedagogical objectives defined by (Saldarriaga, 1996); they can be interpreted as components that allow for the visualization of said elements through the evidences of learning contained within the e-portfolio. From the practical theory of the Design Studio, the definition of pre-indicators may be considered an appropriate way to observe evidences with a high architectonic quality. Based on the results obtained, it can be established that the different categories of evidences (texts, images, online presentation software, video, links, reflections, professor and student commentaries) have the potential to build indicators of learning, but with different features.
The possible features that need to be present in the evidences in order to be considered indicators of architectonic learning relate to the following aspects: -An authentic nature (pertinence); that is, the ability for the resource of the evidence used to communicate the architectonic learning-teaching culture of the Design Studio, and that involves individual, collective and integrative aspects of the formation process. -Presence level (availability); that is, the ability for the resource of the evidence used to keep a constant presence in the development of the architectonic learning process, which allows for the definition of follow-ups by the professor and the paths of the student. -Trustworthiness; that is, the ability for the resource of the evidence used to be able to be categorized within the architectonic learning development process, which allows for the representation of statistical attributes. -Quality (usefulness), that is the ability for the resource of the evidence used to integrate different elements of architectonic learning (components), which allows for the expected learning objectives to be reached.
This research proposal shows the need to be able to count on the theoretical-practical basis for the definition of architectonic indicators of learning that may be observed through the e-portfolio, within the learningteaching process developed in the Design Studio. Nowadays, this requirement is considered to be of the utmost relevance, given the impact of the incorporation of the new technologies and ICT in the professional training processes and the growing need to improve the teaching process of architecture in its didactic-pedagogical aspects.
Based on the results presented by this study, future research ideas can be explored; on the one hand, these relate to implications of the theoretical type, based on the transformation of the evidences of the learning-teaching process into general indicators, supported in the factors and features defined for the fulfillment of the expected pedagogical objectives. On the other hand, the research proposal relates to implications of an empirical type, which moves forward from the use of the factors and features hereby defined to generate indicators of learning from the evidences contained in the e-portfolios or other pedagogical tools in order to unveil good learning-teaching strategies in the Design Studio.

Disclosure statement
There was no sponsor to finance this research and there is no conflict of interests with regard to the objectives and impartial treatment of the data.