Assessment for DE

Notes from the textbook.

A. Role of assessment

• Acts as basis for instruction
• Acts as a tool to achieve the desired learning outcome
• Develop skills in learners (formative assessment).
• Report success or failure (summative assessment).
• Enhance motivation
• Reinforce confidence

B. Characteristics of several types of assessment

• Outcomes-based-assessment - (reflects mastery of skills identified in the objectives).
  
• Norm-referenced scoring - compare students with others who have completed the same assessment. (Report long term trends and comparisons of extremely large group of learners - should never be used to determine grades)
  
• Grading on the curve - reflects how well the learners perform as compared to one another. (not useful to see whether the students have mastered the content).
  
• Criterion referenced scoring - (reflects mastery of course content)

Characteristics of good assessment:

• Validity - provides an accurate estimate of learning gains.
• Reliability - refers to stability and consistency of an instrument despite occasion and frequency of the assessment taken.
• Clarity of expectations - learners could easily understand the instructions given for the assessment.

Note: A useful assessment reflects learners' progress, understanding, transferability of skills and knowledge.

C. Strategies for assessing learners' progress

• Objective measures: multiple choice, true or false, matching, etc.
• Short answer test item: graphic organiser, concept map, fill in the blanks, etc.
• Subjective measures: essay, projects, presentations,
• Alternative assessments: authentic assessment, performance based assessment and constructivist assessment.

Assessment for DE learners:

• Online tests and quizzes
• Asynchronous communication : thought provoking questions, discussion forum/ debate, questions generated from reading, blogs and wiki.
• Synchronous communication: audio conferencing, online chat, instant messaging
• Portfolios: compilation of work consist of variety of materials leading to development of standards.
• Papers and essay
• Presentations
• Problem-based activities
• Journal writing
• Concept map
• Simulation and games

Implementation Strategies:

• On going and non graded assessment measures
• Balancing flexibility and structure
• Using scoring rubrics
• Facilitating students collaboration
• Selecting appropriate assessment measures

D. Related issues to assessment

• Academic misconduct
• Plagiarism
• Cheating

What is ahead?

• evidence-centered design
• automated essay scoring
• accessibility
  

DE- Major issues and challenges by Najib Kofani and Nowduri Srinivas

The paper classified 3 major practical issues that distance learning system (DLS):

1. Seperation of instructor and learner
2. Use of educational media to carry course content
3. Provision of two-way communication between instructor and learner

Prominent issues outlined in the paper are based on the following questions:

1. What is different about teaching at a distance?
2. What are the reasons for teaching at a distance?
3. What are the different ways to improve planning and organisation of DLS?
4. What are the different ways to meet students' needs?
5. What are the guidelines on teaching skills?
6. What are the different methods for improving interaction and feedback?

Four challenging initiatives and required recommendations:

1. Make the university management support as wide as possible.
   
2. Special attention towards the course that have a rich job market.
3. Take a noble look towards the far- reaching academic courses.
4. Establish an attractive faculty/quality programmes for the university staffs

Recommendations to achieve the above target:

1. Improve the organizational (design), financial (productivity) and the physical structure(architecture) of the DLS.
2. Provide high quality student service.
3. Broaden the faculty reward system.

The paper proposed the following factors for an effective DLS:

1. The quality of course materials.
2. The quality of the instructor.
3. The quality of the students.
4. The quality and frequency of the interaction.
5. The quality of curricula preparations, assignments and solutions.

The paper presented a case study, comparing the five aspects of DLS in three universities.

Table: Comparative study among the several DLS

Nature of Functionality	The Technical University of Milano, Italy	Indiana University USA	Warsaw University of Technology, Poland
1. System Philosophy	Easily adoptable to the students with easy student/teacher interaction	Concern more towards establishing higher education and interaction with the software industry	Concern more towards engineering education with different curricula flexibility
2. System Architecture	Simulate the classroom scenario based on intercampus network	Integrate distributed education into mainstream of teaching with a robust design structure.
3. System Evaluation	Reduces teacher-student interaction as little as possible.	More of faculty driven	Comprehensive and flexible framework
4. Programs/Courses currently offered.	Lessons, seminars and tutoring along with graduates and below	Graduates and under graduates	Engineering education courses
5. Number of faculty and participants involved	Several thousands of students per faculty	Fixed number of students per faculty	Quality conscious with flexibility and adoptability

Reflection on Internet based distance education

My humble opinion of Pat's and Khor's presentation is the content covered are very comprehensive until there were too much information for me to digest. Frankly speaking, in trying to filling in the evaluation rubric, I had a hard time recalling the contents presented. What I remembered most would be ISG is not ajinomoto but interactive study guide instead, since it was repeatedly told by Pat during the presentation. After reviewing the slide presentation posted, then only I was able to unpack the whole idea. Indeed, the content covered was very wide, informative as well as very useful. I am able to know about Course management sytem and tools available to be embedded in CMS. The term ISG is new to me and the presentation made me aware of the types of ISG available and their usefulness to enhance learning.

However, I noticed that the presentation outlined 5 learning objectives only but the the content went on beyond 5 I guessed. Besides, comparing the 3 movies presented at the beginning of the presentation as the induction set were not quite relevant to the whole ideas of internet based education besides time consuming. A better idea would we comparing internet based distance education with educational tv programme or learning coursewares. Infact, presenting the statistics of the growth of internet users in charts could also be an eye opener for the presentation.

Proposal for Project

Last week during class, Iza and I discussed superficially the idea of teaching the upper form students to think using Edward de Bono's Six Thinking Hats. We would use freewebs to create the website and use the blog function on freewebs for instructions and submission of writings. Our main objective is to train the students to think critically and be aware of differences in the way people think. By engaging the students in activities and situations using different thinking hats, educators would be able to enhance the students' thinking skills and writing skills as well as soft skills. We do hope to get some feedback to see whether or not our ideas could be accepted or not.

Pedagogical approaches and instructional strategies

Personally, I think Lim and Chun had done tremendously well both in the preparation and presentation of their topic. I respect their initiatives in the effort of presenting the gist of the content of the topic from a different angle to avoid redundancy of what have been covered by the textbook. The textbook gives an explicit guideline whereas Chun and Lim allowed the audience to share their tacit knowledge through the activity introduced such as debate.

Well, thinking of the debate, I would like to express my dissatisfaction towards the judges. No offence is meant but this is my fair and humble opinion. We were supposed to give our opinion with regard to the advantage and disadvantage of brainstorming in a distance learning environment. During the debate, the class was divided into 3 groups. I belonged to the pro, the other group was con and the middle group was supposed to judge our motion and justifications. What happened was, when both groups had presented and argued, the judges were supposed to tell which group won the battle. Instead, the judges took sides and expressed their own opinions. So the battle went to the con. Weren't the judges supposed to judge based on arguments and not on what they believe? After all it is just an activity..small matter though ;)

Reflection on the class presentation

I will break my reflection in two parts: before and after.

Before:
When Izawany and I paired up for the presentation, we decided to be the first..just to get over and done with so that we could focus on other on coming tasks. While preparing, we felt the tense and started wondering why on earth we volunteered to be the first presenters. The tension built up as the date for the presentation was approaching. Iza came to my house and we discussed the big picture of the presentation before we broke it into smaller ideas. Then, we discussed the flow of our presentation and decided on the delegation of work and roles. We created our own wiki as a platform for discussion.

Our main ideas were based on the graphic below as well as from wikieducator:

To be able to understand distance education learners, we need to do the analysis of the five areas. We need to answer the following questions:
1. What are the general characteristics of the learners (age, language ability, etc)?
2. What are their personal or physical characteristics?
3. What are their entry skills?
4. What are their affective characteristics?
5. What are their instructional preference?

In order to make what we are going to preach real, we decided to let our classmates find out about their own learning styles so that they have a hands on experience by giving them a link to VARK questionnaire. We thought of presenting the scores or the VARK questionnaire taken as our induction set for the presentation but later we switched to a quote instead and use the scores from the VARK questionnaire to explain about learning styles.

I realised, once we have completed the presentation successfully, we would be the at the highest level of the revised Bloom's taxonomy which is 'teach' on top of 'create'. We wanted our coursemates to realise that by teaching, we have learnt 95% of the subject matter.
We held on to Gagne's Nine events of instruction as a guide in preparing our presentation. Nevertheless, we found that the 8th event of instruction was not applicable during the 45 minutes presentation. How our flow of presentation fits into gagne's nine events of instruction is shown in the table below.

Gain Attention	Use quote
Inform the learner of objective	Present the outline of the presentation
Stimulate recall of prior knowledge	Bloom's taxonomy
Present new material	Learners' profile and prompt questions for open discussion.
Provide learning guidance	Prepare slides with answers.
Elicit the performance	Provide activities.
Provide feedback	Provide feedback and possible answers to the activities.
Assess performance	not applicable
Enhance retention and transfer	Recap contents

After:
Based on the constructive comments after the presentation that night, we realised that we ended the presentation too abruptly. What we did for the wrapping up was presenting a list of the next steps after understanding the distance education learners. We should have recap the main ideas we have covered to enhance retention and transfer. We even had forgotten to ask the audience for Q/A session since we were so eager to end the presentation.

Iza and I felt so relieved after the presentation ended. We thank our classmates for their moral support and cooperation throughout the presentation.

Epistemology

On 5th of February, Dr Tee mentioned the word epistemology in class and he talked about it. Since I am quite unfamiliar with the word I decided to look it up and explore its meanings.

Wikipedia defines Epistemology as the theory of knowledge and related to questions like:

• What is knowledge?
• How is knowledge acquired?
• What do people know?
• How do we know what we know.

I found a website that explains epistemology using simple explanation if anyone would like to read more about it.

Agree to Disagree

I personally think it takes wisdom and 'gentleman' demeanor for an individual to come to term and agree to disagree with an open heart. Recollecting the past experiences by putting myself in the situation where I agree to disagree, I saw myself in a role of an obedient wife. There are countless time when I disagree with my spouse's decisions, opinions or actions. Knowing his behaviour too well, I would just agree to disagree and accept the fact that he is the boss to avoid conflicts.

In the context of developing the wikispaces collaboratively, I always keep my mind, heart and options open. I accept differences in people's opinions, thoughts, creativity and beliefs. There is no one correct or best way of developing it. However, I do hope that the task could be accomplished on time.

While digging further on the topic, I stumble on a website (the link is given below) while searching for ideas to write about the topic.
Here are the 5 steps involved in how to agree to disagree.
How to agree to disagree

Step1

Be a good sport. In an argument or debate, keep a friendly mind-set. Though some issues may be incredibly important to you, like your religious beliefs, understand that everyone has the right to their own opinion. If everyone believed the same thing life would get boring.

Step2

Stay calm in the face of a debate. Agree to disagree with someone who is able to intelligently converse with you about tough subjects. Keep your cool and simply swap ideas with this person instead of letting it get ugly.

Step3

Use honesty above all. People will tend to throw up their hands and plead that there is no winning or losing the argument when they have clearly lost. This tends to happen to those who have not fully researched or thought out their opinion on the subject. If you find yourself in a situation like this, simply be honest with the other person and let them know you have to do some research before continuing.

Step4

Respect the other person. In a debate, friendly or otherwise, if the person states that you should just agree to disagree, accept it—even if you think they are saying so to avoid admitting defeat.

Step5

Understand that you can't change a mind that does not want to change. Some people will be open to new ways of thinking, and others are steadfast in their beliefs and cling to them as though their lives depended on it. Take this fact to heart, and it will help you be a better debater.

Intersubjectivity

The title is a difficult concept to grasp indeed. When I read my note on intersubjectivity, the definition given in class was 'context where learning from teaching occurs resulting in common understanding of something subjective.' Then , I read again and again while trying to figure out the examples in context of my own learning. Frustrated... I left the concept unattended until Dr Tee brought the title again in class last week on the 5th Feb. He knew that the concept did not transpired in our cognitive domain since nobody wrote about it in our reflection. I couldn't agree more...

So in this entry, I decided to explore the topic of intersubjectivity. Wikipaedia explains it as 'the sharing of subjective states by two or more individuals which is used in 3 ways: when it refers to agreement between people; when it refers to common sense shared by people; when it refers to share divergences of meaning'. Hmm... complicated?

I kept on browsing, then I found this definition of intersubjectivity under social constructivism: 'Intersubjectivity is a shared understanding among individuals whose interaction is based on common interests and assumptions that form the ground for their communication (Rogoff, 1990)'. The article further explains 'Intersubjectivity not only provides the grounds for communication but also supports people to extend their understanding of new information and activities among the group members (Rogoff, 1990; Vygotsky, 1987)'. My understanding of intersubjectivity becomes clearer after I read this explanation: 'The construction of knowledge is also influenced by the intersubjectivity formed by cultural and historical factors of the community (Gredler, 1997; Prawat & Floden, 1994). When the members of the community are aware of their intersubjective meanings, it is easier for them to understand new information and activities that arise in the community'.

Let me put it in a situation based on real life context. For example, when I meet teachers from other school, we share common grounds. We could talk about students, education system, pedagogical problems and so forth since we have shared understanding of the matter discussed. On the other hand, if I were to meet a lawyer, singer, politician and the like, I could not share my thoughts, opinions or feelings toward the education system as a whole since there is no common grounds or understanding but if I talk to them (as a parent) about the issue on parenting, we might be able to share our worries and concerns about the our children and this involve intersubjectivity.

Understanding Distance Education Learners

Tonight, Iza and I are going to present the above topic. As outlined in the course syllabus, the aim for having us presenting is to let us deeply engaged with the content. I think Dr Tee has done a good job in designing the instructions which allow the learners to go through the process of Bloom's taxonomy and Anderson & Krathwohl's below.


I found that we will actually apply the highest stage of both taxonomies which is 'teach' since that is what we are going to do tonight. To be able to teach a content, we need to master and digest the different angles available with regard to the subject matter. After all we tend to remember 95% of what we teach to others. I do hope what we have planned for tonight will become a fruitful discussion.

Frankly speaking, Iza and I are a little bit skeptical about the presentation tonight. We are afraid if our delivery is not up to everyone's expectation.