Evaluation+and+Feedback

http://www.flickr.com/photos/criterion/3417811375/

**EVALUATION AND FEEDBACK (2009 Program)**

 * Please add your thoughts below about various aspects of the LTM program.**


 * MENTOREES **


 * Number of Mentorees** - //(was having 4 - 6 mentorees realistic/manageable? what number would you suggest? more? less?)//

I have found that having 3 or 4 mentees is manageable if together in one work group (Nadine)

I found 6 mentorees manageable - in the end I have had 7 mentorees and two Innovations Projects to manage - however we have met once a fortnight rather than weekly and this has generally worked well (Julie P)

I found having 6 mentees would have been manageable had they all been able to attend regularly-we had weekly video-conferencing sessions initially and will swap to centra for term 2. (Deb H)

I agree that 6 mentees would have been manageable if there was not so much chasing involved. Probably 4 is a better number, especially if they are all working on different projects and group meetings are just not practical or useful. (Karen M)

I found it difficult to keep 6 mentees for the entire project trying to get them together was almost imposible as thy were scattered all over the countryside (I messed up badly there should have kept them in my own workteam) if I had 4 would have been a better number. (Alan B)

I also found working with 6 mentorees manageable as many already had high levels of moodle expertise and required minimal contact allowing extra time for those mentorees requiring more input and a partnership of learning (Deborah).

I found that I just could not get to see my Mentorees enough to be able to do all that much. I did manage to get to most in some form or another but not what we had hoped for. (Ric C)

While I had eight mentorees over the life of the project, over half did very little or nothing worth talking about despite having enthusiasm and a desire to be involved initially. The easiest were those from my own work group (although in different sections) perhaps because I was able to be "in their face" almost daily. Therefore, I would say six is okay but having a strong rapour built up before you actually start the mentoring is the critical issue. (Tony A.)

I guess its like anything & depends on the situation... if 6 mentees were in one group,in one location a small class is quite manageable. Arranging a suitable time for all to attend then becomes an issue Difficulties arise when single mentees at other campuses & involves traveling & session content is being repeated over.(Jim B)


 * Mentorees from same cluster** //(do you think was a good idea? would you rather just choose mentorees from your own work team?)//

Definitely this is my preferred option for time saving and efficiency (Nadine)

All my mentorees were from my program area or location and several were at the same point in their understanding of e-learning so I was able to cluster mentoring to meet these needs (Julie P)

It should have been easy with all mentees from the same cluster however this wasn't the case. The time allocation of the workgroup was difficult to fit into as many of the staff were part time and in the classroom for all of that time. I think in the end it would have been better to have expressions of interest with information about who would commit time and energy to the project and who had capacity. I think having mentees from my own work team would have been easier as we have the same understanding of our time and committments and have a very supportive manager who is prepared to give time to this project (Deb H)

All of my mentees were from my workgroup, although across different campuses. I ended up approaching potential mentees as managers were on leave and unable to identify who should be involved. I deliberately chose people who had not previously had much PD in e-learning and this made for quite labour-intensive work. (Karen M)

As above keep it in the workteam with the utter unreliability of the TAFE network distance just does not work.(Alan B)

I have experienced mixed results from both within and outside of the workgroup, so for me this wasn't a critical factor. I believe that motivation, allocated time and a willingness to collaborate are all contributing factors (Deborah).

I believe that the secret is to have our mentees programmed to be mentored, that way it would solve most problems. It made no difference whether they were from the same workgroup or not. We required more planning. (Ric C)

To save time and the fact that you are seen all the time, even if you are not working with them at that point will prompt a better outcome overall (Tony A.)


 * Fact that mentors had 'paid' time but mentorees didn’t** //(this was flagged by many of you as an inherent problem. Can you suggest ways that this problem might be addressed?)//

So far no one has sadi anything to me about money. Quite frankly I don't think that its an issue if the learning is happening. (Nadine) I haven't found this to be an issue - all my mentorees volunteered to be mentored and I was upfront about their involvment (Julie P)

This is an issue for me as finding back-fill is not an easy task! I think with mentees part of the initial discussion was about blocking out time. (Deb H )

Not sure if there is a solution Managers are not going to release even small numbers of people to do E-Learning with the current financial preasures being placed on us all. In our workteam mentoring goes on all day every day in all sorts of areas so we are allways learning! (Alan B)

Finding time to get moving on these projects was difficult, especially in Term 4 with the pressure of marking and resulting. Many people seemed to be involved in curriculum rewrites and the e-learning got put on the back burner. Providing time out of class, with backfill, would have been of some help - mainly as an indication of support for the mentees and the extra work that they were doing. (Karen M)

I think accountability plays an important part in some mentoring cases, but not all. If reporting is required in a set time frame, then results tend to happen. For the highly motivated mentorees, who were keen to trial the resources with students, much time was spent outside of their allocated hours because of genuine interest (Deborah).

My mentees who stayed the course contributed much of their own time. those who did not stay did not have the time. It was a problem for them, I expect they would have stayed on if the time had been made available. (Andrew K)

I don't think "Paid" had any influence at all, other than us getting the time to do things. If the mentees had volunteered for this, knowing what they were in for, they would have made the time themselves. (Ric C)

Some of my mentorees already had some of the skills they needed they just did not have the time to create things like Moodle pages, etc. They were looking for someone to 'do' the work and not really be mentored too much.(Alex W)

Like some of the other comments here, I don't think being "paid" was a problem. In fact on several occasions, I was arranging things or learning new stuff in my own time too so I could help them which they knew about so it was never raised as an issue. (Tony A.)

I had nine mentees from my own work group, which was OK because they had varying support needs - some very little - otheres quite a lot. My workgroup are all HPIs and they were paid to participate which is appropriate. They have all delivered sound work, some of it is quite inspired. It was genuinely a privilege to work with them. (Liz P)

**EXPECTATIONS**


 * Goals – realistic?** //(is it possible to get 4-6 lecturers doing elearning with students in the given timeframe?)//

The time-frames are in interesting really. I think that a project is somewhat timeless and ongoing with editing and changes happening all the time. I will work with my mentees for as long as it takes to get them where they need and want to go but with solid direction and planning. (Nadine)

I think this depends on what you mean by 'doing elearning with students' - all my mentorees have improved the way they are using elearning technologies over the period but they all still have learning opportunities which is on-going (Julie P)

I think this is unrealistic especially if they are beginning with e-learning. I felt it would have been more realistic if we had staff with a)projects already started or b) ideas that needed to be fleshed out with support. I think in terms of what we have been able to achieve (given that we only started this term) has been useful but not at a stage where it is measureable in outcomes (DebH)

I think the goals were realistic but the timimg of the LTM project wasn't. Terms 2 and 3 would be more productive as the momentum could be maintained and students more accessible for trials (Deborah).

I think the goals were realistic but as "Julie P" says, it depends on what is meant by doing elearning with students". All my mentees and I have had huge teaching programs this semester and elearning gets the bums rush when it's like this. (Ric C)

HI all Im back on deck after a stint in hospital stil a bit shaky and wobbly but ready to put in, I can only speak for my own workteam I think the goals were a little unrealistic given the time of year the project was run and i agree with Ric's comments re the meaning of doing e-learning with students I was in a more fortunate position in that we have been using e-learning with our students almost exclusivly for nearly 3 years so I had a good base to build on. (Alan B)

It was OK for us. Running an online class depends on having a minimum number students so some subjects haven't been run yet, but those that did went remarkably well. (Liz P)


 * What would be realistic goals? **

Realistic goals are to plan for the outcomes I think. My mentees want to create websites for thier own courses and provide support for thier students so that the Moodle doesn't become the main attraction on the course but more supportive of the course with links to relevant materials and website, writing guides, academic rules such as referencing. Creating a basic support site for students is realistic in the time-frame (Nadine).

I think that the focus should be on improving individual skills in using elearning - this is not an area where one can become an expert in 4 - 6 months (Julie P)

I agree with Julie's comments here-6 months would mean more time to get something off the ground and begin to use it, however perhaps a concurrent group of improving general skills in using e-learning-becoming familiar with moodle, centra, movi and similar technology and then beginning to run their own groups from here. The more confident group would have a project in mind or already be working with one that they want to enhance. (Deb H)

The three mentees who have real e-learning happening in their classrooms at the moment were doing nothing like this at this time last year. I feel fairly confident that they will continue, partly because I work with them and will be badgering them in the nicest possible way, but mainly because they have been pleasantly surprised at the positive response from the students, and the ease with which thay have taken to the technology. Probably need to check in at the end of the year to see if people have continued or, better still, have added additional e-learning tools to their repertoire. (Karen M)

It is also worth measuring the soft outcomes such as increased confidence of both mentors and mentorees in new areas of e-learning and the ongoing partnerships to continuously improve course design, delivery and accessibility (Deborah).

I think that "Realistic" will be different for each of us, it will depend on the mentees, hours, previous knowledge, dedhat can run tication, equipment and so on. (Ric C)

One of the most usefull areas we could build on is macking the use of our systems more easily accessible currently much of the great work people do is not used because its 'just too hrd to access',the network does not work, the network is to slow, we dont have computers that can run the programs, or most sad of all and the most frequently heard WE DONT HAVE ANY COMPUTERS!!!!!!!! (Alan B)

**COMMUNITY OF PRACTICE (CoP)**


 * CoP expectation – realistic?** //(what do you think of this idea? did you feel like you were part of a CoP? do you think you contributed to enough to 'our community'?)//

I don't feel I have been able to contribute very much to the CoP - I have met with several other members within my Institute at various activities and have been able to learn/discuss issues with them. I think location and time available has been a restriction but I also believe that working across Institutes is very valuable but how to do this is a challenge for the organisation (Julie P)

I like the idea of a CoP but felt I did not contribute much to it-being a regional member makes it difficult to attend anything face to face-eg the celebrations etc. Time has been a big factor here and again back-filling to enable me to join an issue (Deb H)

I managed to get to all but one of the Centra meetings and I do feel part of a CoP. I have made contacts with people and have found out about their areas of expertise, and feel that I could go to them with questions if I needed to. I am also on the E-Learning Practitioners Group for TAN and feel that this group could have a role to play in setting up a strong CoP, in the North at least. (Karen M)

I have found the CoP invaluable as a support network and have been involved in as much communication via Centra, the wikispace and google groups, as possible, as I have so much to learn. I have also extended my participation via the TASI IT Users group and Moodle Users group as I feel that the current e-learning focus needs to be further developed to continue to provide up-to -date and accessible learning opportunities (Deborah).

I've not felt like I was part of anything much as I've not had the time to be involved with the Centra Meetings or get on here much. Today I was asked to help a lecturer in Port Adelaide with Janison stuff, then I felt part of it all, they really wanted to learn and be involved in elearning. (Ric C)

Unfortunatly I feel much the same as Ric and this has been made worse by the abandonment by TAFE SA of the janison platform just when the federal government through Queenstand tafe has just launched a national Janison based mech eng ste no where near as advanced as the SA site. (Alan B)

Unfortunately I have been away so much and things have been really busy that I have not been able to attend the Centra sessions or see many of the people in the program. I agree that it is a great idea but it just did not work out too well for me this time.(Alex W)


 * Idea of an ongoing CoP?** //(It is the intention that you remain a member of a growing CoP that will include the next round of LTM mentors, and that you continue as an 'elearning resource' for your workteam and others who need support. How do you feel about this?)//

I'm very keen to continue to be an 'elearning resource' for my workteam - providing the funding for me to do this is the challenge as this will have to be informal and with no time allocated to this function (Julie P)

I like the idea of continuing with this especially within my own work team as I feel I can realistically contribute to this in a useful, targeted and meaningful way. In terms of as a member of the whole LTM group yes I would be happy to continue in a limited capacity. (Deb H)

I am keen to continue as an e-learning resource person for my team and others. Finding a way to be available to people will be a challenge with a full lecturing load and limited opportunity for backfill (I would rather not leave my classes anyway). (Karen M)

Definitely! I am keen to be involved as it really has been a learning partnership and provides the impetus to investigate new e-learning resources (Deborah).

Try to stop me, now that we have started I can not have it all just stop. I need to get all the lecturers involved, starting with all the "Subject Masters". We have long term plans for elearning here, this is just the start. (Ric C)

Currently the MET groups moral is at rock bottom and we are looking for alternatives that do NOT involve a network based system and we certainly do not feel like we are part of any ongoing CoP, (Alan B)

I like the idea of CoP, but was only able to participate minimally. I was paid an extra day a week for several months to do our project (for which I am grateful) but combined with the rest of the workload, I still found it was a bit too much in 3 days a week. The google group emails started to get to me because i felt I wasn't really able to give it as much attention as it deserved. But overall I still have strong enthusiasm for mentoring and elearning. (Liz P)


 * REPORTING AND COMMUNICATION**
 * Weekly reporting and sharing to the whole group** //(not many did this effectively - why not? did you not find this useful? is this a reasonable expectation?//

I found the Wikispace to be very useful way for me to record what I did and what needed to be done next with my mentorees - I used this as a reminder of what we discussed - so as I met with my mentorees on a fortnightly basis this has been updated fortnightly (Julie P)

I began this with good intentions but the reality is once I had set up a wikispace for my mentees I did little in the other one! I liked the use of it as a practice site and have continued with this for my mentees. (Deb H)

I did like the routine of reporting in through wikispaces and I am hopeful that it will be a valuable resource when I come to prepare my case studies. (Karen M)

I kept a regular update on my wikispace of all mentoring sessions as an informal record of progress and used the area as an experimental zone for powerpoints and photostories. It provided a timeframe for reporting that was achievable and kept us accountable. Admittedly, much of the reporting couldn't be completed during work hours but this was by choice to have an uninterrupted allocation of time to explore e-learning options (Deborah).

I did very little, but it is a good idea. Maybe next time. (Ric C)

Same as Ric (Alan B)

Like Deb, I had good intentions, but in terms of overall work priorities, it had to slide to the lower end of the list (Liz P)


 * Use of wikispaces** //(how did you find it? did it meet your needs?)//

As above (Julie P)

I found wikispace useful for my mentees (Deb H)

I had not used wikispaces before this project but I have liked the way that it works and one of my mentees is using a wiki with her ESL students. I have begun a wikispace of my own for mentees but have not really developed it enough yet. I will look at it again though after all of this reporting is finished. (Karen M)

This was my first experience with a wikispace and I found it a useful repository for trial materials or useful downloads (Deborah).

Same as above (Ric C)

Maybee its just me but I found it difficult to use and a pain so I didn't (Alan B)

It is a useful tool. (Liz P)

Busy people we all are! I can spend a few moments telling people very little each week or I can spend a few more minutes sharing some good reports of mentee advancements and progress (Nadine)


 * Use of the Google group email list** //(how did you find it? did it meet your needs? did you find it useful to have a means of communication that went to the whole group?)//

This is where I had a challenge - mainly in keeping up with too many spaces to keep checking - thank goodness for the emails through to my usual email space or I probably wouldn't have looked at this at all. (Julie P)

This seemed to work quite well and I received all the updates from others-I did link this to my igoogle page so that I had it open each day and that made a difference to my checking of it, even though emails came through normal channels. (Deb H)

I did not use it much but I followed other peoples messages and responded where appropriate. (Karen M)

This was useful to keep up to speed with other projects and Centra sessions and to seek specific informatiion for my mentorees. I think many of these requests will be answered once the Moodle users group in TASI is fully underway ( Deborah).

Did we use it????? I got nothing on mine (Ric C)

I found this far more usefull than the wiki and was able to contribute having email notifications kept me in cintact and focused, (Alan B)

Like Julie P I had a challenge here. It did my head in a bit because the emails were coming to my private email (which I nominated -so it's my fault) and this was always out of work time. I more or less have to do a lot of extra time to get my core work done and this was OTT for me. (Liz P)

I don't have time sorry to keep going onto blogs (Nadine)


 * Case Study Idea** //(any thoughts?)//

I have two Case Studies which I will be contributing - these show how individual people have improved the way they use elearning. Are they exemplar level, probably not but for me this has not been the point. Building the skills of individuals has been my focus, starting from where they were (Julie P)

I am still trying to work through these as we did not start the project until this term. I think it could be useful, but again with such disparate geography and needs, I will need more time to get this complete. Most of my mentees have either pulled out due to time constraints or are part-time and in class as mentioned. Making a case study with very few photos and no face to face contact is not my preferred option-exemplar level? Not at this point, however as Julie mentions above my focus is building on their current skills and knowledge of the use of technologies. (Deb H)

I will have 2 or 3 case studies that will look at the mentoring process as well as where lecturers are at now and how they are using e-learning with their classes. (Karen M)

I will focus on developing 2 case studies- one about Second Life and the other Moodle. Quite interesting stories to tell. They will be photostories for this reporting purpose but Nadine and I will be continuing to work together on a mini-movie of the Second Life experience (Deborah).

I'm finding it hard to get the time, but I will have a couple ready in time, (I hope). (Ric C)

I will do the best I can I will do only one encompassing the whole group It might be a bit late though we will see (Alan B)

I am working on getting a couple of case studies done, with other work commitments it can be hard to achieve this promptly. (Alex W)

Late to the party of course, but I have one up now. I should have got more feedback from the team for the purposes. I have good feedback from them regarding my support etc but didn't want to add it all to the public arena. (Liz P)


 * Centra meetings** //(did you attend them? find them useful? should we have had more of them?)//

I was never able to attend these as they were scheduled when I had other commitments - I don't work at set times week after week as I work on many different projects at the same time so each week is different for me. This means that I do have to 'bump' priorities all the time and unfortunately this was the one that did get bumped. The one time I was available Centra was not activated on my computer and it took a week to get this going again. Very frustrating. (Julie P)

I attended these as much as I could-sometimes the timing did not work for me but otherwise they worked quite well. I would probably prefer to have some further aspect of e-learning as a part of them rather than just a catch up with the group-overall they worked well. (Deb H)

I attended most of these and found them useful for finding out how other mentors were going. I also used it for my own PD on how to use Centra with classes. (Karen M)

I attended most of the Centra sessions and found the catch-up times useful. I hadn't worked in Centra before, only videoconferencing and appreciated the workshops (Deborah).

I personally like Centra but I could only get to one which was dissappointing, I was hoping to learn much more about it for myself to use and pass on to the lecturers. I want to use it quite a lot in the online teaching of students. (Ric C)

Im with Ric on centra I think it has great potential and i will need to use it soon to run CAD classes in Berri, Barossa and Murray Bridge (Alan B)

I was unable to participate when they were scheduled. (Liz P)

I think I will contribute at least one case study but it is very early days yet (Nadine)


 * FACILITATION ** //(what did you think of the facilitation of this project? did you feel supported? were there things you required of the facilitator that didn't happen?)//

Michael, you did a great job answering questions and providing advice. For me where I found challenges was with the reporting of outcomes. This needed to be clearer at the beginning and there has been a difference of interpretation between the Institutes (how unusual!) (Julie P)

I felt I could ask the facilitator anything without feeling too much of an IT-o-phobe! You were available and provided the necessary support with any issues Michael-thanks. I began this project with no idea about its requirements and found it quite daunting and overwhelming at the beginning. I feel that there needs to be time and thought given to who the mentees will be and what is required of them prior to attending your sessions-perhaps even a meeting with their manager to discuss and spell out the needs of the program. I do not feel I had that at all and therefore felt ill-prepared. I do not believe I will be able to meet the required outcomes but will continue to "give it my best shot". (Deb H)

I don't think that there were too many occasions where I needed to call on Michael for direct support because I often found the answers to my questions through the communications in Google groups and the Wiki - which Michael set up and facilitated anyway. The Centra meetings also kept me connected to the rest of the group as well as drinks at the end of last year. (Karen M)

Michael, I really appreciate your support and patience throughout the LTM project. You have given rapid responses to any queries and have kept us on track throughout (Deborah).

I feel you did a great job Michael, just by the feedback from the others that I did see. If the goal for you was to get the ball rolling in elearning at TAN, then it sure is gathering momentum, thanks. (Ric C)

Michael you did the best you could given the area you had to cover. I would have liked more local involvement from people at your level it would have helped keep the project on track, (Alan B)

I think you did a great job Michael. You helped me out by being one of the "outside experts" for our PD. I wish we could clone you! Sometimes, you answer my questions in my head... ooops! Giving away too much information about my crazy imagination :) (Liz P)

Michael, I think that the support I have received has been up and down so I have always found answers outside the group. Like I have said, we are all busy people (Nadine)


 * OTHER COMMENTS ON ANY ASPECT OF THE PROGRAM **

I have found this program to be very rewarding - both personally and for my mentorees. Many spoke at our recent planning meeting and commended the way this mentoring program has improved their skills in elearning and would like to see this methodology used to support other professional development needs. So thank you Michael for this program. (Julie P)

I have learnt a lot Michael and have enjoyed learning about methodologies I had little idea about before. I would hope that my mentees feel like that at the end as well, however they will need to continue into term 2 for this to be possible (Deb H)

At this stage of the project I can really see the changes that have happened in my mentees' teaching methodologies and the increase in their confidence in using these tools. I have to admit that there were some dark patches, especially in Term 4, where I felt that I was making almost no progress. Ultimately, it has been a great program to be part of, fantastic PD for me as well as benefits for my mentees and wider team. (Karen M).

This program has been an incredible learning journey although initially daunting because of the high level of IT skills of both other Mentors and mentees. Looking back, I have now dabbled in at least 5 new areas of e-learning so the next step is to consolidate these skills and to continue to support the mentorees both within and associated with my workgroup, with blended delivery. (Deborah).

This has been a great chance for me to push the elearning barrow here. As an elearning mentor I have been able to get more support from the managers. I think they now see a bit more of a use for elearning in the future and are starting to push for it. Best of all, I now have some other lecturers that can see the benefits, I'm not alone any more. (Ric C)

There is no doubt in my mind that E-learning is the way ahead however the proponents of the medium i feel need to focus on the learning more and a little less on the technology no sooner has somthing been developed than therr is some new way to present it. My experiance with my student cohort is they do not respond well to constant change and will abandon the e-learning medium for old fashioned books when pushed. (Alan B)

I believe this process is going to be the most effective way to develop elearning (or implement any change processes for that matter). Mentoring is warm and human and connected in a world that is becoming more and more disconnected and so on. We're all time poor, and it does take time but it is more important then ever to work together. (Liz P)

Well here I am in November and I have been so loving working with my colleagues and watching them just flourish with the Moodle sites. Most of my mentees have run with the concept and produced some wonderfully simple and easy to work with sites where their students can access support materials and participate in group discussions and quizzes. They all giggle at the prospect of putting up a photo of themselves but I am working on them. Photos add such colour to an otherwise dull background.