Journal of Instructional Science and Technology ISSN: 1324-0781 Editors-in-Chief: Olugbemiro JEGEDE (jegede@ouhk.edu.hk) and Som NAIDU (s.naidu@meu.unimelb.edu.au)

---

Volume 1 No 4, November 1996
- - - Article 2 - - -

An Evaluation of the Queensland Open Learning Network Audiographic Conferencing Professional Development Programs

by

Olugbemiro J JEGEDE
The University of Southern Queensland, Toowoomba, Australia

Anne GOOLEY & Stephen TOWERS
Queensland Open Learning Network, Brisbane, Australia

[ Abstract | Introduction | Methodology | Results and Discussion | Summary and Conclusion | References ]

Abstract

The literature indicated that although there was evidence that audiographics conferencing was used for education and training purposes, relatively little information on professional development programs was available. The objectives of this study were to (1) evaluate participants' perceived value of the audiographics professional development program, (2) identify areas for revision and improvement, and (3) collect and disseminate information about professional development programs for audiographics conferencing of the Queensland Open Learning nertwork (QOLN). A questionnaire was administered to participants of an audiographic professional development program to gather data for a comprehensive evaluation of the training program. The data was quantitatively and qualitatively analysed. The results of the evaluation indicated, amongst others, that (i) the majority of the participants found the training workshop very beneficial, easier than they had expected, and that computing skills had no relationship with their mastery of the use of audiographic conferencing equipment; (ii) they were pleased with the interactivity characteristic of the audiographic equipment, and the learning environment requirements for the use of audiographic conferencing; and (iii) participants considered audiographic conferencing. Based on the results, a number of recommendations were made relating to policy on training and professional development opportunities for teachers using audiographics conferencing, encouraging teaching staff and instructors to use audiographics conferencing, and the need to disabuse the minds of people of the myth that high computer literacy is very essential for successful audiographic conferencing.

Introduction

In 1993 the Queensland Open Learning Network installed audiographic conferencing facilities in each of its 40 Open Learning Centres (OLC's) to provide a human interface and technology infrastructure for the delivery and reception of learning programs throughout Queensland. Over this period, demand has emerged for personnel to be trained to use audiographic conferencing education and training activities. Apart from familiarisation of the technical aspects of audiographics conferencing there was a lack of professional development programs to train teachers, lecturers and other facilitators how to use the technology appropriately. The aims of the professional development program were to:

1. design, develop, deliver, and evaluate an audiographic-teleconferencing skills training program for a target group of lecturers.
2. pilot and compare open learning and face-to-face delivery methods for the audiographic-teleconferencing skills training program with lecturers from two universities.
3. Refine and produce master copies of training materials.
4. Distribute details of the programs to higher education institutions providing distance education and open learning throughout Australia.

Computer mediated communications have become common in professional development within the past decade especially as their use in distance education and open learning became very pervasive. Such applications have included: continuing education for Francophone and Anglophone addictions workers across Canada ( Burge, 1993); the development of computer conferencing to support a face to face conference in Bangkok (Anderson & Mason, 1993); the Satellite On-line Searching Interactive Conferencing Experiment at the University of Plymouth, United Kingdom (Hughes & Priestley, 1992); for access in remote areas (Robson, 1991); in teacher education (Westerman, 1990); a research-based communication model of professional development (Wilsman, 1988); and, online professional development in instructional design (de Vries, Naidu, Jegede & Collis, 1995). A summary of the existing information on the use of computer mediated communications in professional development indicated that:

• CMC are effective in ensuring equal access to group provision
• CMC benefits those who are disadvantaged by distance and social/economic factors
• CMC help both new and experienced educators develop as reflective practitioners
• CMC are cost effective, and involve real time linking of local groups of professionals with their peers/colleagues and with external expert colleagues
• CMC fostered positive response to the course and the technology as an effective way to learn material otherwise unavailable to them

However, a strong indication was found in the literature (see Johnston, 1992) to have more information and training on CMC. If this could be demanded of the less difficult types of CMC (computer and audioconferencing), the need for training regarding the audiographic conferencing could be well imagined. Audiographics conferencing is the transmission of images and text between computers. Audiographics is a software package that runs on a personal computer and usually uses two standard telephone lines - one for audio and one for data. It is used in conjunction with audio teleconferencing so that participants may hear each other and also view the same information, diagrams or pictures on the computer monitors. While many sites can be linked together, 3-5 sites are common. Audiographics conferencing is often described under the rubric of 'teleconferencing' or sometimes the term 'computer mediated communication'. Audiographic facilities enhance audio conferencing by providing the means to display and share visual images. Audiographic conferencing provides lecturers/teachers and learners with the means to interactively share graphic and auditory information across multiple locations. The shared workspace enables participants from each site to contribute visual as well as audio information. This type of conferencing offers an alternative for the design, development and delivery of education and training programs to remote and decentralised regions of Australia. Programs can be quickly and simply prepared and delivered to multiple locations simultaneously. A geographically distributed target group can be aggregated to increase the cost effectiveness of information dissemination and training cost effective. An important feature of the system is that education and training activities may originate from any location which enables and empowers small and large communities to deliver as well as receive courses.

Audiographics can replace face to face instruction or print-based material as the primary delivery strategy or can be used within a wide range of flexible delivery options to supplement or enhance learning programs (Gooley & Towers, 1996). A typical example is that courses can be delivered to a large audience via satellite broadcast (one-way video, two way audio) followed by a series of small group audiographic conferencing. Much the same way that lectures providing mass information are followed by tutorials to encourage deeper learning and peer interaction.

Although there is evidence that audiographic conferencing has been used in a number of projects for delivery of instruction (Clark, 1989; Murphy, 1988; Ehrmann, 1990; Gilcher & Johnstone, 1988; Martin, 1986), there was nothing to indicate deliberate effort to map out and evaluate its use for training. Especially those who would use the technology on a regular basis to achieve distance education and open learning related objectives. Indeed, Blinn (1989) recommended the evaluation of this technology especially where long term training requirements are conducted on a frequent basis to multiple remote sites. This underlies the objectives behind this project undertaken on the QOLN Audiographic conferencing professional development program.

The objectives of this evaluation were to: evaluate participants' perceived value of the audiographics professional development program; identify areas for revision and improvement; and collect and disseminate information about professional development programs for audiographics conferencing. This evaluation significantly departs from the traditional approach by focussing on the training required to use such technologies for educational applications.

Methodology

Sample

The sample included lecturers/teachers from tertiary education providers who participated in a three day training workshop on audiographics conferencing. All participants of the workshop were invited on a voluntary basis to complete the questionnaire. A total of 30 questionnaires were distributed. 26 questionnaires were returned.

Instrumentation

A questionnaire, The Audiographic Conference Training Program Evaluation Questionnaire (ACTPEQ) was designed for participants undertaking the training program to gather data for a comprehensive evaluation of their training program. The questionnaire comprised three major sections. Section A was for demographic information which included items seeking pertinent information about the participants, that is; keyboarding skills, computing knowledge, status, and course level they currently teach at. Section B contained a total of 55 items in eight factors and sought responses on areas such as expectations of audiographic conferencing prior to, during and after the training program; interactivity; difficulty; learning environment; and education/cognitive values. Responses to items in this section were based on a six point Likert types scale of “Strongly Agree” = 5, “Agree” = 4, Neutral/Not Sure” =3, “Disagree” =2, “Strongly Disagree” =1, and “Does Not Apply” = 0. Section C was in a free response format allowing for comments on: easy and difficult features associated with audiographic conferencing for education and training purposes; outcomes of the workshop; and suggestions for improving future workshops.

The ACTPEQ was developed and refined over two years and underwent several stages of validation and trials. Sections of the questionnaire had been trialed in earlier studies (Jegede & Gooley, 1994; Gooley &, Jegede 1995) and the several drafts of the instrument were reviewed by selected academics from two independent universities, and experts in distance education and open learning, computer mediated communications, social sciences research methodology, and biometrics. The results of the validation and reliability analyses indicated that the instrument exhibited a test-retest and split-half reliability coefficients of 0.92 and 0.94 respectively. The alpha reliability for the items in Section B ranged from 0.88 to 0.97. The major factors within the final instrument correlated significantly well (p< .05) with coefficients ranging from 0.22 to 0.53.

Administration of the Instrument

The ACTPEQ was administered to participants who volunteered to evaluate their training, at the commencement of the training workshop. They were advised to complete it and to return it either to the facilitator or the address provided (as discretion of the respondent) once the workshop was completed. The 26 completed copies of the questionnaires returned were collated ready for data analysis.

Analysis of data

The data collected were quantitatively and qualitatively analysed using the SPSS for Windows and the NU.DIST software respectively. For the quantitative analysis, descriptive statistics, correlation coefficients, paired t-tests, and analysis of variance (ANOVA) were employed. The free response comments on aspects of the training workshop related to: easy features of audiographics; difficult features of audiographics, what was gained from the workshop; and, suggestions for improving future workshops. These were subjected to detailed exploration using the NU.DIST software in order to tease out salient factors and determine any trends in the reactions of the participants to the resources, content and organisation of the workshop.

The 26 respondents comprised 16 males and 9 females. The majority of them 11 (44%) fell within the 36-45 years age bracket, 8 (32%) rated their keyboard skills as good or excellent, 12 (48%) of them rated their computer knowledge as low (see Figures 1 and 2 respectively).

Figure 1 Level of keyboard skills	Figure 2 Computer literacy skills

56% of the respondents had less than 5 years of teaching experience by face to face mode and 76% of them had less than 5 years teaching experience at the distance education/open learning mode. While 52% of them self nominated for the training, and 48% of them were nominated by their supervisors.

Results and Discussion

The following section presents the results of the quantitative and qualitative analysis.

Quantitative analyses

Table 1: Frequency compilation of the responses to items on Section B of the ACTPEQ

The results of the frequency analysis of the responses of the workshop training participants to the items on Section B of the ACTPEQ are as shown in Table 1. A critical examination of the results indicated amongst others, that the participants returned a mean score of 4.0 and above on 17 items (see Table 1, items 10, 11, 14, 15, 16, 17, 29, 30, 34, 36, 37, 41, 42, 48, 49, 50, 55). A composite summary of these positive responses from the participants of the training workshop was that they are particularly pleased with three aspects of the training: what they experienced during the training program; the interactivity characteristic of the audiographic equipment; and the learning environment requirements for the use of audiographic conferencing. Although they felt confident to use audiographics (item 23), they felt they have also acquired the skills to prepare audiographic lessons (item 21). The positive responses given for all the sub-items on item 20, which focused mainly on the training session, indicated in the main that they would have liked the training to be prolonged to enable them to acquire more skills and become more familiar with the technology (see Table 1 items 20i to 20vi).

A further extension of the analysis was to examine what differences, if any, had occurred in their expectations prior to and immediately after the training. In effect, the items on two sub-scales (expectations of audiographics conferencing prior to training, and expectations of audiographics conferencing prior to training immediately after training) were compared. The comparison was undertaken by examining their graphic representations followed by a comparison of the differences in their mean scores using the t-test statistics. The results are as shown on Table 2 below.

Table 2: Comparison of audiographic conferencing trainees expectations prior to and immediately after training.

Before The Training	Immediately After The Training
t-value = .51, p = .61
t-value = .97, p = .34
t-value = 3.00, p = .01
t-value = -.24, p = .81

Before The Training	After The Training
t-value = -1.31, p = .20
t-value = 3.08, p = .01
t-value = .62, p = .54

The results of the paired t-test comparisons indicated that significant differences (p< .05) were found in their expectations for only two items. They are the items dealing with confidence in the use of audiographic conferencing and the difficulty in mastering audiographic conferencing equipment. While their confidence in the use of audiographics dropped after training (t-value = 3.00, P=.006), their expectation that the audiographics will be difficult to learn, significantly dropped after training (t-value = 3.08, p= .005).

As mentioned earlier, a number of independent variables were included in Section A which sought for biographical details of the respondents. The independent variables were gender, age, qualification, keyboard skills, computing knowledge/skills, level of teaching, teaching experience in face-to-face mode, and teaching experience in distance education mode. Each of the variables had several groupings.

An analysis of variance was carried out for each variable versus each factor on the items contained in Section B of the instrument. The results are as shown in Table 3 below.

Table 3: ANOVA of mean differences of selected independent variables in the responses of the audiographic trainees to the various subscales in the questionnaire.

* significant at P< .05.

The results indicated that only two of the analyses were significant. They were the effect of the different levels of keyboarding skills on their perception during the workshop (F=4.55, p<.05), and the effect of their face-to-face teaching experience on their perception of the learning environment (F=11.03, p<.05). These results seemed to indicate that the differences in their keyboarding skills affected their assessment of the training workshop. The more proficient they were with their keyboarding skills, the more positively they expressed their perception of the workshop. Similarly, the more experienced teachers expressed a higher positive assessment of the audiographic learning environment than those less experienced.

(ii) Qualitative analyses

Free responses to Section C (a) and (b)

Free responses (a) and (b) pertaining to 'The difficult features of using this equipment for education and training purposes', and 'The easy features of using this equipment for education and training purposes', which were found to be unique are listed below. There was some correlation in these responses with the computer literate finding all features relatively easy (to master) and those without prior computer knowledge finding them much more difficult.

Free responses to Section C (c)

These free responses pertained to “What have you gained from this workshop?”

The comments:

Basic understanding of the system
Basic introduction
A new way of presenting lessons
Better computer literacy
More confidence
Confidence
Competence
Experience in designing and holding audiographics workshops.
At this stage an understanding (not comprehensive) of audiographics.
An excitement and desire to use audiographics.
Skills in using PowerPoint and audiographics.
Ability to assess aspects of the technology for use in learning and teaching
It will broaden my teaching strategies
More strategies
New insight into doing a couple of courses myself in the future

Free responses to Section C (d)

The following free responses were given to “Suggestions for improving future workshops”. Responses to this section were received from 81% of the participants.

Suggestions:

Some keyboard/compute skills as pre-requisite
More (practical work) on Smart Board
Extra time for course devoted to conferencing techniques
Can't think of anything - was very well done.
Perhaps more time developing slides
Consider a 2-day workshop with “presentations” in afternoon of second day.
“Models of potential teaching techniques/strategies”
“Learn about audiographics through audiographics (understanding the students position)”
It was a great workshop. Thank you!
Need more modelling
Ensure room booked
Provision of tea/coffee & water at midbreak-4 hours is a long time, especially when it is tacked on to the rest of the day. ie. our workshop was 5-9pm Thursday & Friday.
Need more time to get used to all the features.

Summary and Conclusion

This evaluation was primarily focused on audiographic conference training for those who would continue to use them in their teaching, and to gather information on participants' perception of the value of the training package, the delivery mechanisms, the audiographic equipment, and other factors which affect the learning of the applications of the equipment within a distance education and open learning environment. 26 out of 30 participants at the training workshops, located in different sites of the Queensland audiographic conferencing, participated in the evaluation. An instrument, The Audiographic Conference Training Program Evaluation Questionnaire (ACTPEQ) was designed for participants undertaking the training program to gather data for a comprehensive evaluation of their training program.

The results of the evaluation indicated the following:

• the majority of the participants found the training workshop very beneficial, easier than they had expected, and that computing skills had no relationship with their mastery of the use of audiographic conferencing equipment.
• the participants were pleased with the interactivity characteristic of the audiographic equipment, and the learning environment requirements for the use of audiographic conferencing.
• while the confidence level of participants in the use of audiographics decreased immediately after the training, their expectation that the audiographics would be difficult to learn also significantly dropped after training.
• only the independent variables of keyboarding skills and teaching experience in the face-to-face mode showed significant differences in the responses of the participants with regard to the factors of the actual workshop training and the audiographic learning environment respectively.
• generally the participants expressed the view that they would have liked the training to have been longer to enable them to acquire more skills and become more familiar with the technology.

Regarding the trainees' expectations prior to and immediately after audiographic training, the following summaries can be made:

Skills and Confidence

• Participants were more positive towards their acquisition of audiographic conferencing skills before the training. Immediately after training, they were less likely to rate themselves as highly in skill areas. These comments apply to skills in preparing audiographic lessons, delivering audiographic lessons, and confidence in using audiographics.

View of Audiographics Use

• After the training, more participants were inclined to view audiographic conferencing as being able to meet the needs of their students. A marginal increase in the number of participants who believed that audiographic conferencing would improve the effectiveness of their distance education was evident after training.

Learning Audiographic Conferencing

• Fewer participants thought that audiographic conferencing was difficult to learn after the training than before the training.

Based on the results, the following recommendations are made:

1. The findings indicate strong support for the need, value and benefit of audiographics conferencing training. Institutions and organisations as a matter of policy should provide training and professional development opportunities for teachers using audiographics conferencing.
2. Contrary to initial perceptions, users found audiographics conferencing relatively easy to use. Furthermore, low computer literacy skills were not a barrier to users once people had received training. Teachers/lecturers regardless of computer literacy skills should be encouraged to use audiographics conferencing, provided training is available. Furthermore, an awareness campaign should be undertaken to promote that high computer literacy may not be important.
3. Low keyboard skills were considered a barrier to users during training. The training course should include keyboard familiarisation for people with low computer literacy skills.
4. Further longitudinal research work is required and should be conducted in this area to investigate the generalisability of these findings.

References

Anderson, T., & Mason, R. (1993). International computer conferencing for professional development: The Bangkok Project. American Journal of Distance Education, 7 (2), 5-18.

Blinn, C. R. (1989). Developing telecommunication linkages for microcomputer aided instruction. Telecommunication Development Center Research Report No.1. Minnesota University: Telecommunication Development Center.

Burge, E. J. (1993). The Audioconference: delivering continuing education for addictions worker in Canada. Journal of Alcohol and Drug Education, 39 (1), 78-91.

Clark, G. (1989). Distance education in United States schools. Computing Teacher, 16 (6), 7-11.

de Vries, L., Naidu, S., Jegede, O. J., & Collis, B. (1995). Online professional development: An evaluation study. Distance Education, 16 (1), 157-173.

Ehrmann, S. C. (1990). Reaching students, reaching resources: Using technologies to open the college. Annerberg/Corporation for Public Broadcasting project. (ERIC Document No. ED327171).

Gilcher, K. W., & Johnstone, S M. (1988). A critical review of the use of audiographic conferencing systems by selected educational institutions. Maryland University, College Park. (ERIC Document No. ED313003).

Gooley, A., & Towers, S. (1996). Turning ocean liners: Managing interactive technology innovation. Proceedings of the 3rd International Interactive Multimedia Symposium, Perth. 149-159.

Gooley, A., & Jegede, O.J. (1995). A longitudinal study of audiographic conferencing system in open and distance learning in Queensland. In F. Nouwens (Ed.). Crossing Frontiers. Proceedings of the 12th Biennial Forum of the Open and Distance Learning Association of Australia, September, Vanuatu, 17 - 24.

Hughes, A., & Priestley, J. (1992). The SOLS TICE project: satellite television and audioconferencing in continuing professional development for LIS staff. Information Services and Use, 12 (3), 263-274.

Jegede, O. J.& Gooley, A. (1994). A front-end evaluation of audiographic conferencing system in open and distance learning in Queensland. Proceedings of the First International Conference on Open Learning, November, Brisbane, 175-180.

Lundin, R. (1994). Flexible learning processes for the delivery of professional development activities of principles: A national report to the Australian Principals' Associations Professional Development Council (APAPDC) for Project 1. Brisbane: Queensland University of Technology.

Martin, D. J. (1986). Audiographic conferencing in small rural schools in New York state: A mode for developing shared teleconferenced services between rural school districts. (ERIC Document No. ED272336).

Murphy, K. L. (April, 1988). The integration of teleconferencing in distance teaching. Paper presented at the Annual Conference of the Association of European Correspondence Schools. Istanbul, Turkey.

Robson, J. (1991). Remote schooling and information technology (A guide for teachers). Canberra: Australian Catholic University.

Westerman, D. A. (1990). A study of expert and novice teacher decision making: An integrated approach. (ERIC Document No. ED322128).

Wilsman, M. J. (1988). Distance education in action: The Wisconsin rural reading improvement project Report. Madison: Wisconsin Public Radio & Television Networks, Education Services Division.