[CSUSB] >> [CompSci] >> [Dick Botting] >> [Papers & Essays] >> rjb99h.feedback
[Index] || [Contents]
Disclaimer. CSUSB and the CS Dept have no responsibility for it.
This is a Beta Version, read at your own risk.
Copyright. Richard J. Botting(Sun Jul 6 10:03:46 PDT 2003). This paper is being developed for publication. Permission is granted to quote parts of it as long as the source is acknowledged and the author informed.

Contents (index)


    End of Year Report on FEEDBACK99

    Summary

    The FEEDBACK'99 project was designed to be inexpensive (about $1000) and make it easy for the faculty to incorporate anonymous student feedback into their pages on the World Wide Web. The result has been operational for a little less than a month. It is available for review and use at

    [ mkfb.html ]

    Process

    A small amount of money was spent on hiring teaching assistants so that I had time to (1) study the technical options (C/C++, UNIX Bourne Shell, Perl, JavaScript, Java) and (2) think through the risks and benefits of the options. See the Appendix (below) for the technical details of the analysis, design and implementation. A third party reviewed my first attempts, then two colleagues in the Computer Science Dept. Beta tested it. I announced the new system on the CSUSB faculty/staff bulletin board system(bb@csusb.edu).

    Responses

    Just when I advertised the new facility, another faculty member at CSUSB (Dr. D. Schnorr, [ schnorr ] ), quite unexpectedly, published a paper on a similar topic[ref1]. This paper is about giving and getting individualized feedback rather than anonymous surveys. My own experiments with individualized feedback were presented at the Teaching Colloquium at CSUSB in 1997. The notes and demonstrations for this presentation are on the web as "Ten Years of Internet Teaching"[ref2].

    So far members of the faculty have generated 3 bona fide anonymous feedback forms -- there are some tests, duplicates and trial runs. These are for classes with Id starting with HUM, MGMT and EVOC. Two people spotted errors (see technical detail in appendix). One member of the faculty liked the idea but wanted some help to set up a web site with which to begin. Here are two other comments received by EMAIL after the tool had been announced on the faculty and staff bulletin board system at CSUSB:

    Dr. Botting, I'm teaching a course this summer and once I come up with some questions, I'll use your form to create a questionnaire. This is an excellent service you're providing. Michael Casadonte

    I looked at your student feedback survey tool and think it's wonderful. Would it be possible for Physical Plant to use this tool as well for feedback on facilities and necessary repairs and/or service? It would be very valuable to us to establish more direct links with our students. Thanks. -Jim Hansen

    A similar request came from Rob Garcia, Information Technology Consultant, College of Social and Behavioral Sciences. It seems that this idea has a wider use than I expected.

    Effect on Student Learning

    The feedback technique was first used in the Spring 1998 CS320(Programming Languages) class with a manually generated page. It was also used in the Winter 1999 CS320 class. Each time it suggested some small changes for the next class. The SETEs did not change significantly. I have put a short analysis of one set of learning objectives on the web[ref4]. After the Spring section, I reduced the emphasis on a topic that the students felt was overemphasized. They did not feel this in the Winter section. However, their skills had decreased in that area. I will therefore be spending more time on motivating the topic in the future.

    So far no measurements have been taken of learning outcomes from using the computer generated forms. I plan to use to collect some information for presentation in the Fall of 1999.

    References


    (ref1): Schnorr, D. & Hazari S., Leveraging Student Feedback to Improve Teaching in a Web based Course, T.H.E. Journal, V26n11(Jun 1999)pp30-38, [ http://www.thejournal.com/magazine/ ]


    (ref2): Botting, R.J., Ten Years of Internet Teaching, [ rjb98a.tenyears.html ]


    (ref3): Botting, R.J., Notes on Perl, [ perl.html ]


    (ref4): Botting, R.J., JavaScript, [ javascript.html ]


    (ref5): Botting, R.J., Short Report on UML in CS320, [ rjb99a.ROOT320.html ]


    (ref6): Botting, R.J., Rational Rose Model of the FEEDBACK'99 system, [ feedback.mdl ]

    Appendix

    Technical Details As part of surveying technical options I made detailed notes on the languages Perl and JavaScript. I have published them on the world wide web[ref3,ref4].

    My experience administrating the computer science network has convinced me that risk analysis is the most important part of developing an Internet system. In one case we published an apparently safe page only to have a friendly hacker prove to us that they could take over the whole server by abusing it. Analyzing the risks in the FEEDBACK'99 project reduced the options drastically and determined much of the design. First, only the UNIX Bourne Shell made it simple to avoid the risk of a hacker breaking the program and gaining control of the WWW server. In the other languages either the language did not support sending Email or it had features that opened invisible security problems. Second, the design of the pages must allow for the fact that Word Wide Web users can save any page, modify it, and use the result in its place. For example, if the form contains a command that sends anonymous Email then students can change a copy to execute any command instead. If the faculty's Email address is in the form, an abusive student could download a copy, change the address and so be able harass anybody on the Internet.

    So the feedback form could not contain the full address of the faculty member. By putting only the faculty Email ID on the form and supplying the command and domain(csusb.edu) I could limit the possible damage to people on campus. Such abuse is harder if only an encrypted version of the faculty's address is on the page. However, this design would mean the developing an encryption scheme. Risk analysis lead to the idea of minimizing the information put into the feedback form.

    Two other risks had to be thought about. First, some means was needed to stop people abusing the service by creating an anonymous mailer. This would be an attractive target for unscrupulous Internet marketeers. A hacker once hijacked our Computer Science system to send mass unsolicited Email. Thus the faculty page can only create forms that deliver mail on campus. Second, having a way of turning off a form would also be useful in case a student copies it and stores for future abuse.

    A simple and secure plan reduces the information on the form to a minimum. This would be a single meaningless number that acts as a key to a private data base. The information identifying the faculty could be kept in a private area on the server. The student's access a feedback form that contains a hidden number that identifies one item of information in this file. The server would use the number as a key to lookup the command that transmits the response to the faculty member. All the sensitive information is hidden by the server where it cannot be abused. As a worst case scenario an abuser could change the form to send email to an unknown person with a feedback form. The hidden file of commands would be accessible only via the "make feedback" ( mkfb.html ) page. This would be designed so that only local Email could be stored.

    Keeping a private list of commands also makes it easy to make a page inactive by deleting the record of the command.

    I sketched a model of the system using the industry standard Unified Modeling Language[ref6]. I developed the necessary pages and scripts incrementally. I added user documentation, tested the scripts, and had a third party review the page. I improved it and invited two colleagues in the Computer Science Dept. ( Drs. Mendoza and Karant ) to Beta test the system. Their tests worked. I then announced the new system on the CSUSB faculty/staff bulletin board system(bb@csusb.edu). Several faculty and staff responded rapidly. Charles Tabbut of the library pointed out a small error and one bug was found by Dr. Ronald Pendleton. These have been fixed.

    . . . . . . . . . ( end of section End of Year Report on FEEDBACK99) <<Contents | Index>>


Formulae and Definitions in Alphabetical Order