Appendix 3 -- Details of the Development of IS'97

Return to Table of Contents

This appendix explains the development process for IS'97. A few key principles guided the effort:

1. The documents produced and the thinking involved should represent a consensus from IS faculty, chairs, and industry leaders.

2. The documents should be designed to help IS faculty produce competent and confident entry level graduates well suited to work-place responsibilities.

3. The documents should not be prescriptive, but should allow faculty to see clearly the depth of understanding and behaviors expected of graduates, and to build their own courses within the guidelines of the documents.

4. The documents should be based on sound educational methodology and make appropriate recommendations for consideration by IS faculty.

5. The model should be flexible and adaptable to most IS programs. It should be useful for IS programs in different parts of the university. Survey data indicated almost 50% of IS programs occur in schools of business, while the rest occur in a number of other areas.

The Development Process

There were meetings of the entire task force, of the co-chairs, and of the IS community at formal panels, keynotes, and other presentations at the annual meetings of IACIS, ISECON, DSI, IAIM, ICIS, and SIGCSE. Agenda items (Clawson and Bostrom 1991) and group methods were considered in detail, and methods were selected to take maximum advantage of the various assembled groups.

In meetings, facilitated discussion was frequently used. As the group talked through agenda items, an almost verbatim transcript was produced and projected on a screen for all to view. This written record prevented any blocking behavior and provided a mechanism for developing shared vision, a major goal at every meeting.

Group systems tools were used for several electronic meetings in which information was collected from group members.

1. A group meeting tool (GROUP_OUTLINER) was used for an electronic discussion forum and idea generation. The tool allowed developing an organized list/outline (with directions) for electronic discussion by the participants. Printouts were given to the group frequently during such meetings. This allowed for group and individual learning and sharing of ideas.

2. A group meeting software function (VOTE) was used to test opinions and to determine the relative strength of ideas. The tool was not used as a means to divide or eliminate discussion. The tool was not used to develop or enforce a consensus.

3. A group meeting software function (GROUP-MATRIX) was used by the task force for level setting with lengthy multi-column lists. Significant variances in opinions were explored through face-to-face discussion during the sessions. The tool was used to encourage consensus.

Idea development was usually initiated in large groups with collection of data. Small teams or individuals sifted through the ideas to extract the central meanings and present abstractions for review by the group. Idea initiators were asked to bring reference material to ensure completeness.

Abstractions were reviewed and evaluated. Surveys were developed to involve a larger group in reviewing the developing ideas and concepts. The surveys were considered useful forward-looking input because:

1. Faculty completing the surveys knew that their answers would be used to reflect the future.

2. Consistency in importance of survey items over time identifies items having long-term value. For example, the importance of systems theory was in all surveys going back to 1988. Such a constant observation leads to the recommendation to include systems theory within the IS'97 curriculum.

3. Important unmet needs emerge. Where a significant percent of the community accepts or rejects an idea, the recommendations reflect such concerns. For example, a very high percent of surveyed participants suggested that the curriculum should present options involving cooperation with Computer Science units.

In using the survey data, questions were posed that tested many of the ideas the task group felt were important. The results were shared with the IS community to test the suggestions. Such data are an explicit part of IS'97.

The co-chairs and task force members are well known IS faculty and/or professionals. ACM Education Board, AIS, DPMA top management, DSI, IAIM, ISECON, and IACIS management have provided critical review and the opportunity for refinement of these materials. Reviewers have completed surveys and provided commentary to ensure the validity and accuracy of this document.

In identifying participants for the survey, the MISRC/McGraw-Hill Directory of Management Information Systems Faculty was used for tracking IS departments, IS faculty, and IS chairs. We feel this collection represents about 40% of the IS community. Other lists from meeting participants, the DPMA, and Peterson Guide were used. Overall, several thousand IS programs within the United States, Canada and from other parts of the world were contacted.

Exit Objectives for Graduates

In a systems approach to the IS curriculum, the definition of exit objectives identifies the output of the system. The input is assumed to be high school graduates capable of entering college.

The basic idea is that graduates of IS programs should have competencies, skills and attitudes that are necessary for success in the workplace and life-long learning as an IS professional or provide the basis for graduate programs. Surveys of the task force and of others have identified and prioritized these characteristics. Many of these competencies are shared within the computing professions, as indicated by this data.

Given these exit characteristics, there exists a sequence of learning units that produce graduates with the desired competencies. The task force described these learning units and their sequencing. This was accomplished in a top down manner by describing broad curriculum areas, formulating courses to implement these areas, and specifying learning units to implement the courses.

Depth of Knowledge Metrics

The depth of knowledge metrics of IS'90 were adopted by the IS'97 task force. There are five levels of depth of knowledge competencies explained in Appendix 4. The IS'97 group extended the IS'90 observations by reviewing the appropriate pedagogy, particularly the issues regarding the successful application of cooperative learning mechanisms. These are particularly important in team environments. The competency levels were used to differentiate introductory explanation of topics from later depth coverage. A backtracking process was used to assure that correct sequencing of material was achieved and appropriate prerequisite material had been defined.

Body of Information Systems Knowledge

The body of computing knowledge represents a synthesis of the IS'90 body of knowledge updated by the IS'97 task force. Also incorporated were elements of the ACM-Computer Science knowledge elements from CS'91 and other curriculum proposals as well as all of the knowledge elements from the Software Engineering Institute documents (see Appendix 7 for detail). In addition, the document has been modified to reflect the work of the NSF sponsored task force describing a tenth area for CS'91.

In order to match and include the ACM and other knowledge elements, all competency statements were removed, leaving just the knowledge element. The IS body of knowledge is essentially a three level hierarchy of broad subject areas. Most of the CS and SE elements are added as the fourth level to further elaborate the detailed topics. They add a considerable richness to the hierarchy and, in fact, define a computing body of knowledge. The ethics components from an NSF task force for a tenth area of CS'91 were added as elements at the fourth level and required renaming several third level elements for completeness.

The mapping process was done one element at a time, searching to see where the element belonged. The process was repeated for each element. Some elements matched an existing third level element.

Development of Courses

Selecting the names and number of courses was one of the most controversial issues for the task force. Although there was a final concession to agree to adopt ten courses and one prerequisite, it is doubtful there was a true consensus on the validity of the entire course set. For example, in AACSB accredited schools, a maximum of eight courses is permitted. The task force therefore used the courses as a vehicle to determine the goals and objectives that all graduates needed to learn rather than a unified prescription.

Curriculum areas were a source of discussion but there was good agreement on these areas. The task force used Group Systems to collect potential course names within each curriculum area. These lists were organized and the number of courses that could be taught within each area were identified. Once the number of courses was fixed, the group selected course titles. Reasonable consensus was reached through face to face discussion of the lists and the course titles presented in this document were adopted.

Development of Learning Units

The systems approach of Gagne, Briggs and Wager (1988) based on a strategic sequencing of learning objectives requiring performance at increasing levels of Bloom's (1956) competencies was chosen. A modification of Bloom levels was used with four levels of competency. Learning units were developed to encapsulate goals and objectives for transferring elements of the IS body of computing knowledge. IS'90 (Longenecker and Feinstein 1991c) developed a language for specification of goals and objective statements.

Once course titles were fixed, the task force again used group meeting software to collect goal and objective statements for each course. Task force members had previous curriculum documents, access to texts, the body of knowledge, and a variety of recent publications in addition to their own expertise.

Several thousand goal and objective statement were collected. These were organized into groups manually and rewritten giving a list of 127 goal statements and 250 uncorrelated objective statements. Both lists were entered into a database. Each goal and objective was classified with keywords. The keywords were used to partially order the two lists. One or more objective statement was matched to a goal statement. Some goal statements had no corresponding objectives and goals were written for these. Each goal and set of objectives was then reviewed for completeness, given a name and a depth level based on the language of the goal and objectives.

Using the keywords for each goal and objective, the body of knowledge was searched manually for appropriate elements. The search was done by looking primarily for third level knowledge statements. For each third level element, at least one, and preferably more than one, goal-objectives set, or "learning unit," was isolated for each depth of knowledge level. The mapping of the body of knowledge to learning units is shown both in Appendices 8 and 9. In Appendix 7, the learning units are shown. In Appendix 8, the goal and objective sets are shown, along with the relevant body of knowledge elements.

Return to Table of Contents

Appendix 4 ­ Depth of Knowledge Metrics and Related Pedagogy

Return to Table of Contents

A key ingredient in IS'97 is a competency or depth of knowledge metric with five levels (with four levels specified in the curriculum). This metric is based on but not identical to the work of Bloom (1956), which describes a six level metric. The metric makes it possible to communicate specifications and expectations.

Depth of Knowledge Metric

Table A4.1 is a summary of the depth of knowledge metric. Note that there are conceptually five levels for depth of knowledge in IS'97 but only the first four are used for an undergraduate program. The IS'97 levels differ from Bloom levels in that Bloom's level 1 is divided into IS'97 levels 1 and 2 and Bloom levels 4, 5, and 6 are mapped to IS'97 level 5.

The characteristics of the metric include

­ the definition of the levels of knowledge,

­ the behavior to be demonstrated by those completing learning units of the curriculum,

­ how goals and objectives are developed compatible with each knowledge level,

­ how to determine the level of knowledge from previously defined goal and objective statements (reverse engineer knowledge levels from existing documentation),

­ how material at a given level can be delivered to students, and

­ how learning at given level can be assessed.

The Joint IS'97 Curriculum Task Force used the taxonomy of knowledge description adopted by IS'90 (Longenecker and Feinstein 1991c; Longenecker, Clark et al. 1994) summarized above. The IS'97 task force used the template shown in column 3 for use in writing behavioral objective and goal statements; these statements allow authors and faculty to be more precise in communicating expectations for both students and teachers.

Identifying Expectations

The statements of characteristics of graduates contain "keywords" that can be detected using the template of the metric. For example, if the expectation is to "apply problem solving techniques in configuring a local area network," this is the equivalent of a level 4 objective. The knowledge levels specified within IS'97 are compatible with the definitions of Table A4.1. The exit objectives of the goals and objectives have been checked and verified to assure consistency with the expectations of industry and academics.

Content Analysis of Statements of Expectation

The knowledge levels of IS'97 are designed to give guidance to educators in planning as well as in the analysis of outcomes. Column 3 of Table A4.1 describes a template for writing objectives. This template was originally defined in IS'90 and has been expanded in the present context. The language used in writing a behavioral objectives was derived from the Bloom taxonomy. The template may be used prescriptively in writing presentation goals and student performance objectives to ensure that the implied level of difficulty is presented. Likewise, given the objective, the student's behavior can be observed and compared with the objective statement to ensure that the students achieve the desired results of the presentation goal statements.

Learning Techniques for Different Levels

Learning techniques often differ for different levels. Level 1 knowledge in IS'97 (awareness) is knowledge that is immediately apparent. Given a visual stimulus, it is knowledge that is recalled. IS'97 level 2 knowledge (literacy) requires not only recognition, but recognition of the context of the knowledge; that is, the knowledge element and its parents and descendants should be familiar to the learner. Classroom activity or participative learning strategies are sufficient in transferring this level of knowledge, although level 2 activity is enhanced in the lab. Although knowledge at levels 1 and 2 is relatively low, these levels should be mastered before higher levels can be achieved. It is the "revisiting" of previously presented and learned knowledge that is implied in the organization of learning units.

The more complex IS'97 level 3 (usage/comprehension) requires considerable practice and creative repetition. Level 4 (application) requires unsupervised practice. Team work, project work, and other participative learning facilitate achieving these levels. Proper sequencing is an important factor in achieving student success. Project laboratories are ideal for this level of student activity. In fact, these laboratories are beneficial at all levels of instruction (Doran, Longenecker and Pardu 1994; Dutt 1994). Some institutions have been successful with total participatory project environments (Holland College 1993).

The cooperative paradigm (Litchfield 1996; Johnson, Johnson and Houlubec 1993) offers many advantages to learners, although it requires considerable change on the part of faculty. The cooperative paradigm greatly increases student motivation and better simulates the work environment in which graduates are expected to perform. The cooperative paradigm supports well the development of application level competencies.

Table A4.1 -- Knowledge Levels, Templates for Objective Writing, and Meaning of the Depth Levels with Associated Learning Activities

Return to Table of Contents