COLLAGEN: Java Middleware for Collaborative Agents
COLLAGEN (for COLLaborative AGENt) is Java middleware for building collaborative agents. A collaborative agent is a software program that helps users solve problems, especially in complex or unfamiliar domains, by correcting errors, suggesting what to do next, and taking care of low-level details. A collaborative agent can be added to an existing graphical user interface, such as a software simulator, or integrated into the design of a new hardware device, such as a home appliance.
Background & Objective: The theoretical foundations of COLLAGEN derive from the study of naturally occurring human collaboration, such as two people assembling a complex mechanical device or two computer users working on a spreadsheet together. The practical objective of the project is to maximize the software reuse in building collaborative agents for many different applications.
Technical Discussion: As shown in the diagram above, a collaborative agent in general communicates with the user (using either natural or artificial language), manipulates some shared hardware or software artifact, and observes the user's manipulation of the shared artifact.
The key to COLLAGEN's application-independence is an abstract, hierarchical representation, called the "task model," of the sequences of actions typically performed to achieve goals in a particular domain. The task model captures all of the knowledge that is specific to a particular application. In essence, COLLAGEN is an "interpreter" for task models.
COLLAGEN's representation of the current state of a collaborative dialogue consists of a plan tree, which tracks the status of steps in the task model, and a focus stack, which tracks the current focus of attention. COLLAGEN automatically updates these data structures whenever either the user or the agent speaks or performs a manipulation. The agent then uses these data structures to determine what to do or say in response.
Publications:
Lesh, N.; Marks, J.; Rich, C.; Sidner, C.L., “Man-Computer Symbiosis Revisited: Achieving Natural Communication and Collaboration with Computers”, Transactions on Electronics (IEICE), December 2004 (IEICE 2004 , TR2004-053) |
|
|
DeVault, D.; Rich, C.; Sidner, C.L., “Natural Language Generation and Discourse Context: Computing Distractor Sets from the Focus Stack”, International Florida Artificial Intelligence Research Symposium (FLAIRS), May 2004 (FLAIRS 2004, TR2004-004) |
|
|
Garland, A.; Lesh, N.B.; Rich, C., “Responding to and Recovering from Mistakes During Collaboration”, International Joint Conference on Artificial Intelligence (IJCAI), August 2003 (IJCAI 2003, TR2003-017) |
|
|
Rich, C.; Lesh, N.B.; Rickel, J.; Garland, A., “A Plug-in Architecture for Generating Collaborative Agent Responses”, International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), ISBN: 1-58113-480-0, pp. 782-789, July 2002 (ACM Press, TR2002-010) |
|
|
Eisenstein, J.; Rich, C., “Agents and GUI's from Task Models”, ACM International Conference on Intelligent User Interfaces, ISBN: 1-58113-382-0, pps 47-54, January 2002 (Proc ACM Press, TR2002-016) |
|
|
Garland, A.; Ryall, K.; Rich, C., “Learning Hierarchical Task Models by Defining and Refining Examples”, ACM International Conference on Knowledge Capture (KCAP), ISBN: 1-58113-380-4, pps 44-51, October 2001 (Proc ACM Press, TR2001-026) |
|
|
Lesh, N.B.; Rich, C.; Sidner, C.L., “Collaborating with Focused and Unfocused Users Under Imperfect Communication”, International Conference on User Modeling (UM), pp. 63-74, July 2001 (UM 2001) |
|
|
Cassell, J.; Nakano, Y.I.; Bickmore, T.W.; Sidner, C.L.; Rich, C., “Non-Verbal Cues for Discourse Structure”, Association for Computational Linguistics Annual Conference (ACL), pp. 106-115, July 2001 (ACL 2001, TR2001-024) |
|
|
Rickel, J.; Lesh, N.B.; Rich, C.; Sidner, C.L.; Gertner, A., “Building a Bridge Between Intelligent Tutoring and Collaborative Dialogue Systems”, International Conference on Artificial Intelligence in Education (AI-ED), Outstanding Paper Award, pps 592-594, May 2001 (AI-ED 2001, TR2000-040) |
|
|
Davies, J.R.; Gertner, A.; Lesh, N.B.; Rich, C.; Rickel, J.; Sidner, C.L., “Incorporating Tutorial Strategies into an Intelligent Assistant”, ACM International Conference on Intelligent User Interfaces, ISBN: 1-58113-325-1, pps 53-56, January 2001 (ACM Press, TR2000-030) |
|
|
Lesh, N.B.; Rich, C.; Sidner, C.L., “Using Plan Recognition in Human-Computer Collaboration”, International Conference on User Modeling (UM), pps 23-32, June 1999 (UM99, TR1998-023) |
|
|
Rich, C.; Sidner, C.L., “COLLAGEN: A Collaboration Manager for Software Interface Agents”, An International Journal: User Modeling and User-Adapted Interaction, Vol. 8, Issue 3/4, pps 315-350, January 1998 (Kluwer Online, TR1997-21a) |
| Technical Reports: | |
| The Role of Dialog in Human Robot Interaction | |
| Building Spoken Language Collaborative Interface Agents | |
| Plan Evaluation with Incomplete Action Descriptions | |
| Learning Hierarchical Task Models by Demonstration | |
| Hosting Activities: Experience with and Future Directions for a Robot Agent Host | |
| COLLAGEN: Applying Collaborative Discourse Theory to Human-Computer Interaction | |
| Task-Oriented Tutorial Dialogue: Issues and Agents | |
| COLLAGEN: When Agents Collaborate with People | |
| Segmented Interaction History in a Collaborative Agent | |
| Adding a Collaborative Agent to Graphical User Interfaces | |
| Window Sharing with Collaborative Interface Agents | |
Research Areas:
Off the Desktop Interaction and Display
Artificial Intelligence
Spoken Language Interfaces
Modification Date: September 14, 2007