Publications of George Avrunin

Formalization and Analysis of Human-Intensive Processes

Jake Cyr, Georgios Karagkiaouris, Stefan C. Christov, Heather M. Conboy, George S. Avrunin, Lori A. Clarke, Leon J. Osterweil, Elizabeth A. Henneman, Jenna L. Marquard, and Nancy Famigletti. Web-based smart checklists for guiding performers of safety-critical human-intensive processes. Poster at American Society for Engineering Education Northeast Section (ASEE-NE) Annual Conference, April 2017.

Leon J. Osterweil, Matt Bishop, Heather M. Conboy, Huong Phan, Borislava I. Simidchieva, George S. Avrunin, Lori A. Clarke, and Sean Peisert. Iterative analysis to improve key properties of critical human-intensive processes: An election security example. ACM Transactions on Privacy and Security, 20(2):Article 5, 31 pages, March 2017. [ DOI | .pdf ]

Heather M. Conboy, George S. Avrunin, Lori A. Clarke, Leon J. Osterweil, Julian M. Goldman, Steven J. Yule, Marco A. Zenati, and Stefan C. Christov. Cognitive support during high-consequence episodes of care in cardiovascular surgery. In Proceedings of the 2017 IEEE Conference on Cognitive and Computational Aspects of Situation Management (CogSIMA), Savannah, GA, 2017. IEEE. [ .pdf ]

Stefan C. Christov, Jenna L. Marquard, George S. Avrunin, and Lori A. Clarke. Assessing the effectiveness of five process elicitation methods: A case study of chemotherapy treatment plan review. Journal of Applied Ergonomics, 59:364-376, 2017. [ DOI | .pdf ]

Stefan C. Christov, Heather M. Conboy, Nancy Famigletti, George S. Avrunin, Lori A. Clarke, and Leon J. Osterweil. Smart checklists to improve healthcare outcomes. In Proceedings of the 2016 International Workshop on Software Engineering in Healthcare Systems, pages 54-57, Austin, TX, 2016. [ .pdf ]

Stefan C. Christov, George S. Avrunin, and Lori A. Clarke. Online deviation detection for medical processes. In American Medical Informatics Association Annual Symposium, pages 395-404, November 2014.

Heather M. Conboy, Jason K. Maron, Stefan C. Christov, George S. Avrunin, Lori A. Clarke, Leon J. Osterweil, and Marco A. Zenati. Process modelling of aortic cannulation in cardiac surgery: Toward a smart checklist to mitigate the risk of stroke. In Proceedings of the Fifth Workshop on Modeling and Monitoring of Computer Assisted Interventions (M2CAI '14), page 10 pages, Cambridge, MA, September 2014. [ .pdf ]

Preventable adverse events related to surgery account for two thirds of hospital complications. Adherence to recommended processes of care has been suggested as a strategy to improve patient safety in surgery. This paper presents preliminary work that is exploring the use of a semantically rich process-modelling notation to describe and inform critical phases of common procedures in cardiac surgery. This work focuses on reducing stokes, a catastrophic and often preventable adverse event. The well-defined semantics of the process-modelling notation allow rigorous analysis techniques to be applied. In our work, model checking is applied to determine if the process as defined by the process model always adheres to event sequence requirements and fault-tree analysis is applied to determine where the process is vulnerable to performance failures. The results from these analyses lead to validated and improved process models that are then used to generate context-sensitive, dynamic “smart” checklists. Future work will evaluate whether the introduction of dynamic checklists based on these models will reduce the number and severity of errors in cardiac surgery.

Matt Bishop, Heather M. Conboy, Huong Phan, Borislava I. Simidchieva, George S. Avrunin, Lori A. Clarke, Leon J. Ostwerweil, and Sean Peisert. Insider threat identification by process analysis. In Workshop on Research for Insider Threat, IEEE Computer Society Security and Privacy Workshops (SPW14), San Jose, CA, 2014. [ .pdf ]

The insider threat is one of the most pernicious in computer security. Traditional approaches typically instrument systems with decoys or intrusion detection mechanisms to detect individuals who abuse their privileges (the quintessential “insider”). Such an attack requires that these agents have access to resources or data in order to corrupt or disclose them. In this work, we examine the application of process modeling and subsequent analyses to the insider problem. With process modeling, we first describe how a process works in formal terms. We then look at the agents who are carrying out particular tasks, perform different analyses to determine how the process can be compromised, and suggest countermeasures that can be incorporated into the process model to improve its resistance to insider attack.

Stefan C. Christov, George S. Avrunin, and Lori A. Clarke. Considerations for online deviation detection in medical processes. In SEHC '13: Proceedings of the 2013 ICSE Workshop on Software Engineering in Health Care, pages 50-56, San Francisco, May 2013. [ .pdf ]

Medical errors are a major cause of unnecessary suffering and even death. To address this problem, we are investigating an approach for automatically detecting when an executing process deviates from a set of recommended ways to perform that process. Such deviations could represent errors and, thus, detecting and reporting deviations as they occur could help catch errors before something bad happens. This paper presents the proposed deviation detection approach, identifies some of the major research issues that arise, and discusses strategies to address these issues. A preliminary evaluation is performed by applying the approach to a part of a detailed process model. This model has been developed in an in-depth case study on modeling and analyzing a blood transfusion process.

Heather M. Conboy, George S. Avrunin, and Lori A. Clarke. Modal abstraction view of requirements for medical devices used in healthcare processes. In SEHC '13: Proceedings of the 2013 ICSE Workshop on Software Engineering in Health Care, pages 24-27, San Francisco, May 2013. [ .pdf ]

Medical device requirements often depend on the healthcare processes in which the device is to be used. Since such processes may be complex, critical requirements may be specified inaccurately, or even missed altogether. We are investigating an automated requirement derivation approach that takes as input a model of the healthcare process along with a model of the device and tries to derive the requirements for that device. Our initial experience with this approach has shown that when the process and device involve complex behaviors, the derived requirements are also often complex and difficult to understand. In this paper, we describe an approach for creating a modal abstraction view of the derived requirements that decomposes each requirement based on it.

Wilson C. Mertens, Stefan C. Christov, George S. Avrunin, Lori A. Clarke, Leon J. Osterweil, Lucinda J. Cassells, and Jenna L. Marquard. Using process elicitation and validation to understand and improve chemotherapy ordering and delivery. The Joint Commission Journal on Quality and Patient Safety, 38(11):497-505, November 2012.

Background: Chemotherapy ordering and administration, in which errors have potentially severe consequences, was quantitatively and qualitatively evaluated by employing process formalism (or formal process definition), a technique derived from software engineering, to elicit and rigorously describe the process, after which validation techniques were applied to confirm the accuracy of the described process.

Methods: The chemotherapy ordering and administration process, including exceptional situations and individuals' recognition of and responses to those situations, was elicited through informal, unstructured interviews with members of an interdisciplinary team. The process description (or process definition), written in a notation developed for software quality assessment purposes, guided process validation (which consisted of direct observations and semistructured interviews to confirm the elicited details for the treatment plan portion of the process).

Results: The overall process definition yielded 467 steps; 207 steps (44%) were dedicated to handling 59 exceptional situations. Validation yielded 82 unique process events (35 new expected but not yet described steps, 16 new exceptional situations, and 31 new steps in response to exceptional situations). Process participants actively altered the process as ambiguities and conflicts were discovered by the elicitation and validation components of the study. Chemotherapy error rates declined significantly during and after the project, which was conducted from October 2007 through August 2008.

Discussion: Each elicitation method and the subsequent validation discussions contributed uniquely to understanding the chemotherapy treatment plan review process, supporting rapid adoption of changes, improved communication regarding the process, and ensuing error reduction.

Huong Phan, George S. Avrunin, Matt Bishop, Lori A. Clarke, and Leon J. Osterweil. A systematic process-model-based approach for synthesizing attacks and evaluating them. In Proceedings of the 2012 Electronic Voting Technology Workshop/Workshop on Trustworthy Elections, Bellevue, WA, August 2012. [ .pdf ]

George S. Avrunin, Lori A. Clarke, Leon J. Osterweil, Julian M. Goldman, and Tracy Rausch. Smart checklists for human-intensive medical systems. In Proceedings of the Workshop on Open Resilient human-aware Cyberphysical Systems (WORCS-2012), June 2012. [ DOI | .pdf ]

Human-intensive cyber-physical systems involve software applications and hardware devices, but also depend upon the expertise of human participants to achieve their goal. In this paper, we describe a project we have started to improve the effectiveness of such systems by providing Smart Checklists to support and guide human participants in carrying out their tasks, including their interactions with the devices and software applications.

George S. Avrunin, Lori A. Clarke, Leon J. Osterweil, Stefan C. Christov, Bin Chen, Elizabeth A. Henneman, Philip L. Henneman, Lucinda Cassells, and Wilson Mertens. Experience modeling and analyzing medical processes: UMass/Baystate medical safety project overview. In 1st ACM International Health Informatics Symposium, pages 316-325, Arlington, VA, November 2010. [ DOI | .pdf ]

This paper provides an overview of the UMass/Baystate Medical Safety project, which has been developing and evaluating tools and technology for modeling and analyzing medical processes. We describe the tools that currently comprise the Process Improvement Environment, PIE. For each tool, we illustrate the kinds of information that it provides and discuss how that information can be used to improve the modeled process as well as provide useful information that other tools in the environment can leverage. Because the process modeling notation that we use has rigorously defined semantics and supports creating relatively detailed process models (for example, our models can specify alternative ways of dealing with exceptional behavior and concurrency), a number of powerful analysis techniques can be applied. The cost of eliciting and maintaining such a detailed model is amortized over the range of analyses that can be applied to detect errors, vulnerabilities, and inefficiencies in an existing process or in proposed process modifications before they are deployed.

Heather M. Conboy, George S. Avrunin, and Lori A. Clarke. Process-based derivation of requirements for medical devices. In 1st ACM International Health Informatics Symposium, pages 656-665, Arlington, VA, November 2010. [ DOI | .pdf ]

One goal of medical device certification is to show that a given medical device satisfies its requirements. But the requirements that should be met by such a device depend on the clinical processes in which the device is to be used, and such processes are increasingly large and complex. Critical requirements may thus be specified inaccurately or incompletely, or even missed altogether, and the use of the devices may lead to harm. Thus, we investigated a process-based requirement derivation approach that inputs a model that captures a particular medical process and a requirement that should be satisfied by that process, and outputs a derived requirement of the medical device that is sufficient to prevent any violations of the process requirement. Our approach combines an approach for generating assumptions for assume-guarantee reasoning with one for interface synthesis to automate the derivation of the medical device requirements. The proposed approach iteratively performs the requirement derivation by employing a model checker and a learning algorithm. We implemented this approach and evaluated our approach by applying it to two small case studies. Our experiences showed that the proposed approach could be successfully applied to abstract models of portions of real medical processes and that the derived requirements of the medical devices appeared useful and understandable.

Lori A. Clarke, Leon J. Osterweil, and George S. Avrunin. Supporting human-intensive systems. In Proceedings of 2010 FSE/SDP Workshop on the Future of Software Engineering Research, pages 87-91, Santa Fe, NM, November 2010. ACM.

Danhua Wang, Jingui Pan, George S. Avrunin, Lori A. Clarke, and Bin Chen. An automatic failure mode and effect analysis technique for processes defined in the Little-JIL process definition language. In 22nd International Conference on Software Engineering and Knowledge Engineering, pages 765-770, July 2010. [ .pdf ]

Many processes are safety critical and therefore could benefit from proactive safety analysis techniques that attempt to identify weaknesses of such processes before they are put into use. In this paper, we propose an approach that automatically derives Failure Mode and Effect Analysis (FMEA) information from processes modeled in the Little-JIL process definition language. Typically FMEA information is created manually by skilled experts, an approach that is usually considered to be time-consuming, error-prone, and tedious when applied to complex processes. Although great care must be taken in creating an accurate process definition, with our approach this definition can then be used to create FMEA representations for a wide range of potential failures. In addition, our approach provides a complementary Fault Tree Analysis (FTA), thereby supporting two of the most widely used safety analysis techniques.

Stefan Christov, George S. Avrunin, Lori A. Clarke, Leon J. Osterweil, and Elizabeth A. Henneman. A benchmark for evaluating software engineering techniques for improving medical processes. In Lori A. Clarke and Jens Weber-Jahnke, editors, SEHC '10: Proceedings of the 2010 ICSE Workshop on Software Engineering in Health Care, pages 50-56, Cape Town, South Africa, May 2010. [ DOI | .pdf ]

The software engineering and medical informatics communities have been developing a range of approaches for reasoning about medical processes. To facilitate the comparison of such approaches, it would be desirable to have a set of medical examples, or benchmarks, that are easily available, described in considerable detail, and characterized in terms of the real-world complexities they capture. This paper presents one such benchmark and discusses a list of desiderata that medical benchmarks can be evaluated against.

Leon J. Osterweil, Lori A. Clarke, and George S. Avrunin. An integrated collection of tools for continuously improving the processes by which health care is delivered: A tool report. In Third International Workshop on Process-Oriented Information Systems in Healthcare (ProHealth '09), Ulm, Germany, September 2009. [ DOI | .pdf ]

This report will present a collection of tools that supports the precise definition, careful analysis, and execution of processes that coordinate the actions of humans, automated devices, and software systems for the delivery of health care. The tools have been developed over the past several years and are currently being evaluated through their application to four health care processes, blood transfusion, chemotherapy, emergency department operations, and identity verification. The tools are integrated with each other using the Eclipse framework or through the sharing of artifacts so that the internal representations generated by one might be used to expedite the actions of others. This integrated collection of tools is intended to support the continuous improvement of health care delivery processes. The process definitions developed through this framework are executable and are intended for eventual use in helping to guide actual health care workers in the performance of their activities, including the utilization of medical devices and information systems.

Jenna L. Marquard, Stefan Christov, Philip L. Henneman, Lori A. Clarke, Leon J. Osterweil, George S. Avrunin, Donald L. Fisher, Elizabeth A. Henneman, Megan M. Campbell, Tuan A. Pham, and Qi Ming Lin. Studying rigorously defined health care processes using a formal process modeling language, clinical simulation, observation, and eye tracking. In Proceedings of NDM9, the 9th International Conference on Naturalistic Decision Making, pages 239-240, London, June 2009. [ .pdf ]

Motivation: The complex nature of health care processes requires new methods for describing, capturing and improving these processes. Research approach: We deployed a novel combination of methods-formal process modeling using a language called Little-JIL, simulations with embedded errors, observations, and eye tracking technology-to gauge how health care providers complete one complex process, patient identification. Findings/Design: These methods allowed us to thoroughly analyze how health care providers completed the patient identification process with and without embedded errors, and to record exactly what participants looked at during the simulations. Research limitations/Implications: We have used this set of methods to analyze only one type of health care process to-date. Originality/Value: We can use these approaches to inform health care provider training, process redesign, and the design of technologies to support health care providers as they verify patients' identities.

Stefan Christov, George S. Avrunin, Lori A. Clarke, Philip L. Henneman, Jenna L. Marquard, and Leon J. Osterweil. Using event streams to validate process definitions. Technical Report UM-CS-2009-004, Department of Computer Science, University of Massachusetts, January 2009. [ .pdf ]

This paper describes preliminary work on validating process definitions by comparing a process definition to event streams derived from an actual execution of that process. The goal of this work is to find and remove discrepancies in the process definition before using that definition as the basis for various forms of analysis and decision making. The paper outlines mportant issues that need to be addressed and suggests possible approaches. The example used in this paper is based on a process from the medical domain.

Elizabeth A. Henneman, Rachel Cobleigh, George S. Avrunin, Lori A. Clarke, Leon J. Osterweil, and Philip L. Henneman. Designing property specifications to improve the safety of the blood transfusion process. Transfusion Medicine Reviews, pages 1-9, June 2008. [ DOI ]

Computer scientists use a number of well-established techniques that have the potential to improve the safety of patient care processes. One is the formal definition of the properties of a process. Even highly regulated processes, such as laboratory specimen acquisition and transfusion therapy, use guidelines that may be vague, misunderstood, and hence erratically implemented. Examining processes in a systematic way has led us to appreciate the potential variability in routine health care practical and the impact of this variability on patient safety in the clinical setting. The purpose of this article is to discuss the use of innovative computer science techniques as a means of formally defining and specifying certain desirable goals of common, high-risk, patient care processes. Our focus is on describing the specification of process properties, that is, the high-level goals of a process that ultimately dictate why a process should be performed in a given manner.

Bin Chen, George S. Avrunin, Elizabeth A. Henneman, Lori A. Clarke, Leon J. Osterweil, and Philip L. Henneman. Analyzing medical processes. In ICSE '08: Proceedings of the 30th International Conference on Software Engineering, pages 623-632. ACM, May 2008. [ DOI | .pdf ]

This paper shows how software engineering technologies used to define and analyze complex software systems can also be effective in detecting defects in human-intensive processes used to administer healthcare. The work described here builds upon earlier work demonstrating that healthcare processes can be defined precisely. This paper describes how finite-state verification can be used to help find defects in such processes as well as find errors in the process definitions and property specifications. The paper includes a detailed example, based upon a real-world process for transfusing blood, where the process defects that were found led to improvements in the process.

Lori A. Clarke, George S. Avrunin, and Leon J. Osterweil. Using software engineering technology to improve the quality of medical processes. In ICSE Companion '08: Companion of the 30th International Conference on Software Engineering, pages 889-898. ACM, May 2008. (Invited keynote address.). [ DOI | .pdf ]

In this paper, we describe some of the key observations resulting from our work on using software engineering technologies to help detect errors in medical processes. In many ways, medical processes are similar to distributed systems in their complexity and proneness to contain errors. We have been investigating the application of a continuous process improvement approach to medical processes in which detailed and semantically rich models of the medical processes are created and then subjected to rigorous analyses. The technologies we applied helped improve understanding about the processes and led to the detection of errors and subsequent improvements to those processes. This work is still preliminary, but is suggesting new research directions for medical process improvement, software engineering technologies, and the applicability of these technologies to other domains involving human-intensive processes.

Stefan Christov, Bin Chen, George S. Avrunin, Lori A. Clarke, Leon J. Osterweil, David Brown, Lucinda Cassells, and Wilson Mertens. Formally defining medical processes. Methods of Information in Medicine, 47(5):392-398, 2008. [ DOI | .pdf ]

Stefan Christov, Bin Chen, George S. Avrunin, Lori A. Clarke, Leon J. Osterweil, David Brown, Lucinda Cassells, and Wilson Merten. Rigorously defining and analyzing medical processes: An experience report. In Models in Software Engineering: Workshops and Symposia at MoDELS 2007, Reports and Revised Selected Papers, number 5002 in LNCS, pages 118-131, Nashville, TN, September 2007. [ DOI | .pdf ]

This paper describes experiences in using the precise definition of a process for chemotherapy administration and as the basis for analyses aimed at finding and correcting defects, leading to improvements in efficiency and patient safety. The work is a collaboration between Computer Science researchers and members of the professional staff of a major regional cancer center. The work entails the use of the Little-JIL process definition language for creating the precise definitions, the PROPEL system for creating precise specifications of process requirements, and the FLAVERS systems for analyzing process definitions. The paper describes the details of using these technologies, by demonstrating how they have been applied to successfully identify defects in the chemotherapy process. Although this work is still ongoing, early experiences suggest that our approach is viable and promising. The work has also helped us to learn about the desiderata for process definition and analysis technologies that are expected to be more broadly applicable to other domains.

Leon J. Osterweil, George S. Avrunin, Bin Chen, Lori A. Clarke, Rachel L. Cobleigh, Elizabeth A. Henneman, and Philip L. Henneman. Engineering medical processes to improve their safety: An experience report. In J. Ralyte, S. Brinkemper, and B. Henderson-Seelers, editors, Situational Method Engineering: Fundamentals and Experiences, pages 267-282, Geneva, September 2007. Springer. [ DOI | .pdf ]

This paper describes experiences in using precise definitions of medical processes as the basis for analyses aimed at finding and correcting defects leading to improvements in patient safety. The work entails the use of the Little-JIL process definition language for creating the precise definitions, the Propel system for creating precise specifications of process requirements, and the FLAVERS systems for analyzing process definitions. The paper describes the details of using these technologies, employing a blood transfusion process as an example. Although this work is still ongoing, early experiences suggest that our approach is viable and promising. The work has also helped us to learn about the desiderata for process definition and analysis technologies that are intended to be used to engineer methods.

Elizabeth A. Henneman, Rachel Cobleigh, Kimberly Frederick, Ethan Katz-Bassett, George S. Avrunin, Lori A Clarke, Leon J. Osterweil, Chester Andrzejewski, Jr., Karen Merrigan, and Phillip L. Henneman. Increasing patient safety and efficiency in transfusion therapy using formal process definitions. Transfusion Medicine Reviews, 21(1):49-57, January 2007. [ DOI ]

The administration of blood products is a common, resource intensive, potentially problem-prone area that may place patients at elevated risk in the clinical setting. Much of the emphasis in transfusion safety has been targeted towards quality control measures in laboratory settings where blood products are prepared for administration as well as in automation of certain laboratory processes. In contrast, the process of transfusing blood in the clinical setting (i.e., at the point of care) has essentially remained unchanged over the past several decades.

Many of the currently available methods for improving the quality and safety of blood transfusions in the clinical setting rely on informal process descriptions, such as flow charts and medical algorithms, to describe medical processes. These informal descriptions, while useful in presenting an overview of standard processes, can be ambiguous or incomplete. For example, they often describe only the standard process and leave out how to handle possible failures or exceptions.

One alternative to these informal descriptions is to use formal process definitions, which can serve as the basis for a variety of analyses because these formal definitions offer precision in the representation of all possible ways that a process can be carried out in both standard and exceptional situations. Formal process definitions have not previously been used to describe and improve medical processes. The use of such formal definitions to prospectively identify potential error and improve the transfusion process has not previously been reported.

The purpose of this paper is to introduce the concept of formally defining processes and to describe how formal definitions of blood transfusion processes can be used to detect and correct transfusion process errors in ways not currently possible using existing quality improvement methods.

Bin Chen, George S. Avrunin, Lori A. Clarke, and Leon J. Osterweil. Automatic fault-tree derivation from Little-JIL process definitions. In Qing Wang, Dietmar Pfahl, David M. Raffo, and Paul Werinck, editors, Proceedings of SPW/ProSim 2006, number 3966 in LNCS, pages 150-158, Shanghai, May 2006. [ DOI | .pdf ]

Defects in safety critical processes can lead to accidents that result in harm to people or damage to property. Therefore, it is important to find ways to detect and remove defects from such processes. Earlier work has shown that Fault Tree Analysis (FTA) can be effective in detecting safety critical process defects. Unfortunately, it is difficult to build a comprehensive set of Fault Trees for a complex process, especially if this process is not completely well-defined. The Little-JIL process definition language has been shown to be effective for defining complex processes clearly and precisely at whatever level of granularity is desired. In this work, we present an algorithm for generating Fault Trees from Little-JIL process definitions. We demonstrate the value of this work by showing how FTA can identify safety defects in the process from which the Fault Trees were automatically derived.

George S. Avrunin, Lori A. Clarke, Elizabeth A. Henneman, and Leon J. Osterweil. Complex medical processes as context for embedded systems. ACM SIGBED Review, 3(4):9-14, 2006. [ DOI | .pdf ]

Many embedded systems are intended for use in complex and highly concurrent processes with multiple human agents. In these cases, the requirements for the system depend critically on the details of the process. If certification is to be useful for such systems, it must take the details of the pro- cess into account. In this paper, we describe some current research involving the formal definition and analysis of complex medical processes. We discuss the ways in which this work may provide a basis for a more complete understanding of the behavior of medical devices in the context of the processes in which they are used, and thus for certification methods for sophisticated embedded systems.

Lori A. Clarke, Yao Chen, George S. Avrunin, Bin Chen, Rachel Cobleigh, Kim Frederick, Elizabeth A. Henneman, and Leon J. Osterweil. Process programming to support medical safety. In Mingshu Li, Barry Boehm, and Leon J. Osterweil, editors, Unifying the Software Process Spectrum: International Software Process Workshop, SPW 2005, number 3840 in LNCS, pages 347-359, Beijing, May 2005. [ DOI | .pdf ]

Medical errors are now recognized as a major cause of untimely deaths or other adverse medical outcomes. To reduce the number of medical errors, the Medical Safety Project at the University of Massachusetts is exploring using a process programming language to define medical processes, a requirements elicitation framework for specifying important medical properties, and finite-state verification tools to evaluate whether the process definitions adhere to these properties. In this paper, we describe our experiences to date. Although our findings are preliminary, we have found that defining and evaluating processes helps to detect weaknesses in these processes and leads to improved medical processes definitions.

This file was generated by bibtex2html 1.97.

[Back] Back to George Avrunin's home page