Shumin Zhai


Shumin Zhai is an American-Canadian-Chinese human-computer interaction research scientist and inventor. He is known for his research specifically on input devices and interaction methods, swipe-gesture-based touchscreen keyboards, eye-tracking interfaces, and models of human performance in human-computer interaction. His studies have contributed to both foundational models and understandings of HCI and practical user interface designs and flagship products. He previously worked at IBM where he invented the ShapeWriter text entry method for smartphones, which is a predecessor to the modern Swype keyboard. Dr. Zhai's publications have won the ACM UIST Lasting Impact Award and the IEEE Computer Society Best Paper Award, among others, and he is most known for his research specifically on input devices and interaction methods, swipe-gesture-based touchscreen keyboards, eye-tracking interfaces, and models of human performance in human-computer interaction. Dr. Zhai is currently a Principal Scientist at Google where he leads and directs research, design, and development of human-device input methods and haptics systems.

Education

Born in Harbin, China in 1961, Dr. Zhai received his bachelor's degree in Electrical Engineering in 1982, and his master's degree in Computer Science in 1984 from Xidian University. After that, he served on the faculty of the Northwest Institute of Telecommunication Engineering in Xi'an, China where he taught and conducted research in computer control systems until 1989. In 1995, received his PhD degree in Human Factors Engineering at the University of Toronto.

Career

From 2001 to 2007, Dr. Zhai was a visiting adjunct professor in the Department of Computer and Information Science at Linköping University, where he also supervised graduate research.
He was a consultant at Autodesk in 1995 before joining IBM Almaden Research Center in 1996.
From 1996 to 2011, he worked at the IBM Almaden Research Center. In January 2007, he originated and led the SHARK/ShapeWriter project at IBM Research and a start-up company that pioneered the touchscreen word-gesture keyboard paradigm, filing the first patents of this paradigm, publishing the first generation of scientific papers. In 2010, ShapeWriter was acquired by Nuance Communications, and taken off the market. During his tenure at IBM, Dr. Zhai also worked with a team of engineers from IBM and IBM vendors to bring the ScrollPoint mouse from research to market, and received a CES award and millions of users.
From 2009 to 2015, Dr. Zhai was also the Editor-in-Chief of the ACM Transactions on Computer-Human Interaction. At the time he had been deeply involved in both the conference side and the journal side of publishing HCI research as an author, reviewer, editor, committee member, and papers chair.
From 2011 till present, Dr. Zhai has been working at Google as a Principal Scientist, where he leads and directs research, design, and development of human-device input methods and haptics systems. Specifically, he has led research and design of Google's keyboard products, Pixel phone haptics, and novel Google Assistant invocation methods. Notably, Dr. Zhai led the design of Active Edge, a headline feature of Google Pixel 2, which enables the user to reach Google Assistant faster and more intuitively using a gentle device squeeze rather than the touch screen.

Work

Dr. Zhai researches primarily in human-computer interaction, and is currently working on the research, design and development of manual and text input methods and haptics systems. Besides text input and haptics, his other research interests include system user interface design, human-performance modeling, multi-modal interaction, computer input devices and methods, and theories of human-computer interaction. He has published over 200 research papers and received 30 patents.

Word-gesture keyboard

In 2003, Dr. Zhai and Per Ola Kristensson proposed a method of speed-writing for pen-based computing, SHARK, which augments stylus keyboarding with shorthand gesturing. SHARK defines a shorthand symbol for each word according to its movement pattern on an optimized stylus keyboard. In 2004, they presented SHARK2 that increased recognition accuracy and relaxed precision requirements by using the shape and location of gestures in addition to context based language models. In doing so, Dr. Zhai and Kristensson delivered a paradigm of touch screen gesture typing as an efficient method for text entry that has continued to drive the development of mobile text entry across the industry. One of the most important rationales of gesture keyboards is facilitating transition from primarily visual-guidance drive letter-to-letter tracing to memory-recall driven gesturing. By releasing the first word-gesture keyboard in 2004 through IBM AlphaWorks and a top ranked iPhone app called ShapeWriter WritingPad in 2008, Dr. Zhai and his colleagues were able to facilitate this transition and brought the invention from the laboratory to real world users.

Laws and models of action

One of Dr. Zhai's main HCI research threads is Fitts’ law type of human performance models. From 1996, Dr. Zhai, alongside his colleagues, has pursued research on “Laws of Action” that attempted to carry the spirit of Fitts' law forward. In the HCI context, Fitts' law can be considered the “Law of Pointing”, while they believe there are other robust human performance regularities in action. The two new classes of action relevant to user interface design and evaluation that they have explored are crossing and steering.
Dr. Zhai started working on multiple degrees of freedom input during his graduate years at the University of Toronto. In his Ph.D. thesis, he systematically examined human performance as a function of design variations of a 6 DOF control device, such as control resistance, transfer function, muscle groups used, and display format. He investigated people's ability to coordinate multiple degrees of freedom, based on three ways of quantification: simultaneous time-on-target, error correlation, and efficiency.

Eye-tracking augmented user interfaces

Dr. Zhai has been involved in two applications about eye-tracking augmented user interfaces, MAGIC pointing and RealTourist.
In 1999, he worked together with his colleagues at IBM Almaden Research Center and published a paper Manual and gaze input cascaded pointing. This work explored a new direction in utilizing eye gaze for computer input, showing that the MAGIC pointing techniques might offer many advantages, including less physical effort and fatigue than traditional manual pointing, greater accuracy and naturalness than traditional gaze pointing, and possibly faster speed than manual pointing.
In 2005, he developed and studied an experimental system, RealTourist, with Pernilla Qvarfordt and David Beymer. RealTourist lets a user to plan a conference trip with the help of a remote tourist consultant who could view the tourist's eye-gaze superimposed onto a shared map. Data collected from the experiment were analyzed in conjunction with literature review on speech and eye-gaze patterns. This inspective, exploratory research identified various functions of gaze-overlay on shared spatial material including: accurate and direct display of partner's eye-gaze, implicit deictic referencing, interest detection, common focus and topic switching, increased redundancy and ambiguity reduction, and an increase of assurance, confidence, and understanding. This study identified patterns that can serve as a basis for designing multimodal human-computer dialogue systems with eye-gaze locus as a contributing channel, and investigated how computer-mediated communication can be supported by the display of the partner's eye-gaze.

FonePal

FonePal is a system developed to improve the experience of accessing call centers or help desks. Known as "touchtone hell", voice menu navigation has long been recognized as a frustrating user experience due to the nature of voice presentation. In contrast, FonePal allows a user to scan and select from a visual menu at the user's own pace, typically much faster than waiting for the voice menus to be spoken. FonePal uses the Internet infrastructure, specifically Instant Messaging, to deliver a visual menu on a nearby computer screen simultaneously with the voice menu over the phone.
In 2005 and 2006, Dr. Zhai and his colleague Min Yin at IBM Almaden Research Center published two papers about this project. Their study shows that FonePal enables easier navigation of IVR phone tree, higher navigation speed, less routing error and greater satisfaction. FonePal can also seamlessly bridge the caller to a searchable web knowledge base, promoting relevant self-help and reducing call center operation cost.

Awards and honors

Dr.Zhai is a Fellow of the Association for Computing Machinery and a Member of the CHI Academy. He has received many awards and honors. Among them: