Question

Wanted by the Police: A Good Interface

By KATIE HAFNER NOV. 11, 2004

From: The New York Times, Technology section, not-for profit classroom used.

SAN JOSE, Calif. - SAN JOSE has a reputation as one of the safest large cities in the nation, with the fewest police officers per capita.

Yet a number of the 1,000 officers in this city of 925,000 in the heart of Silicon Valley have been worrying about their own safety of late. Since June, the police department has been using a new mobile dispatch system that includes a Windows-based touch-screen computer in every patrol car. But officers have said the system is so complex and difficult to use that it is jeopardizing their ability to do their jobs.

Officers complain that routine tasks are so difficult to perform that they are discouraged from doing them. And they say that the most vital safety feature -- a "call for assistance" command that officers use when they are in danger -- is needlessly complicated.

"Do you think if you're hunkered down and someone's shooting at you in your car, you're going to be able to sit there and look for Control or Alt or Function?" said Sgt. Don DeMers, president of the San Jose Police Officers' Association, the local union and the most vocal opponent of the new system. "No, you're going to look for the red button."

Officers also say they were not consulted about the design of the user interface -- how information is presented and how commands are executed using on-screen and keyboard buttons. Many have said they wish the department had retained and upgraded the old system, in place since 1990.

Such complaints have a familiar ring. Anyone who encounters technology daily -- that is to say, just about everyone -- has a story of new hardware or software, at work or at home, that is poorly designed, hard to use and seemingly worse than what it was intended to replace. Yet because the safety of police officers and the public is involved, the problems in San Jose are of particular concern.

At the heart of the dispute is the question of how much the technology itself is to blame, how much is a training problem and how much can be attributed to the predictable pains associated with learning something new.

Any new technology, whether it is a microwave oven or the controls of a Boeing 777, has a learning curve. And often the user interface, the all-important gateway between person and machine, is a dizzying array of buttons or keys that have to be used in combinations. It can take weeks, sometimes months, of training and adapting for people to become comfortable with a new system.

Police department officials in San Jose have acknowledged that the off-the-shelf system, which cost $4.7 million, has had some bugs, yet they say the software vendor, Intergraph Corporation, of Huntsville, Ala., has fixed many of them.

"The city and Intergraph have worked together to iron out the software and work-flow issues that sometimes accompany the introduction of a new system," said Alice Dilbeck, vice president for customer services at Intergraph.

And at public safety agencies elsewhere in the country where similar software has been introduced, employees have eventually grown used to the new technology.

Still, questions and complaints remain, not only among patrol officers but among dispatchers who say that with the new system, unlike the old, they are unable to perform several tasks at one time.

With the system, officers in the field can receive orders, send messages, write reports, call up maps of the city and, using the Global Positioning System, see not only where they are but where other patrol cars are at any given time.

When first installed, the system was unstable. A day or two after the new system went into operation, it crashed, and for several days it was periodically down. "That didn't engender a lot of trust," said Sergeant DeMers of the police union.

Ms. Dilbeck acknowledged, "That was a really bad start."

When the system was running again, a number of bugs were discovered, said Aaron Marcus, president of Aaron Marcus & Associates, a user-interface design consulting firm in Berkeley, Calif., that studied the new system at the request of the union.

Some of the map information, it turned out, was inaccurate, screens were cluttered with unnecessary information, the on-screen type was difficult to read and officers could not easily perform one of the most basic tasks -- the license-plate check.

"This is almost a casebook study of what not to do and how to do it wrong," Mr. Marcus said.

Perhaps the biggest misstep of all, Mr. Marcus said, was that the officers themselves were not consulted beforehand, especially when it came to the design of the interface.

Jakob Nielsen, a principal of the Nielsen Norman Group, a technology consulting company in Fremont, Calif., agreed.

"It's a prescription for disaster to develop a big system without testing it with users before it's launched," Mr. Nielsen said. "There are always issues in the user interface that need to be smoothed over."

The San Jose police chief, Rob Davis, said that those who were in charge of planning for the new system "have reviewed it and in retrospect would probably agree that if they had involved more of the end users during the planning phase it would have made the rollout easier."

Since the complaints first arose, Intergraph has fixed bugs and streamlined some of the more cumbersome tasks such as the license plate checks. Ms. Dilbeck and others have spent weeks at a time in San Jose working to eliminate bugs.

"I'm getting very good feedback about the upgrades," Chief Davis said.

Part of the problem stemmed from the fact that the San Jose officers had grown used to the city's 14-year-old text-based system.

"It's a debatable issue as to whether you should fix the old or go for a new paradigm," Mr. Marcus said, "because the old software wasn't off the shelf, it was customized."

The amount of training that was initially given to officers, three hours, was considered by many to be inadequate. "You expect our officers to be able to operate in life-and-death situations with three hours' training?" Sergeant DeMers said.

Sgt. Thomas Navin, supervisor of the department's systems unit and the person who has been most responsible for training on the mobile systems, acknowledged that training was "bare-bones basic." Additional training has since been offered.

But the fact that the system is based on Windows complicated the issue, since not everyone was familiar with pull-down menus and other basic features. "There are people who are Windows savvy and those who aren't," Sergeant Navin said. Officers in their late 50's and early 60's tend to resist the new system more than younger officers do.

Also, officers were trained on desktop computers with track pads on the keyboards, not the touch screens they would eventually be using.

Another point of controversy was the red Code 99 command, used when an officer is in danger and needs help. Originally the system had one key for Code 99, but it was later changed to a two-key combination because the single button code was resulting in too many false alarms. Now it is the two-key method that prompts some complaints.

Over all, the new system is an improvement over the old, some department officials say, in part because it contains a mapping feature based on global positioning data provided by the city. But the maps contain errors, Sergeant DeMers said.

Officers say they are being distracted by the tasks they are expected to perform on the new system when their full attention should be given to what is happening outside the patrol car. Sergeant DeMers said one officer recently was so distracted by what he was doing on the 12-inch touch screen that he crashed into a parked car.

During a recent tour of the system with a reporter in the passenger's seat, Sergeant Navin typed in a message to another officer, Sgt. David Bacigalupi, asking his opinion of the Intergraph system. "You can't print what I think," came the officer's response.

Later, as Sergeant Navin drove through the streets of San Jose, his taps on the screen inevitably led to some swerving, inadvertently bearing out his colleagues' claims -- even though he was clearly well versed with the ins and outs of the system.

"In practical reality, especially when responding to an emergency call, they have to do some of these things while en route," Mr. Marcus said.

The fact that the officers and police dispatchers were not consulted about their preferences and requirements has come back to haunt the city. In July, the union asked for meetings to discuss the new system, saying it was having an adverse impact on officer safety. "Legally, they can't just implement something like this unilaterally," said John Tennant, general counsel for the union.

Even after some extensive tweaking, there still seem to be some fundamental bugs, Mr. Marcus said. "Much of the design was incorporating a Windows desktop graphical user interface with complex menu hierarchies, which just doesn't make sense in a vehicle."

Dispatchers have been similarly unhappy, citing delays with the new system that could endanger officers.

It takes longer to give officers information about the prior arrest record of someone they have just caught, said Melissa Albrecht, a San Jose dispatcher for 15 years. "Does that two extra minutes make a difference when they're standing there with a felon?" she asked. "It could." In September, Ms. Albrecht sent a six-page memorandum to the police chief listing her concerns.

She credits Intergraph with many improvements. But the system still does not allow dispatchers to perform several tasks simultaneously, and this causes delays. "What they keep throwing at us is that the system works as designed, and my question for them is, 'Does this design work for us?"' she said.

For perspective the San Jose department might do well to borrow a page from a city to the south.

The San Diego Sheriff's Department has had the Intergraph touch-screen system in place for six years, and although there were bugs and resistance at first, the kinks have been ironed out and the deputies are now used to it.

"Some of our people had never done anything with a computer," said Hanan Harb, who manages the department's dispatch center. "We had to do basic Windows training, and it's hard to make that leap if you're not computer literate to begin with. It's a big learning curve." Now that people have grown used to it, and now that this is what they know, "it's very easy for them," Ms. Harb said.

Dr. Nielsen said the Chicago Police Department had similar problems in 1999 when it rolled out an ambitious computer system without having tested it with on-the-beat police officers first.

"Chicago learned its lesson and now has a much better system, developed with user involvement," Dr. Nielsen said. "It's sad that the San Jose Police Department had to learn the same lesson all over again. Those who don't know history are doomed to repeat it."

1. Why was the system found to be too complex and difficult to use?

2. Identify the first point at which you think the project began to be headed for trouble. What would do you think should have been done differently at this point?

3. Based on our discussion of the process of interactive design, and our identification of the steps (establishing requirements, designing alternatives, prototyping, and evaluation), what step or steps were neglected or poorly done?

4. Provide at least three examples of where the Eight Golden Rules were not followed.

5. This case study is from a system built some time ago. Can you identify a system since 2005 which has done as poorly as this system, or been far more successful than the case study presented? Identify your system, indicate if you are providing an example of inadequate or superior interface design and provide three examples of how the system you have selected supports specific golden rules.

Answer 1

Ans 1) The system was found to be too complex and difficult to use because:

i) the officers were not familiar with the new mobile dispatch system that used windows base touch screen system

ii) officers needed proper time and training to be acquainted with the on-screen and keyboard buttons.

iii) they were searching for shortcuts rather than juggling with information overload or a set of commands to persecute simple and regular tasks that were used frequently, for example- call for assistance command.

Ans 2) The very first point were the project began to be headed for the trouble was the instability of the system with its launch. Even after the system began to operate, it crashed again and remained down for several days.

Ans 3) This issue described in the above answer should have been tackled differently. All the bugs should have been fixed before the re-launch and before beginning the operations.

Ans 4) The important steps that were neglected or poorly designed while considering our discussion of the process of interactive design, and our identification of the steps (establishing requirements, designing alternatives, prototyping, and evaluation) have been discussed as below:

i) not involving officers during the design process as they were the end users

ii) giving a very short period of training to the officers to be adjustable with the new technology without acknowledging their level of friendliness with the technology and the crisis of situation

iii) not checking for the bugs before the full operationality of the system launch.

Ans 5) The three examples were the eight golden rules were not followed in the above case study are discussed as below:

i) not striving for consistency: shift from touch-screen based systems to windows based touch screen computers was a major change for officers especially for those who were not computer literates.

ii) Inability of frequent users to use shortcuts:

the most vital safety feature -- a "call for assistance" command that officers asked for a lot of commands to operate rather than doing it with a simple red button.

iii) Inability to reduce short term memory load: Displays were full of a lot of commands and not consolidated.

iv) feedback: after the system crashed on its launch and went on to be down again after coming in operations, it was still digonised with a lot of bugs.

Ans 6) Google Wave (2009) is an example of such failure because they forgot about user experience much like the example presented in the above case study. The inadequate system that we have taken as an example here, also failed because its utility was not clear to the common user.

The three ways it followed 8 golden rules are :

i) internal control: google had a locus command.

ii) reduced short term memory load: display was interactive and inviting.

iii) easy reversal of activities: users could back.