Search
  • David Shapiro

Measuring Usability


When dealing with the implementation of any new system within an organization, it is usually management and senior staff who will be making the final decisions. This is of course understandable since it is not possible for everyone in the organization to have their say. However, in the next few lines we will see why it is so important to make the users of a new system an essential part of the decision-making process.

​​​Disclaimer: Before going any further, it is important to make very clear that getting users involved in the decision making process can have devastating consequences. The suggestions made in this article are being made only after management has laid down some very clear rules to any users participating in the procurement process, including:


  • Clearly stating that the purpose for the users’ participation is solely to evaluate usability and suitability of the new system.

  • Clearly outlining to participating users the corporate goals behind introducing the new technology in the first place.

  • Ensuring that participating users are fully aware that the introduction of any new technology will most probably mean changes to working processes and workflows.

If you feel that your users would not be able to accept any of the above terms, or that such a process would not be feasible within your organization, then this article is not for you.

Don't Judge by the Cover (or what the Vendor has told you)

When you select a new system you want to ensure that it not only has all the necessary features, but that it is also easy to use. In any market in general, including the broadcast business management systems market that I specialize in, there are some very feature-rich systems, but it is not until the system starts getting used that you find some unexpected deficiencies. Although it does everything that was promised and more, simple straight forward, day-to-day tasks take longer to perform than the users would have expected.

Think of it like buying a new car that has all the latest features; Heated leather seats, built-in GPS, reverse camera, state-of-the-arts sound system, the works. You walk into the dealer to pick up your new car, excited with your new purchase, dreaming of the many enjoyable hours of driving that lie ahead. However, it is only when you actually drive out of the showroom do you realize that the car’s suspension is far from good, fuel consumption is very high and the legroom is far from generous. As the car owner, you would now be very disappointed and frustrated with your new purchase.

So too will users of a newly acquired system that does not give them the ‘smooth ride’ they had would expect. Furthermore, when you sum up the time spent by multiple users performing simple tasks that instead of taking 1-2 minutes, are now taking 3-4 minutes, the extra time translates into a lot of money, leading to frustration not only on the part of the users, but also on management who will feel that perhaps their expensive investment was not so worthwhile after all.

So how to you ensure that you get the shiny features AND a smooth ride?

The future users of a system should be given the chance to contribute their input when deciding on which system to choose. Following the selection of a short list of systems for consideration, handpicked users and super users, who you feel have the capability, should be involved in the process of final selection of a system. If you have gone through a tender process, leaving the final choice to ONLY management and senior staff will not necessarily ensure that the best system has been selected.

Measuring Usability

During the tender process, one of the things most difficult to measure is the usability of the systems you are evaluating. There are several reasons for this:

  1. At this stage, you will not have a local installation of the system. It is therefore not possible for the users to get an accurate feel of the systems. Yes – once you have selected a vendor you will be able to request a 3-month pilot – but it is not until then that the users will actually be in a good position to test drive the system.

  2. When evaluating the various systems, the main emphasis will be placed on the functionality checklist that you will have prepared ahead of time, and less attention is paid to how that functionality was achieved through the GUI (graphical user interface). Even when the system is being displayed to you by the respective vendors, and you are ticking off your long list of items on your wish list, you will never be able to evaluate the usability because the presenter, who is fluent in the system he is running for you, will always be able to make everything look so easy.

  3. You will have sat through several of these presentations, and although you can compare functionality between the different systems you are looking at, it is very difficult to measure and compare usability. On top of this, as the presentation drags on during the course of the day, you become more and more pressured to get through your function checklist.

Engaging the Users

So how exactly can you measure the usability of the system, and how can the users assist in this process?

The users will already have contributed to the requirements during the requirement gathering process. They will have been interviewed, and possibly their workflows and processes would have been properly documented; but their role should not end there. The users are, in the final analysis, the ones who will be using the system. They need to feel comfortable with the product on which a lot of money has just been spent. I have seen entire projects fail, not because the system did not do what was promised, but because the users were not happy with how it did it. This is a scenario you want to avoid.

It will therefore be sensible to request senior users to participate in the presentations of the selected, short listed vendors. I specifically say senior ‘users’, and not their bosses, because management will have been focusing more on features and the checklist prepared during the RFP process, whereas in contrast, the users’ role will be more focused on usability and how the system gets the job done.

Before attending the presentation sessions, the users should make a list of the key work processes. These should be recorded on a high level, without going into too much detail. From experience, users who are not necessarily technical will tend to get into the minor details (e.g. John will receive a tape and will prepare a bar-code, and then will send an email to Jane who will fill out a ‘New Tape Form’, etc., instead of : Tape Received, New Tape Form completed….).

On top of the current workflows, the users should also try to suggest ways in which the processes may be improved. From experience, this is easier said than done because users tend to get set in their ways. However, encouraging them to think of process improvements has several, very important benefits:

  1. New ideas are presented to the vendor during the evaluation process, adding this to the list of issues that will need to be discussed during the gap analysis.

  2. This will help the users to think outside the box, which is very important when implementing a new system. Users need to be open to new ideas and working methods and by receiving support from management for this initiative will help contribute to the success of the project.

  3. Such encouragement to the users will induce creative thinking on their side.

Thinking outside the box

Although we may want to, you cannot expect users to think outside the box. However, using the right people, and by using the right methods, extra effort should be made to encourage the users to travel the extra mile. You could arrange workshops, brainstorming sessions, bring in external process consultants. Whatever you do, the effort and investment will be very worthwhile in the long run.

Once the vendors have complete

d their presentations, the users’ thoughts and impressions should be carefully heard and given careful consideration. As mentioned previously, it is the users who will be using the system, and the success of the project lies not only with the efficiency of any new solution that is introduced, but also very much so on the willingness of the users to accept change and ensuring that they feel comfortable with the newly implemented system. Making them a vital part of the decision making process should contribute greatly to the success of a project.


0 views0 comments