Usability Testing

- Evaluation & redesign
of the BBC Sport website - 

- Usability evaluation & redesign
of the BBC Sport website - 

'Evaluate the usability of a website through user testing, to establish issues with the current design and use these to create an improved one'

Methodology and Data Collection

The evaluation of the site first required identifying a sample of representative users, who were then asked to undertake three separate tasks that covered different areas of BBC Sport’s functionality. Performance data (time take & errors made), user feedback and satisfaction ratings were then collected, along with eye tracking images and a brief mobile test, to provide a clear basis of comparison between the two designs.



Improvements and Redesign

The data and feedback received highlighted a number of areas for improvement; both in the context of the tasks, and more general changes to page layouts. These changes were implemented, before retesting the site with a new set of participants.

Results

Despite two further areas of improvement being identified within the redesigned, it did perform better in the usability evaluation overall; producing quicker completion times, fewer errors and higher satisfaction ratings; indicating improved usability in comparison with the original site. 

'Evaluate the usability of a website through user testing, to establish issues with the current design and use these to create an improved one'

Methodology and Data Collection

The evaluation of the site first required identifying a sample of representative users, who were then asked to undertake three separate tasks that covered different areas of BBC Sport’s functionality. Performance data (time take & errors made), user feedback and satisfaction ratings were then collected, along with eye tracking images and a brief mobile test, to provide a clear basis of comparison between the two designs.

Improvements and Redesign

The data and feedback received highlighted a number of areas for improvement; both in the context of the tasks, and more general changes to page layouts. These changes were implemented, before retesting the site with a new set of participants.

Results

Despite two further areas of improvement being identified within the redesigned, it did perform better in the usability evaluation overall; producing quicker completion times, fewer errors and higher satisfaction ratings; indicating improved usability in comparison with the original site. 

project summary picture

THE PROCESS

The Website 

BBC Sport is a site that provides “breaking news, results, videos, audio and analysis on football, cricket, rugby, golf, tennis, F1 and all the main world sports” (BBC, 2017). It is a relatively complex website with a huge range and depth of content, and users of varying ages, backgrounds and levels of internet experience.

The Website 

BBC Sport is a site that provides “breaking news, results, videos, audio and analysis on football, cricket, rugby, golf, tennis, F1 and all the main world sports” (BBC, 2017). It is a relatively complex website with a huge range and depth of content, and users of varying ages, backgrounds and levels of internet experience.

Screen Shot 2018-02-22 at 14.48.05

METHODOLOGY 

To carry out the evaluation of the site we had to test it with a sample of representative users, who were asked to undertake three separate tasks within the sites functionality - from which a variety of data points would be collected.

Participant Sampling

The initial evaluation procedure consisted of 8 participants, ranging from 21-25 years old, who were studying a variety of different subjects at university and were all experienced in using technology and web browsing, along with having at least a decent level of knowledge about the websites topic focus. A further 8 participants were then used to evaluate the website redesign, to eliminate any bias. 

All participants were selected using purposive sampling; a "non-probability sample selected based on characteristics of a population and the objective of the study" (ThoughtCo,2017), to ensure they were representative of the target users of the website. 

Participant Sampling

The initial evaluation procedure consisted of 8 participants, ranging from 21-25 years old, who were studying a variety of different subjects at university and were all experienced in using technology and web browsing, along with having at least a decent level of knowledge about the websites topic focus. A further 8 participants were then used to evaluate the website redesign, to eliminate any bias. 

All participants were selected using purposive sampling; a "non-probability sample selected based on characteristics of a population and the objective of the study" (ThoughtCo, 2017), to ensure they were representative of the target users of the website. 

list of participants

The Three Tasks 

To measure the websites usability, and then effectively compare the redesign to the original, some control tasks were needed that the user could complete on both designs. The tasks were chosen to explore BBC Sports different features and content, and were as follows:

Task 1: Find Football Fixtures

"You've decided to watch an Ipswich Town football match, but you’re unsure when they are next playing at home. Please check their fixtures to find out when their next league home match is, who they’re playing and what time it kicks off"

Task 2: Get Inspired

"Having followed Team GB’s success in the women’s hockey at the Rio 2016 Olympics, you have been inspired to take up the sport. Please use the BBC Sport ‘Get Inspired’ service to search for the next hockey session taking place near you, and find the contact number for the organiser"

Task 3: Customise My Sport

"You haven’t updated your 'My Sport' page in a while. Please log in, go to your page and add a new sport. Once you have done this please read the most recent story published about the sport. You will be provided with an email address and password with which to log in"

DATA COLLECTION 

In order to provide a basis for comparison between the two designs, performance data, user feedback and satisfaction ratings were collected along with eye tracking data for one participant and a brief mobile test. 

Performance Data

The first type of data collected during this study were three raw values relating to how well the users could complete each task:

  • Time taken to complete the task
  • Errors made during this time
  • Any help or corrections from the investigator that required

This objective data would then be used to provide the initial focus of comparison between the original site and its redesigned prototype, showing how well the user completed each task rather than their interpretation of how they did; as the data isn't influenced by personal feelings, interpretations, or prejudice; but based on facts.

Participant Feedback

A series of questions were also produced to generate some more subjective data; focusing on the user experience and how each participant felt completing each task. The aim of these were to allow the user to express: 

  • Parts of the website they particularly liked or disliked.
  • How easy they felt each of the tasks were and why. 
  • Any suggestions they had on how to improve the site.

Each user was asked the following questions after completing each task:

  • How did you find the task? And why?
  • How did you find how the information was laid out? And why?
  • What did you like about the website on this task?
  • How would you improve the website on this task?
  • What did you like about the website on this task?
  • Any additional comments?

Satisfaction Ratings

In addition to the interview questions, a Linkert scale was used to measure the user’s attitudes to the website in the context of the tasks, and the website as a whole. They were given statements and asked to give a rating of 1-5 to show whether they agreed or disagreed with them:

1 = Strongly Disagree       2 = Disagree       3 = Neutral       4 = Agree       5 = Strongly Agree

This type of scale was chosen as it assumes that the strength or intensity of experience is linear; on a continuum from strongly agree to strongly disagree, and this measurement provides further basis for comparison of how usable the participants feel each website design is, when completing each task. The participants were given four statements to answer:

"I found the task easy to complete"            "I found the site easy to navigate"
 

            "I like the layout of the site"            "I found the site visually appealing"

Satisfaction Ratings

In addition to the interview questions, a Linkert scale was used to measure the user’s attitudes to the website in the context of the tasks, and the website as a whole. They were given statements and asked to give a rating of 1-5 to show whether they agreed or disagreed with them:

1 = Strongly Disagree       2 = Disagree       3 = Neutral       4 = Agree       5 = Strongly Agree

This type of scale was chosen as it assumes that the strength or intensity of experience is linear; on a continuum from strongly agree to strongly disagree, and this measurement provides further basis for comparison of how usable the participants feel each website design is, when completing each task. The participants were given four statements to answer:

"I found the task easy to complete"            "I found the site easy to navigate"
 

            "I like the layout of the site"            "I found the site visually appealing"

Eye Tracking

Eye tracking data was generated for one participant when completing each of the three tasks, with the aim of providing a more visual representation of how the user reacts to specific parts of the site as they use it. This information helped inform the redesign process; showing particular focus points or confusing areas at various stages of each task and giving clarity as to why participants might have difficulty performing them

Mobile Test 

I carried out a short evaluation to establish how well the website works on a mobile device, this was again completed in the context of the three tasks set out at the beginning of the study and used the rating scales and interview questions to give feedback.

EVALUATION RESULTS  

Both the performance data and satisfaction ratings were put into graphs for easy comparison, with the interview responses and eye tracking data being used to understand and explain the raw data.   

Task 1 

Completed with ease by all the participants with only two very minor errors made; reflected by the rating scales generating scores of mainly 4 or 5. This, combined with the relatively high time data ; an average of 44.5 seconds, indicated that the current process might take longer than it should

“It feels like there’s a lot of clicking involved and it could be quicker.” 

- Sam, Stage 1 Participant

task 1 – time taken
task 1 – errors made
task 1 – satisfaction

There were three main problems highlighted by the participants in this task;

  • Not knowing which league a team's in
  • Display of football fixtures
  • Home page search bar

Not knowing which league the team plays in causes difficulty:

“I would have liked to see the teams in the league under the league name”

Felt that the ease of completing the task relied on knowing which league Ipswich are in beforehand.

Not knowing which league the team plays in causes difficulty:

“I would have liked to see the teams in the league under the league name”

Felt that the ease of completing the task relied on knowing which league Ipswich are in beforehand.

The fixtures for each team were displayed in long list form:

“The team names all blend into one big list that’s much harder to read”

The layout makes it much harder to find the specific fixture they were looking for. 

The fixtures for each team were displayed in long list form:

“The team names all blend into one big list that’s much harder to read”

The layout makes it much harder to find the specific fixture they were looking for. 

The home page search bar gives too much information:

“It doesn't show me what I’m actually looking for; a link to the team page”

Didnt like the fact that it gave results from the whole of the BBC, rather than just Sport.

The home page search bar gives too much information:

It doesn't show me what I’m actually looking for; a link to the team page”

Didnt like the fact that it gave results from the whole of the BBC, rather than just Sport.

The eye tracking evaluation showed how the user scanned across the page when looking for the desired fixture, and that there was nothing that particularly grabbed their attention. They also didn’t see the target fixture when looking through all the information on this page.

Task 2

All the participants struggled with task 2 in some way, and this was reflected by the high number of errors made and help given; 21 in total, paired with the very negative rating scale scores being mainly 2’s and 1’s. The difficulty was also shown in the high time data; no users were able to complete the task in under 1 minute, the average time being 159.5 seconds.

“The Get Inspired section was quite hard to find initially, and once in it wasn’t immediately obvious that you use categories to select the target sport ”

- John, Stage 1 Participant

task 2 – time taken
task 2 – errors made
task 2 – satisfaction ratings

Again, the issues with this task could be refined down to three main areas:

1. Struggled to find the 'Get Inspired' section on the homepage:

“The link was tucked away at the bottom of the screen and hard to find”

Either required help, or previous knowledge of where to look, to find what they were looking for. 

1. Struggled to find the 'Get Inspired' section on the homepage:

“The link was tucked away at the bottom of the screen and hard to find”

Either required help, or previous knowledge of where to look, to find what they were looking for. 

2. Filters not prominent enough on the page, and no search feature:

“Why can't I just search for the sport? It would make the page far clearer”

Didn't realise they could filter the search initially, and were then confused by the categories menu.

2. Filters not prominent enough on the page, and no search feature:

“Why can't I just search for the sport? It would make the page far clearer”

Didn't realise they could filter the search initially, and were then confused by the categories menu.

3. Unnecessary duplicate activity sessions being displayed:

“There's only 2 different sessions on the first two pages, it's confusing”

Made finding a suitable activity session really time consuming, looking through multiple pages.

3. Unnecessary duplicate activity sessions being displayed:

“There's only 2 different sessions on the first two pages, it's confusing”

Made finding a suitable activity session really time consuming, looking through multiple pages.

The eye tracking results for this task showed how the very busy and content heavy ‘Get Inspired’ page meant the users focus jumped around a lot and was never really focused on the areas of the page that would help them complete the task. This is seen further by the main eye focus being on the different activity panels, rather than the filters on the right-hand side that the participant needed to refine their search and complete the task.

Task 3

Similarly, to task 1, all the participants completed task 3 without any real difficulty or errors, but the average time taken was 84.8 seconds; with only one user completing the task in under 1 minute. In addition, satisfaction scores for ease of completion were almost all 4’s, showing that users found the task easy to complete. Again, this indicates steps could be taken to speed up the task:

“It was easy enough to find and add the sport, but I wasn’t sure if it had
actually been added so I had to go back and check.”

- Tom, Stage 1 Participant

task 3 – time taken
task 3 – errors made
task 3 – satisfaction ratings

The issues raised here focused more on layout than the actual process:

1. No discernible order to selections, didn't make sense:

“I thought it would be in alphabetical order or most recent first”

Some form of order on the page would make it much easy to navigate and complete the task. 

1. No discernible order to selections, didn't make sense:

“I thought it would be in alphabetical order or most recent first”

Some form of order on the page would make it much easy to navigate and complete the task. 

2. Tab system didn't update with the 'My Sport' page selections:

“It would make more sense to add it to the tabs at the top of the page”

Felt that this feature would make navigation through the page to specific selections much easier. 

2. Tab system didn't update with the 'My Sport' page selections:

“It would make more sense to add it to the tabs at the top of the page”

Felt that this feature would make navigation through the page to specific selections much easier. 

3. The titles of each section weren't clear enough:

“The sections blend into one another, they aren't separate enough”

Sections need to be separated more through the visual design and layout of the page. 

3. The titles of each section weren't clear enough:

“The sections blend into one another, they aren't separate enough”

Sections need to be separated more through the visual design and layout of the page. 

General Feedback and Mobile Test 

When asked about the site in general, in terms of layout, ease of navigation and visual appeal, participants felt like the various pages could be a little more in keeping with one another.

“Why are some of the pages narrower? It feels like the page is too cluttered” 

General Feedback and Mobile Test 

When asked about the site in general, in terms of layout, ease of navigation and visual appeal, participants felt like the various pages could be a little more in keeping with one another.

“Why are some of the pages narrower? It feels like the page is too cluttered” 

- Rory, Stage 1 Participant

This issue, combined with the low visual appeal scores of mainly 2’s, showed users wanted more consistency between pages. These layout issues also reflected rating scores of 3’s and 4’s; acceptable but room for improvement. The navigation scores of 1’s and 2’s were almost entirely due to task 2; where all the participants struggled the most; a primary area for improvement.

general – layout
general – ease of navigation
general – visual appeal

On the mobile site it was very simple completing task 1 and 3, reflecting the satisfaction ratings of 4 for each and the quicker than average times. Task 2 again reflected the feedback given by participants on the normal site, but was even harder to complete due to the inability to select a specific sport on the get inspired page. The general layout was similar, but a lack of tabs on the main bar slowed down each task in comparison to the desktop site.

REDESIGNING THE WEBSITE

The evaluation results and feedback I received from the participants highlighted a number of areas for improvement within the website; both in the context of the tasks, and more general changes focusing on page layout.  

Improvements Identif i e d 

The analysis of the results above generated a series of focus points for the website redesign to improve its usability and enhance the users experience when completing the three tasks. The chosen improvements were:

  • BBC Sport specific search bar on home page that also gives better search results
  • Make ‘Get Inspired’ easier to access from home page
  • Clearer display of fixtures to make it easier to find a specific team within a specific league
  • Clearer and more structured layout on ‘Get Inspired’ page, accentuating the filters
  • More effective method of finding and choosing the desired sport on ‘Get Inspired’ page
  • More diverse range of activity results with no repetitions on ‘Get Inspired’ page
  • ‘Get inspired’ more in keeping with the rest of the site
  • More organised layout of 'My Sport' page and individual sections
  • Updating sub-tabs depending on 'My Sport' page selections
  • BBC Sport specific search bar on home page that also gives better search results
  • Make ‘Get Inspired’ easier to access from home page
  • Clearer display of fixtures to make it easier to find a specific team within a specific league
  • Clearer and more structured layout on ‘Get Inspired’ page, accentuating the filters
  • More effective method of finding and choosing the desired sport on ‘Get Inspired’ page
  • More diverse range of activity results with no repetitions on ‘Get Inspired’ page
  • ‘Get inspired’ more in keeping with the rest of the site
  • More organised layout of 'My Sport' page and individual sections
  • Updating sub-tabs depending on 'My Sport' page selections

DESIGN CHANGES

General Site

In terms of the general site, there were only minor changes made but these focused on a BBC sport only search bar, moved down into the main menu area to make it easier to see. In addition to this, a get inspired tab was added to the menu to make this feature easier to find.



Football Fixture Page

fixture page redesign

The football fixture page was redesigned to make it easier to differentiate between leagues as the user doesn’t have to look through a large list to find what they’re looking for. The week’s fixtures for each league are displayed, with buttons at the bottom that load more fixtures in that league. The fixtures on the individual team page were also redesigned to combine fixtures, results and table in one clear and organised page.



'Get Inspired' Page

get inspired page redesign

The layout of the get inspired page was changed to make it clearer and accentuate the various filters the user can refine their search with. The results are displayed in a simpler and more spaced out way, and the page generally looks more in keeping with the rest of the site. A search bar to select the sport saves time and is more effective than the previous dropdown menu.



'Get Inspired' Activity Page

hockey session page redesign

The activity pages were also redesigned to change the layout and present the information in a clearer and more succinct way. Multiple dates of the same club were combined and presented in list form as opposed to having multiple results displayed on the previous page



My Sport Page

my sport page redesign

The My Sport page layout was updated to show more results, retaining the main story on the left-hand side. A sub-tab list at the top in black, that updates as users add new selections to their page, allows for quick and easy navigation to each section without having to scroll down the page.

REDESIGN EVALUATION RESULTS 

Once the design changes had been made, the redesigned website was tested using the same procedure as the first testing process; the only difference being that the prototype was tested using PowerPoint, rather than being a live website. 

Task 1

Similarly to the original website, there were very few mistakes made during task 1, however a vast improvement on performance can be seen by the drop in completion times; an average time of 19.4 seconds; down from 44.5 seconds in the first evaluation and no participants taking longer than 25 seconds.

redesign task 1 – time taken
redesign task 1 – errors made

In addition, the satisfaction results increased to all 5’s, indicating the users felt the redesign made the task easier to complete. This was supported by positive feedback on the effectiveness of the search bar and its search content:

“I like how it gives multiple suggestions, like fixtures and results, from just the team name - rather than having to search for them after finding the team"

 

- Oli, Stage 2 Participant

The alternative method of completing the task, going through the football fixtures tab as opposed to the search bar, was also seen as an improvement. The individual boxes that separate each league, and show the next fixtures within that league, also made it easier for the user to find the match they’re looking for quickly and efficiently.

redesign task 1 – satisfaction

Task 2

Task 2 showed the biggest improvement between the original site and the redesign, with errors dropping from 21 in the original test, to 4 for the redesign, suggesting that the redesign was far clearer and less confusing. The time data supported this further, as the average time decreased from 159.5 seconds to 36.25 seconds.

redesign task 2 – time taken
redesign task 2 – errors made

When combining these two with satisfaction ratings that increased to all 4’s and 5’s there was a clear improvement in the site’s usability with this task in mind. The user comments and feedback during the second evaluation supported this. All participants noted how easy the get inspired page was to find; an issue that was prominent in the initial website evaluation.

redesign task 2 – satisfaction

In comparison to the original website, the filters were used much more effectively. All four of the errors made were the same one; selecting the wrong hockey session, something the participants felt t could be solved by “allowing results to be filtered by ability level or age/gender”.

 “The filters make it really easy to find what you’re looking for”

- Tom, Stage 2 Participant

On the hockey session page, the key information was laid out in a much clearer and more succinct way without the needed to scroll down the page to find it; “the page had everything they needed to know pretty much straight away - and this would have again contributed to the improved user performances on this task.

Task 3

Similarly to the first two tasks, the average time taken decreased with the redesign; from 84.8 seconds to 42.4 seconds and satisfaction ratings went up to almost all 5’s; suggesting the redesign makes it easier to complete the task. However, the number of errors made actually increased, and this was down to some confusion over the my sport sub-tab’s in black at the top of the screen.

“They almost blended into the top menu tabs, rather than
clearly being a part of the 'My Sport' page”

- Matthew, Stage 2 Participant

redesign task 3 – time taken
redesign task 3 – errors made
redesign task 3 – satisfaction

As with the original site, the My Sport page was easy to find and the sport easy to add for all participants, but there was notable positive feedback about the new layout of the my sport page.

 “I liked the layout and how clear each story was from one another, and the way the subtabs update with your selections is really good”

- Rich, Stage 2 Participant

General Feedbac

In terms of the layout of the site, ratings scores did generally increase and this was reflected by the positive feedback given by the participants on each task. This was seen to an even greater extent when focusing on ease of navigation, where scores increased to nearly all 5’s, and this was likely down to the improvements made on the Get Inspired page specifically.

redesign general – layout
redesign general – ease of navigation

Visually, the ratings scores were generally quite similar to the original site and this was likely down to the fact that the improvements made focused on layout and navigation rather than the visual appeal of the site.

redesign general – visual appeal

Conclusion

This study showed me that while the BBC Sport website is very popular amongst its users, and may initially seem like it provides a very positive user experience, there were several areas that needed attention to improve its overall usability. Overall, the redesign did perform better in the evaluation; producing quicker completion times, fewer errors and higher satisfaction ratings; indicating improved usability in comparison with the original site.

However despite the improvements made to the site, there were two key further issues identified by the redesign evaluation process. The first of these was a result of the errors made in task 2, and looking to avoid the user clicking on a session that’s unsuitable for them. To avoid this, a consideration for the future might be to introduce a filter that allows the user to refine their search by ability and age.

The second issue focused on the confusion on the my sport page as to which set of sub-tabs are part of that page, compared to the tabs on the main menu. This would be corrected by making the my sport tabs more separate from the main menu; possibly changing the colour from black to a more stand out choice. Considering the future, the website could be developed further with the above problems in mind.

Project Date: 3rd October 2016 - 23rd January 2017
Project Team: Solo Project