Study Procedure

At the beginning of the study, I introduced myself and the purpose of this study to participants. Participants were told to think aloud during the test. If they were not sure how to do think aloud, I would make a demonstration to show how to do it. To make participants comfortable to think aloud, they were explicitly told that it is the test for software not them.

​

Tasks

The tasks are addressed below [3]:

• Download and installation: Now, as you can see on the browser, this is the homepage of Tor. Now please download and install the software.

• Open the browser: Please open the browser and look at its Homepage. Is there anything that you find confusing?

• Change security setting:

  1. Now, please click ‘open security settings’ and change your setting to ‘Medium’ level. Do you find anything confusing?

  2. Now, please change your setting to ‘High’ level. Do you find anything confusing?

• Search the flight ticket: Now please search the flight ticket to Iceland on Expedia.com

​

Background of Users

Two participants were recruited for this study, whom will be referred as P1 and P2 in the following analysis [1]. Both participants are female, users of MAC OS X system, and graduate students in engineering school [2]. They did not have using experience with Tor Browser before this study [2]. The average age is 29.5 [2].

​

Results

Installation of Tor Browser

Participants were asked to download and install Tor Browser on Tor’s homepage. Since this was their first time to use Tor, both participants were curious about the tool and try to distinguish what the differences between Tor and other general browsers [4]. When trying to download the package, P1 found there are two packages, ‘Stable’ and ‘Experimental’ (Figure 1). P1 was wondering what are differences between them and eventually she chose the ‘Stable’ one [4]. However, P2 did not encounter this issue [6]. P2 clicked the download button that led her to the ‘Stable’ one and downloaded it.

​

​

​

​

​

​

​

Figure 1. The interface of download page

 

When installing Tor, both participants struggled with understanding and choosing network setting [4,5]. As shown in Figure 2, Tor network setting provides two choices, one is to make the direct connection to the Tor network and another one is to connect through bridge or local proxy. Participants (P1 and P2) were really confused by the wording. P2 asked what the ‘proxy’ and ‘censored’ means and needed to look up the terms in dictionary. She also asked whether all network is censored to certain extent. Then she looked at these choices again and said ‘I don’t know’. Eventually she selected the first choice to connect Tor network directly.

 

For P1, she was also confused by the term but she sought for the assistance on Tor’s support page [4]. However, she did not find the information page helpful. Then she went back to the setting and select the second choice to connect the Tor network via bridge.

​

​

​

​

​

​

​

​

​

​

​

​

Figure 2. The interface for Tor network settings in the process of installation. 

​

Use of Tor Browser

Interface of  Homepage

Participants were asked to explore the interface of homepage (Figure 3) and found if anything confused them. P1 clicked ‘Test Tor Network Settings’, which led her to another page informing her IP address (Figure 4) [4]. P1 was confused by this IP address and not sure where this IP address came from [4]. P2 was trying to understand what Tor is and recognized that she could search on the search bar [4]. She found no confusion on the homepage [6].

 

 

​

​

​

​

​

​

​

​

​

 

​

​

​

 

Figure 3. Homepage of Tor Users

 

​

​

​

​

​

​

​

​

​

​

​

​

​

Figure 4. Confirmation page of security settings

​

Security Settings

Tor’s security settings allow users to adjust the level of security from low to high and provides the description about what each security level does and its outcome. Participants were asked to open the dialogue (Figure 5) and adjusted the setting from low (the default) to medium and high. P1 read the description for each setting but she could not fathom what do they mean [4]. She said: “I don’t know what the outcome of each one…” and she decided to select the medium security level [4]. For P2, while looking at the security level, she immediately decided she wanted the highest level of security. Then she read the description about the function of setting. She found that the highest level of security would not allow her to access math equation so she changed to the medium level [4,5].

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

 

Figure 5. Dialogue for Tor Browser Security Settings

 

Using Tor for Search

In this task, participants were asked to search flight ticket on Expedia.com on Tor. P1 typed the Expedia in search bar, which led her to the search result page of Duckduckgo (Figure 6) [4]. She then said: “Why it is on Duckduckgo…I like Google…I want to use Google” [4,5]. She opened another tab and typed Google.com and tried to search Expedia on Google [4]. However, she found it was slow on Google and she thought it was because she was using Google [4]. Thus, she changed it back to Duckduckgo. While waiting the result on Duckduckgo, she went back to Google and checked the results [4]. She found an abnormal warning message shown on Google. She said: “why I can’t search on Google? What happened? What did I do?“. P1 looked frustrated when she could not access Google and did not know why [5]. She started questioning herself if she did anything wrong [5]. Then she returned to Duckduckgo and clicked the Expedia page. However, she got the ‘Access Denied’ message (Figure 7). At this point, P1 kept saying “Why this is happening?’. I don’t know what to do“ [5].

 

Interestingly, she began tried different approaches to access Expedia, even though she felt frustrated [5,8]. For instance, she copied and pasted the URL of Expedia to the search bar, but it did not work. Then she went back to the search page on Duckduckgo and tried different links. One of the link worked but only for united state airline. And P1 still could not get to the Expedia site. Therefore, she started searching different websites and tried to figure out if it was the problem of the Expedia or the browser [4,5].

 

For P2, she also input the keyword ‘Expedia’ on the search bar and tried to access the site. She noticed the search engine was not Google and also wondered why [4]. When she received the access denied message from Expedia, similar to P1, she kept asking “Why…Why? Is because of the setting” [4]. Then she went to the security setting and adjusted it to the default (low) and searched it again [4,5]. Yet she still could not access the site. She then went to the preference setting to see if she could adjust some settings to make her access the website [4,5]. After trying for a while, P2 gave up the search [4].

 

It seems that participants exhibited different strategies while encountering difficulty to access the website [8]. Another interesting result is that both participants thought that they were using google while they were on the search page, although they noticed that it is Duckduckgo at first [6,7].

​

​

​

​

​

 

​

​

​

​

​

​

​

​

​

Figure 6. Search page on Duckduckgo

​

​

​

​

​

Figure 7. Access Denied after attempt to search for Expedia

​

Discussion

According to the results, there are three main usability problems of Tor Browser.

​

Technical Terms Make Users 'Guessing'

The results show that both participants were confused by the technical terms in the network (Figure 2) and security settings (Figure ?) [4]. When users do not understand the technical terms, they would try to look for helps [4]. For example, when trying to decide which network setting to choose, one participant looked up the help document provided by Tor but she did not find useful information, another participant used the dictionary and still did not know what the technical terms mean. In another word, Tor Browser not only violates Nielson’s heuristic guideline of help and documentation but also Clark et al. [2007]’s guideline of terminology [9]. When uses cannot have a better understanding about the terms, the best thing they can do is to ‘guess’ and just randomly choose one of settings that ‘looks right’ or ‘feels right’ to them [8].

 

Error Message Make Users ‘Trying’

Both participants felt frustrated and somewhat surprising when encountering the error message [4]. Interestingly, both participants started ‘trying’ to solve this issue [6]. It seems that users will attempt to diagnosis and find different solutions to access the website instead of giving up [5,6,7]. One participants tried different search approach and wanted to figure out whether it is the problem of the website or the browser [4]. Another one tried to adjust the setting and to see if it made any change [4]. That is, although users try to figure out what happens, they employ different strategies by following their logics [8]. Furthermore, it also means that Tor Browser does not help user to diagnose and recover from errors, which violates Clark et al. (2007)’s guideline of helping users diagnosis and recover from errors [9].

 

Uncommon Search Engine Interfere Habitual Searching

Another interesting finding is that both participants were surprised by the search engine, Duckduckgo [5]. One of them even felt frustrated and questioned why the browser did not use Google search. Users seem not to be open to the new search engine [7]. A possible explanation is that users have established a habitual searching behavior on Google, and they do not like their habits being interfered [8]. Users like something they have been used to [8].

​

Overview of User Flow

Figure 8 shows users' usage flow of Tor Browser.

​

​

​

​

​

​

​

​

​

​

​

​

​

Conclusion

According to the results, technical terms in network and security settings and unclear error message are significant usability problems to users. Therefore, my redesign will focus on the following two aspects [10]:

1. Change the terminology in the dialogue of network and security settings: I will come up with different ways to present the technical information and guide users to walk through the process of installation and the change of security settings.

2. Design an error-support message: I will design a pop-up error-support message to help users to solve the problems.

​

​

References

1. Clark, J., Van Oorschot, P. C., & Adams, C. (2007, July). Usability of anonymous web browsing: an examination of tor interfaces and deployability. In Proceedings of the 3rd symposium on Usable privacy and security (pp. 41-51). ACM.

2. Nielsen, J. (1995). 10 usability heuristics for user interface design. Fremont: Nielsen Norman Group.[Consult. 20 maio 2014]. Disponível na Internet.