GitLab is a remote-only organization and just like our team, our users are spread across the globe. Conducting remote UX research allows us to quickly connect with GitLab users anywhere in the world. It provides us with the opportunity to gather insight into users’ behaviors, motivations and goals when using GitLab. This helps us to determine what features should be built and how they should behave. But how do we do all this remotely?
These are some of the remote UX research methods we use at GitLab.
Card sorting
Card sorting is a research method for discovering how people understand and categorize information. Each card represents an item or a topic and we ask users to group the cards in a way that makes sense to them. We may also ask them to help us label these groups.
Card sorting can be used to:
- Help design the information architecture of your application
- Establish what information should be on a page and in what order that information should appear
- Provide a ranking for items or topics based on a set criteria
When analyzing a card sort, we look for common patterns such as which cards appear together the most and which cards are labeled in a similar way.
At GitLab, we’re currently using card sorting to restructure the sidebar navigation at a project and group level. We want to understand how you, our users, would expect our features to be grouped and classified. Our aim is to improve the ease and the speed at which you navigate around GitLab. We conduct remote card sorting via Optimal Workshop.
First-click testing
First-click testing explores what users click on first when completing a task within an interface. It tells us whether users are able to find what they’re looking for quickly and easily. This research method is based on the principle that users are two to three times more likely to find what they are looking for if their initial click is correct, rather than a click in the wrong direction.
We’ve used first-click testing at GitLab to quickly evaluate multiple design ideas against one another. We share our designs with users via UsabilityHub. We measure whether users take the correct path and how long it takes them to decide where to click. A slower click time would suggest a user has hesitated about where to click.
First-click testing is great for providing an indication of whether a design is intuitive to users and helps us to quickly narrow down multiple design concepts.
Surveys
Surveys are used to investigate the opinions or experiences of users by asking them questions through an online form. A survey invites people to share open and honest feedback. Some people find them less intimidating than other forms of research as there is the option to remain anonymous when providing answers. They also allow us to track how the attitudes and behaviors of our users change over time.
We’ve used surveys to understand our users and form personas, to generate new ideas for future GitLab improvements and to help measure users’ satisfaction with our existing features.
User interviews
If you take part in a user interview at GitLab, you’ll usually be speaking one on one with a UX researcher. In order to do this, you’ll need a desktop or laptop computer and a headset with a microphone.
We find that most of our users like to talk with us on their lunch break at their work station, whether situated at home or in an office. We love this, as it provides some insight into the environment in which you use GitLab.
Often our interviews are focused on you! We’ll ask you to chat about things such as your background, occupation and experience with GitLab. Sometimes we might have a particular topic we’d like to discuss, such as how you’ve incorporated GitLab into your workflow. We’ll always tell you our intentions ahead of the call so you have time to think about what you’d like to contribute to the discussion. We also welcome you to share your screen with us during the call. We understand that it is sometimes easier to show and demonstrate something than it is to just talk about it!
We’ve used feedback from user interviews to:
- Inform our personas
- Follow up on survey answers
- Understand and develop objectives and goals for features
Usability testing
Usability testing is a technique used to evaluate a product by testing it with representative users. Usability testing can be divided into two categories: moderated and unmoderated research.
Moderated
If you participate in moderated usability testing at GitLab, you’ll complete a series of tasks whilst being observed by one of our UX researchers. In order to see what you're doing, we'll ask you to share your screen with us. We use Zoom to run our moderated usability testing sessions.
As you use GitLab, we’ll ask you to try and think out loud: tell us what you’re looking at, what you’re trying to do and what you’re thinking. We’re interested in hearing your honest feedback. Sound scary? It really isn’t! It’s important to remember that we’re testing GitLab, not you. You can’t say or do anything wrong during a study.
Moderated research allows for conversation between a user and the UX researcher, because both are online simultaneously. It gives the researcher the opportunity to ask a user follow-up questions regarding something they’ve said or done. Subsequently, moderated research provides us with a lot of in-depth qualitative research about our users’ needs. It can help us to uncover usability problems that we weren’t aware of and to generate solutions to solve these problems.
Unmoderated
Unlike moderated research, unmoderated research doesn't involve any conversation between a user and a UX researcher. Instead, unmoderated usability testing sessions are completed alone by a user. As users can complete sessions at their own convenience and studies can be run simultaneously, they're good for collecting data quickly.
We use Validately to serve the tasks to you and to record your actions. We then analyze the data collected asynchronously. It is, however, still very helpful to us if you try and think out loud while you’re completing tasks.
Unmoderated research can provide some qualitative data. However, as there’s no opportunity to ask users follow-up questions related to their actions, the study should focus on a few specific elements or relatively minor changes. Unmoderated research is usually better at addressing specific quantitative questions, such as:
- What percentage of users successfully completed the task?
- How long did it take users to complete the task?
As a researcher cannot view an unmoderated usability testing session until it's completed, there's a risk of a study being unusable if the user didn't complete the tasks as specified or if they ran into technical difficulties.
We conduct both moderated and unmoderated usability testing sessions at GitLab to test new features and changes to existing features.
How can I get involved?
We’re always looking for people to participate in our research, whether you're a GitLab user or not. You can get involved by signing up for GitLab First Look, a comprehensive research program that will help us ship the features and fixes you need to do your best work. Besides being instrumental in shaping the future of GitLab, you’ll have the opportunity to earn gift cards and win awesome tech prizes by sharing your feedback with us.
