Unlock the Insight of Users

Optimize the user insight delivery in Trend Micro

Formed by Summer UXR Intern:

Monly Zhuang, Howsh Jhuang

Mentor: Joseph Yeh, Sopjia

Research Summary

Project Duration: 2 months
Conducted 8 in-depth interviews with 2 different stakeholders including 3 countries.
Responsible for a research workshop with nearly 20 participants.
Clarify the E2E processes, sweet and pain points of the entire insight delivery.
Clarify 3 key issues and analyze their severity for internal operations.
Proposed 5 correlated suggestions and divided into short and long-term plans.

Background

Many were complaining about the difficulty of reaching out to User Insight.

In recent years, user researchers have utilized SharePoint web pages to store and deliver user data, reports, etc.

However, despite its longstanding use, many stakeholders are unfamiliar with it and prefer privately messaging researchers for file retrieval or directly asking relevant questions.

This situation not only hinders individuals who may be hesitant to inquire but also frequently interrupts researchers' workflow."

PMs and UIDs are the 2 key stakeholders.

Understand the user needs/pains and other data.
Delivery user insight to stakeholders.

UXR

Insight

Make decisions with user insights.

UID

Design UI with user insight.

User researchers have identified product/project managers (PMs) within the company and user interface designers (UIDs) as the primary users of user insights.

Therefore, this study will focus on addressing the problems and needs of these two roles to identify the most appropriate solutions.

Research Objective

Currently, PMs and UIDs cannot access the necessary insight efficiently.

Therefore, to address this obstacle, we want to...

1. Understand their Motivation and Purpose for requiring user insight.

2. Understand how they Obtain and Work with user insight.

3. Discover the Pain points, Needs, and so the best Solutions.

Research Method

From general pattern to deep understanding

Research

Workshop with UIDs
Interview with PMs/UIDs

1 month

Proposed

Solutions

.5 month

UID Workshop:

Find out the common situations of UIDs' Pain & Needs

To find out the UIDs' common needs and the challenges of using user insights, we invited all UIDs (15+ members) to join in this online workshop.

We focus on 3 questions throughout the overall design process

(Before design / Within Design / After Design)

What information do they need?
How do they access to the information?
What is the challenge during design?

1 month

1:1 Interview with PMs and UIDs

Deep understanding of how they obtain, and work with user insight.

At this stage, we conduct one-on-one interviews to gain an in-depth understanding of where PMs and UIDs require User Insight in their work, how they acquire and utilize it, as well as the difficulties and challenges they face.

To comprehensively grasp these details, the interview is divided into two parts:

1) Self-introduction: personal background and project experience.

2) Case-sharing: share the actual cases with the needs of user insight.

Prompting them to describe in detail the process from requirement emergence to acquisition and utilization.

＊Recruitment Rule: To understand the pain and need for user insight delivery, we select the interviewee(PM/UID) who has worked with UXR, and who has a large demand for user insights.

Discussion Guide:

For PM, we focus on understanding the channel they obtain user insight and the difference after absorbing it (how to apply/if they share or not...)

Section1. Background

A. Seniority

What position before becoming a PM?

How long have you been a PM?

B. What projects are you responsible for now?

1. New / Old product?

2. How long does the project take to develop?

3. Team functions.

Section2. Use Case (5W1H)

A. Please share your past or current project experience (share specific projects directly)

How do you open a spec, and what do you consider when releasing? (user, business…)
When will you need user insight, why?
What customer needs are important to you, and how can these insights help?
How to obtain customers needs? (who & where) & the difficulties you've met.
How to use customers’ needs after obtaining them? What impact did it have on the project?
Will you share the customer needs you have obtained?
- What kind of information would make you want to share?
- When do you like to share?
- Who will you share it with?
- How will you share?

B. How did you work with UXR?

Under what circumstances and when would you want UXR to join?
What assistance will be needed from them? What kind of tasks are assigned to UXR?

C. Is there any situation where customers’ needs cannot be obtained? What will you do at this time?

Who will you contact? What will you do?

Product / Project Manager (TW/JP/US)

UI Designer (TW)

Discussion Guide:

For UID, we focus on understanding the current demand situation for user insight, the project type that demands, and how to achieve this demand.

Section1. Background

1. Seniority

How long have you been a UID?

2. What projects are you responsible for now?

How many project do you participate in now?
How frequently does TOI and new projects occur?
Team function(with or without UXR).
How to communicate with your team? (Scheduled weekly meetings, customary communication channels)

Section2. Use Case (5W1H)

A. Please share your current project experience

Is there any UXR in this project? How UXR work with the team?
When is research insight needed? What insights are important to you? Why?
How can these insights help you work?
How to obtain these insights?
How to apply insight? What impact did it bring to the project?

B. Any situation that insight cannot be obtained? What will you do at this time?

Who will you contact?and where will you search?

C. How would you classify the insight information?

Research Insight

Insight Delivery E2E process / Pain & Sweet Points

New Key Stakeholders

The customer feedback from the Field Team is also

a kind of vital user information to the development.

Customer Feedback

*The firsthand user feedback information that hasn't been organized into user insight.

Field Team

Here, the Field Team generally refers to the customer-facing functions, including the sales team(SAL) that sells products, the sales engineer(SE) team that teaches product use, and the PM ops team that serves as a bridge between customers and product PMs.

These functions regularly interact with customers, gather firsthand feedback, and relay it to the development unit to address customer needs and issues.

How stakeholders interact

PMs convert the user data from UXRs and Field team

into design requests to UIDs.

There are two providers of user-related data, 1) UXR who deliver information in a unified manner (user insight), and 2) the field team who bring customers' feedback directly back to the team.

Besides UXR will deliver the user insight to anyone who needs it, once the PMs have all this information, they will consolidate and convert it into requests for the development team, especially the UIDs, as they have to create a design that will lead to a good user experience.

*User Insight: The user information is in a unified manner which is clarified and organized, mainly provided by UXRs.

*Customer Feedback: The first-hand user data that is raw and unorganized, mainly provided by field team.

Provider

UXR

Converter

User Insight

Customer

Feedback

( unified manner )

Field Team

( firsthand info )

User Insight

Receiver

UID

User Insight,

Customer Feedback

& Request

2 main user insight delivery scenario

Different product maturity will

strongly impact the insight delivery path

The user insight delivery mainly relies on the provision of UXRs. Therefore, whether there are dedicated researchers in the project will greatly affect the way the insight is delivered.
The allocation of researchers for the project can mainly be based on the maturity of the product. In general, researchers will be invited to join if the product is in the early stages of development which needs to further explore the development directions and strategies. On the contrary, there's no dedicated UXRs in the project when the product is relatively mature which only needs enhancements.

New Product: Exploring Phase

With dedicated UXR

UXR will conduct user-related research based on project needs, including interviews, competitive product analysis, etc., and organize the feedback data collected by the field team to integrate it into user insights, then convey them to the PM and development team.

Mature Product: Enhancing Stage

Without dedicated UXR

In projects where the product is in the later stages of enhancements, without a dedicated UXR, most user data (such as customer feedback) will be provided by the on-site team or simply find relevant past user insights themselves. They will contact user reviewers directly only if necessary.

The Sweet and Pain in the Working Process

How the main receiver "UID" work with user insight

Receiver

UID

Based on the results of the workshops and interviews, we found that user insights are closely related to the work of UID. They need to understand the user's information in the early stages to design for the user's context.

Therefore, we use the UIDs' workflow as the basic structure to integrate the needs and pain points that stakeholders will encounter at each stage.

UIDs' Workflow

Tools

1. Understanding

Search: Mesaage / Past Research / Youtube

2. UI Design

Figma

Dedicated UXR

External UXR

Message / Meeting / Work page

Message / Meeting

Research

Message

Message / Meeting

Exploring Stage

Message

Enhancing Stage

3. UI Validation

Interview

4. Recording

The interaction that would happen in general.

The interaction would happen only if necessary.

The workflow of UID can be mainly divided into the following steps:

(1) Understanding

In the early stages of a project, UIDs need to understand the project background and relevant user information, which will usually be provided by the PM, and UXR(if there's a dedicated researcher), search historical records or ask for more information by themself.

(2) UI design

After understanding, UIDs usually start preliminary design first to meet the development schedule. And participate in the research process (such as interviews, workshops, etc.) to speed up the acquisition of new user information. After the research is completed, UIDs will optimize the design according to the user insight report.

(3) UI validation (optional)

PM and UID will conduct validation after the design draft is completed, including user interviews or testing if time is available.

(4) Recording (optional)

After the design is approved and released, UID usually organizes the project, explains the function development process, and attaches brief user information to facilitate future handover if time is available..

Stage 1 :Understanding

Sweet and Pain points

2 key critical information before design:

1) Who is the User/Why & How do they act: Persona, Use case, User Scenario, Competitor

2) Why the feature came out: Design Context

Usually stored in Figma provided by the former UID, listed the design developing process, discussion context, and user info.

The user info was provided by PM, and UXR, which was shared in the meeting time. Sometimes PM will record on the project page online.

Exploring Stage

1) The related user info is usually well organized and delivered quickly.

UXR will sort out the related user information and share with the team in the meeting, so UID no need to organize by their own, PM can also receive some information.

2) It's easier to align members thought of user with UXR

The project direction would be sure to consider the user, which is a critical point of UID's work.

Quickly understand users to move towards the right project direction.

"UXR is very helpful in clarifying goals, because the user scenario information will be much richer"

Sweet point

Self-reliance, soul-searching (霊魂拷問 ) to collect pieces of information, since the user's voice is counting on me

Enhancing Stage

The user info was mainly provided by PM. If sometimes PM doesn't, UIDs need to soul-searching PM or seek external UXRs for more information.

Pain point

1) The delivered info from PM vary greatly, taking time to adapt

Each PM are vary, sometimes the request will be sent out with rich user information but sometimes with no related content.

2) For UIDs, it takes time to assist in delivering User Insight.

UIDs need to organize the user related questions from the development team, then message UXR for the needed files or arrange the meeting for explaining. After understanding the context, UID has to transfer to the team by message or meeting.

"If there is no information, I'll go soul searching the PM, keep asking about what is our goal, why this feature, the use case, competitor..."

"UID is like the contact person of HIE, if the team has questions, then I'll organize them then message UXR, to reduce the time occupied of UXRs, but it would also take my times"

3) UIDs lack of the user information or cannot easily be found

The required information is scattered around platforms, thus it's difficult to find.

Don't want to bother, but still need to contact external UXR to get more information.

"The information needed is scattered in different places, someone put on Teams chat, Wiki, recording, etc. often spend time recalling"

"I wanted to find some user info of XDR in Insight Repository, so I clicked into the file I thought was related, but I couldn't figure out which could fit my needs, so I quit."

"Because UXRs are responsible for lots of projects which make them really busy, so I feel embarrassed to bother them further"

"Even every member has to understand users, but I really cannot consume it..."

Exploring / Enhancing Stage

Customer feedback from field teams can be obtained quickly but can be difficult to organize and act on.

Sweet point

1) Can fast gain the real customer feedback

PM would receive a lot of raw customer data from field team. It's still a precious resource when UXR cannot contact the real user.
The user insight analyzed from research usually takes much more time, thus may not fit the development process, while the field data can provide some help immediately.

2) For PMs, it's hard to differentiate the key user needs and prioritize them

The quantity of needs delivered to the PMs are very often, so it's hard to decide which should be handled first.
The standard of deciding the features priority remains unclear, causing confusion from UIDs and RDs.

3) For UIDs, the information is hard to transfer into design

"Feedback" information is too specific to understand what they really want/need.
Some requests from field is only about technical problems but not related to UI design.

"I'm still learning about how to decide data priority, it's often just up to our(PMs) experience"

Pain point

Stage 2 : UI Design

Sweet and Pain points

The key information before design:

How users do/feel toward the design: Use Case, User Story

The user info was provided by PM and UXR in the meeting or Teams message.

Sweet point

1) UXRs make up for the lack of research, feel confident in designing for UIDs

If there aren't enough user info related to the project sometimes, UXR can research about the unknown area to assist in understanding the user.

2) For UIDs, the required user info and suggestion can be fast obtained

UID can reach the team UXR directly to gain needed information or suggestions, letting them make better design choices.

3) The product release cycle won't wait for the research, UIDs still have to design first

Research usually takes a long time, sometimes UID still has to design without new research information.

Pain point

"We're in a hurry to release design, and the goal was also unclear,

so I feel difficult to understand the whole context about why we have to do this"

Exploring Stage

Make sure to receive timely access to user information and suggestion, helping to make better design choices.

Exploring Stage

UID is no longer just a designer,

but are also responsible for insight delivery and user testing.

The user info was usually provided by PM in the meeting or Teams message.

If needed, UID has to take the initiative to message the external UXR for more info.

Pain point

1) For UIDs, it takes time to assist in delivering User Insight.

2) UIDs lack of the user information or cannot easily be found.

The required information is scattered(meeting recording, Teams chat, Wiki, JIRA, Insight repository, UID's Figma), it's hard to find out quickly.
The information found is outdated or not relevant, not available to be used in the current situation, can only rely on PM's experience to conduct design.

The design will be the part of testing usually, because there're no relevant research.

3) The mature products would change the strategy in some situations, but have no .

The enhancing mature products need to explore again when the strategy is changed, but the UXRs will not join in the project of the enhancing product, causing the team hard time aligning the thinking of the direction and the problems.

"The feature is evolving and the previous research does not cover the new target we are currently focusing on (and the project does not have a dedicated UXR)"

"Sometimes the persona or use case is not very clear. Then we need to make the hypothesis, and come out design based on that hypothesis."

"MSP is in the maintenance phase, but required to reach a new goal. PMs and RDs were mostly discussing the functions then, but the real question was what value could MSP bring to the client, so we needed to discuss with HIE"

Stage 3: UI Validation

UID serve as a researcher, responsible for tracking data, conducting interviews with PM to verify the effectiveness of the design.

The key information in the validation stage:

Effectiveness of UI design:

User Interview: To simply verify the effectiveness of UI design, if there's time left before release, PM will conduct a simple interview with users with UID, gathering some qualitative information before release.
UBT data: During the later discussion in the design phase, the team will usually ask the data team to set up UBT point, so let UID can follow up with the user using status of the feature.

Stage 4: Recording

Benefit the all UIDs, the stones of those hills may be used to polish gems

The key information in the recording stage:

Why the feature came out

Design History: It's common for UID to TOI the project, and for the next member to better understand the content, the UID usually delivers the records of the process of the project in Figma(e.g. release time, related files, discussion content, features...), also be called "Time machine" in TOI.

Summary | The difficulties of insight delivery

c. The user insight of the requests provided by PMs is often not enough for UIDs to work.

a. The customer feedback is too raw and hasn't been well organized.

b. The data haven't been well stored, so it's hard for UIDs to search the related insight

The severity of the problems

Since this is an internal operation project, and considering that colleagues need to spend their spare time to implement future proposed projects, we also analyze the severity of each problem here.

The analysis factors include the type of project affected, the stakeholders, and the difficulty of their current solutions:

*Ex: exploring phase / En: enhancing phase / T: Type / S: Stakeholders

We multiply the project type and the number of stakeholder impacts as the total impact amount of the problem.

Then put the current solution difficulty together on the graphical line to compare the severity (point P is the most serious).

Looking at the straight-line distance between points, we get the result that the severity is a>c>b.

That is, "a" is the most urgent problem to be solved for now.

High

Severity

a. The customer feedback is too raw and hasn't been well organized.

c. The user insight of the requests provided by PMs is often not enough for UIDs to work.

b. The data haven't been well stored, so it's hard for UIDs to search the related insight.

Proposed Solutions

Suggestions and correlated ideas

Convert the field data into user insight in systematic ways.

Problem a | Customer feedback hasn't been well organized.

Build up a collaborative platform for PMs to provide customer feedback from the field team, and UXR can organize and convert it into user insight directly.

customer

feedback

Collaborate

UXR

UID

UXR

Interview

organize

Research democracy, coaching the PMs and UIDs to interview and organize the field data themselves on the simple task in comparison (like UI validation).

Problem c | PM does not provide enough user information.

Let PMs understand which info is needed by to UIDs' work clearly.

Provide a request template to PMs that

has the title of the element that UIDs need

(e.g. persona, use case...)

One platform(channel) for one purpose,

and build up an easy-to-find storing rule.

Problem b | The existing user insight is hard to reach out to.

Construct a delivering rule, let the type of information delivered on each channel/platform be clear.

(e.g. JIRA for request, Teams chat for discuss feature...)

Optimize the storage logic of the current insight storage platform(Insight Repository) by UID preference, involving the categories, naming rules, and metadata.

Solutions from short term to longterm

In the previous section, we analyzed the severity of the problem. Here we also use the difficulty of the solution to distinguish the most suitable short-term or long-term goals and list the relevant stakeholders and outputs.

Project Outcome

Through a series of workshops involving 20 participants and 8 interviews, this project successfully clarified the transmission of information, and stakeholders involved, and identified pain points within the workflow. Additionally, it prioritized the severity of issues and proposed five corresponding recommendations categorized into short-, medium-, and long-term optimization goals.

Despite being a two-month summer internship research project, the outcomes of this project received high praise from over 20 colleagues within the department. Departmental supervisors also expressed the possibility of incorporating its findings into future optimization.

Personal Reflection

Explore the Customer Needs in the Future

In this project, I collaborated with another intern on various tasks, including research planning, workshops, interviews, converging results, and proposing recommendations. It was my first time participating fully in a high-level user research project, which provided me with a deeper understanding of rigorous and comprehensive research methods and processes.

Through this study, I also conducted cross-national interviews in English for the first time, which, despite being challenging, I found extremely valuable.

However, I feel it's unfortunate that in this research, we primarily focused on exploring user insights provided by user researchers without delving into the circulation process and impact of "customer needs" brought by the field team. Although we acknowledged its importance in this study, given ample time, I believe we could further explore the patterns, formats, and delivery channel of this information. Additionally, interviewing the members who collect and bring back this crucial customer needs data could potentially suggest more concrete ways to translate and utilize these equally significant insights.