Wrong UX Ways to Improve your Product

Summary:  Gathering user feedback can seem straightforward: just ask users what they think! Polls and surveys are quick, scalable, and seem democratic to product teams. But there’s a big problem: these tools often fail to provide the insights needed for real product improvements. Relying on surveys and polls alone can leave you with biased, superficial data that doesn’t reveal how users truly interact with your product.

What’s wrong with Elon Musk’s first move?

When Elon Musk bought Twitter (now X), one of his first moves was to put a poll on his new platform. The poll asked Twitter users if they wanted this or that feature. It went something like, “Should we remove this button?”. Asking users to vote on feature decisions is like asking restaurant customers to vote on the ingredients of a dish.

The big deal:

  • How you ask, the way you ask, and what techniques you use are critical for UX outcomes.
  • Asking questions the wrong way can lead to misguided product decisions.
  • Understanding user adoption and behavior requires a more nuanced approach.

Polls oversimplify user research, making polling seems like a good thing: fast, cheap, democratic! However, polls are easy to ignore: Elon Musk is even known for not paying attention to his own polls. Not long after the product poll, he tweeted:

elon musk poll should i step down shows yes

Polls and Surveys: not enough for real behavioral insights

Polls and surveys give the illusion of user understanding, but they often only scratch the surface. They’re useful for getting a temperature check—like understanding overall satisfaction or consensus on pain points. However, these commonly overused tools usually fail to capture the complexities of actual user behavior. Here’s why:

  1. Users don’t always know what they want or why
    Users often don’t have the self-awareness to explain their behavior accurately. When asked why they don’t use a feature, for instance, they may attribute it to factors they think sound logical, like “I’m too busy.” But the real reason could be something they’re unaware of, like a confusing interface or lack of perceived value.
  2. Surveys can be biased by your questionnaire design
    Polls and surveys are influenced by how questions are framed and what’s asked. If a survey question suggests that a product feature should be valuable, users may respond positively relative to their perceptions of what’s considered poor UX. They might blame themselves, not the UX. See: When surveys don’t work for UX insights
  3. Surveys and polls reflect Intent, not Action
    Surveys capture user intent, but they rarely reveal what users actually do. For example, 80% of users might say they’d use a new feature based on survey responses, but only a fraction will adopt it in practice. The intention-to-action gap is especially common in UX and product design, where features might sound helpful but aren’t used due to hidden friction points or differences in how users interpret their usefulness.
  4. Self-reported data can be misleading
    Self-reported responses are vulnerable to biases like the “social desirability bias,” where survey respondents provide answers according to societal expectations, rather than their own beliefs or experiences. This can lead to skewed data, painting an inaccurate picture of what users actually do or value. The issue becomes compounded when product teams make decisions based on these skewed insights.

What’s needed instead:

Context intelligence: Repeatedly referencing actual user behavior or being insights-led is critical. Building this skill in your team is called ‘contextual intelligence’. It’s essential to good design decision-making.

Stories: Provide high-value understanding by gaining a journey-wide view of goals, tasks, and pain points. Stories communicate rich insights. They are also valuable learning mechanisms for product teams. Agile teams should refer to the parent of their Epics, which are Sagas: journey-wide stories of user problem-solving.

Scenarios: Usage scenarios are like Agile User Stories but based on real insights and with motivation and added emotional value. Extreme user scenarios can help you understand your Inclusion Innovation opportunities, for example.

Why asking the right questions leads to defining the right problems

If you decide to use surveys and polls, carefully consider how you word questions. Even small wording changes can drastically impact responses, leading to a completely different picture of user sentiment and behavior. Some principles to keep in mind:

  1. Avoid leading questions
    Leading questions prompt users to answer in a certain way. For instance, asking, “How helpful did you find the onboarding process?” assumes that the onboarding was helpful in some way. A better approach is to ask open-ended questions like, “How would you describe your onboarding experience?” or “What, if anything, was helpful during onboarding?”
  2. Be specific, avoid vague wording
    General questions lead to general answers, which are rarely useful. For example, a question like, “Do you find the app easy to use?” may seem valuable but doesn’t yield any actionable insights. A more specific question would be, “Did you encounter any issues completing your first task?” or “What steps felt challenging when you first used the app?”
  3. Use behavioral prompts instead of hypotheticals
    Avoid hypothetical questions, such as, “Would you use a feature that lets you do X?” Instead, focus on what users have actually done, like, “When was the last time you used [similar existing feature]? Why or why not?” This approach grounds responses in real behavior and experience rather than abstract preference.
  4. Keep it short to avoid fatigue, use logic-branching to boost relevance
    Lengthy surveys with complex or numerous questions can lead to survey fatigue, which decreases response accuracy. Not using logic-branching is an amateur move- all surveys should include logic-branching. See my Masterclass: Using UX Surveys in UX like a Pro
Alternatives to polls and surveys for better UX insights

While polls and surveys have their place, deeper user insights require additional tools and methods. Here are some techniques that provide a richer understanding of user behavior:

  1. User Interviews and Contextual Inquiry
    Direct, open-ended interviews let you explore user behaviors, challenges, and preferences. Contextual inquiry—observing users in their natural setting—allows you to see firsthand how they interact with the product and identify potential pain points. It provides real, actionable insights rather than assumptions.
  2. Usability Testing
    By watching users interact with your product, you can observe their behavior in real time. Usability tests uncover friction points and moments of confusion, and they show what works well. This is especially helpful for identifying why users may abandon a task or miss key features. Watching users navigate your product is far more illuminating than asking them to describe it.
  3. Behavioral Analytics
    Tracking user behavior through analytics tools gives concrete, unbiased data on how users navigate the product. Analytics reveal user paths, time spent on features, and drop-off points, providing insights that self-reported survey data cannot. For example, analytics might show that users frequently exit at a certain step in a workflow, indicating a pain point to investigate further.
  4. A/B Testing
    A/B tests let you see how real users react to different versions of a feature in real-world conditions. This helps you understand user preferences without relying on self-reported data. When combined with analytics, A/B testing provides strong evidence for which designs or features best meet user needs.
  5. Diary Studies
    In a diary study, participants record their experiences with a product over a set period. This long-term insight helps reveal how users’ needs and behaviors evolve, uncovering patterns that short surveys or one-time usability tests miss.
Bottom line: A Balanced Approach to UX Research

Polls and surveys can provide a snapshot of user sentiment, but they’re limited in scope and susceptible to bias. For true UX improvements, you need a balanced approach that includes observation, behavioral analytics, and thoughtful user interviews. While surveys can serve as a starting point, they should be supplemented by deeper research methods that capture how users actually interact with your product.

What’s at stake here? Sloppy techniques or asking questions the wrong way can lead to weak and watery insights. When it comes to making smart product decisions, you need solid data. Then, defining the right problems to solve becomes a lot easier.

Interested in that last part? Join me for: Are you Solving the Right Problems? masterclass

Leave a Reply

Your email address will not be published. Required fields are marked *


The reCAPTCHA verification period has expired. Please reload the page.

Recent Posts

Scroll to top

Get a quote or discuss your project

Tell us about your project

Arrange a 30 min call

Project in mind?

logoblack

Fight for the rights of your users. We'll show you how.

Read more articles like this for exclusive insights into the best ways to approach UX and Service Design challenges. Find out when events occur first. Privacy protected, no exceptions.

Subscribing indicates your consent to our Privacy Policy

Should we add you to our email list?

Privacy protected-You can unsubscribe at any time.

Download the Better UX kit