Over the past 6 months I ran multiple beta tests for my product and in the process learnt a couple of important things about the users of a software product. However, most of the lessons can also be applied to any other software product.
It's always good to carry out tests but at the same time we can't test everything out because of limited number of active users that we have especially in an early stage product. That's the reason, knowing in advance what are going to be the most critical lessons and applying them to our products before the beta tests begin can save us a lot of time and will give us the opportunity to test other important things that are really critical.
In this post I've summarized these generic lessons that can be applied to most products in the hope that it'll help some founders and product managers to get more value out of their beta tests. It will help them achieve higher retention among early users (and the better word of mouth) which is a direct result of better user experience.
Lesson 1: Before Starting Your Beta Test With Actual Users
If you have invested your time in building something, you might know by now where your ideal users hang out. But before you reach out to them, go through this simple step to avoid the initial friction with your actual users. Share your product with your friends and acquaintances, at least a few of them. The only valuable and honest answer you can expect from them is this:
What are the things about the product that are not obvious?
Now this is a big question about the product that we as creators can't answer. We as the creator of a product are so indulged that we often underestimate the learning curve a product might have. If you don't have a friend who can understand the product then reach out to people in your industry who can help.
The question you ask them can be as simple as- how do you think this product can be useful to you? Find out if they mention about the differentiation that you have in mind.
At a more complex level the questions will be- what changes will you suggest in the UI, what is it that turns you off, where does it fall behind the most popular product in the market?
But this has to be one-on-one in the form of a direct interview. If you proceed without this step, you'll muddle the results from your beta test with actual users because even though you'll know what people are doing with your product, you won't know why they are doing it.
Lesson 2: Features And Pricing
In most cases it's not easy to understand what features your users like best and what are turning them off. And finding answer to that question should be the biggest goal of a beta test. But there are some obvious things (in the hindsight) that would have saved me a lot of time. Here I'm listing them down as a checklist:
- Info tooltips: Make use of info tooltips, help articles, FAQs as generously as possible to answer the tiniest doubt a user might have. You should carry out this research with one-on-one interviews to understand the common doubts regarding your product even before you launch the beta test and make the information easily accessible.
- Free trial: If your goal is to get feedback, not asking users to enter a credit card is important in the early phase. Another important point is offering a long enough trial period. If it's too short people might feel that they can't evaluate the product properly during this period.
- Make key features immediately visible: Making users hunt for a critical information is an immediate turn off. Understanding what's most critical for you users and making them easily available and visible is the key. Decide the features that can hook your users and make them prominent. If you make sure the hook features are free (at least for a limited time) and are the ones that immediately catches the attention of a new user, you have a chance to hook them for a long time.
- Overly complicated metrics/features: another big mistake that I learnt from experience is that simplicity is often underrated. Sometimes we offer something valuable in the form of an overly complicated feature and we are so proud of it, we don't bother to loosen it up or break it down into something simple. We must understand that most users are not ready for an advanced feature from a new product, especially if it involves a learning curve.
- Pricing: Make sure you communicate the pricing very clearly especially if you are offering a limited period free trial. People often need a clear visibility before they start trial of a product unless they are just casually trying it out. The later type are of no value to your business.
Lesson 3: Reducing Friction
The most common points of friction for a new tool are related to privacy, load time, setup and maintenance, cancellation and transition. Here are some pointers on each of these:
- Privacy- If you value user's privacy, highlight it in your promotional content as well as within the product. Various privacy related pointers are not tracking users, no data-sharing or selling to 3rd parties, being GDPR compliant etc.
- Load time- This is absolutely critical. One of our initial products was feature heavy but it took so much time to load that the user retention was abysmally low. When we made a trade off, got rid of some features and reduced load time the performance went up significantly.
- Setup and Maintenance- People don’t want to fiddle with configuration and patches/updates especially for a new products where they are not yet sure about the returns. No-fuss setup and maintenance makes and breaks a product. Best way to measure this is to look at your immediate competitors and make sure your setup/maintenance are relatively lower.
- Cancellation- If you're offering a paid service (or free trial with credit card) state very clearly how people can cancel the trial or subscription in your FAQs as well as other places.
- Transition- If you expect your users to come from a different service, offering them a free transition (possibly in the form of data transfer etc.) can help build their trust and make the transition easy for them.
Lesson 4: Readiness To Measure Certain Critical Metrics
This is important because users coming from different platforms are different. If you are trying to measure whether users are signing up, how they use your product, how long they use it, whether they are coming back to the product- you should be ready to measure these parameters using an analytics tool. Usually most of us rely on Google Analytics to measure these things but as I learnt from my own experience, there are other free tools out there which can yield better results. Here is a paid analytics tool with a free monthly trial that you may try.
But irrespective of the tools you use, list down what you really want to measure using a beta test, what can be measured using your tool and if there is a way to segment users based on the source they are coming from so that you can measure your goals for each of these traffic sources.
Trust me you'll be surprised to find out how much user behavior vary with traffic sources. I started measuring this quite late in my journey and that's my greatest regret. I would have only focused on sources that bring quality traffic to my product and catered to their specific needs, rather than dilute my findings by measuring the KPIs at an aggregated level.
Lesson 5: Setting The Right User Expectation
While promoting our products we often tend to be a click-baity. Sometimes we end up over-selling the product. That's another mistake I made with my earlier products. Although it may work when we have a tried and tested product in the market, it's often not the case with a new product. Try to set realistic expectations while inviting users for a beta test. At times I even undersell a new product so that when people actually use the product it exceeds their expectation.
Lesson 6: Product Performance At Scale
Many software makers (especially bootstrapped) don't get a chance to test their product at scale before the beta test begins. The importance of stress testing is to make sure your product works when it's being used by tens or even hundreds of users. Otherwise it may so happen that when users visit your page, the page is down and they don't come back again.
In case you don't have a way to stress test your product here's a hack that worked for me. I acquired users from facebook with display ads (from geographies with very low cost per click) and I use another automated tool that tracks page downtime.
This tool sends constant feedback on where the page fails to load at a certain point. Since you'll get a high influx of users at a short interval, it will give you a good idea about performance under scale. Although it's not a perfect stress-test, it gives you a good understanding of your product's performance at scale.
Lesson 7: Compatibility With Browsers and Devices
It's important to test your product under different browsers and different devices before the launch. Common browsers that you should test are Google Chrome, Firefox and Edge.
If you don't have access to different devices, here are some emulators that will do the trick while testing for different devices.
Genymotion is an Android emulator mostly for developers. It lets you test your apps on a variety of devices without owning them. You can configure the emulator for a variety of devices with various versions of Android to help suit your needs.
While Appetize.io is a browser-based iPhone emulator for PCs, Ripple does its job in the form of a Chrome extension. It’s popular for testing cross-platform mobile and HTML5 applications without going through a complicated setup process.
Lesson 8: Having A Great Onboarding Experience
Even today a lot of users visit my homepage and bounce off from there without trying the actual tools. They judge us by the cover. And I believe that's the case for 30-70% users of any website depending on how good their copy is.
Investing time on landing page copy will always pay off. The trick here is to know what is a good copy for the home/landing page.
The trade-off is whether to test out many possible alternatives (and lose potential customers) or hire a high-end copywriter who knows how to convert people from homepage to product.
Lesson 9: Having A Blog Or Other Form Of Content
One thing I realized quite late in the journey was that a lot of people were searching for the keyword "useropinion blog" or visiting the blog pages from google search when they searched for "useropinion". Since at that point we didn't have elaborate blog content, we lost a lot of users (or failed to gain their trust). Having a blog with a few posts on the usability of our products and detailed explanation of our mission, values and differentiators would have gone a long way in retaining our initial users.
Audience Research
We have developed an audience research tool, in the form of an one-click survey app to help you research visitors by using non-intrusive single question surveys. It gives you a deeper understanding of the problems that are bothering your users. It also helps you unveil many use cases of your product that you were possibly not aware of.