App Development

Mobile App Health Check: 10 Diagnostic Exercises for Roadmap Prioritization

When a mobile app experience is good enough, it’s tempting for brands to take their foot off the gas. But the mobile app landscape constantly evolves due to new technologies and metrics for success. While certain industry shifts or design trends might not require a significant app overhaul, others, such as the recent proliferation of generative AI, at least warrant a closer look at your application development process.

So, how do you know it's time to implement an upgrade? We’ve compiled the Mobile App Health Check below with some of WillowTree’s most effective indicators of mobile app health. Whether you’re long overdue for an evaluation or have recently launched a new feature, running these diagnostics can help you understand your product backlog and prioritize your mobile app development roadmap.

And if your team doesn’t have the bandwidth to track down all this data, we can run the assessment for you.

Download our Mobile App Health Check worksheet

Phase One: Diagnosing Surface-Level Symptoms

Kick off your Mobile App Health Check by taking a fresh look at data and KPIs that are easy to access. Use this information to understand how your app is actually doing in the wild.

1. Evaluate your app store optimization status

In the realms of Google and Bing, we all know that SEO is king, but in the App Store and Google Play Store, app store optimization (ASO) matters. Your ratings and reviews (including the volume of reviews) influence your ASO, and low ASO equates to poor placement in search results, which reduces visibility.

Our recommendation

Here’s a quick and easy exercise to gauge your app’s ASO performance. Conduct a keyword search (or several) to see where your app falls in the search results. Ideal placement is in positions 1 through 5.

Run this exercise for all target keywords someone might use to find your app. The more keywords your app ranks for, the higher its visibility.

Keyword 1: _____________
Results Placement: ____________

Keyword 2: _____________
Results Placement: ____________

2. Audit your app store rating

Your app store rating impacts a user's decision to download more than your feature set, carefully crafted app store cards, or any other aspect of your mobile app. So, no more resting on laurels. The important thing is to understand how that shiny 4.3-star rating has trended over time.

If this more comprehensive narrative shows a steadily increasing average rating, you’re doing something right. However, if your current 4.3-star rating has eroded from 4.8 stars, that warrants serious attention and could mean it’s time for an update.

While mapping your app store rating directly to revenue might be challenging, it’s hard to dismiss a correlation. A Harvard Business School study found that an increase of a single star in an overall rating on Yelp can boost a restaurant’s revenue by 5% to 9%.

Our recommendation

Review your average app store rating by quarter, starting two years prior. Sustaining an average score of 3.9 or below suggests you might have solid fundamentals, but you should look closer. Sustaining an average of 4.0 or above is excellent.

If you’re unsure how to find this information, your developer console should show reviews by version and allow filtering by rating and other criteria. To view an average rating tied to a particular date range or version, you may need an API to download the data in its raw format and analyze it elsewhere.

Average rating:
Q1 2022 __________
Q2 2022 __________
Q3 2022 __________
Q4 2022 __________
Q1 2023 __________
Q2 2023 __________
Q3 2023 __________
Q4 2023 __________

3. Assess your app store reviews

You may have an untapped wealth of real-world insight in the form of app store reviews. If you’ve conducted user testing, you already know the value of thoughtful feedback from your user base.

Carefully review recent and historical reviews for clues on where you might be satisfying or shortchanging customer expectations. Are you hearing the same complaints repeatedly? Are dozens of people requesting a particular new feature? It’s critical to address user needs like these, and as we mentioned above, the reviews directly impact a prospective user’s decision to download the app in the first place.

Our recommendation

Read over some of your lowest- and highest-rating reviews from two years ago and from today. Copy-paste some key complaints or highlights and summarize some themes and trends.

Save time by using AI to synthesize reviews into insightful takeaways.

3 Low-Rating Reviews (2 Years Ago):

1. __________
2. __________
3. __________

3 Low-Rating Reviews (Recent):

1. __________
2. __________
3. __________

Thematic Observations from Low-Rating Reviews: ______________________________

3 High-Rating Reviews (2 Years Ago):

1. __________
2. __________
3. __________

3 High-Rating Reviews (Recent):

1. __________
2. __________
3. __________

Thematic Observations from High-Rating Reviews: ______________________________

4. Reflect on your backlog

Are the tickets in your product backlog trying to tell you something? If your Jira board has gotten stale, it might be time to reconsider the vitality of your team’s proposed final vision.

At WillowTree, we refer to a product strategy declared early and rarely revisited as a “Waterfall strategy.” Like Waterfall product development, the shelf life is limited when the product roadmap is rigid and inflexible. Use this opportunity to return to the waterfall’s source and revisit your app's relevance and timeline.

Our recommendation

Freshen up your perspective with a game of backlog Mad Libs™. As a strategy and research practice, we frequently use a fill-in-the-blank method to surface alignment, or lack thereof, amongst stakeholders.

Challenge your colleagues to complete the Mad Libs exercise below. Embrace the simplicity.

Fill in the blank: “Once we launch (feature or capability), our app will be truly outstanding.”

5. Review your app performance

User tolerance for general app underperformance (e.g., long load times or a high crash rate) is simply nonexistent. According to Google, 53% of visits will likely be abandoned if pages take longer than 3 seconds to load. And, apart from obvious user-experience metrics (engagement, session length, etc.), app stores prioritize reliable products that provide an excellent user experience.

Our recommendation

Review the current crash rate of your app (on both iOS and Android) with particular attention to its relationship to core functions. While crash rate is one of many performance markers, it’s easy to assess quickly.

An acceptable crash rate for mobile apps varies depending on the type of app, its complexity, and the target audience. Even a crash rate of less than 1% may not be acceptable for banking or medical apps where reliability is critical. A slightly higher crash rate may be more tolerable for other app categories, such as games or entertainment. However, a general target is a crash rate of less than 1% per user per session.

Current crash rate (iOS):  _________
Current crash rate (Android): _________

Phase Two: Exploring Underlying Causes

Okay, we’ve walked through exercises to assess how your app is currently performing. Now, let’s explore more significant considerations that could improve your app’s health by leaps and bounds.

These experiments may require broader stakeholder participation and thoughtful facilitation of the exercises described. Again, WillowTree’s strategists can support such efforts.

6. Thoughtfully explore AI

It’s no surprise generative and agentic AI made this list, right? Exploring AI correlates with a brand’s appetite for learning and experimentation. Your brand may or may not have an immediate application for AI, but it’s worth getting clarity and alignment on this issue with your team. Spend extra cycles exploring your options, and give yourself the freedom to imagine how you might leverage this technology, even in far-fetched ways.

Our recommendation

Take a How Might We (HMW) approach to activating AI during ideation:

  1. Identify 10 user goals in your app (purchase a product, stream a video, etc.).
  2. With a multidisciplinary team to fuel creative thinking, ideate “How might we use genAI to help a user achieve goal #1?”
  3. Jot down the prospective genAI solutions.
  4. Rinse and repeat through all 10 goals.
  5. Prioritize all the solutions proposed against one another in terms of which will have the greatest impact and ROI.
  6. If all of this feels a bit daunting, reach out and learn more about our Agentic AI Accelerator and we'll help facilitate this thinking.

7. Embrace competitive benchmarking

If you don’t have your competitors' apps downloaded on your device, do it now. Why? Suppose a competitor suddenly offers a novel feature that saves time or money or otherwise changes the game. In that case, you should reconsider the velocity of your mobile app development roadmap because you are now behind the curve.

If you’re not spotting game-changing features on their apps, there’s still more to do. In the App Store or Google Play Store, survey the version history of some of your competitor’s mobile products and their updates. If most have released new versions with substantial changes (beyond bug fixes) in the past six months, it may be time to follow suit.

Our recommendation

WillowTree teams regularly run an activity called a Competitive Feature Audit. Each row of the x-axis consists of an app feature, while the y-axis positions your app alongside your competitors. If you convert your results to a heat map, you’ll have a fresh way to look at broader themes.

8. Conduct adjacent market assessments

With the release of curbside returns in Target's native app, their customers gained the ability to place Starbucks orders for curbside delivery while waiting for their returns to be processed. Whether or not your brand is connected to big-box retail, anyone can understand the potential consumer value of this convenience.

Why does this matter? The Anchoring Principle dictates that consumers are industry-agnostic in their critique of digital interactions with a brand. On the heels of an easy Target return and a hot latte, the bar is high for seamless, intuitive digital experiences no matter the industry.

Our recommendation

Rather than overlook adjacent market activities, gather your team and conduct this thought exercise:

What would it look like if (brand) were to launch an app in our industry?

(Insert brands like Target, Amazon, Disney, Apple, and Goldman Sachs. Your answers to the question could unlock ideas for your app that you might have never imagined.)

9. Audit your UI

The visual appeal of your app goes hand in hand with its overall usability. So, your hamburger menu might not be cutting it anymore.

UI needs to feel fresh while enabling the user to efficiently navigate the app out of the gate. Gone are the days when users patiently read through pop-up instructions to understand how an app works. Clarity is essential.

Our recommendation

Conduct a "Quick-Hit Usability Assessment" using tap count:

  1. Identify three core actions in your app (e.g., purchasing a ticket, posting a message, or earning a reward).
  2. Tally the number of taps it takes to complete that core action.
  3. Conduct the same study with your competitors’ apps, and be sure to start from an authenticated state for a fair comparison.
  4. Assess how your app’s usability stacks up against competitors.

10. Get up to speed on usability & accessibility

1.6 billion people worldwide have disabilities affecting cognitive, neurological, speech, physical or visual capabilities. That number gets even more significant when considering other dimensions, such as age, temporary health conditions, and environmental factors.

As WillowTree Managing Director of Design Alex Carr said when introducing our accessibility Figma plugin Contrast:

“If firms don’t comply with web accessibility standards, they purposefully exclude billions of individuals. On a human level, that’s unacceptable. From a business perspective, that leads to an estimated $10-16 billion in revenue loss and a $3-6 billion risk in legal fees. Moreover, if assistive technologies like screen readers can’t effectively engage with content, neither can search engines indexing pages — leading to negative SEO ramifications. Organizations creating more inclusive digital experiences are the companies that will win.”

Our recommendation

Conduct an accessibility assessment using the P.O.U.R. principles. If you have a project under development, we encourage you to use Alex’s Figma plugin to ensure your designs meet Web Content Accessibility Guidelines (WCAG), Android, and Apple standards.

Don’t delay your evaluation

It’s an ever-evolving mobile app landscape out there, which makes staying proactive about the health of your mobile app crucial for its success and the satisfaction of your users.

By assessing key markers such as app store performance, ratings and reviews, backlog reflection, app performance, and competitive benchmarking, you can identify areas for improvement and determine when it’s time to reinvest in your app. Additionally, you can maintain your competitive edge by exploring the potential application of generative AI and ensuring usability and accessibility.

But it all starts with evaluation and collaboration. Use these ten exercises to begin discussions with product managers, development teams, and other team members and stakeholders to prioritize and kickstart your next major app upgrade.

Reach out for a mobile app assessment
Table of Contents
Read the Video Transcript

One email, once a month.

Our latest thinking—delivered.
Thank you! You have been successfully added to our monthly email list.
Oops! Something went wrong while submitting the form.
More content

Let’s talk.

Wherever you are on your journey, we can help. Let’s have a conversation.