--- title: A/B testing in ASO description: How to provide A/B test of graphics in the App Store and Google Play --- [![AppFollow](https://cdn.appfollow.io/assets/site/img/logo.svg) ![AppFollow](https://cdn.appfollow.io/assets/site/img/logo-mini.svg)](https://appfollow.io) Platform [App Monitor Discover areas for improvement and differentiation from user feedback and competitor insights.](https://appfollow.io/monitor-app-performance) [ASO Tools Harness valuable market intelligence for targeted messaging and better organic reach.](https://appfollow.io/aso) [AI-Powered Review Management Take the next step in how your business engages with user reviews.](https://appfollow.io/app-review-management-tool) [Workflow Automation Respond to reviews, sort and report offensive ones automatically. Streamline your support workflows.](https://appfollow.io/workflow-automation) [Integrations Process critical app data on reviews, ratings, and ASO in email, Slack, Zendesk, Tableau, Webhook, and over 20 more services.](https://appfollow.io/integrations) [AppFollow MCP & AI Toolkit AI assistant access to your app’s reviews, ratings, and ASO data, plus ready-made analysis workflows, with no dashboard or code required.](https://appfollow.io/mcp) Solutions [Gaming Keep players happy at scale. Extract insights and automate support to maintain high ratings.](https://appfollow.io/gaming) [Finance Catch complaints before they hurt your rating. Talk to users in any language instantly.](https://appfollow.io/finance) [Customer Support Handle tickets faster. Save 4 hours daily per agent and respond 50% faster.](https://appfollow.io/support-teams) [ASO & Marketing Stop optimizing in the dark. Get higher rankings through data and competitive intelligence.](https://appfollow.io/marketing-teams) [Enterprise Get 99.5% uptime, enterprise security, and dashboards that connect sentiment to business metrics.](https://appfollow.io/enterprise) [See more Explore solutions by industry, team & scale.](https://appfollow.io/solutions) [Consultancy](https://appfollow.io/asoconsulting) [Customers](https://appfollow.io/customers) Resources [Blog Explore expert articles and case studies on app growth, ASO, user retention, and mobile SaaS trends.](https://appfollow.io/blog) [E-books, guides & more Enhance your app reputation management skills with our collection of valuable resources.](https://appfollow.io/learning-hub) [Webinars Attend insightful webinars covering mobile app reputation, customer satisfaction, ASO, and more.](https://appfollow.io/webinars) [API for Developers Leverage our API to seamlessly integrate AppFollow with other services.](https://appfollow.io/appfollow-api) [Glossary Find explanation of every term about your app performance on app stores from AppFollow experts.](https://appfollow.io/glossary) [Help Center Find expert tips and solutions for utilizing AppFollow to its maximum potential.](https://support.appfollow.io) [Academy Explore AppFollow functions to master your business goals with comprehensive ASO and review management tutorials.](https://appfollow.io/appfollow-for-reviews-management-aso) [Trust Center Learn how AppFollow protects your data with industry-leading security practices, ISO 27001 certification, and trusted infrastructure.](https://support.appfollow.io/hc/en-us/articles/360020832997-AppFollow-s-Trust-Center-Data-Security) [Pricing](https://appfollow.io/pricing) Contact sales [Log in](https://watch.appfollow.io) [Sign up](https://watch.appfollow.io/signup) [![AppFollow](https://cdn.appfollow.io/assets/site/img/logo.svg)](https://appfollow.io) Platform ![](https://cdn.appfollow.io/assets/site/img/union.svg) [App Monitor Discover areas for improvement and differentiation from user feedback and competitor insights.](https://appfollow.io/monitor-app-performance) [ASO Tools Harness valuable market intelligence for targeted messaging and better organic reach.](https://appfollow.io/aso) [AI-Powered Review Management Take the next step in how your business engages with user reviews.](https://appfollow.io/app-review-management-tool) [Workflow Automation Respond to reviews, sort and report offensive ones automatically. Streamline your support workflows.](https://appfollow.io/workflow-automation) [Integrations Process critical app data on reviews, ratings, and ASO in email, Slack, Zendesk, Tableau, Webhook, and over 20 more services.](https://appfollow.io/integrations) [AppFollow MCP & AI Toolkit AI assistant access to your app’s reviews, ratings, and ASO data, plus ready-made analysis workflows, with no dashboard or code required.](https://appfollow.io/mcp) Solutions ![](https://cdn.appfollow.io/assets/site/img/union.svg) [Gaming Keep players happy at scale. Extract insights and automate support to maintain high ratings.](https://appfollow.io/gaming) [Finance Catch complaints before they hurt your rating. Talk to users in any language instantly.](https://appfollow.io/finance) [Customer Support Handle tickets faster. Save 4 hours daily per agent and respond 50% faster.](https://appfollow.io/support-teams) [ASO & Marketing Stop optimizing in the dark. Get higher rankings through data and competitive intelligence.](https://appfollow.io/marketing-teams) [Enterprise Get 99.5% uptime, enterprise security, and dashboards that connect sentiment to business metrics.](https://appfollow.io/enterprise) [See more Explore solutions by industry, team & scale.](https://appfollow.io/solutions) [Consultancy](https://appfollow.io/asoconsulting) [Customers](https://appfollow.io/customers) Resources ![](https://cdn.appfollow.io/assets/site/img/union.svg) [Blog Explore expert articles and case studies on app growth, ASO, user retention, and mobile SaaS trends.](https://appfollow.io/blog) [E-books, guides & more Enhance your app reputation management skills with our collection of valuable resources.](https://appfollow.io/learning-hub) [Webinars Attend insightful webinars covering mobile app reputation, customer satisfaction, ASO, and more.](https://appfollow.io/webinars) [API for Developers Leverage our API to seamlessly integrate AppFollow with other services.](https://appfollow.io/appfollow-api) [Glossary Find explanation of every term about your app performance on app stores from AppFollow experts.](https://appfollow.io/glossary) [Help Center Find expert tips and solutions for utilizing AppFollow to its maximum potential.](https://support.appfollow.io) [Academy Explore AppFollow functions to master your business goals with comprehensive ASO and review management tutorials.](https://appfollow.io/appfollow-for-reviews-management-aso) [Trust Center Learn how AppFollow protects your data with industry-leading security practices, ISO 27001 certification, and trusted infrastructure.](https://support.appfollow.io/hc/en-us/articles/360020832997-AppFollow-s-Trust-Center-Data-Security) [Pricing](https://appfollow.io/pricing) Contact sales [Log in](https://watch.appfollow.io) [Sign up](https://watch.appfollow.io/signup) [← Back](https://get.appfollow.io/aso-appfollow?hsLang=en) ASO Vault / Lesson #7 How to provide A/B test of graphics in the App Store and Google Play? ## Intro The main two tasks of ASO are to boost the app’s visibility and improve the conversion rate. In our previous articles, we’ve talked about how to boost the app’s visibility to get more impressions and page visits. In this article, you will learn how to boost the conversion to install rate. ## A/B Testing process The testing of app page elements is done in order to reduce the risks of a sharp change in conversion rate after an app update. The principles of testing are in the distribution of tested elements (2 or more) to the target audience of the application. With the help of A/B testing, it's possible to predict the most likely result of metadata updates of the app page. #### App testing tools **Google A/B Testing tool** is the most common tool for A/B testing of metadata. Pros: it’s completely free and you can find it in the “Store listings experiments” in Google Play Console. With its help, you can test up to 3 variations of metadata for a localized app version, globally or at the special app page. Below are the elements available for testing: - App icon - Screenshots - Video - Description picture - Short description - Full description Cons: Performing testing on all traffic channels (both organic and advertising campaigns) and the confidence interval of the test result both have a certain margin of error. After the applied result, you have to monitor the real changes in conversion rate. Another disadvantage is the inability to test graphics other than those meant for Google Play, since this is a platform-specific limitation. **Apple Search Ads.** Testing graphics in the App Store is done by using Apple Search Ads. With its Creative Sets, you can configure up to 10 sets of app screenshots and videos from those uploaded to the current app version, and see with which sets of graphical assets convert better. There are a few notable differences: 1. You can only test graphical assets - screenshots and videos 2. The testing is done only within the ad itself. The app metadata will remain the same. 3. All metadata for testing must be present on the app page already, so if you have a complex app page design, there will be only so much you can truly test. **Third-party testing solutions.** These will help you build a separate app landing page that emulates a given app store page that will receive traffic in order to conduct testing. Some of these solutions will present raw data (the number of page views and the number of conversions per install), and leave you to interpret the result yourself (ASO Giraffe). Some have an analytical platform (Splitmetrics) that is able to conduct testing, present raw data, and calculate the expected result based on input data. Among the advantages is the testing of any part of the App Store or Google Play app page on any type of user (which is especially convenient if you already know your target audience). A disadvantage is in the high cost for working with platforms and a separate fee for attracting traffic to the landing page, which sometimes is far from cheap. **Testing changes on the “live” app page.** If no other method is available, it's possible to conduct tests not only in the format of simultaneous testing but also by gradually changing graphic elements at equal time intervals. This method is the riskiest because you force your users to accept the current graphic assets without any alternative. Therefore, this method can be tried when you have test results on Google Play and you don't have the resources to test graphics for the App Store in some other way. #### Rules for conducting testing Almost all of the rules below will apply to the Google A/B testing tool as it's the most common metadata testing tool. ![pic](https://get.appfollow.io/hubfs/appfollow_59026766-1fcc-435b-98e5-ee46c71d946c.jpg "pic") *Source: appfollow.io* **1. Each experiment should be conducted over 7 days.** Some users behave differently in the store depending on the day of the week - thus giving root to the definition of Weekly fluctuations. For instance, on Monday, the popularity of sports apps grows, and on Friday, the popularity of taxis does the same thing. To account for these fluctuations, conduct tests for at least 7 days - despite your app category and other factors. **2. Plan hypotheses from general to specific.** To avoid repetition, it’s necessary to distribute hypotheses from large-scale (background, location of mockups, colors) to small (slogans, highlighting elements, etc.) during planning. 3\. **Test one element in one testing scenario**. In order to interpret the results more easily, it’s necessary to test one element in a single test scenario. Testing two or more elements (icon + screenshots) can lead to a situation where the icon increases conversion and the screenshots decrease it. You will not get hard data this way. 4\. **Conduct testing with a similar number of users.** It’s done with the same goal - to reduce the margin of error during testing. 5\. **Keep paid traffic flow at the same level.** When testing with the built-in Google Play tool, the presented options are split equally across traffic from all traffic sources. It’s not worthwhile to increase or decrease the number of purchased installations for a more accurate result. 6\. **Validate your tests**. To validate your tests, you can run reverse tests (B/A), or tests with a large number of items (A/B/B, A/A/B). If the result is not false positive, it will be confirmed during testing. Google A/B testing tool lets you test up to 3 test sets of elements + the current one simultaneously. When setting up a hypothesis, make sure to have more than one option. In a single test case, it’s necessary to run tests in A/B /B format. If you are unsure of the test result, run the validation test in the same format. This approach can help you to avoid a false-positive test result since the efficiency is calculated with a confidence interval of 90% (for conventional instruments, a confidence interval of 95%). #### Interpreting the results Depending on the testing tool used, you will have various options to interpret the results. Some tools provide you with raw data that you need to interpret yourself using an Excel spreadsheet. Some of them have functionality that will present you with the result of conversion growth. Test interpretation in Google Play leaves a lot of questions unanswered. Once each tested variant has the necessary number of installs to interpret the results, you can see an indicator of effectiveness. **An indicator of effectiveness** is the percentage of a possible change in the number of installs when applying the current option with a 90% probability. For example, an indicator of effectiveness of +8% at 90% probability means that if you apply this variant, installs are expected to increase by roughly 8%, and there's a 90% chance this result isn't just random noise. ![pic](https://get.appfollow.io/hubfs/appfollow_adaae823-a609-4951-9aef-130b3a458ade.jpg "pic") The data you get from a given tool is an average indicator and it must be compared with the current position of traffic across all channels. If you are actively running advertising campaigns, you need to pay attention to the entirety of the product funnel - from the conversion of ad creatives to retention indicators. ![pic](https://get.appfollow.io/hubfs/appfollow_3e04c963-230c-45e1-b92f-9cb2facc031f.jpg "pic") ## Conclusion Testing the elements of an application page is crucial to the success of any application. By conducting testing, you can cover a variety of tasks: audience research, the need for certain app mechanics, current trends, as well as increasing the app conversion itself. Prioritize the elements you intend to test if you need to adjust your app conversion. Decide which elements need to be improved first and foremost, and outline the goals the testing is trying to achieve. Formulating clear hypotheses and building a test plan will help you stay focused on the research. To get ideas, use all possible channels, don’t focus only on competitors' references. Make sure to choose the tool that will suit your needs, convenience, and budget. Don't forget that it’s easier to conduct an experiment and interpret the results in a 7-day time frame, with stable traffic, and with well-defined hypotheses. Compliance with all the rules above will help you get more realistic results, and reduce the likelihood of a poor user experience overall. In short: test one element at a time, run each experiment for at least 7 days with stable traffic, and validate a positive result with a follow-up A/A or A/B/B test before rolling it out. ### Read other posts from our academy [![Lesson #8](https://get.appfollow.io/hubfs/Lesson%20%238-Dec-15-2023-05-41-11-6132-PM.jpg "Lesson #8")](https://get.appfollow.io/how-to-analyze-text-change-results-the-good-and-the-bad-kpis-of-aso-work?hsLang=en) LESSON #8 [Measuring ASO Performance: KPIs for App Store Optimization](https://get.appfollow.io/how-to-analyze-text-change-results-the-good-and-the-bad-kpis-of-aso-work?hsLang=en) [![Lesson #9](https://get.appfollow.io/hubfs/Lesson%20%239-Dec-15-2023-05-41-11-6215-PM.jpg "Lesson #9")](https://get.appfollow.io/how-ratings-reviews-help-with-aso?hsLang=en) LESSON #9 [How Ratings & Reviews Help with ASO](https://get.appfollow.io/how-ratings-reviews-help-with-aso?hsLang=en) [![Intro](https://get.appfollow.io/hubfs/IntrO-1.jpg "Intro")](https://get.appfollow.io/the-importance-of-aso-in-mobile-marketing-strategy-and-where-to-start-if-you-havent-done-it-before?hsLang=en) INTRO [Why you need ASO in 2023, and how to start](https://get.appfollow.io/the-importance-of-aso-in-mobile-marketing-strategy-and-where-to-start-if-you-havent-done-it-before?hsLang=en) React to user feedback and market trends faster Get started for free Company - [About](https://appfollow.io/about) - [Pricing](https://appfollow.io/pricing) - [Consultancy](https://appfollow.io/asoconsulting) - [Customers](https://appfollow.io/customers) - [Team](https://appfollow.io/team) - [Contact us](https://appfollow.io/contact) - [For partners](https://appfollow.io/partners) - [AI & LLM Guide](https://appfollow.io/ai-llm-guide) Platform - [Monitor app performance](https://appfollow.io/monitor-app-performance) - [App Store Optimization](https://appfollow.io/aso) - [AI Review Management](https://appfollow.io/app-review-management-tool) - [Workflow Automation](https://appfollow.io/workflow-automation) - [Consoles Data Aggregator](https://appfollow.io/consoles) - [App Competitor Analysis](https://appfollow.io/app-competitor-analysis) - [Product Roadmaps](https://appfollow.io/product-roadmaps-for-mobile-app-market) - [Top Chart Rankings](https://appfollow.io/rankings) - [AppFollow MCP & AI Toolkit](https://appfollow.io/mcp) Solutions - [Customer Support](https://appfollow.io/customer-support) - [ASO & Marketing](https://appfollow.io/marketing) - [Product Managers](https://appfollow.io/product-managers) - [Gaming Companies](https://appfollow.io/gaming) - [Enterprise](https://appfollow.io/enterprise) Resources - [Blog](https://appfollow.io/blog) - [Webinars](https://appfollow.io/webinars) - [Academy](https://appfollow.io/appfollow-for-reviews-management-aso) - [Glossary](https://appfollow.io/glossary) - [Help Center](https://support.appfollow.io) - [Trust Center](https://support.appfollow.io/hc/en-us/articles/360020832997-AppFollow-s-Trust-Center-Data-Security) - [API](https://appfollow.io/appfollow-api) - [Free ASO Tools](https://appfollow.io/free-aso-tools) How we compare - [AppFollow vs Native consoles](https://appfollow.io/compare-appfollow-vs-native-consoles) - [AppFollow vs App Store Connect](https://appfollow.io/compare-appfollow-vs-app-store-connect) - [AppFollow vs Google Play Console](https://appfollow.io/compare-appfollow-to-google-play-console) - [AppFollow vs In-house](https://appfollow.io/compare-appfollow-to-in-house-solutions) - [AppFollow vs Appfigures](https://appfollow.io/compare-appfollow-vs-appfigures) - [AppFollow vs Appbot](https://appfollow.io/compare-appfollow-vs-appbot) - [AppFollow vs AppTweak](https://appfollow.io/compare-appfollow-vs-apptweak) - [More](https://appfollow.io/appfollow-comparisons) Integrations - [App Store Connect](https://appfollow.io/ios-app-analytics) - [Google Play Console](https://appfollow.io/googleplay-analytics) - [Steam](https://appfollow.io/steam) - [Trustpilot](https://appfollow.io/trustpilot) - [Zendesk](https://appfollow.io/zendesk) - [Slack](https://appfollow.io/slack) - [Salesforce](https://appfollow.io/salesforce) - [Helpshift](https://appfollow.io/helpshift) - [More](https://appfollow.io/integrations) [System status](https://status.appfollow.io) [Terms](https://appfollow.io/terms) [Privacy policy](https://appfollow.io/privacy) Join our communities: [![AppFollow logo](https://cdn.appfollow.io/assets/site/img/purple-logo.svg)](https://appfollow.io) AppFollow © 2015-2026. Built with ![love](https://cdn.appfollow.io/assets/2/img/white-heart.png) remotely The ultimate platform to help your product thrive in the mobile market Elevate your review management strategy. Communicate with users from a single place. Gain actionable product insights. Improve visibility and increase organic growth. Track key metrics. Stay ahead of the competition. Be the first to know about mobile trends and market changes. Trusted by 100,000+ product teams worldwide ![get-in-touch-logos](https://cdn.appfollow.io/assets/site/img/get-in-touch/logos_v2.png) Contact us ✉️