Rumours a plenty that there will be a facebook phone with claims that they are hiring ex-Apple engineers who worked on the iPhone. Other suggestions that they should acquire a hardware arm in a similar way to how Google acquired Motorola.
The reason for these rumours are that Facebook users are increasingly moving to mobile with almost 500 million people using facebook via mobile. Whilst saturation rates are high in the developed world, extension to developing countries would likely be via mobile as the phone increasingly becomes the first access point to the internet and web.
With all the momentum moving mobile, Facebook lose out on potential ad revenues and currently have not monetized from their smartphone apps or mobile presence. This leaves a huge potential revenue stream untapped and a Facebook Phone is one way that could be realised.
Analysts have generally been negative about the rumour -
“I don’t see a huge demand for a specific Facebook phone, for the single reason that Facebook is already available on every smartphone,” said Francisco Jeronimo, research manager at IDC.
“It is a hugely intensive undertaking to develop phones, because life cycles are extremely short particularly in the Android world; there is an awful lot of margin pressure; and you have to ensure that you are competitive every release. It is absolutely relentless,” said Blaber.
Whilst the hardware play is not in facebook’s realm of expertise, making a platform/OS may make sense here’s why -
- No need to create, sell, distribute phone hardware. Rely on manufacturers to use the platform
- Facebook already have an App Store. It’s a potential monetization route
- They already have in-app purchases in the form of facebook credits. Tying it into mobile payments makes sense
- PIM (Personal Information Management) apps already exist. Contacts, events(Calendar), messaging, photo gallery
- Camera is already taken care of.
- No one owns or drives HTML5 app development, or owns a store to make it easily monetizable or accesible. Facebook could lead this charge and own it.
- A developer and app ecosystem already exist and it gives their developers who have not created mobile apps, and easy way to move their audience mobile without adding complicated hurdles that exist when using another platform.
- Does not alienate other manufacturers, which provides potential access to more carriers.
Bring that all together and you have a potential way to go mobile, but who would be the manufacturers? As with any new entrant in the mobile world, it’s best to align with companies that are in a similar situation. A struggling manufacturer, a local manufacturer looking to break through internationally, a hardware company looking to move into mobile, a carrier who wants differentiation.
It’s certainly food for thought.
The technology used for oversampling, and the inspiration taken from satellite imaging techniques is the main innovation. It needed a 41MP sensor to be able to create high quality 5/8MP photos. Strategically, this makes perfect sense, where everyone else in the smartphone battle has been focused on screen size, CPUs and other iterative minor improvements, this implementation has made people stand up and take notice.
We must remember that this is a Technology innovation, therefore it can be carried forward to any of Nokia’s future portfolio, as such, if they have ringfenced the IP around the tech, it bodes well as a competitive differentiator.
Focus on Camera
In terms of people not buying this because of the current OS that the tech sits on, this only affects one part of the mobile phone purchasing market. This is about targeting a certain niche. In terms of buyer preferences, different segments have different critical choices. If the rest of the package is good enough (Speed, Social, connectivity), those who hold a higher purchasing priority for camera will go for the camera capabilities. Likewise, those who prioritize apps will go for iOS or Android (or Windows Phone). In this respect the rest of the phone just needs to be good enough to capture the target camera market, which Symbian Belle seems to be (not outstanding but good enough).
This has been a mixed bag. Many initial reports focused on the 41MP headline. and failed to read further into where the real innovation was. For example, this is the first time i recall digital camera fanzines reporting about a smartphone camera. However, many tech ‘blogs’ have dismissed it due to the headline marketing number.
But! The headline marketing is essentially “linkbait”. To market pureview as producing super-high quality 5/8MP photos wouldn’t have the same marketing impact. Therefore, to get more eyeballs, and to attract the correct niche market, the headline MP count was the most effective. The con is that people report about the MP rather than the technology. The major pro is that it grabs headlines.
The technology has been in development for 5 years, waiting to release it on Windows Phone would mean delays. The technology is almost ready, and the window to announce was good. Amongst all the quad-core CPUs and high resolution screens, this announcement clearly grabbed mindshare for Nokia. Therefore it was important to announce the capability that Nokia still has, and the value they add as a manufacturer (irrelevant of platform) to smartphones. By announcing now, any future product’s camera by any manufacturer will be compared to the pureview. It has become the ‘de-facto’ comparison. Effectively giving Nokia some long-tail marketing.
Setting up the pipeline
Hidden amongst all the messages is that Pureview technology is that it will be released on future devices in different configurations. Effectively opening up an ‘upgrade’ path for people who want the camera tech. Want it now, buy the Symbian device, App ecosystem too important that you want to skip it for now, wait for it on another platform. Announcing it now builds up anticipation for future releases of pureview, essentially giving people a different reason to buy a Nokia Windows Phone. It also give Microsoft impetus to ensure their platform is capable of handling the hardware requirements for this tech.
Releasing on Symbian means that sales expectation can be lowered. If this was released on Windows Phone, expectation of sales would be higher. This gives Nokia a leeway for sale figures, allows the to gauge initial demand with lower risk than delaying, potentially 1 year, and then announcing it to an unknown market situation for Windows Phone. I would expect Nokia would use sales of the N8 as a benchmark for sales of this device.
The fact that it was released on a symbian phone is irrelevant. The technology is not binded to Symbian only, it is Nokia IP and means whatever platform they develop on, potentially has this technology differentiator. It has now become the de-facto comparison that smartphone cameras have to aspire to and ensures Nokia remain relevant, so strategically, it was absolutely the right thing to do. The cons are minor (negative press about it being released on Symbian, the 41MP headline vs the technology implementation) when compared to the longtail benefits.
Image from http://conversations.nokia.com
Nokia recently announced the 808 Pureview in Barcelona at the Mobile World Conference and have drawn excitement to their pureview camera technology. They’ve re-imagined how zoom works by working backwards from a large Megapixel picture and focusing in on an area of the sensor. Zooming in expands the area of the sensor which you are looking it, which means a bulky optical zoom mechanism is not required. This works also for video recording. How have cameraphone evolved to reach this point?
The very first commercial cameraphone
In 2000 Sharp produced the J-SH04 for J-Phone (now softbank) in Japan. It cost $500 and produced 110,000 pixel images (0.1MP). It was followed by J-SH05 which used a 65,536-color semi-transmissive TFT LCD on a flip phone. By 2002, Sharp had sold 5 million camera phones covering 40% of J-Phones users.
Out of Japan and turning international
In 2002, Vodafone picked up the Sharp cameraphone and remodelled it into the GX10 for European release. One of the few times that a Japanese phone manufacturer successfully sold their mobile phone offerings to a non-Japanese carrier.
Sprint brought the Sanyo 5300 to the US market in November 2002.
By 2003, KDDI released the first CCD 1 megapixel cameraphone, the Casio A5401CA.
In 2004, Sony Ericsson released the S700 with a 1.3MP CCD Camera, Nokia, Motorola and Samsung quickly followed suit,
but in 2005, Samsung went a little crazy releasing the SCH-V770…. a 7MP camera phone with 3x optical zoom and autofocus. Take a look at the picture to see how ‘ridiculed’ it was.
Nokia vs Sony Ericsson enter the battle
By 2004, Nokia had become the worlds most sold “digital camera” brand, but did not have the “best” camera phones. Sony Ericsson took that mantle when it released the S750 in 2005 which could be considered the first ‘regular’ camera phone that had autofocus… with a 2MP sensor added.
Nokia’s attempt to take the lead came via the N90 which used Carl Zeiss lens with their 2MP auto-focus phone with a rotatable barrel.
But Sony Ericsson’s response was the K800, the first with a xenon flash. Released in July 2006
Nokia March On
First the N73 with 3.2MP, followed by the N93 with 3x Optical Zoom and the first camcorder phone with Carl Zeiss lens both in the Autumn of 2006. (photo is of the N93i)
Than in 2007, the N95 with 5MP and the N82 with 5MP and Xenon Flash were released. By now the megapixel battle was in full swing, but we all know that Megapixels are not the only equation in cameras. Sony Ericsson released the K850 with a poor 5MP camera.
Samsung Enter the frame
With the i850 and M8800 Pixon the 8MP phase of the camera phone battle opened up. Nokia upped the technical capabilities by introducing variable apertures with their 8MP N86 in 2009.
Samsung released the first 12MP camera phone with 28mm wide-angled lens and Xenon with the M8910 Pixon12.
The current best camera phone was released in 2010. This is the Nokia N8, with a very large 1/1.83” sensor.
Nothing was released in 2011 that could beat the camera quality of the N8, but 2012 sees a quantum leap with the release of e Nokia Pureview.
Skype will kill the telco, you’ve probably heard it many times across many tech blogs. Well, they’re wrong. There’s one fundamental problem with this logic.
Skype needs telcos for it to work.
Without the telco, whether that is a mobile carrier or a fixed line provider, Skype need their infrastructure. How else would you be able to Skype someone if there wasn’t a data pipe wired or wireless? Moreover, Skype rely on telcos for their revenues. Let’s look at how Skype makes money - https://support.skype.com/en-us/faq/FA335/How-does-Skype-make-money
- Calls to phones and mobiles
- Sending text messages
- Skype to go
- Skype Wifi
- Skype Premium (Group chatting)
Each and everyone of these services relies on telcos and carriers, whether directly or indirectly.
Calls to phones and mobiles receive an interconnect fee, Skype pays telcos for these fees, reducing cost by making it a ‘local’ call, therefore you are only charged local interconnect fees. The telco make revenue from this, and Skype make money from the extra they charge on top of this fee. Use Skype voice or video? No revenue for Skype.
Sending SMS’s goes through the carriers, Skype will pay the carrier and charge you a little more. Use Skype messaging instead of SMS? No revenue for Skype
Skype to go, making international calls local. Again, paying interconnect fees and charging a premium on top.
Cannibalizing Network Effects
The more successful Skype is at gaining users, the more they will kill these 3 revenue streams. Skype to Skype messaging, voice and video calls are all free. This means network effects work against Skypes revenue model.
The last two revenue streams are not affected by Skype user growth. Skype Wifi, utilises the service of existing wifi providers. Skype Premium is the only revenue stream from the above that doesn’t directly rely on a partnership with an external service provider. It’s just utilizing the existing infrastructure
Kill the Telco
If Skype were to kill the telco, they effectively kill off the infrastructure they rely on, yet they need substantial amounts of money to maintain and improve their services for ever-increasing-data-hungry services from mobile phones and telecoms disruptions. The disruptions that are killing telco revenues are also rely on telco infrastructure.
$400B needed for LTE rollout and maintenance in 2012 alone. Kill the telco and Skype effectively kill themselves. Who will be left to takeover infrastructure? No Internet or consumer electronics company want to touch this because they will then come under scrutiny of regulation from government organizations such as the FCC. Google Voice got into trouble for not covering rural areas which telcos have to do.
This is the big paradox that Skype are contributing too. Marginalize telco services by disrupting their revenues, yet still rely on them to improve their infrastructure that Skype rely on.
(Image by Solarbotics)
The royalty rate that Motorola Mobiity wants from Apple for using its patents is 2.25% and Google are stating they will not change the policy. Cries of “unfair”, they are of a FRAND (Fair, Reasonable and Non Discriminatory) nature, so is 2.25% excessive? Even Samsung are demanding 2.4%!
Apple have complained to ETSI (European Telecommunications Standards Institute) and Microsoft are supporting Apple’s complaint claiming that the rate is far too high.
So the real question is what is considered fair and is Apple’s complaint valid?
Why isn’t mobile be the same as the wired world?
Mobile has been different from the outset. Spectrum is owned by governments, is licensed for specific uses and is limited in its resources, this means that the technology required for delivering larger amounts of data on such a narrow amount of available frequencies requires investment and a return on the investment.
Apple and Microsoft do not have substantial patents in this area. They have both come from the wired world which has a different value chain and different (strong) players at each layer. There is no equivalent of Qualcomm in the wired world, there are no gate keepers that can prevent your mobile phone from being easily supported such as carriers.
Is it really fair, what has history told us?
Apple previously lost a patent dispute with Nokia. Probably involving patents of a FRAND nature. The value is around 4.5% of the cost of building the device. Previous patent agreements have been around 5%
Qualcomm won a patent case against Nokia, with Nokia settling for an alleged $1.8 Billion in patents and agreeing to pay a royalty of approx 2%. Qualcomm’s average royalty rate is around 3.5% according to this chart . At the time, Nokia were also complaining that Qualcomm were not complying to FRAND.
The main thing to note is that these royalty percentages are above what Motorola and Samsung are demanding. In this context they seem fair……. however, the main difference (in the Motorola case) is a % of what? Motorola are asking for a % of the average sales price, whereas previous cases are about a % of the cost of building a device.
Here is a BOM breakdown of the iPhone 4S -
WIth a sales price of $599 for the 16GB and $699 for the 32GB version that means Motorola are asking for around $13.48 and $15.72 per device respectively. Whereas at Qualcomm’s estimated 3.5% of BOM, that figure is $6.58 and $7.25 respectively. This is likely the driver of the ‘unfairness’ claim.
It’s not the % in itself, it’s what it’s a % of.
How much are they asking?
Smartphone growth is forecasted to slow to 22% in 2012. If Apple maintain their current market share then the amount they are looking for in 2012 is worth $1.7B.
Compare that to the BOM version which is worth only half a billion and is likely to be less due to deceasing prices of components over time.
Number of Patents
If we look at the LTE patent pool and see who owns what, this is another argument for the unfairness of the request. Although the dispute is not about LTE (where Apple has acquired Nortels patent pool), it puts the size of the patent pool and the amount Qualcomm request vs what Motorola/Google are requesting into perspective.
Is asking for $1.7B fair?
(Image from 401K)
One of the most important things when developing your mobile app (or web app) is determining how many people are returning to use it and how to measure if the changes you are making is really improving it for the users. The number of registered users is not a great measure of how your app is engaging users. You could have 1 million users signing up but none of them may return to use the app after the first time.
A better measure is to see how many people are actively using your app. However, this number can get confused with the number of people who are new users, so you need to work out how many people stop using your app for each group of new users and when they stop using it. This makes it complicated to forecast the growth (or non-growth) of your app, making it hard to measure if your changes are having the desired effect of making your app sticky.
When is it time to get more users?
If your losing lots of users, you may not want to focus on acquiring lots of new users because they will likely only use your app once, so you need a good indication of when to start applying resources to get more users.
I’ve created a basic google spreadsheet that lets you to set up some active user goals and makes some forecasts for you. You can setup some targets for a future number of active users for a certain point in the future and understand how many people you need to acquire to hit your goal.
This lets you decide if it is an appropriate time to turn on the taps for user acquisition. e.g. if achieving 100,000 active users in 50 weeks means acquiring 10,000,000 new users, it’s clearly not the time to plough financial resources into acquiring new users.
It will produce a chart like the above.
- The Redline is the forecast of active users.
- The orange line is your real data.
- The blue bars are the number of new users you are acquiring for each data point (day, week, month.. it’s up to you).
- Target Active Users is the number of active users you hope to achieve
- Target Period is the time by which you are looking to reach that number of active users
- User Acqusition Growth rate is the linear % increase of the number of users you need to acquire for each time period.
- Total Users is the number of overall users you need to have acquired by the final period to hit your target number of active users
What do you need to Input?
You need to type in the Target number of active users and the target period you hope to achieve it by. To generate the graph you need to input into the fields described below
The first 2 fields are your baseline. It’s the number of new users you acquired in the last period and the number of active users you had in the last period.
Churn rate is the % of users you lose every period. e.g If you acquired 100 users in a period and in the next period, 30 of them are still active, your churn rate is 70%. However, usually the churn rate is high for the first period and subsequent periods are lower, this is because in the first period people are trying out your app and quite a few of them will decide it is not for them. The subsequent periods tend to have a more ‘natural’ churn.
So in this chart you need to apply a first period churn rate and a second churn rate for all other periods. You should be able to extrapolate these figures form your analytics tool.
New Users and Active Users
The last things to fill out is the number of new users you get every period. You can start by plugging in old data to see if your churn rate estimates are accurate. Fill in the actual number of active users and you can see how well the prediction is based on your inputted churn rates.
Fill in your new user forecasts and it should give you a forecast on the number of active users.
Testing your performance
When you get new data for a period, you can input the number of new users and the actual active user number. Look at the numbers and the chart and if it starts diverging upwards, then the changes you have made to your app are having a positive effect on your churn rate.
You can reuse the chart with your new churn rates to re-estimate the targets you are trying to achieve.
If you find this useful, or know someone who may find this useful, spread it around.
The example spreadsheet can be found here
Image from Cillian Storm
Blackberry is dead! Not enough Apps! Actually, part of the reasons for their demise is because changing enterprise and carrier trends caught up with RIM’s previous decisions, so let’s begin by delving into RIMs history
History & Strategic Shift
RIM focused on enterprise early, originally a 2-way paging device, they created push email and transformed into a smartphone company in 2003 by focusing on push e-mail. This was their first critical decision, they moved from reseller channels and completely relied on carriers to sell and distribute their phones.
RIM positioned its BlackBerry as a carrier‐friendly platform via its data efficiency. Most of the world had low bandwidth data connectivity and blackberry was dominant with their compression algorithms.
With its smaller bandwidth footprint, it costs enterprises less when there is no all you can eat data plan and carriers save money because they do not need to spend as much money to increase the capacity of their network capacity for this service (see this article for more info - http://www.rysavy.com/Articles/2009_01_27_Rysavy_EMail_Efficiency.pdf)
Then they introduced Blackberry Messenger (BBM) in 2006 an excellent solution for enterprise, it allowed secure messaging for enterprise customers reducing their costs which increased incentive for data packages to be purchased from the carriers. Security and email became synonymous with Blackberry.
Chasing the golden egg
RIM started to chase the consumer market and relied heavily on BBM to attract the youth market. The attraction for this market was reduced SMS costs. RIM got caught up in the consumer smartphone battle losing market share.
By spreading themselves out, they lost focus on how the smartphone world changed causing delays in execution.
- Enterprise is shifting away from owning the mobile device for its employees and allowing employees to us their own phones.
- BBM as a youth market cannibalizes carrier SMS revenues reducing the incentive for carriers to push blackberries. After Android came along, Verizon didn’t push Blackberries with the same vigor.
- Apps such as WhatsApp, Line, Talkbox etc. give the youth a ‘newer’ phone with an equivalent low cost messaging system so they can shift away from BBM. Moreover, it’s not locked down to only one manufacturer which means they can ‘free message’ friends using different devices.
The new CEO has stated they will have a consumer focus, particularly at the entry point where people upgrade from feature phones to smartphones. This will be incredibly tough for Blackberry who have to garner more support for the consumer app ecosystem. Smaller development houses do not have the resources to support multiple platforms and they will be selective over which platform to develop for first. Blackberry is not the first on the list.
Part of reasoning for their focus on consumer is because they rely heavily on carriers to sell their devices and the carrier battlefield is currently focused on the consumer rather than the enterprise market. What might exaceberate the problem is that they plough their resources into consumers and end up losing the enterprise customer.
Cloud, multiple devices and security.
These are areas that are becoming increasingly important. For enterprise, the deployment of private cloud solutions, online collaboration and working on the go. For security, the increase in cyber attacks and ‘cyber espionage’ is on the increase, and for the prosumer, carrying multiple devices is a pain. 3 distinct areas that need addressing which RIM can approach.
Here’s what they could try -
- Expand the scope of their cloud services beyond messaging, email and IT administration. Consider CRM, or 3rd party developer solutions that can wrap around their cloud architecture
- Wrap up corporate collaborative productivity tools within their ‘secure cloud’. They already have a great reputation for security.
- Split the blackberry so that it’s both a personal and enterprise tool. If a phone gets stolen, there is extra security for the ‘enterprise’ half of the phone. The prosumer can then install apps etc. onto the ‘consumer’ half of the phone. Many people don’t want to mix their professional contacts with personal friends.
- Separate billing for different profiles on one device. An extra value service which carriers can benefit from which provides effective cost management for IT departments, and still allows the phone to be used as a personal device.
This tries to address a relevant market for RIM with their blackberry products and targets both the enterprise, prosumer markets with incentives for carriers to continue to push the device. If more data solutions go via blackberries cloud, it provides more incentive for carriers because of RIM’s data efficiency. Even recently, carriers such as Docomo are looking for data efficient solutions
Photo by arrayexception
There was a time when smartphones had different types of form-factors sometimes iconic, sometimes innovative, even crazy designs. However, in the past few years designs have become boring. Almost every new smartphone in the past 3 years years is either a black/white touchscreen slab or a QWERTY keypad.
I will look back at some of the designs that made people look twice at your smartphone, maybe in awe, sometimes in comedy. (I’ll try not to repeat similar designs)
The Kyocera 6035, flip phone with touch screen (left)
The Nokia 9210 is a clamshell with keyboard (right)
Sony Ericsson P800 a flip phone with a touchscreen (left)
One of the first camera enabled smartphones was the Nokia 7650 with a slide (right)
Nokia 3650 with its crazy rotary keypad. (Left)
Palm Treo 180 with a full QWERTY keyboard and see through flip. (Middle)
Blackberry 5810, full QWERTY slab.
Nokia N-Gage aka the Taco. Designed to be a handheld gaming device (left)
Motorola A920 - Touchscreen Slab (Middle)
Siemens SX1 - With the numerical keypad on the sides (right)
Nokia 7610 - Although candbar form factor, funky keypad design.
Nokia N90 is a clamshell with a rotatable camera barrel and rotatable screen.
HTC startrek - thin clamshell smartphone (Left)
Nokia N93 - Transformer phone, camcorder, flip and laptop modes with a 3X optical zoom
Nokia N95 - Double slider
iPhone 2G - Slab
T-Mobile G1 (HTC Dream) - Sliding QWERTY
This is the era of Touchscreen Slabs.
I think it’s about time smartphone industrial design had a shake-up and we get some of the crazy, interesting head-turning designs back.
In my previous post. I talked about getting retention and engagement metrics out of split testing.
Here’s a practical example of how to do A/B testing using Flurry.
Create an App_Launch event that happens whenever your app is started or brought back from the background
When you log the event, pass it with a parameter A(name of split test) or B(name of split test). You can decide in advance if the app should use the ‘A’ version or the ‘B’ version using some device variable such as the MAC address or UDID.
For the purpose of this post, I will use a conventional ‘marketing’ conversion split test. The position of the in-app purchase button as the illustration of the ‘split test’. However, it could be for anything, the number of coins a new user receives on starting a game, the order of the tabs at the bottom, the layout of a particular screen, etc. In this example, people in group A have the in-app purchase in the top of the screen. People in group B have it pop-up. Marketing wants to know which positioning maximises conversions…but we want to also see the impact on engagement and retention, which I will talk about in a later post.
1) Create 2 segments inside Flurry
- Go to the Manage -> Segments and press Create New Segment.
- Press Add Custom Event and click on Only include users who triggered the event with these parameters and values
- In the triggering event name put in App_Launch (or whatever you called it)
- In the Parameter name put in A(name of split test)
- Do the same for B
You have now created 2 segments that can dissect the user behaviour of both of these parties.
2) Check the conversion
- Go to the Usage -> New Users Table and select the App version for the split test. This gives you the total number of new users who used the app with the split test in place. But you want to segment them, so select the ‘A’ Segment. In my example there are 6604 new users in segment ‘A’.
- Go to the Events Summary screen and select the A segment. Click on the Event Statistics icon and you will get the number of people who where part of Group A who clicked on the in-app purchase
This shows 608 people converted. That’s just below 10% in conversion
Do the same for B, and now you can compare the conversion rates. In my example, B has 6320 and 478 conversions. Use a tool such as this online calculator and we find that it is statistically significant.
3) The Bonus Engagement and Metrics
If you go back to the event summary, you can download all the A and B data in CSV format. Go ahead and do that.
Then create a spreadsheet with 3 sheets. A, B and statistical significance. Then you can create a spreadsheet that can test your whole app across all its events, by adding a statistical test to each event. I used this basic formula-
=(((0.5*(‘Split Test A’!B2+’Split Test B’!B2))-‘Split Test A’!B2)^2)/(0.5*(‘Split Test A’!B2+’Split Test B’!B2))+(((0.5*(‘Split Test A’!B2+’Split Test B’!B2))-‘Split Test B’!B2)^2)/(0.5*(‘Split Test A’!B2+’Split Test B’!B2))
But there are plenty of other formulas that maybe more suitable for you..
Next, colour code the spreadsheet so that any significant differences are highlighted and then you can see the impact of the A/B test beyond the scope of just the conversion.
This can bring you many different insights. For example, conversion maybe higher for in app purchases but the number of people recommending or sharing the app using a tweet or facebook button decreases for that group.
Remember to check in which direction the result is statistically significant.
For most people, A/B testing analytics (or split testing) is all about conversion. How can I redesign this webpage to get more people to click through to my goal? How can I get more people to sign up. It’s all about acquisition, acquisition, acquisition… actually there’s more to it than that.
For mobile app analytics, A/B testing tools generally support campaign or content optimization focused on conversion, clicks or triggers. This is fine if what you are doing is marketing focused, acquisition focused or activation focused.. but it doesn’t really help much with engagement or retention. What if you wanted to find out the impact of :-
- Giving new players to your game different starting statistics?
- Change the order that the app screens are presented?
- Modify a tutorial page
etc. what change will increase retention, not just conversions. What change will increase use of other features?
What conventional A/B testing doesn’t tell you -
- Engagement - Which users are more engaged to my app as a whole, or feature X of my app because of A or B (besides the feature/layout under A/B test)?
- Retention - Which users are more likely to be retained (increase retention) because of A or B
- Features - Given that A or B increased retention, what features are used less or more with each group?
- Deeper Activity - Does A or B increase activity anywhere else in my app?
In fact there’s a very easy way to do this using event attributes or event parameters (depends on the app analytics tool you use). In your app framework, assign people who are in the ‘A’ group with an event attibute/parameter as ‘Test Group A’ and likewise for ‘B’ in their App Launch event or whatever naming convention makes sense for you. You can give more unique names for different tests.
By doing this, you have segmented your users into A & B and now you can test the impact across every event/metric in your app.
Segment and extract all the data into a spreadsheet and you can then statistically test them like this -
In the above chart anything in Green was tested as statistically significant. This gives a deeper insight into what other changes occured due to the A/B test and means you can make A/B tests beyond thinking about ‘click’ or ‘tap’ conversions at the top level, remove features that don’t add to engagement, retention or revenue.
Secondarily, using tools which provide retention or lifecycle metrics, you can create a segment using the event parameter and see which version really does provide more retention and allows you to make more discoveries
- Conversions where higher with A, but yielded lower retention. Customer Lifetime Value may be lower because of this so long term revenue could decrease
- Conversion was higher with A, but higher value conversion with a deeper feature is higher in B
- There is no significant difference in conversion between A or B, but retention is higher with A
- There is no significant difference in conversion but engagement is higher with B
These are some of the potential discoveries just looking beyond the basic A/B test.
Do you have any mobile analytics tricks to share?
Image by Search Engine People Blog