In our previous post, we provided 4 insights to increasing the success rate of mobility projects. One of the key insights that led to these is learning from failure. So, if learning from failure is so important then why are all case studies about success?
In this article, we will summarize 3 mobility projects that failed and what we can learn from them. The projects include:
- one of the top global beer brands
- a leading European real estate agency
- a major carrier/mobile operator
Note: We avoid using the name of the customer to protect the people involved.
The Global Beer Brand
Beer commercials are probably the most well researched, invested and tested marketing campaigns of any industry. The brands and agencies often spend 6 months or more on market research, customer personas, brand studies, insights and more before the creative process begins. Our customer was no exception.
One would assume that they would take a digital platform for a sponsorship with potential reach of 100 million or more customers just as seriously.
Planning for the event started almost 18 months before. This included evaluations from previous events, campaigns and research. Early on there was a high-level concept agreed for mobile and web. The concept was a platform to make the viewers and attendees of the event interact. We requested to start 12 months before with prototyping and user testing to refine the concept.
4 months before the event we were finally given a go ahead for the project as the overall creative had been signed off. Now there was no time for testing or refining the concept, defining the success factors and aligning teams. Instead we went straight on to development and finished two weeks before the event.
The agreed concept was delivered but:
A) The concept didn’t really solve a core customer problem/need which became apparent in the conversion rate from customers being informed
B) Customers didn’t understand the service at first due to some issues that we would have easily solved with user testing
C) We achieved a few hundred thousand active users which was considered a partial success but we believe they could have achieved tens of millions with the right concept and aligning success factors/targets across teams
D) There were a considerable number of technical issues/bugs at launch since the implementation was done under stress with limited time for testing
E) The service didn’t have a material impact early on the overall event and therefore didn’t get the promotional support it needed to reach its potential
What we learned
- Getting it right for a big event/launch takes time no matter what agencies and developers say. One year is minimum for a service with potential to reach 100 million people or more.
- Any service that doesn’t solve a customer problem/need is likely to fail even if the concept seems good
- Always test the concept before developing and launching as this will help refine the core use cases and UX
- Setting targets and aligning across stakeholders is key to keep support beyond launch
The Leading Real Estate Agency
Property is big business and although digital channels have a big impact on finding a property it’s still a face-to-face trade. Therefore, understanding customer needs is key to success.
The brief we received from the real estate company was to create a light version of its website for mobile including app and web and replicate this across the different brands. In the initial meetings, we discussed how we could address specific mobile needs in terms of core use cases. But we were told by the management that the first priority was to deliver what was already available on the web and then we could discuss other things.
Consequently, we went ahead and implemented what we had been told for mobile web and as native apps for Android and iPhone. Some limited user testing was carried out during the acceptance testing but by then it was too late to make changes.
A) Customers immediately complained that the functionality on mobile was limited compared to web
B) Downloads and engagement of the apps was about 1/100 compared to the property search competitors (which were considerable smaller businesses)
C) Mobile web ended up outperforming normal web within a couple of years and due to the original decision to just replicate the website it didn’t meet customer needs
D) It was unclear to the customer if the project was a success or a failure and therefore they couldn’t assess the ROI and thus motivate additional funding
E) Three years later the apps were killed as they didn’t get enough traction
What we learned
Replicating a website as a light version for mobile users is generally doomed to fail
- Apps that have the same functionality and UX as their web counterpart are generally doomed to fail as they don’t leverage the mobile specific use cases (better to not do apps at all)
- Not setting targets for a project makes it difficult to assess success/failure and thus justify additional investments
- To compete with digital companies the incumbents need to reimagine their business in a mobile-only world to succeed
The Major Carrier/Mobile Operator
Mobile voice and data services is a very competitive business. Most countries have three or more players and the main competitive drivers are price, network coverage and device offers. Any other service that can help differentiate can be a huge advantage to a business. For O2 in the UK it’s been O2 Priority Moments and for T-mobile US, the Uncarrier initiatives. Both were based on extensive research, testing and investments in propositions that would meet customer needs.
Our brief from the mobile operator was to look at the main problems customers call customer care about or ask for help with in the stores and solve these problems through a mobile app and responsive website. This is a great starting point as it was clearly based on customer needs. However, in a mobile world you can do so much more.
Jointly we designed a mobile interface that would make it easy for customers to solve their problems without having to call or enter a store. We also created a preliminary roadmap for the first year.
The customer set a target of engaging 10% with the users in the first year, partially it was set low to ensure the target was achieved. No KPIs/targets were set to calculate revenue uplift and cost savings from the service.
The service was launched successfully about 12 months after the project started.
A) The initial scope defined by the customer was too complex. The project team was tired and demotivated by the end of the project as it took so long. In addition to this we found a lot of issues after launch which could have been avoided with a phased approach.
B) About 15% of the customers engaged with the mobile service in the first year but after this it flattened out
C) We estimate that the customer got a return of about $5m the first year after launch based on an initial investment of $750k. However, the customer didn’t define KPIs so it’s unclear what the real impact was.
D) Due to the lack of metrics the customer didn’t continue to invest in improving the service despite a potential of over $100m in incremental revenue and cost savings over the next 3 years. They merely kept the service running with some minor enhancements.
What we learned
- Our preference was to start with a MVP and launch 5-6 months after the project start to achieve quick results. Instead we agreed to the bigger scope. We should have pushed back harder.
- The lack of clear success factors/ metrics resulted in reduced buy-in from management to continue investing despite the initial success
- It would have been better to focus on web or mobile apps initially instead of doing all at the same time
In summary, we learn a lot from failures and we’ve used these learnings to improve our engagement model. As per the previous blog, our customer success rate has increased significantly over the past 3 years.
Keep on sharing and learning from mistakes!
Magnus Jern, President DMI International