Best practices for deploying apps on mobile
By Andrew Morgan
Patience for buggy software is at an all-time low. With millions of apps to choose from, they have become the ultimate disposable commodity. How long does it take you to delete an app that doesn’t work perfectly, right out of the gate?
The truth is that the only apps that can possibly succeed now are those that fulfill their intended function every time, while delivering an outstanding user experience.
In today’s digital economy every company is becoming a software and data company. Software engineers in these companies have a pivotal role as they shape the touchpoints between a brand and its customers. The pressure is on, to deliver quality apps that meet users’ expectations, fulfill the intended functionality, and deliver the required outcomes.
They have to deliver new functionality and support for new devices on a regular basis – working in ‘agile mode’ is a new mandate to keep users happy. To get there, app developers must make quality assurance (QA) and testing a robust and fundamental part of the design and development process by working hand-in-hand with appropriate teams and taking continuous testing principles seriously.
Here are three critical factors for deploying apps on mobile and developing a test strategy that will put you ahead of the pack.
Factor 1: Test automation and DevOps to increase coverage
The demand for faster release schedules and shorter development cycles for deploying apps on mobile has critically weakened the viability of manual testing for professional developers. App developers that still rely upon manual testing are constantly faced with hard choices about which tests to prioritise, severely limiting the overall scope of the testing function.
Automation, when enabled correctly, addresses these challenges and achieves higher test coverage.
As well as increasing the scope of testing, automation enforces rigour and a structure that accelerates QA cycles, allowing you to shift-left. With DevOps you can integrate automated test scripts with every build and introduce testing earlier in the cycle resulting in better releases throughout the lifecycle. It can also enforce homogeneity by uniting web, mobile and web services testing, delivering consistency across multiple builds.
Factor 2: Diversification of device selection for testing
Developers sometimes claim they can only test a few devices during development cycles because of limited budgets and time pressures. But this is an inherently risky proposition, leaving you plenty of scope for catastrophic failure.
A handful of cloud based mobile device labs have matured offerings, which can vastly widen the scope of testing through cloud-based services. When mobile testing labs are used wisely in conjunction with automation tools, testing becomes quicker and more robust, by orders of magnitude.
Factor 3: Introduction of chaos to produce real-life conditions
If lab conditions seem too perfect, it’s time to introduce chaos into the scenario to develop a better indication of real- world possibilities. Unreliable signals, low battery, access to phone functions like speaker-phone camera, background apps and out-of-date systems can all potentially wreak havoc on an app.
Not to mention the types of issues we have all seen in real- world conditions around faulty Bluetooth connections and NFC proximity for mobile payments. To mitigate the impact of these scenarios, developers need to test with these sub-optimal conditions in mind. What you don’t know will probably hurt you – or at least make your consumers unsatisfied if they aren’t prepared.
The same testing labs mentioned above can simulate a wide variety of factors that replicate typical conditions that end-users face in the real world. Only when you have tested for poor conditions and understand the real-world impact of interactions between the app, the device and the network, can you consider that you have truly tested an app in the wild. Another form of delivering real-world feedback is through crowdsourced testing, or crowdtesting.
Likewise, there are mature service providers which can deliver valuable feedback from real world users on real networks on real devices.
Deploying apps on mobile: best practice in QA & test management
For those teams that are relatively new to the vagaries and challenges of the mobile development process, you may not be prepared for the expanded complexity and scope involved in mobile versus other media. Having clarified the three major success factors, here are seven additional best practices for mobile functional testing, polished from multiple years of mobile test experience.
1. Collaborate on the requirements for testing
Without clarity on what needs to be tested, there can be no test plan. And without a test plan, there should be no testing. Collaboration among business- dev-QA-operation teams is key to avoiding confusion about application functionality and scope of testing.
The development team (and the operations team in a DevOps environment) will have a handle on which user commands, integrations, processes and screens will be most critical for an app.
2. Create a test plan and prioritise tests
It’s both surprising and worrying that this step is sometimes left out entirely. The test plan is not a boring task for simply writing down what you already know. Instead, a good test plan maps out an actual strategy for how testing should be done. Typically, the hardest part of creating a test plan is the mental gymnastics that go into the test strategy, rather than physically writing down the plan. A best- practice testing plan should incorporate the objectives and scope of testing, the test resources required (including people, software tools and hardware) and a test schedule. Additionally, the plan should prioritise and rank test cases that are to be developed.
Not every test is of equal importance to every other test.
3. Determine what can be automated
As noted above, test automation can be an important component of delivering quality software. Determining what can be automated should be a considered part of the test plan creation, especially for mobile app development. In that case, the rule should be to automate as much as possible – when it makes business sense.
Testing automation improves software quality, while reducing time to market. But common sense has to be applied, as well. If a manual test can accomplish an outcome more cheaply and effectively than an automated test, it absolutely makes sense to do so.
This blog provides guidelines on knowing when to automate. Additionally, if a functionality is mission critical with limited devices and can severely impact an organisation, it shouldn’t be automated. It’s also important to think long term when considering automation. While test automation may cost slightly more in the short term, the long-term financial benefits can be significant.
Our research has shown that with the right support, most enterprises can comfortably automate about 80% of their testing. And those savings only multiply once you can factoring the data and insights gained (and efforts saved) from automation.
4. Execute your tests under real user conditions wherever possible
At the risk of repetition, it’s important to find a way to test under real user conditions. Web developers don’t have to concern themselves with what happens when they go out of data coverage or when a user receives an SMS the same way mobile app developers do.
Without question, this increases the number of test cases, but this is simply a reality for mobile app development. However, once again, I would highlight that test automation is incredibly useful when considering the functional requirements under various app conditions.
5. Manage defects with appropriate sophistication
At a basic level, an issue tracking system will help any team with the simple task of centralising the test repository. It can help with the assigning of the tasks, and it can provide a simple one-to-one relationship with an issue or bug and its test case.
Depending upon the level of sophistication in an organisation, that may be enough. Typically, a straight issue tracker becomes less capable the more sophisticated the scenario, especially those with multiple configurations, different workflows and different data sets. For more sophisticated environments, investment in a mature mobile test management system is required. Some examples include tools by Tricentis, QMetry, Plutora, and Jira.
6. Simplify reporting of test results
In theory, the management of results should be one of the easiest elements to handle in the testing process. However, in reality, this is dependent upon the test management tools that are used.
A sophisticated test management system makes the abstraction and presentation of test results very straightforward. They are automatically created, and a dashboard is available to stakeholders at any time. Meanwhile, an Excel-based system typically requires hours – or even days – of admin, with additional formatting required at reporting time.
In an agile setup, a distinctly un-agile method of reporting is hardly the right fit. Regardless of the system used, the reporting element needs to communicate both the status and progress of testing in such a way that’s comprehensible by both development and management. It also needs to pave the way for useful analysis. It seems like common sense, but I can’t even count the number of times that this is overlooked and not made a priority.
7. Analyse the test metrics
Last, but most definitely not the least, is analytics. Depending upon the sophistication of the testing and the scope of the project, analysing test metrics is imperative to the continuous performance and improvement of any solution. In smaller setups, the analysis won’t go far beyond presenting what has been tested and what doesn’t work.
In more sophisticated operations, the analysis should highlight problems and opportunities in both the development and testing processes. Doing so provides a source of constant improvements to ensure the operation becomes much smarter over time. This means incorporating everything from functional and API tests, and mapping them back to UX requirements, data sets, and app analytics.
There is always more that could go into an article of this nature, but nonetheless, following these guidelines for deploying apps on mobile will go a long way to getting your mobile testing initiative up to standard – and making it far less likely that your app is one of those that gets deleted in minutes, or seconds after download.