Solving FUT pain points
It’s not all good news. Friendly User Test (FUT) programs need significant management resources and friendly user persistence to drive the potential benefits. And even then results can be disappointing: data is inaccurate or incomplete, errors are difficult or impossible to reproduce and weekly feedback reports arrive too late. Which means that, ultimately, many FUT programs fail to help the product management team gain the valuable insights they need to support their prioritisation, escalation and release decisions.
Our software addresses these problems by providing device and user management tools, real-time feedback and automated tests that can be event, user or developer driven.
User & smartphone device management
Benefit ⋕1: Essential administrative control
Every FUT program needs an administrator to manage participants (users), devices (smartphones) and feedback. This can range from tracking equipment and people to chasing feedback reports in a timely fashion. It’s a full-time job to manage even small groups in closely defined geographies.
Our software (MRiX) solves these problems and allows product development teams to scale field tests to large groups spread across geographies and time zones. We help administrators with tools that manage devices, query usage statistics, accurately record feedback and even profile users.
Issue latency
Benefit ⋕2: Shorter TTF (Time-To-Fix)
MRiX treats subscribers, devices and users as unique identities. And any identity can communicate with another, if they have permission, in real time, using instant messaging (IM) protocols.
This significantly reduces the communication time between identities, for example using “Publish and Subscribe”, devices on a South American network may publish WiFi problems to a node, a developer who’s interested in these problems can subscribe to this node and instantly be notified of problems occurring, and even directly access a device to diagnose a problem. This has a dramatic impact on the time taken to report an issue, the time taken to replicate an issue and ultimately the time taken to fix that issue. We call this ‘issue latency’ and anecdotally (at least) we know this can save product development teams weeks and months.
Automated test capability
Benefit ⋕3: Control of the field environment
Automation saves time and increases accuracy – particularly for complex, concurrent test cases. MRiX supports a scripting language to facilitate development of new tests. ‘Scripting’ allows office-based users to augment the friendly users’ normal activity with automated mobile tests.
Scripts can be automatically triggered at specific events, for example after a phone call, panic (crash) or time interval. But it’s also possible to set up complex smartphone tasks that conduct automated stress tests without the user’s direct involvement. Which is useful to product teams that want to understand issues by replicating them across other users and devices.
Statistically significant samples
Benefit ⋕4: Release confidence
Our software simplifies FUT administration, gathers data efficiently, enables effortlessly addition of further tests and analyses the data, which allows Product Managers to target larger groups of friendly testers inside the organisation and even their supplier and customer channels.
Not only does this build statistically significant sample sizes, it also helps to target specific user behaviours and demographics through profiling methods. So with MRiX it becomes simple to to stress test how your Facebook-centric smartphone works with ‘Angry birds’ followers on the East coast of America.