The Digital Quality Handbook (with Eran Kinsbruner from Perforce Software)

Read The Transcript:

We’re trying out transcribing our podcasts using a software program. Please forgive any typos as the bot isn’t correct 100% of the time.

Intro In the digital reality, evolution over revolution prevails. The QA approaches and techniques that worked yesterday will fail you tomorrow, so free your mind. The automation cyborg has been sent back in time. TED speaker, Jonathon Wright's, mission is to help you save the future from bad software.

Jonathon Wright Hey, welcome to the show. Today, I have a very special guest, who I've known for a very long time. He's a good friend of mine, Eran. He's the Chief Evangelist and author for Perfecto and also, which is now Perforce Software. He's written a number of books. I was lucky to be part of The Digital Quality Handbook. He's just put the latest one out on Continuous Testing for DevOps Professional. I definitely recommend checking that one out. He's the most knowledgeable person in the world when it comes to mobile. We're going to find out what he's working on now, new books, next-generation technology, exciting times. Hey, Eran.

Eran Kinsbruner Hey, Jonathon. Excited to be here. Thank you for inviting me. Hopefully, it will be a nice Q&A. Bring it on.

Jonathon Wright Absolutely. You were the first person I reached out to when I was thinking about setting up this podcast. I sat watching your trends for 2020 and I was just blown away with some of the stuff you've been working on in AI and testing in QA. Would you be able to tell the listeners a little bit about the kind of exciting things you've been working on?

Eran Kinsbruner Sure. Obviously, I'm working on a third book, which I cannot say that much about right now, but stay tuned, follow me. The thing that I am able to talk about and now, obviously, becoming reality and will becoming even more real to all of the testing practitioners out there is everything that is related to AI and machine learning. This is no longer a buzzword or a trend. I actually am working and involved in projects that are AI, machine learning-driven.

As mentioned, I am working right now, and for the past eight years, at Perfecto, which is a Perforce company. In the Perfecto product itself, I can mention that we have in the reporting, automated root cause analysis that is driven by machine learning algorithms. We have codeless Selenium, which is based on record and playback with a self-healing mechanism, so it's also machine learning. It's not just about what I work on in Perfecto. We see a lot of new technologies and tools.

KK from Cloudbase just announced that he's launching a scriptless test automation creation that is risk-based. It can scan through Jenkins and all the other code changes that are happening in the pipeline and based on the changes, create relevant scripts that will save you time from developing, running, and maintaining scripts that are no maybe any longer relevant. AI machine learning and testing is definitely happening and will receive a huge boost in 2020.

The second thing that I'm looking at right now, I started looking into it in early 2019, and I start to see it becoming even more real, is everything that is about the conversion between mobile and web, in the form of progressive websites. This is another topic that is becoming popular, PWA. The third one is around affordable smartphones and how they evolve and become an equal citizen in the mobile testing or mobile industry that we live in today.

Jonathon, I'm happy to uncover specifics of each of these three threads. There are many more, but if you want, we can start by talking about one or a few of these ones that I just mentioned.

Jonathon Wright Absolutely. I'm really excited about foldable. I know there are probably only a few people that are, but it always reminds me of one of the great interactions I had with you guys. I was working for a large pizza retailer and we were looking at, if you remember, back in the time when the IOS watch came out. I remember saying to you, "How quickly can you get one of those devices and get it docked into your system?" You guys were the first that were there.

Part of what the company wanted, the pizza company, is they wanted to be the first on the app store. It's kind of, to me, that it was like a second screen technology because even though the functionality of this pizza app was literally just clicked it on, and open the app open, and do a repeat order, we found that, actually, it was really interesting. Because we'd been using the emulators and I think emulators only get you so far. That's great for writing the scripts, but you've got to test it on the real thing, right?

Eran Kinsbruner Definitely, yeah.

Jonathon Wright You've already got foldable phones in your labs, right?

Eran Kinsbruner Yes, yes. We have, actually, the two Samsung foldable smartphones, the Galaxy Fold X and the Galaxy Fold Z. These are two different foldable smartphones running on Android 10. They are not the only ones. LG just launched in the canceled Mobile World Congress event. It was canceled, but launches did happen, so LG, Samsung, and Huawei are the three early adopters of foldable smartphones. Android 10, as an operating system, was the first operating system in the market. IOS is still, I would say, lagging behind. Android 10 came out with a baked-in Samsung emulator just for developers and testers who want to get started with these new and exciting devices.

Definitely, this is a challenge. These devices have a large price tag on them, over $2,000, which is not something that anyone can and wants to afford, but they serve a specific purpose. They serve as a twin one, a smartphone and a tablet, both in one device. They also provide users a much more flexible working environment, if you like, so they can work on multiple apps as they would do on a normal desktop.

They can work on a smartphone, and then they move to the working environment, which is the tablet, and then they have three applications running in the foreground, in parallel. For those of you who are listening to this podcast and you are Android users, you know that once you are putting an app in the foreground, the others go to the background. With foldable and Android 10, all the three applications are running at the same time, actually competing on resources at the same time.

You can think about the browser, which you are looking for a restaurant, you talked about the pizza delivery, so you are looking for a pizza. You are opening the Chrome browser on the Samsung Galaxy Fold, and it launches, let's say, the Waze or the Google Maps application, they are all running in parallel so you can see the menu on the browser, and you can navigate to this location on the GPS application all in the same time, all in the foreground.

Jonathon Wright That sounds like there's a whole stack of new possible challenges. I remember we'd been daunted by IRQs when they came out, and they gave you that kind of ability to have a channel to request them, but if you're sharing resources like the camera, like the GPX information and the GPS, all these sensors, how well are they going to react with multiple applications accessing them at the same time?

Eran Kinsbruner Yeah, yeah. The resource allocation, resource management is becoming extremely challenging. Also, keep in mind, Jonathon, the event management. When you have three applications running, not in the background, in the foreground, and you are also getting a phone call or a push notification from an app or a message, everything happens at once for you. This is, not just from a testing perspective is a headache, but also from a user perspective, users need to learn a new way of engaging with smartphones, because things happen in parallel and they need to prioritize.

This is why, to your point about resource management, if an app that will be, maybe, docked as an always-on application won't behave properly, will drain too many resources and stuff like that. This app will be undocked real fast and allocate room for a different one, which behaves more nicely from an end-user perspective. This is definitely a mindset changing for both application developers, testers, but also for the users who now have a new way of engaging with apps.

Jonathon Wright It's unbelievable how far things have come. I remember this great example when I was working with you guys and we were building a Smart City application. It was for the City of Copenhagen. It was the ability to help them become carbon neutral by 2020. Then part of what it was doing was recommending the most carbon-neutral route to work. We paired up with you guys because there was no possible way that A, were going to get devices, that amount of device that we needed with that variety but also, we needed this ability to inject the GPX, so we could actually see these phones, believing that they're moving around the City of Copenhagen.

I remember working with you guys and [Uzi 00:10:38] around extending that capability, which was just feeding that metadata, and then we could run those devices, even though they weren't there. A bit like with the Pokémon kind of thing, where they could actually feel like they were walking around the city. This was fascinating. One of the things we did and we learned was because we looked at crowd testing, and what we asked the Copenhagen University students to do is actually to have a GPX logger on there and start tracking, taking sample data, so we could then create ... Out of that delta, we could generate four billion different historical transactions for the Smart City data exchange, so that way it had historical information in there of different routes and recommendations that it made.

One of the things we identified with the ... I don't think this would have been possible without you guys was the sheer battery drain. Because, obviously, we were sampling data and GPX information. Nobody likes an app that drains the battery. If you've got three apps open, all looking and trying to access resources, and some of these things are quite heavy, especially when you start looking at augmented reality, like what Pokémon Go did. Part of it is it starts draining because it's using the camera. The processors are getting absolutely hammered.

There are all these different aspects, which I don't think people are completely familiar with, especially when they think about the everyday use of their mobile app and what that actually means as far as resources. Now, do you find that there are either more metadata coming out with these devices that, actually, you might want to interact with? Like, I guess, you always used to be able to tell ... Can you tell the individual dock screens? Or is there more functionality, like being able to allocate it to a particular window?

Eran Kinsbruner Yes, yes. I can share with you, Jonathon, for you to share also with your audience, I actually ran a webinar where I drafted an entire test run, if you like, for testing foldable smartphones. In order for Android developers who are listening to this podcast, to have them join the party of foldable smartphones because if you don't do anything to your app, the app will when someone launches it, it will be launched, and when you close the screen or unfold it or fold it, it will probably go to the background.

From Android Q, Android 10 and above, you have a flag in your manifest, in the Android application, to allow, enable, if you like, multi-resume capability. It's like a new capability added to the manifest file of the Android app that enables multi-resume. Multi-resume is the enabler of three applications running in parallel. You can see that an app can move from a pause to a resume state upon folding and unfolding the device. If the multi-resume flag is not enabled, set to do, it won't resume automatically.

Going back to your question, you can define a specific app, up to three applications, located in a specific area on the screen. Once that you enable the multi-resume, they will always be thereupon closing the device or unfolding the device, so this is something that most Android developers and testers need to know, both from development, but then also from a testing perspective.

Split window, multi-resume, these are new terms that any Android application developer and tester needs to start getting used to from Android Q, Android 10 and above. Jonathon, we are now in days that Android 11 is already in beta two, so we are expecting late summer to see a new Android being released to the market, maybe with more foldable being also introduced.

Jonathon Wright That's really interesting because I know the new IOS, they've got the new headphones that are coming on because they notice the new icon. Obviously, if you've got the AirPods, you see the icon, they've got noise-canceling capabilities, so you have the ability to do audio validation input from the microphone. I know you guys support biometrics as well, things like face ID. I saw the webinar you guys did with the guy from ThoughtWorks. These are all different types of inputs, and the ability to inject video and images. Also, IOS, looking at the new things like the iGlasses, so you're going to start having AR as well and MR, so mixed reality capabilities were, actually, apps are going to start docking on the screen. As a proud previous owner of the Google Glasses-

Eran Kinsbruner Oh, yeah.

Jonathon Wright ... That never really worked out, but this time 'round, do you see that actually, do you predict that you're going to start seeing the mix between mixed reality start coming through down the line? Maybe 2022 or something?

Eran Kinsbruner That's actually, I didn't start talking about 5G, but with 5G being deployed all around the globe, I think that you will start seeing much more advanced technologies including AR, VR, large video streaming capabilities. Smart Cities and everything connected are becoming more advanced and more adopted. I think that in the past, technology wasn't ready for devices like Google Glasses and other connected devices, IOT devices.

I think that as 5G becomes widely spread, and I think we are already, I would say, in a 30 to 40% global spread of 5G. It's not perfect yet, but it's getting there. I would say late 2020, as 5G is totally there. You start seeing, even last February, Samsung and Apple, by the way, is also talking about a new iPhone that is being launched in September with a tag 5G. It's going to be iPhone Pro 5G.

I think with all these 5G enablers, you will start seeing, maybe in 2021 to be more realistic, a new range of technologies, connected devices, connected cities, [F-Car 00:17:31], cars, everything that was talked about in the past few years as part of the digital transformation, going back to your chapter by the way, in my book, in our book, I think that this happening and I think that the 5G will be the barrier that ... Or the un-blocker to this evolution that we'll see.

Jonathon Wright I completely agree. I definitely recommend for anyone, spin-up Amazon now. Type in "The Digital Quality Handbook." You can get it on both Kindle, you can also get it as a physical book, and also in color. I'm sure, if you ask Eran nicely, he'll probably give us an audiobook one day as well. Jokes aside. I know, as a Tesla owner now, that the technology that's in there for self-driving, fair enough there's still a long way before we're going to get to be completely autonomous, but even the mini ...

You've starting to see these heads up displays, but also with the mini glasses actually have information about, coming from the car, around the speed, around the sat-nav, all overlaid into the glasses. These things aren't that far away. Once the technology's there, people will come up with new and exciting, creative ways of doing that. You're absolutely right. You need that 5G bandwidth to remove the latency.

I remember, if you've seen my TED Talk, in the background, there's this really good example around Car-to-X and Infrastructure-to-X, where people are walking around a city, and it's giving you information about, "Keep off the road because there's a car coming," so it goes red. There's heads up information about the person who's coming towards you that you know, or maybe some of the predictive crime stuff that we did. Or even coronavirus, it's saying, "Please keep a certain distance away from it." All that information is overlaid.

The latencies have to be reduced. 4G was never really fit for that kind of thing. 5G is going to definitely see the increase of infrastructure to M2M connections and this kind of support for IOT. Do you see that there are a lot more device interactions that are upstream and downstream from a mobile device that people are doing? Like accessing their TV, accessing their smart home, their smart office. Do people test these to the level that they need to? Connected car.

I know the guys when I talked to them at one of the German manufacturers, they still go in and manually test their systems or use an emulator because it's difficult to have the entire car sat in your test lab. For me, there's no substitute for physical hardware and not over-emulation. Is that what you're seeing as well?

Eran Kinsbruner I see a mix, actually, of things. Some of our customers, without naming them, from the automotive industry are testing their smart box, if you like, in our Cloud, so they actually have a simulator that is basically the screen with all the technology that you have in the cars, hosted in the Cloud, connecting via Bluetooth to devices, to a Perfecto or Cloud of devices.

They are doing a lot of, not just voice, but also Google testing, synchronization testing between the devices and the smart box that is baked into each of these cars. I see that, and this is not the device or the mobile phone tested against the real car. It's against the box that is built into the car. It's as close as possible, and it reduces a bit the overload that you have when testing it just manually.

I see a mix of complementary manual testing that is done within a specific segment. You mentioned automotive. It can be also healthcare. We are also doing testing for healthcare with smart glucose connectors that are Bluetooth-based, BLE devices that are connecting with smartphones to report back to the healthcare provider about your glucose levels, diabetes and stuff like that. I think I'm starting to see, going back to your initial question, I'm starting to see more and more use cases that are across the verticals. Healthcare, automotive, retail as well.

One of the retailers that work with us has kind of a GPS-based capability in his mobile app that when you get into the stores you can actually, based on your interests and previous purchases can guide you like a GPS application to the exact same zone so it will save you time when you're going to buy a specific artifact. I start seeing much more advanced capabilities in mobile that are also connecting to IOT devices, so definitely the innovation never stops, especially these days. Going back to the 5G comment, 5G network, resilient networks are enablers for such innovation.

Jonathon Wright When the launch of 5G came out, obviously, I was really excited. I also remember the 4G rollout for when I was living in New Zealand.

Eran Kinsbruner Oh, yeah.

Jonathon Wright There was a really good case, which they had, which they used as a test location. It was one of the first places to roll out 4G. There was this small problem. That everything rolled out perfectly, they got great connections but then the emergency services weren't able to dial through, so people actually died from the rollout of 4G over in New Zealand.

Eran Kinsbruner Oh, boy.

Jonathon Wright You look at that kind of side of things of new versus old technology. We were talking a little about it today in the sense of, unfortunately, [GT 00:24:11], the coronavirus, a lot of what was retail systems are getting a lot more traffic, they're under a lot more load. I remember, we're going to get Todd DeCapua on the show, and partly, he came from Shunra for the network virtualization. The first-ever demo he showed me, which I couldn't believe my eyes with, was around just the additional load that was put on by latency. Because the latency, what was associated with mobile traffic meant that all your hardware is stuck waiting for that response to come through and while it's waiting, it's not able to get on with other things.

Now, yes, architecture's slightly changed now, but at the same time, what 5Gs doing, in essence, is reducing that latency and that amount of time. It's actually, it's like a desktop equivalent. You're actually going to start getting more load on systems with less latency, which means there's going to be a different type of traffic profile. Are you starting to see that as far as what the data throughputs are on a device that is throttled at 4G versus 5G, and what that actually means for the round trip of a request going through to an end system?

Eran Kinsbruner Yes, totally. I think that even before that, what I'm mostly concerned about is we are discussing the network contribution to user experience issues, the adoption of 5G to enable technologies. If you look specifically, also, from your own daily experience, Jonathon, we've just seen today some specific retail and healthcare website. I think that this concept of shift left and shift right as a whole topic, that's what concerns me the most today.

I think that we're actually advancing this discussion, while the market today, in my mind, doesn't really consider non-functional testing. It's not just about performance. It's performance, and security, and accessibility as something that is equal in priority and importance, like any functional and regression testing that you are doing. I see it over, and over again.

You don't need to wait for this Black Fridays and other major events to see that websites, mobile web applications, hybrid applications are not ready for, not even prime time, for just something above normal load perspective. This is my major concern today. What do you see from your own projects that you are involved in today?

Jonathon Wright I'd say probably exactly the same thing. There are two dangers which I see. The first one is that mobile development. Because of the technology that you're doing doesn't always share the same as the full-stack teams. There is some re-use, so obviously, with React Native. I'm doing a project at the moment with React Native. The e-commerce site is built on a different type of architecture. Maybe that's Ruby on Rails. Maybe that's some other MVC kind of thing. The app is a separate team, so they're going at different speeds.

From a business perspective, whether or not they're doing bimodal, trimodal kind of approach to how fast they're releasing, the kind of OpsDev approach of, "What is the operational state that you have to understand?" The big one that I found the hard way as Google Analytics doesn't support information around mobile. You have to go through Firebase, so you're getting different metrics, which are the same metrics for the business.

The business care about if conversion rates, and retention, and people coming back, but they're getting data from two different sources. Then what you're having to do is you're then having to pull that data together to then give you a view of why mobile and web should be a single pane. I think that's a real challenge because that's where people have got things like Snowflake and they're starting to have data scientists and data engineering. Data engineering and test roles emerging where they're having to get information from different sources of truth.

From an operational standpoint, the Ops guys really need to understand, well, what is the end state? What are they looking for? What are they looking for as far as, you might have a McAfee from a security perspective, you may have some really snazzy APM? AI Ops is obviously a big trend, so your Dynatraces, your New Relics, your [Op-Tees 00:29:10], your Splunk's.

They're taking all this data and doing really cool stuff with it, but they're still treating mobile and web as separate streams of information. That doesn't help when it comes to cascading KPIs, and exec dashboards, and understanding operational statuses of, "Well, what happens if the system goes down? Well, how does the systemic failure work?" The site reliability engineering and chaos engineering is something which I think people aren't really extending out to mobile yet. Yes, they may be thinking about the web. They might think, "Well, what happens if they pull down this particular node? Does it spin back up? What kind of data? How does it, the fallback or self-heal?" I think mobile is still, is territory that people aren't completely familiar with.

I think I was talking to you about doing some work around telemetry beforehand. Embedding telemetry in React Native so that I can understand more about what's going on with the user. What activities are they doing? What are they interacting for? How long are they waiting for things to become enabled? What's the end experience?"

I think digital experience analytics, to me, it's the big thing where they pull in all these different data sources from APNs, from all these different tools from the IT operations landscape, and they start seeing a single pane of glass of what's happening operationally, and with the Cloud infrastructure, the multi-Cloud across a number of different providers, as well as hybrid as well from our own end just to confuse things.

I think there's still a massive divide between ADMs, application delivery management, and Dev, shall we call it, then the Op side of things and the IT operations management, and the speeds which both of those, and the innovation that happens between the both of those. That's where I'm seeing it. I know you're at the forefront of AI, and AI within the test but I guess, you're also seeing what the AI in operations can also do and how that data has to come together from a shift left and a shift right perspective.

Eran Kinsbruner I definitely think so. Yeah, I definitely think so, Jonathon. I just wanted to say something about the previous comment, the previous discussion on the network. I forgot to mention an important asset or an important resource that I tend to go to when I'm discussing with clients on things that are shifting left, performance testing, user experience.

I don't know if you are familiar, you are using the open signal website, but open signal, it's free. It's a very good resource that shows you at any given time pair geography, UK, Germany, Australia, whatever location that you think about. The current benchmark, distributed by different telcos. Let's say for the U.S., you will get Verizon, T-Mobile, AT&T, and Sprint differentiated by download speed, by 4G adoption, 5G, and other metrics.

It gives like a scale, a benchmark on each category of network performance and A, you can choose the top-performing one and the least performing one as part of your testing strategy but also, you can understand some of the implications on your end-users if you do not really test against these providers because they will definitely suffer. You have the ability to optimize, on your end, some of the calls, the load, or whatever optimizations that you can think of in your app.

Jonathon Wright I think that's really great advice. It's interesting when it gets down to this network level. I was in France with someone who was talking about just the difference between network switches and different sites for different organizations. People are VPN'ing in now into their work offices by working remotely. You're starting to add these additional layers, whether that be Cisco or it could be your Citrix, thick kind of virtual desktop on-demand.

They've got all these different aspects, which you've got to think about, but people are expecting them to be able to run at these fast response times. You've got things like speed test and there's a number of apps out there where you can capture what the actual traffic is like on a site. I had this where I was sticking a Raspberry Pi at each one of our locations. It was a building, construction company. Some of them were building in the middle of nowhere, so as you could probably expect, a bit like what we did with Copenhagen.

When people got on the train from the airport, everyone was on the shared, one main connection that was free WiFi on the train. Packet loss, time to re-try. Apps just weren't dealing very well, because not many of them have this kind of catch for if they lose signal. You just get these really obscure kinds of error messages. I think we can't get away with those things anymore because people have less and less tolerance for mobile applications just not performing as they should do. Especially recently, with everything that's going on, there's a lot more load coming through and you've got people ...

We had Disney+ launch recently, like this week in the UK. I know when it launched over in the States, they had a challenge with the service being unavailable. This is big brand damage for some of these kinds of organizations. I think a focus maybe needs to move and shift away from the website side of things, and focus more towards the mobile aspect because we're getting to that turning point where there are more people using mobile.

Yes, they may be using your website, but I remember looking on your site, a Note 10, which had a huge resolution. Therefore, when I opened Chrome up and went to the mobile version of a website, it was just physically unreadable. It didn't scale. I couldn't actually interact with any of the menus because they were too small. I think people still think that the easy way around is just to have a website that works with mobile. I think it's getting more complex than that.

I talked, said about how such a big advocate I am for physical devices. We caught this error which the emulators just wouldn't pick up, which was we do 90% of our advertising through Facebook. Facebook loads a browser, which is in a Windows browser, which is a cut down version of Safari. It's not the whole ... It's not Chrome or anything like that. On certain Android devices, on certain phone factors, it literally wouldn't load. It was because of the capability of that end browser on that particular release of the phone.

I just think there's a lot of cross-browser testing focus but nobody, apart from you guys, has really nailed the cross-mobile or high-performance mobile testing. I really think that most organizations that are out there really need to think about, how do they make sure they know, at a bare minimum ... You're going onto your Google Analytics, which everyone's got. You go in there, you're pulling new devices. If you've not downloaded already, I'd recommend downloading the mobile test coverage that you guys put out to tell you what kind of devices are typically in your region, what kind of split, and literally make sure that you're getting the right type of devices.

I've got behind me an IOS SE, because the screen factor on there is much smaller, and some apps just don't load. It's amazing that these things are still happening. When I spoke to someone recently about it and I said to him, "Well, Android still represents one-third of your users." They said, "Ah, well, we'll do the IOS first, because that's the one that everyone uses." I was like, "Well, no. Because if one-third of your customers, that's one-third of our revenues."

Equally, if you're looking at the two-thirds of your revenue, which is IOS, apparently, then look at the breakdown of devices and you'll find that, actually, your older devices, your IOS 6s and your 7s, which are starting to get restricted on IOS version upgrades, are literally kind of getting end of life, and are still having a really bad experience with the mobile app.

There are so many things that people need to take into consideration. I think the mobile test coverage guide is an essential piece of work because it changes from country to country. I remember the last time I was in ... It was a Canadian podcast there. In Canada, people still use BlackBerrys because it's still a device that's registered on there. Whereas, we've seen the end of the Windows phones. It is a two-horse race at the moment. Do you see things like Android variation for your Huawei's, who are kind of getting pulled from being able to use Google? Do you think things are going to start changing, you'll see even more variation in the Android landscape?

Eran Kinsbruner The fact that you mentioned the coverage and your experience, especially with the SE. As you speak, these days, I'm authoring a new version of the Index, of the Coverage Index, we call. Yeah, I think at the end of the day, the end-user does not care which device he's using or which browser he's opening. He's always going to blame the application vendor or the website designer if something doesn't work. He won't go back to Apple and say, "Listen, I'm using the iPhone SE and the app doesn't look nice on my device." It's the application developers' and testers' responsibility to ensure that the app works.

You mentioned Google Analytics. The developers, the business knows, at any given time, which chunk of the industry, the target market uses which devices, so it needs to be part of a testing strategy that accommodates the coverage considerations. That's definitely a valid point. Stay tuned for the next edition of the Index. It will be a 2020 edition with all the considerations.

Back to your point on the mobile and cross-mobile testing, Jonathon. Perfecto gives you not just the mobile testing, but we also have a Selenium grid in the Cloud so you can actually test across both mobile devices and desktop browsers. That's the virtual machines in parallel at the same time, so think about having a responsive web application today that, obviously, needs to run on any Android IOS, tablets, and desktops. You can test all of them at once using Selenium on the Perfecto Cloud, so it's not just mobile, it's mobile and web, which is the [contriarity 00:41:15], as we mentioned earlier. 

Jonathon Wright I know the Continuous Testing book, which you wrote, is a fantastic resource. I was a big fan of the quantum framework when it first came out with this kind of, the idea of having something like BDD and Cucumber in this case, and really start baking it into your developers' kind of landscape. I think it's something that I personally, though recently, got a bit more hands-on with. There's so much variation of people using different IDEs, using different versions of Java, whether they're using Maven, they're using Gradle.

There's so much variation out there and also in the CI, the continuous integration and continuous delivery toolsets. It's great to have this kind of low-code approach to be able to actually start running against so many tests very quickly. If you go into the Perfecto Cloud, you could literally run, click on the different gestures, you can do whatever you want and then you can capture out the actual script out the back of it. I know you've done a lot around this kind of scriptless landscape which is, it's fantastic because otherwise, there's just no way to keep up.

Eran Kinsbruner Definitely, definitely. I second that point.

Jonathon Wright I know one of the other things you talk about is the shift, potentially, from Appium to potentially looking at Flutter as a new way of doing cross-platform app development, which is done by Google. Are you seeing people, it's starting to ramp up the number of people using the SDK for that?

Eran Kinsbruner It's being adopted slowly. I think that today, still the majority relies on Appium. Appium has been out there for a long while, and people have the existing scripts, but I can tell you that for application developers, that are just now starting to write a whole new mobile application, native applications or hybrid as well, they are looking at both or multiple technologies.

Flutter, in one hand, has the ability not just to provide you a cross-platform development framework so you can actually write with one singular code base and add the trans on IOS and Android, but you can also get underneath from Google, a supporting test automation framework, quite similar to the Appium driver, so it's an alternative for new application developers. For the ones that are existing or they are maintaining existing mobile applications, I still see Appium, Espresso and XCUI test as the leading frameworks of choice.

Jonathon Wright I love that kind of flexibility, which you've got. I love how easy it is also to connect. A, to reserve a device, which you've got, and then and be able to run that as part of your pipeline against a selection of devices that you just pass in the codes for. That makes it really nice and simple, especially within Appium.

One of the challenges that I think we were talking about before, but with this kind of, well, things like crash analytics and stuff, you're starting to see a lot of the APMs out there offering telemetry information to give you more information about what happens when a crash happens, give more information about battery drain and device debugging information. Do you think that's going to be something we're going to start seeing more of as well, is this kind of more information and insight that's coming out on user behavior, of what's actually happening on a physical device?

Maybe some of these APMs moving from synthetic tests, which maybe don't really prove anything, to actually start using real devices to make sure that the services are up and they are able to connect to maybe your secondary sign-in kind of capabilities with two-factor authentication, which you get with Apple and stuff. Do you think they're going to start using real devices to prove the system is working and the mobile apps, loading the apps, and the pages, and the screens in a reasonable amount of time? 

Eran Kinsbruner Yeah. I think that both myself and I think you as well, Jonathon, was talking about this for multiple years now. I think that as this entire industry shifts to digital-only, especially these days when we are discussing in the podcast. I think that digital is clearly one, and when we talk about digital, we're talking about physical devices, physical platforms that need to work at any given time across different networks, capabilities, screen sizes, operating system versions, and stuff like that.

I think that we are past the excuses time and it's now every user expectation that things will simply work reliably with good performance, good functionality, regardless of the type of device. Whether it's an iPhone SE with a four-inch screen size, or a foldable, or a new Samsung, which has a 6.9 inches screen. It's no longer an issue, it simply needs to work. That's it.

Jonathon Wright Absolutely. Where people have spent so much focus on making sure that their website is working and people optimizing, maybe the checkout, because of the bounce rate's high. You're going to start seeing that too. Some people actually understand down to, "Well, what are the activities that people are trying to do on their mobile phone? What's the user behavior changing? How's that behavior changing as time evolves?"

Because I think a lot of people miss this kind of, well, early adopters, they're more tolerant, typically, of issues, but then if you've ever seen, read the book, Bridging the Chasm, part of it is, how do you get across to the early majority, and the late majority, and the laggards where the big adoption is? I see so many new apps on the App Store that are incredibly promising, and then if they don't work, they just get deleted. It's a fire and forget kind of approach. Some apps make it, some apps don't.

If you really want that engagement, and you want people to keep on coming back, you've got to build better apps. I think the only way they could do that is with real devices and utilizing the latest capabilities, which you guys are offering. I know we could literally talk about this all day, but I know you've got an upcoming event, which you're doing on the first of April, which is going to be around mobile and web.

Could you give the listeners some of the things that they can start signing-up for, ways to get in touch with you, some of the little projects that you're working on that you can share with people, and the best way to contact you?

Eran Kinsbruner Sure. Sure, Jonathon. Just to your previous comment, I would say the one chance that you have with mobile apps. That's behind the second book that I authored around continuous testing. Continuous testings enable you to keep up with our end-users' expectations. Because you can get it right for version 1.2 and everyone would like it, will write it five stars. If 1.3 or 2.0 doesn't work fine, they will delete it, so you don't enjoy maintaining quality. You need to continuously maintain it to gain continuous customer retention. You fail once, maybe they will forgive you. You fail twice, definitely, your app will be deleted. That's just, back to your previous comment.

Regarding the upcoming things that I'm involved in, and for those of you who are not following me on Twitter, you can search Eran Kinsbruner on Twitter or LinkedIn. Feel free to connect with me. I accept everyone, so this is one. I'm going to run next week, April 1st, a webinar about why it's so important, especially due to the recent events that we are all facing, it's important to move your testing into a Cloud infrastructure, whether it's mobile or web.

You need to have a continuous testing mechanism engine that works regardless of anything else that happens around you, in your business, in the world. It's all going to be focused on the benefits of a Cloud solution as part of your testing strategy, and how to actually get started with testing in the Cloud. That's the upcoming webinar.

Mentioned earlier as well, if you care about test automation coverage, as Jonathon highlighted, I'm authoring, and it's going to be probably out in May, a new version of the Test Coverage Index, featuring nine different countries across all continents. If you have a multi-regional application or website that you want to know on which platforms web browsers and mobile devices to test against, stay tuned for the list of the next index. Also, stay tuned for, potentially, upcoming new book later this year. I will provide comments as soon as I can on that as well.

Jonathon Wright Wonderful. Well, it's been an absolute pleasure to have you on the show. If you've not checked out Perfecto yet, go to Perfecto.io to find out. It's a free trial. Literally, you can click up, register, and have access straight to the labs and devices, totally free. I definitely recommend. It's the best in class when it comes to mobile testing. Thanks so much for being on the show. We will have to get you back talking about more future trends later in the year.

Eran Kinsbruner Looking forward. Thank you for having me, Jonathon. Thank you so much.