The Generation Of Autonomous Automation And What It Looks Like (with Bertold Kolics from mabl)

Read The Transcript:

Jonathon Wright In the digital reality, evolution over revolution prevails. The QA approaches and techniques that worked yesterday will fail you tomorrow. So free your mind. The automation cyborg has been sent back in time. Ted speaker, Jonathon Wright's mission is to help you save the future from bad software.

Welcome to theqalead.com.

Today, I'm joined by a very special guest from Mabl all the way from Texas who was sponsoring our testingfestival.com event, which is now available on-demand. And you can go and register and you can also become a member of The QA Lead community for special time offers. So, go check that out and now onto the show.

So, I want to welcome my guest who's going to be talking to us about autonomous automation and what the next generation of automation looks like. So, welcome to the show.

Bertold Kolics Yeah. I'm so happy to be here and so happy to talk to the listeners of this podcast. I work in Mabl in the engineering team.

I am, hold myself to the question asker, maybe right now we don't have a dedicated testing team. So I work with our engineering team to help them you know, shine a light on the right areas to focus on when it comes to testing. How the tooling kind of lower the barrier to make sure that the, you know, capture the right information for, like defects, make sure you have the right process or tooling in place to support us.

And then, you know, up to developers to focus on what they like doing best, you know, writing code and writing high-quality code. So, I'm working in that capacity and the Mabl is a little bit also very special or, you know, end-users are our testing engineers themselves. I often have a chance to work in my, you know, pulling in similar roles or similar areas that they are working on.

So, that makes my job even more interesting because it almost feels like I have an extended team of testers that I can also understand their concerns. But the other unique aspect of my role at Mabl is that I often act as a subject matter expert and provide feedback on the product itself because I am also one of those end users of the tool.

We also use Mabl for testing our own tool.

Jonathon Wright That's really cool. And I must admit, I love your profile and, you know, I was looking through it and I noticed that you talk about doing internal end-to-end testing off Mabl, right? Which I'm guessing you use Mabl for it. I always think it's a really, you know, and we used to use the acronym, you know, drinking your own champagne or eating your own dog food to use, you know, the proof in the pudding using your own tool to your ultimate, testing your tool, which I think is the infinite paradigm, right?

And you could get to the point where you're talking about inception, you know, a story within a story, but I was really excited cause I looked at your profile. You said you used the dogfooding technique. Well, what is that? Because that sounds really awesome.

Bertold Kolics Yeah. I mean, that's been my not so nice phrase for their drinking their own champagne, like you know, pretty much testing your tool with the tool that you are paying and you know, learning from and there are a couple of reasons why I feel this is really powerful and I hope more and more companies do that.

In general, first because you can catch a little bit of defects, even testing your tool or interface is also has its challenges. And then it's not just a simple tool. It has a lot of dynamic elements, dynamic nature that you would see in real-world application. They're also using a popular framework on the front-end that many of our customers are using react on the front end.

So, testing our own tool is a beneficial, the benefit indirectly those customers as well. The other reason why I really feel so strongly about drinking your own champagne is that especially when you're a double poorer or you're working in an organization, then a better functional testing, feature testing is you know, really the responsibility of the developers is that drinking your own champagne forces you to put your end-user hat on.

And you know, feel, and empathize with figure end users because you're using the exact same tool that they would do, or at least automating it, test for it and, you know, mapping those user stories or into a task and then implementing them. So it just helps you build that outside-in perspective that this sometimes it's hard to you know, maintain when you are working deep down in the code. And you're thinking in terms of the code. So, it was building empathy, and understanding the pain points that end-users go through is another benefit.

Jonathon Wright I love that empathy kind of thing around, you know, again, bucket, you know, where you've kind of put, said, you know, were you know, using CircleCI, you're using Octopus, you're using Jenkins.

[00:05:21] And you're so used to doing that for your own internal engineering. You can kind of have empathy for the people who have got to build Mabl into their continuous testing framework, right? And you know, one of the things I really wanted to pick your brain about, which I think is one of Mabl's is probably massively strong areas around kind of low code is, you know, I keep on thinking there's not many tools on the market that can, you can onboard or get people to start using the tool and seeing benefits from, you know, within minutes and hours, right?

Because usually it's, you've got to load a tool, install it, get you to use it, whereas your tool, you know, whether it's, you know, a Chrome plugin or whatever it may be to start your journey. The journey to actually start being able to automate is so much easier with Mabl, you know. Do you think that is one of the reasons for huge success within your organization?

Is that low touch and low code combination?

Bertold Kolics I mean, what we see with our customers, especially where we have you know, and use it for the, you know, a lot of manual testers is that local platforms is like, easier transitioning to learning more about automation and, you know, that is beneficial for many reasons. If you can, you know, automate certain tasks that should be automated because they are repetitive. They take a long time to do, they are error prone because you're repeating the same past. And you know, you will probably want to consider doing automation portals, and users. So the few and the impactful ones.

And it, and if you are a manual task, I'll call, do you get started without having to, you know, understand the full stack, as you say, you know, create a source code for a field, a framework for it, use the editor, learn the programming language and so on. Then you have a tool that allows you to, you know, create those tasks weekly because you can pretty much record as you go through your application.

As we try to mimic the user flow through your application, and then, you know, replay it and tweak it because the two creates the task for you. And then you can start learning some of those concepts that we are the important later on if you want to go deeper into, you know, regular automation coding, if you want it to. But that's the stuff you can't learn the concepts without, you know, spending a month or so in a kind of a bootcamp before you can get productive The other reason why it is, you know?

Yeah. And I hope that makes sense. So, this is why I feel it's necessarily great. The other reason I feel it is really great is that tools like ourselves or our tool at Mabl is, can also help not just, you know, creating the test itself, but then you are running these tasks and some of these tests are failing.

So again, you want to reduce the amount of time you spend maintaining your tasks, right? So, some of these smarts that we can build into the platform for many things. Adjusting the task automatically for three-year-old change like a locator has changed or a text has changed on some, hang on. You don't want to have to go back and I need to tap.

So there is a lower internet aspect to it as well. But again, it just helps to saves time. So that's I feel is one market of. The other is again, there comes the analysis part is that's also what should make the life of a tester easier, or even double pursue will create this task.

Is that, you know, then when you have a system or tool like RRC, we record a lot of information about each and every stock when you're creating an Antron task. So if something breaks, you again, another big time sink is how quickly can you identify what has gone wrong. And if your framework, you know, records and collects all that information for you, like the natural activity log or create screenshots, highlights how it used to be and how it is now, because you can compare, you know, the screenshots of those areas or just record you know, how the page load times has changed.

Record, you know, any kind of cons or the errors that you would see, then it makes the life of professor or the developer who is looking into an issue much easier because everything is in on place. And then take it to further, you know, as a tester you often work with, you know, issue tracking system. You sure who will allows you to, you know, send all that data into your issue with tracking system, that's even better.

Average thing is collected in one place, you have one source of truth. And it makes that life, they makes that you know, fixing that defect much easier. The third bucket I kind of mentioned is that the local tools, I feel it is much easier for non-technical folks to understand what the testers are doing.

If I were to show somebody like code that would be the equivalent of using any of these any popular, you know, testing libraries. And it's, I'm not trying to, you know, say that, you know, they are not useful in certain contexts, but it's much easier for like a business analyst or product manager to look at maybe a test recording a local tool and understand what it is doing in a high level.

Cause Mabl like this makes a big effort to make sure that, you know, you can annotate these stats. It's easy. What you see in the description of each step what it is doing. And then you can follow along with the screenshots when you have recorded a task, then it makes it quite clear of what is being called the hurt and what is being executed. So there's a much, you know, you can build a much bigger confidence that, you know, what you're testing from the end-user perspective is what you actually expected to be tested.

Jonathon Wright Yeah, definitely. I love those kinds of three use cases. You know, because if your clarity for each one of them, right? Is this kind of, you know, we've all been burned by maybe over-promising kind of automation and I, and part of what I think Mabl revolutionizes the industry and is that how quickly you can get started.

So kind of that, not to ten kinds of term of just getting it some value to demonstrate potentially the power of the tool and the use case of where it's going to save you a whole stack of time. And then we all know the next big challenge really is once you've developed these tasks, which, you know, Mabl is incredibly good at, you know, rapidly doing.

You know, part of it is maintaining those and then getting burnt with the maintenance burden, right? Which again, I think Mabl's really got with this whole safe self-healing tech capability where you can, you know,  you said breaks in selectors or whatever it may be really quickly.

And that self-healing capability means that you're going to be able to reduce the actual effort as far as maintenance is concerned. And then, you know, leading onto that, because you're capturing so much information like you said, the screen, what it's doing at the same time, it makes it so much easier to do that root cause analysis, pinpoint failure analysis.

To be able to diagnose what's wrong, avoid the false positives, if you know something slightly out, but actually get to the point where people can identify where there's potential risk and get the real value from automation. I think that is a really unique offering that Mabl has combining kind of low code with AI to really change the way that people automate, right? At its core level.

And I really liked that kind of that area. And I liked the fact that, like you said, it's also, you know, connecting into your tool for defect tracking, whether that be, you know, Juro or Zephyr or whatever tool you use, or I noticed you had some amazing ones for Google Error Reporting and you know, Google BigQuery and all the things that you're using to analyze the Google cloud services, which you build it on.

But, you know, being able to get that information so that actionable bugs, so you can actually then fix them. And then again, retest and I think that's just such a powerful message, you know. Do you think that actually what Mabl's done is lower the boundary that allows testers and maybe non-technical people with the Loco capabilities to actually be able to get value out of using a tool like Mabl?

Bertold Kolics Yeah. Yes. Absolutely. You know, this is really no, the message you articulated very well Jonathon that they're trying to go after is that by using a tool, you know, you save time and you can focus your efforts on, you know, higher-value activities that, you know, you couldn't do.

If you've learned using a tool like, or so, because again if you don't have to spend as much time on test maintenance, you don't have to spend that much time on reading those tests in the first place. And you know, diagnostic is also lower than that. You know, you can spend that budget on creating new features, which make your development team go faster and you can deliver value faster to your customers.

Or you can, you know, if you are a tester you can spend the time on different types of activities that Mabl may not be able to cover, you know. We are not saying that engine testing is the be all and I know of all the testing, you know. There are many other ask her for testing that should happen prior, and then maybe in parallel to the antenna testing But definitely helps you to spend less time on something that can be automated today.

The other, to me, also the another very exciting aspect of the integrations that we do have and I feel that you know, this really ties into what QA engineers and those working in testing roles really faced with is that, you know, you want to understand.

You want to spend your testing dollars or testing Euros, pounds on the most important activities. And the only way to do that is to understand what your users are doing and using. So, for example, we do have an integration with segment that last year to see within our tool, what pages your users are interacting with the most?

What are the most beaten path in your application and compare and contrast is your test coverage, right? And then and not fair you can tell the tape this beaten path is not as much palmer. This is where I should be spending more time on testing or Hey, this area has many of these defects, because I can see that night CA what tests have associated defects and then maybe I should be creating more tests in this area that may, those tests may not be Anton test, but at least you got real data pulls into the tool and that tells you about the usage.

If you didn't have to have the so easy access to in the past. So, this is, I think is really important. It's really hard to argue against data. Then you have to convince your manager or some other stakeholders, why you should be spending time on, you know, automating certain areas or testing certain areas because it's the data that tells you.

Jonathon Wright Yeah, absolutely. Well, I'm a massive advocate for that chef right approach to look at what's happening, what behaviors are happening on the right-hand side and, you know, things like Google Analytics and Firebase and everything else. It just seems like there's so much opportunity to pull that information across to understand what, where, you know, you should, where your users are actually going, and the behaviors that they're doing.

And then of course catch some of those crash analytics and issues that are potentially, you know, happening so you can address those. But you know, I think, you know, what I really like about that kind of message what you said, and I don't, I've not heard the term before, but the testing dollars kind of side. I was reading some market research, which says, you know, testing represents 25% of the SDLC, the Software Development Life Cycle.

And I was thinking, you know if you are a manager and you're thinking, or you're, you know, you're part of a testing or quality effort, you know, you will need to justify, you know, by buying the right tools, getting the right staff, making the right case for, you know, quality. And as you're doing that, you know, that number, which is a huge percent, 25%, if you think of the total spend, you know, really helps kind of justify well where all the dollars you should be spending it?

And I think, you know, and we, I know people have done this many times before, but you know, if you have those monopoly money testing dollars right now, and you thought, you know what, I can spend this on my exploratory. I can expend this on my automation and regression. And you divide that up. You probably realize that you actually.

There's a, there was a use case there for Mabl type tool to enable continuous testing as part of your pipeline, part of your regression, you know. And there's a number of use cases where you could really benefit with spending those books and in a more effective way. And I think that's difficult because I think there's a paradigm between kind of maybe a lot of the people who are listening now who are either in a kind of a startup kind of mode where there's a, you know, a very fast kind of fail fast learning rapid to the opposite end, where people are doing, wanting to scale, scale automation, you know, enterprise level.

And, you know, maybe you know, people wouldn't normally associate that with Mabl, but you know, I've noticed that you've done a lot of work, especially obviously starting with the Ministry of Testing and getting the Boston meetup as a backup and gay. But you're also speaking a couple of days on building and scaling functional test automation.

I think that is a really key challenge that a lot of people are having is yes, strategically it might work for a certain segment of three organizations, but how do they scale that automation? And then all the technologies that you've just kind of mentioned the low code approach and the self-healing and all these kind of really smart technology is actually going to allow it to, for an enterprise customer to actually be able to adopt this at a greater scale.

So, you know, part of that without giving too much away for the MIT event, but you know, what you seeing there for organizations that are enterprise and they want to scale automation, you know, what advice would you have for them?

Bertold Kolics To me you know, obviously there are a few tips of course that could have them scale these automation efforts.

And then obviously someone with what they've seen is very well in some of these organizations, is that, you know, with one team adopts at all, then you know how you can transplant that to working in other teams as well at once. Once you see the benefit and then you create some of those irons that can work very well in other parts of the organization. You can measure the benefit of other important aspects, you know, scaling these efforts in the organization is just like the money as our practices is, you know.

I've found just reviewing and adjusting the needs, the tools, understanding if you're a business context shifting your user base is shifting how you should adjust your strategy or what needs to be tested and how it should be approached that. You know, again, what I mentioned, if you don't already do leveraging user data usage statistics, either from Google or from other tools so that you know that once you have a base, you know, set of tests, where should you be going next?

Another area that I see is I feel like often forgotten is that, you know, even it's very easy to get into a habit, maybe creating way too many tasks, but you only want to keep as many as it makes sense and not more because say if that's don't you know, add value and they should be removed, as we remove features from our product that are no longer necessary or have been, you know, outdated or replaced by other features, then you should just remove them because that's dead weight.

You don't want to carry that on. Same with your automated test suite. If you regularly review what you have and remove all those that branches, and then you will keep your test suite focused and so on. There also comes a time when the special venue and train the automation with larger teams that that will be sort of flows certain snaps in your testing that could be shared across multiple tasks.

Let's say logging into your application. It could be worn or just setting up as the sort of test environment for you and populating it has data. So you can start thinking about creating those reusable pieces that possibly other team members or other teams can use within your organization.

That is, you know, some of these can be specific to your application. And that goes along those lines that I mention initially when you're moving from manual to learn more about automation. These scales are going to be beneficial even if you go further down the road on becoming a full automation engineer. You understand how you can reuse certain pieces of your tasks or across the, across the test suite.

Just again, just to make it easier to maintain the task that you have and then make it easier for others to create those tests in the first place. Now this one to that, you know, and maybe important is to introduce if you have not introduced in a larger organization, work with developers how they could you know, integrate some of the sentence testings into their day-to-day flow.

So day to day activities since Mabl now can allow you to run those tests locally as well. And you're not, there's few years ago it used to be on that maybe some of these antigen tests only happened that maybe after you deployed your part for the environment, like a double. First case, you tested in production and but now what we did, for example, with our tool, you can allow developers to run those tasks locally and then even create them if they are all for it, which is great because the sooner you can, you know, find out these problems that maybe only surface upon the Anton Bible the better it is for everybody.

And then when you are deploying that code in, even in the staging environment, there are fewer surprises. So, but expanding who is doing those testing, who is creating those testing. So, another way to scale your automation effort.

Jonathon Wright No, that sounds really good. And it's kind of bringing the shift left capability and supporting the dev advocacy, right? Which I think is a really sensible approach. And, you know, just looping back, cause I know you're a bit of a BDD advocate, right? So, you know, I like what you mentioned before around this, you know, being able to get subject matter experts, but also people from the business so they're a bit more readable.

You know, I've spent a bit of time with Dan North in the past and, you know, these approaches that make things a little bit more, you know, you can understand exactly what tests you're, what are the main tests that you want to run and pull those back to, like you said, you know, feature-driven.

So, you know, well, these are the main ones that are getting changed. Maybe adding to run these ones that are kind of a risk-based approach. You know, and I also this kind of ability to bring in, you know, new QA testing, younger testers into it, you know, definitely the kind of the Ministry of Tests kind of background and help you know, upskill them to be able to start creating the building foundations, whether that, like you said, it could be something that they can re-share and reuse, or maybe they can get help within that.

But with their age-established automation engineers, to give them the mentorship that they need to kind of to get started, right? You know, are you seeing that as well? That actually, you know, you're catering for, of course the dev-centric people, the test-centric, the automation-centric people, but also the test-centric kind of manual testers who really want to get started.

You know, do you have any kind of tips for those on how they could start using the tools today?

Bertold Kolics Yeah. I mean, I think of what the test is in general. I think the good test is are you usually have a trait that they are naturally curious and then these local tools, you know, pride themselves on being easy to use and easy to get started. Well, it's quite a few resources for our new users to get started being like Mabl University of every Harper.

South is training classes that pretty much holds your hand and lets you get started and show you what we can ask well it seems simple cases the most have a little sandbox application that you can. Well, this quality that training ground that you can also adjust. That's a different kind of component that you find typically in there about applications.

And you know, I think that's definitely one resource. And then fortunately there are so many great automation-related courses on the internet that can help you get started. The other thing is I feel, as you know, of course, you can always learn on your own, but I think it does even matter when you, as a tester can, you know, understand how the application is expected to work and then figure out what to work together to come up with your good test cases that would be great to have on the and level. And then that's it. So probably that I will prevail, appreciate your work more than you can implement those tasks. And it gives you the opportunity to be creating impactful tasks in the first place.

Yeah, I feel that you know, yeah, these are probably the most important one then. And your testers can also reach out to or maybe participate in some of these user interviews when you watch some of your end-users to do the activities in your application. See what they are struggling and what is hard for them to understand the first place and think about how you could you know, create or think for test cases, how what would being be greater to incorporating your test. So just that there are so many different ways to learn how to create tasks and how to create impactful test. It will make a difference.

Jonathon Wright No, definitely. I love the leaving on that high, you know, the impactful tests statement a bit like the dollar testing, right? Is that it sounds so sensible, you know, this idea of, you know, spend a bit of time with the dev centric side of things to understand, well, what are the changes they're making?

What's the new features? What does that mean? You know, is there any kind of areas that might need a bit more attention this time? And then also the flip side of that to the kind of, what was the business acceptance testing, but they actually use or UAT kind of phases where people are going, this is how I use the system.

This is, you know, what you know, that maybe some of their bugbears and get that kind of thing back of your head saying, well, what are the test that I should be running for that for the end-user and also for, you know, testing that functionality. And I think that's a really strong message.

And I love the fact that you've kind of got, and we'll make sure within the show notes, the Mabl university links are on there and also you know, make sure we've got that. I love that sandbox kind of environment. We ran a, I have a testathon the other day, and we use CandyMapper, which is a website which you can go in and automate, and it has different popups.

And you know, part of it is, I think people have got to learn it, have that playground, have that safe space where they can try something that isn't maybe what they're too close to, which is like the mobile app, or the web app they use in at the moment for their office, you know, actually give them that time to have a little play and see how they deal with different things.

And I think that's really good that you've got that sandbox kind of app, as well as the university, can give them the support. And then of course, you know, they can reach out directly to you guys and to find out more. So, you know, with that kind of closing off, do you know what's the best way would you recommend for people to kind of find out more about Mabl or, you know, is it visit the website?

You know, do you have YouTube videos? What's the best way for someone to find out how to leverage Mabl's capabilities?

Bertold Kolics Yeah, definitely check out the website regularly and posts on our blog. Look out for events that hosting that'd been ours. We also have, you know, a user conference or go to testing conferences as well.

And I think that's the best way to reach us.

Jonathon Wright Perfect. Well, it was as always, it was fantastic having Mabl's sponsor the Testing Festival, which was the biggest testing conference. 20,000 people and the next one at which we're doing in September is going to be a DevOps one. So we'll probably invite you back to do that at DevOps Festival, but it's been an absolute pleasure and I will make sure that all the links are below and check out the Mabl website today.

Bertold Kolics Thank you for having me, Jonathon. It's been a pleasure.

Up for learning from an internationally renowned, award-winning software engineering consultant, author, and coach? Check this article out: LEADERSHIP IN TEST: TESTING TOOLS