The Dark Arts Wizard of Quality (With Paul Grossman)

Jonathon Wright Hello, and welcome to the show. Today, I have a very good friend, and also the dark art wizard is upon us.

Paul Grossman Yes, thank you, Jonathon. Do you know that even dark wizards are now practicing social distancing? Allow me to demonstrate, [inaudible 00:00:49], now we are 30 feet in the air, so you have nothing to fear unless of course, you have a fear of heights, Mr. Wright. Hey Jonathon, how you doing, it's Paul.

Jonathon Wright Hey Paul, it's great to have you on the show, I know you messaged me on Twitter the other day, and I think I said, "Which is the worst? The killing curse, or the coronavirus?" I think it's 50/50 for Voldemort. He probably started it in all fairness.

Paul Grossman Absolutely, I think he did, and the major problem you've got is the only spell that we know of that might work against protecting people from a variety of substances, is impervious, but it doesn't seem to work with that. Best advice everybody, wash hands, and I hate to turn into a PSA, but here's a bit of advice that I got from working a medical facility, back when I first started automation. When you're in the washroom, obviously you've washed your hands, but you get a paper towel, and you go to dry off your hands, use that paper towel to open up that door out of that washroom, and then use it to get to your desk, open up whatever the two or three doors to get to from there. That was some advice I got years ago, and it actually cut down my colds on a yearly basis from about two down to one, so it's about 50%, just making that sort of change. I try to share that with everybody out there, but let's talk about just automation.

Jonathon Wright Absolutely, but that's some great advice. So you've got one of the most interesting careers that we have in test automation. You might even be classed as now the godfather of test automation. I don't know if you can do that voice, but you're dating back all the way to, [inaudible 00:02:44] Mercury [Instructor 00:02:46], you did Mercury World, I know you're a fellow Mercury Man Award winner, so what started your career in testing and QA?

Paul Grossman Well, this story has been told before, so I'll give you a brief version of it. I was working for that medical device manufacturer company, as I mentioned, and I was working on a system that would shut down and turn on, it would take about 10 minutes to recycle, really big, the awesome system at the time, and I had 10 minutes of downtime, so I decided to practice ... I actually do magic on the side, so I had to practice some putting knots into ropes, and my manager had perfect timing. She walked in and saw me doing that and said, "Is it possible you could be doing something more productive, rather than these magic tricks? We hired you to test our hardware," and I said, "Sure," and the great thing was is that she had perfect timing. She had walked in about 20 seconds before my system was about to restart. You powered it down, and then it came right back up.

So I said, "Yes, I'm doing a Lazarus test," and I basically shocked my system and it powered right off, right on time, and then she looked at that and went, "Is that supposed to do that?" I said, "No, absolutely," I snap my fingers, and we had a universal power supply on the back of this thing, a UPS, that would detect if the system had shut down, and it would send a wake-on-LAN command across the comport until the system will boot up. As soon as I snapped my fingers, the system started powering up and showing that it was starting up, and it would be another five minutes before I was ready, and she looked at me, she looked at the rest of us, and she says, "You QA people, you are a little off the charts here. I'm never coming up to this lab again." True to her word, she did that.

The following day, I kind of went over to her office, and I said I wanted to try and do some automation, because I had seen someone using a WinRunner box, and I told her, I said, "I opened the box, I did it at home, I did not do this on company time," and I automated a game site.

I basically went to Pogo.com, and I automated their Turbo 21 game, which is basically Blackjack with four hands. I did that with WinRunner, and the interesting thing about that was that I didn't know it at the time because I was brand new, you couldn't look at any of the objects or elements on that game.

All I had was pixels, and I figured out these interesting ways of identifying the suits and the values of cards based on two or three pixels I saw on the screen. I show that to my manager, and she said, "That's cool," and I said, "I'd like to join the automation team. I want to be the guy who learns it all, and learns all this stuff," and she says, "Great, you can join the team," but she said, "There's one thing. Do you know who the project lead is?" I said, "I don't know, I think I saw the guy walking around, but I'm like a sponge, I'll suck it in," and she goes, "Well, you're the project lead now, because you opened the box," and as it turned out, she didn't like that little trick I had done in the lab the day before, and she basically put me onto a defunct project.

There were three members on the team, one had retired, another one moved to another department, the third one basically wasn't really on the project anymore, and all I was left with was some of the code and the documentation of what they were doing. I had to basically start on my own. I do not hold anything against her for that, she set me on the path that I am today, and I thank her for what she did. That's basically where I started, getting automation with WinRunner.

Jonathon Wright Yeah, and you've got a YouTube [inaudible 00:07:47] of the three-pixel playing card value identification, and I think it's fascinating because the use cases for WinRunner have changed over the years, this all exciting and new way to utilize these automation tools. I [inaudible 00:08:06] they call it now business process automation, right? It's interesting because of that same example when I was in New Zealand, and I was working with WinRunner again, and my friend played online Poki, and he said to me, he said, "I [inaudible 00:08:23] 10 hands, I want to play 10 tables at once. However, there's this small little problem. It's not illegal to ..." because they get the rate back, so you're allowed to play as many tables as you want, but you have to be active at your desktop.

What will happen is they'll send you messages and make sure you're there, so it might be like, "Well what's the weather in Toronto look like today," and they expect a reasonable request. I said to him, "Well, you could just grab that and get WinRunner to keep on looking at that box, and then send you a message for your SMTP, or for your text, and then you could automatically then respond. It'll wait for the response, and then read it in."

Absolutely incorrect use for automation tools, but if you think about that today, that's what a business process tool would do, is waiting for two-factor authentication, it needs to get a code from your mobile to then continue. It's interesting that those use cases, actually what you're talking about when you started off, are actually as important now for things like computer vision, and they are more complicated systems, which includes image-based testing.

I know you are the wizard of object-based testing, and I'm sure we've got some really interesting stuff to talk about later on when we talk about the Magic Object Model, but as you've evolved into this capability, what have you learned along the way, for tools and tips that people who are listening might benefit from?

Paul Grossman Well, one of the things I've learned is to be a little open to all sorts of different tools that are out there. One of the tools that I loved using in both WinRunner and in UFT, is AutoIt, it's also known as auto-script or auto [inaudible 00:10:18] three. Some people call it auto IT, and it's a nice little utility because first of all, it's mostly for automating or interacting with elements on your desktop. If you have an application that needs to interact with an application that's locally installed, rather than a web, you can actually get auto [inaudible 00:10:43] to do some of that interaction.

One of the demos I did that actually got me a client many years ago, was that I had intentionally written a script in WinRunner, that would basically ... it was a recursive function, it would basically continuously call itself and call itself, and of course, when you write a recursive loop like that, it eventually does a couple of weird things. It generally eats up all your memory, and the problem was that, at the point where it ate up all the memory, you would just basically throw up a window that said, "Oh, I can't go anymore because I've run out of memory," and then how do you get your automation scripts to recover from that if you've done something like that?

What I did in the demo is that I had AutoIt, which was a nice little ... It's a basic language, but it has the advantage of being complied, so you can have an executable of your code. I had that running on a separate thread, sitting there watching for this window to pop, and then when it did, it would essentially grab the text of the window, which told us what line number the error occurred at, and then click on the, I think it was a cancel button on there, that would stop the execution.

It was a really cool, that's one cool tool that's out there that I don't know that a lot of people know about. It did help me a lot when I switched from WinRunner over to QTP. WinRunner, I loved, for a couple of reasons. One, you could edit the code while you're running. That's something you could, obviously, people working Eclipse and IntelliJ, that's nothing new, but that tool could do that.

Also, when you ran it in step-by-step mode, and you had a step breakpoint, you could drag the arrow back up and re-execute code, and you could change it on the fly and re-execute it. I loved that. When I went over to QTP, one of the things I was missing was that WinRunner had this wonderful little ability to put up a dialog box, and you could put in radio buttons and checkboxes and all sorts of stuff, and you could set up your own little execution dialog that said, "Which test cases do you want to run?" I couldn't do that in UFT. UFT, all it had was a message box and an input box, but what I was able to do is basically write the same thing in AutoIt, and then call it, and then pop up my little message boxes that said, "Which test cases do you want to execute," and run from there.

That's one tool that's out there, that I would recommend people take a look at. It's old, but if you go looking in the help directory of AutoIt, and I'm not being facetious here, there are about 1000 different scripts already pre-written for automation with that tool, and you can find just about anything that you need to do. The only limitation is it doesn't work really well with web browsers, it's not specific to web browsing. So that's-

Jonathon Wright Yeah, [crosstalk 00:13:46].

Paul Grossman There you go.

Jonathon Wright That's really good advice, and it's interesting because I know definitely now, you're kind of the Selenium master, right? Part of it is, it's great that that's there, and we're both coming from Xrunner or WinRunner days. I love the idea, like you said, with the AutoIt, you've got things like what you've done in the past with the [inaudible 00:14:08], right?

Paul Grossman Yes.

Jonathon Wright It's simple things like that to keep sessions alive. I've used that before, for weird versions of Oracle, Java popups that appear, and you're thinking to yourself, "How do I hook into this," when your primary technology stack is, maybe you're only using WebDriver, so anything. You go to save a file in a browser, and you're having to navigate through your Windows objects, straight away you start getting problems, or it opens up into Adobe and you need to export it out into a text file or something. There are so many workarounds, that tools like that are incredibly powerful. I think probably a lot of listeners have been exposed to the landscape of web applications, but maybe don't remember those Bluezone [inaudible 00:15:06] emulation tools, which the mainframes have still got.

Paul Grossman You just read my mind, Jonathon. That's one of the challenges that I've got moving to Selenium. I will tell you, some of the things I may say here may sound like heresy. One of the things I do say is that I'm not entirely a fan of open-source stuff. Now that I've said it, hopefully, the podcast hasn't ended here, but the thing I find is that when we have clients that they say, "Okay, I want to save on all the licensing that I'm paying for these tools, and there's an open-source, there's a Selenium out here, we'll just switch over to that," it doesn't necessarily mean you've got all the functionality that you've got from the tool that you're leaving.

I'll tell you, one thing that I really love about UFT is that the interaction it does with Bluezone Terminal Emulators, interacting with mainframes, I always tell somebody, "It just works." You just plugin, hit the API, and it can interact with the terminal emulator to get your data from a mainframe, and interact with it, submit jobs, that's basically what we're doing is just submitting jobs and jobs and jobs.

That's one of the great things about UFT, QTP, is that it has that built-in ability to work with different toolsets, and because it has such longevity, I mean there's a lot of old rituals out there, and if you got the new stuff, then you got the new toolsets of Selenium and all that stuff, it may not necessarily interact with the older stuff that's out there. You got to kind of [inaudible 00:16:51] it up, so that's one of the things I just want to say, I just love about UFT, that you can just interact with just about any element that's out there, just about any browser.

Jonathon Wright I know we [inaudible 00:17:07] talking before that you've joined a company that you wrote the QTP and UFT scripts, and now they've moved into more open-source. The example you've just given, I remembered the [HAL 00:17:22] API, which meant that it wasn't just text-only support for those green screens, you could actually communicate at the API level, and API's been a big thing now, but you don't really think about that when Cobalt, 90% of all traffic still in the world is generated by Cobalt, which unless you've got Cobalt in the cloud, fortunately, you're back to that kind of green-screen systems.

I was amazed when I was in Silicon Valley, that your people like PayPal are still running on X.com's mainframe systems, but we look at them, that kind of organizations, and go, "Oh yeah, they must be all microservices and open APIs, and all have swagger specs for everything, and it's nice and easy," but the pain for most of those people, and most of probably the listeners here, is they're testing the states which are 20, 30 years old. It's a real challenge. What kind of challenges are you finding now, going back into an estate where you've gone from an enterprise tool, to open-source? Is there a lot more custom [inaudible 00:18:31], and how much effort is there?

Paul Grossman At the moment, I actually have team members researching on how to interact with the Bluezone Terminal Emulator, and I'm hearing stuff back, so they're looking at screen scrapers. I certainly hope that they can do a solution, going through the HAL DLL API, and go that way. I'm trying to think of what other challenges we have. My major challenge is just trying to translate the cool tricks that I did in VBScript for 18 years, into Java.

Just this week, I was trying to ... The requirement was let's open up a list, a dropdown list, and verify that the items are in order. What we found is that, for most test cases here, for most test cases, you're generally selecting an item from a list. Almost nobody ever goes and says, "I just want to open the list and find out what's inside of it and verify what's in there." I think I spent two hours trying to figure out the right way to go and just open a list, and try and click on it, no the click doesn't work. Can I drop down the JavaScript? No, that doesn't work.

After about two hours, I actually went back to one of my team members and I said, "What's your idea? Throw something out to me." She said, "Have you tried sending a space?" I went, "Oh my god, I did that like four or five years ago with VBScript," and sure enough, I did a [inaudible 00:20:09] space, and like magic, the list opened up and showed me everything I needed there, and so that's the challenges I've got working on that, getting that taken care of.

The last [inaudible 00:20:23] I did mention, I think I did at the top of your show, is like speed, just trying to get execution speed faster. If you work on frameworks, you can end up over-complicating it, over-thinking everything that's in there. I've done that myself in UFT, trying to put in a lot of cool stuff, and then realizing, "Oh my gosh, I just put about a 20% load on the execution time of my framework," whatever this was that I was trying to put in there. It just isn't worth my time.

One of the things I'm doing now on a project is just trying to find ways to speed up the execution, populate our fields faster, validate our elements faster. I can explain one of those topics on that, that I'm kind of working at. Generally, when you're working in Selenium, you're looking at expected states of elements, and this basically goes back to my first mentor, it was Linda Hayes, who works over at ... she has the certified product. She taught me, she said, "If you're writing automation framework, the first thing you want to do is, once you get an element, verify that it exists and that it's enabled," and that way you're not throwing errors that you couldn't enter any value in there, you couldn't click on it. I thought that was great.

I find a different approach on that, is that you can validate that with the first element right after you do a click, or a select because those are the two things that are most likely updating your webpage. Within the second and third and the fourth element that you've got, the pages most likely are already built, so maybe we don't need to do those validations, does it exist, is it visible. Why don't we just go in, interact with the element, we want to make sure it's visible on that, but then basically just throw out data in the fields in order to interact with those? And then if we then throw an error that says, "I'm sorry, the button doesn't exist," or an element is sitting on top of your element, at that point, do a catch and say, "Okay, let me try a different approach on that."

I'm the biggest supporter of second chances, so if we're throwing an error, and something happened with the element, and it's something on top of it, maybe we need to take a different approach at how we're going to interact with that element, taking a second or third try at that.

Jonathon Wright Yeah, it makes a lot of sense. I remember one of my first ever bit of code I wrote in PSL, was a popup handler, which was embedded in whatever dynamic, push-dynamic button, or clicks the dynamic button. What we were doing, and my mentor back in the 90s was a guy called [inaudible 00:23:25] [Tobosky 00:23:25], who wrote quite a lot of patterns for Siemens on WinRunner frameworks, really interesting, I'll send you the links.

Part of it was, even back then, what he was doing was taking the object repository, exporting it out into CSV, and then using that as the values that would create is data-driven, dynamic object repository. Part of it is obviously part of the problem with [inaudible 00:23:54], which could be the name on the button, and then part of it is, it would do all that kind of state, and that was the same learning that I was taught. Make sure the DOM, that is visible, make sure it's enabled, make sure you can interact with it, and it's not read-only or something, then put the value in, and then also once you've finished inserting the value, using whichever adding the type of function you need to use, then check the value is being put in there correctly if it had some weird ISO characters or something.

So you've got all these pre-imposed checkings on just a single click, and I was quite impressed by what you were saying about just how much time you've been able to save by looking at ... Maybe some of that lazy stuff where people are putting weights in there, which obviously should never happen, but that idea of being able to see if the object exists yet, and then its rate, and keep on checking. Do you find that as a problem when it comes to running large volumes of automation?

Paul Grossman I hate to say, most of the test cases that we do aren't anymore large-volumes, and we do have large volumes of test cases that execute, but for the most part, they are short test cases that just are 30 seconds to two minutes at most, and then we're basically refreshing the browser and starting all over again. It does remind me of an interesting thing back when I first started, and this goes back again to my WinRunner days, when I first started doing automation, we had one system setup that was basically just registering patients all day long, it was dedicated. Once I had basically shown me... I got some street cred with the developers that, "Hey, this guy knows what he's doing," they actually dedicated one of our four systems to just running, registering patients, all day long, all night long, and we basically registered about 24000 patients of random data in there over the course of two weeks, running this over and over again.

In that case, when we first came up with the idea, I'm like, "Okay, this is about seven lines of code, just populate the fields, hit the submit, populate the fields, hit submit," and the first time we ran it, we actually identified a memory leak in that. The memory leak popped within about five minutes. Once the developers came back, looked at it like, "Oh, yes, we found that," we set it up again and ran it again, and that point, the memory leak popped up after 20 minutes. They went and looked back through their code, "Oh, well yeah, we forgot to ... we were depending on some garbage cleanup, where we forgot to destroy some objects along the way."

I try it again, and over the course of like three days, we finally got it to the point where it was not eating up all the memory, but they had a great ROI on this, because the application we're working with, they're used in the medical industry, and they're used constantly. If you're having a medical procedure being done, you really don't want to have an X-ray machine that is just eating up all your memory in the middle of a procedure. That was kind of a big point on that.

Today, back to your question about long-term runs, we have lots of tests, but the long-term runs, I try to work on essentially making sure that we have either really well-defined element descriptions, or a smart way of self-healing our objects if the objects tend to change over the course of time.

Jonathon Wright So this is the Magic Object Model design that happened that you've got.

Paul Grossman Yeah, it's the ... Go ahead, I'm sorry.

Jonathon Wright Yeah, could you tell us a little bit more about that, and where you've used it, and how [inaudible 00:28:03] the list is leveraged [inaudible 00:28:04] that kind of capability?

Paul Grossman Sure. The Magic Object Model basically came from an idea from a problem that we had. I'll give you a little background on it. We were working on an application that had been basically ... We had automated everything on it, it was working smoothly, we ran it about every three months, and identified one or two elements. The problem was at that point, they changed the underlying architecture of the application, which changed the class of just about every link that we had there to a web element, and of course, if you're working with UFT, and by the way, this isn't specific to UFT, this is almost in any application. Selenium, or TestComplete, any tool that's out there. If you have a repository, if you're changing the properties internally, that's an easy fix, but if your class changes from a link to a button or something like that, that's a little bit more challenging in order to fix, and in our case, what we found was that there were about 50 different elements in our repository that had changed the class.

The longer links now, that'd be described as web elements, and not only that, they were all referenced, since this was one of our older projects, the beginning of that project had a little bit of copy/pasting going on, so for every element, we had four references to that. We had about 200 different places in the code, plus 50 elements to fix, and we estimated that it would take us about two days to fix that element, that process. I looked at my partner and I said, "Maybe we can try what I call class switching," which is kind of an underlying supporting portion of the Magic Object Model.

Let's say if I get this element, and I check to see if it exists, and it does not then let's see if we can build an element on the fly, but rather than as a link, write it as a web element, and then see if that exists. If that goes, works fine, then continue on and just replace that element. That worked out really great. Instead of having two days of a lot of work, within about 90 minutes, we had 95 of our 100 test cases up and running, and the last five, the problems that I identified, the three of them were just an element that just was so weird, that this trick just didn't work. We just had to fix one element in the object repository.

The last two that failed were actually identifying an error in the application, and we just had a 90-minute delay on that. That's our backstory, so we go forward a little bit and I thought, "Well, what if we could basically take that and describe elements just on the fly," and let's say I just create an object factory that I say, "I'm looking for a link," and it has added to cart inside in there somewhere, and the object factory goes out to our DOM and says, "Okay, give me back all of the links. All right, now just give me the links that have to add to cart inside of a description somewhere. Oh, and also, only the links that have a width greater than zero, so it's not the invisible ones.

90% of the time, it narrows it down to one element on your page. There will be some exceptions, obviously with checkboxes, and radio buttons in that case, but in this case, you can just go ask the factory, the Magic Object Model, curious that the two descriptions I'm looking for a link, and add to cart, and see if you can find me the element. If you can't, let's go click on it, and it'll work on it from there.

The underlying part of that is, of course, people listening to this, and I get this all the time is like, "Wait for a second, that's not going to work for every single element. There are multiple developers on your team, and not everybody is creating their element the same way," which is okay, because you implement this and then you come across another button that is describing some other weird way, like a refund, let's say we're looking for a refund button, but instead of being inside of the text, maybe it's inside of the ID, or it's embedded somewhere inside of the inner HTML, then basically you do the same thing.

You say, "Okay, I try this first description of this element, and I couldn't find it." Well, let's try a second description, a secondary version. "Oh, I did find it that way," and then you basically start building off your framework in that approach, and then as things change, it tends to basically remove the maintenance that you've got inside the framework.

Now, to be clear, I'm not looking at using this to replace the Page Object Model, I think that actually works really, really well, putting our find-bys in there, to find your elements. I'm looking at it as kind of support to, in case our find-bys don't work, we have a safety net that you can go back to with the Magic Object Model and say, "Okay, I identified the find-by, let's go and use this element factory to go see if we can find it through that. I've already got the descriptions of how I originally described the element, let's go feed it over to this thing, see if we can find it there.

So that's kind of the overview of the Magic Object Model, and I'd also mention class switching in there, and then just recently, I found myself unemployed for several months, and with nothing to do except invent projects for myself. So I made friends with the people over at Test Project, and they wanted me to see if I wanted to contribute maybe an add-in to their project over there. I was talking with the founder, Mark Kardashov, and Karen Teboulle, and they said, "Would you like to try and do an add-in?" I'm like, "Well, I got to learn Java better, why not?"

So if you do go over to the Test Project IO, you'll find that there is an add-in out there that's written by me, Paul Grossman, also this is the dark arts wizard, and it's actually a natural language processing engine, which sits on top of the Magic Object Model of course, and the idea is, and this sounds crazy, you just type in what you want to do. Click on the okay button, and then the natural language processing engine breaks that down and says, "Okay, I know I need to do a click, I know I'm looking for a button, and it's got okay somewhere inside the element." Let me put that through the Magic Object Model, see if I can find an element, I found one, cool, and it's feasible, and it's enabled, let's go click on it, what's our next step?" Then basically process the sentence as it goes along.

It was a fun little project, I'm still working on it, and I will tell people out there, most people I describe this too, they're like, "That's really way out there. We'll stick with the Page Object Model," but if anybody wants to join on this project, I am willing to put the Magic Object Model out there on Git to kind of add to it. I can open-source that, it's a cool little tool, so I guess I'm just saying, I'm looking for other people to add more ideas to it because I think it's a neat approach that could definitely be leveraged in the future.

Jonathon Wright Yeah, definitely, and I will definitely, if you put that onto GitHub and make that available,

[inaudible 00:36:02] on the show notes, but also happy to have a look and contribute. That is such a big thing. I think you're probably the master now of ROI, but part of where I've always found automation's really difficult is this, how do you get rid of that maintenance there? How do you get rid of all those objects changing, and things breaking? I think there are many different ways that we've done it.

I know when we did the last [inaudible 00:36:32] event together, part of it was Micro Focus UFT had just launched their AI capabilities, which I did have a little play with, and I kind of see what they're trying to do, if they're blending a bit like what Perfecto did, and we got [Eran 00:36:47] on the show in a few weeks, what Perfecto have done with this hybrid model of being able to switch between image and the object OBT technology, so image-based technology, and an object-based technology.

That was great for mobile because some of the most native apps when you didn't have that same level of DOM, and it's really interesting because I don't think anyone's really fixed it. The way they were trying to do it was just text identification, OCR, Tesseract kind of stuff. I think there is a lot of work that still needs to be done on that, and what we used to do because we used to use fuzzy logic to identify the structure of containing structure, to understand how many object types are there.

To identify, yeah, there are three buttons on there, and there are two list boxes, okay. If you remember the WinRunner days, used to have that radius burn, which would kind of say, "If the description for that box was 99 pixels away, then we'll [inaudible 00:37:57]," you'd tweak it a little bit and it would work. The same kind of thing is that you look at something like [Appio 00:38:03], and you'll find that the actual wrapper within the container is three or four levels up before you get down to the object class, and these are rules that we're having to deal with.

If you remember Flex when Flex kind of hit the street, that was the same thing. You'd find Flex, four to five, would change the object model, would change the number of levels the containers were, so it got more and more complex. I know visual testing is a big thing at the moment, and I've seen what the Applitool guys are doing is really cool. I love the TestProject.io that you mentioned because that is Selenium on steroids.

You've got a great demo on your YouTube, around four minutes on showing how you could actually get it working with continuous testing. I think a lot of people are struggling with these areas, and I know you've done a lot of Udemy stuff. What would you recommend to people who are trying to learn more about automation and testing?

Paul Grossman Okay, well I'm going to go right back over to Test Project to start off with because it has four really good reasons for beginners to start playing around with Selenium. To be honest, like I said, I'm a VBScript guy, and jumping over to Java was no small feat for me to get into.

One of the things I love about the Test Project is that you don't have to worry about what version of your Java files is in there. If you've got everything on the path, all you do is you just basically sign up, and by the way, it's free. I don't know how they're paying everybody, but it's free, and you download a test agent. The agent, once you execute it, it sets up all the file folders and where everything's supposed to be set, and you don't have to worry about it. And they send out new ones, says there's a new release of different Java files are available, they put out new releases. All you have to do is do the installation.

The second thing about it is that it starts off code-less. I know everybody talks about, "Oh, record and playback, it's horrible and terrible," unless you don't actually know how to write code in an automation framework, and in a programming language. Then record and playback is actually a really great tool for showing you, how do I write an element so that I can go click on it? It actually has that code-less ability, so that you can go and hit your website, click around, build up executable statements that will execute against your website, and then it also has a modular design. You can break everything into small modular designs and start doing builds on that.

But then, it also allows you, if you're an advanced developer, you're like, "Okay, wait for a second, I've been working in Java, I've been working in C# for years, I'm not going to be going back to record and playback." The great thing about it is that once you've built off this code, you can actually export it, and export out the code into your Java or C#, and then you can start looking at it like, "What did I do? What was the actual code that I created when I started working with this," and start building your framework off of that?

As I mentioned, it's free, and as you mentioned, I've got a demo where it actually has a built-in CI. It's very similar to Jenkins, it's very easy to set up the execution of a test, and the tests run on any system that happens to be in your group that's logged into your project. My laptop here, your laptop over there, wherever you're at, and Joe's laptop, they all could be used as executing systems are available to run test cases on.

By the way, that's not to say that there's anything wrong with Jenkins. Jenkins is great, because they have a plugin for it, so you could actually do this and run it out of Jenkins, so what it basically becomes is an automation platform that allows you to start as a beginner, work your way up to more advanced levels, and even you can do a proof of concept pretty easily on it, and show people, "This is what it's going to look like when we really get going here. Let me think, what was the question you were asking me? I know I got ...

Jonathon Wright No, you've answered the question, I think the great thing about that as well, when you mentioned Joe there, there are some great virtual conferences, like the Automation Guild, there's the Test Guild for the podcasts on both automation performance and security, I think you've done some great test [inaudible 00:42:56] in the past. Would you have any other recommendations for things that you're looking forward to? Are you going to be on the Automation Guild this year?

Paul Grossman I'll give it another shot. I've actually got something to show. One of the things that I have, mentioned earlier that there are two things that I do, and I didn't actually get to say the second thing. The second thing is, whenever I present at a conference, my rule is that I show something live. I don't do PowerPoints, I don't do prerecorded video, and it's basically I have a system up and running, and I'll show you something really cool, and then walk through it. Then thing is, if something goes wrong, live, I have to go either explain it, or generally, I'll say, "That's intentional, let me show you why."

For other places you can go, I mean Udemy obviously is a great place for learning code. There are great courses there to learn Selenium. Let me think ... There's a neat little website, I'm not sure if it's free or not, but it's ... Coding Game, I'm sorry, there we go.

Jonathon Wright Oh, Coding Game, [crosstalk 00:44:16], so [crosstalk 00:44:16]-

Paul Grossman Coding Game, yeah, so if you want to learn a language, and you want to learn the basics of it, but coding is not really all that much fun, Coding Game has this IDE where you can write code and try to solve little things, and then it'll execute and use the output from what you're writing, in order to do a little pod-racer game, and trying to get your pod around a race fast, or trying to get Batman to find where Joker has hidden a bomb inside of a building, most efficiently.

They have 22 different programming language, so you name the programming language, you could probably learn it. If you know one language, you could go and transfer your knowledge to another programming language in Coding Game and have a whole ton of fun doing it at the same time. That's one I definitely recommend checking out. Let me see if I think of ... You mentioned somebody else just a moment ago.

Jonathon Wright Yes, so I know you've done the Alan Richardson, who I should get definitely on the show because he just lives down the road from me, and he's the-

Paul Grossman Yeah, Evil Tester.

Jonathon Wright Yeah, Evil Tester, and he's got hundreds of hours of Selenium videos I think on his website.

Paul Grossman Yes, I agree with that, and I also have at least one of his books sitting around on my bookshelf out here. [inaudible 00:45:45], to take a look at it, if you ever get to Joe Colantonio's site, he always broadcasts with his bookshelf behind the scene, and he broadcasts in high-def, so occasionally I'll take a screen capture and starting looking through, "Okay, I got that book, I got that book, I got to get that one, okay," and go through.

We have a lot of great authors out there, who write fantastic stuff. If you're still working with UFT, QTP, there's Tarun Lalwani, who wrote four books on test automation, on how to use QTP, and he's got some of the most amazing bits of code that's hiding out there. I'll also share with you one other place you can go to. This is, if you're working in VBScript, there's a website called the ...

The website is called Psacake, and it's spelled P-S-A-C-A-K-E, dot-com. I don't know the guy who runs it off the top of my head. It's a little out-of-date, but it's still a great place. There's a link on there that's something like 1000 VBScript code snippets. So if you want to figure out how to do something in UFT, QTP, you go and find that code library that's out on Psacake. That will be just incredibly on that.

What else can I offer for learning? Oh, ExecuteAutomation, that's Karthik, who ... he does a lot of great demos on almost every tool that's out there. I know he's working with Cypress, Cypress is cool. I've seen it just interacting with

[inaudible 00:47:41],it is so cool, and it's got a player that will playback your execution, so you can kind of slide back and forth, that's built-in. The guys put in a lot of really, really good design decisions, and when it reports out errors, it's not like a billion lines of code that you've got to sift through to figure out what happened. It's really well-written English statements in there.

Let's see, I mentioned Cypress because of ... Oh yeah, ExecuteAutomation run by Karthik, and of course one other one I'll throw out there is, out on Facebook, I've got a Facebook group, and it's called Advanced Test Automation, and if you look into that, there is constantly people putting out videos showing what cool things that they do there. We have about 4000 people I think in the group, so if you throw out any questions on any tool, on how to do things, there's usually somebody out there who can figure it out and work with you on that, [crosstalk 00:48:49]-

Jonathon Wright And you've got 11000 followers on Twitter as well, so [crosstalk 00:48:53]-

Paul Grossman I do.

Jonathon Wright You keep me entertained with most of my daily tweets, so is that the easiest way to get in touch with you? Do you find more and more people are sending you instant messaging, DMs on Twitter, or what's the best way to contact you?

Paul Grossman You can DM me on Twitter, you can send an email to thedarkartswizard@gmail.com, I'd be more than happy to do that. You can check out my YouTube channel, although I think that's under QTP and Grossman, so the response through that, and of course the Facebook group, Advanced Test Automation, that's a good place to find me hanging out from time to time. Occasionally, I'll write a Medium article, so you can search out on Medium and see things up there. That's I think, I've covered all my social media aspects. It's amazing I get any work done with all those areas. I have to hold off of Snapchat, I'm not on Snapchat, I'm not on Instagram, don't go looking for me out there. Well actually, I am on Instagram, I've got two posts from four years ago.

Jonathon Wright Snapchat would be a really difficult one to deliver VBS scripts and tips though wouldn't it? Because you'd only see them for 30 seconds, and it'd just disappear. So you'd be like, "I better write this down quickly."

Paul Grossman Yeah, sometimes I record videos from my phone and post them over to Twitter and, "Hey, look at this bug in this game I found, man!" In fact, I got one coming up, I'll have one of those coming up just short in just a couple of minutes here, that is how to cheat one of these rocket launch games very easily. So look for that.

Jonathon Wright That's what I like to hear.

Paul Grossman Anything else?

Jonathon Wright [crosstalk 00:50:50] ...

Paul Grossman Any other questions?

Jonathon Wright No, it's been such a pleasure to have you on there, and obviously reach out to Paul, he is the wizard of the dark arts, and it's so great to have you on the show, man, and we will put everything that you've put in on the podcast notes, and you have to follow this guy. He is an automation legend, the godfather of automation, you heard it here first.

Paul Grossman Excellent! All right, that's as close as I get to James Brown. Thank you so much, Jonathon, it has been an absolute pleasure speaking with you, and I hope to do this again sometime really soon.