5 Years Of Assure For 5 Million Customers (with Milan Kuveljic from N26 Group)

Intro In the digital reality, evolution over revolution prevents the QA approaches and techniques that worked yesterday. We'll see you tomorrow. So free your mind. The automation cyborg has been sent back in time. Ted Speaker Jonathon Wright's mission is to help you save the future from bad software. 

Jonathon Wright This podcast is brought to you by Eggplant. Eggplant helps businesses to test, monitor, and analyze their end to end customer experience and continuously improve their business outcomes. 

Thank you so much for joining this podcast. We're really excited to have you on the show. 

Get your current role as head of quality engineering, which fits perfectly with the theme of leadership as well as quality assurance. 

For those out there, listeners out there could get a little bit of a brief intro of what you'd be doing and what you're working on at the moment. 

Milan Kuveljic Yeah, sure. Thanks for the invite. Happy to contribute to this wonderful idea. Maybe I can go a bit back in the past and quickly just go through my history as an engineer. So I started my career for almost twelve years, twelve years ago. I was really coincidentally bumped into a person in my hometown who wanted to build a small team in a small company, an international company with him in Utah who actually wanted to bring a new discipline into software development.

At the time, it was not quite new, but it was new in a way that software development went through in agile disruption. So basically, the company was all about software testing and how we can bring software testing to different organizations in a way that it's it is more around automation rather than out manual testing. 

So that was my first touch with the software industry where I started learning about basically how to sell software to stink to organizations and all the time, like at the same time learning about it and going to potential clients, about it was a cost of control. And that was a perfect start because I didn't know much about it. 

And I had very good mentors with a very long experience in the industry. And I had very good colleagues as well. So we were basically in a very nice journey without so much stress on development or deliveries of just about learning about where the software testing is and what is going. And this took me almost two years.

I worked in various different organizations, starting from traditional banks, insurance companies, investment companies, and some Ospel some healths organizations where we were basically establishing automation as a process, for instance, or testing and all development by the day. So at that time, it was still using a very traditional way to develop software.

What does it mean? Traditional means they were still using tools such as IBM or HP to develop that package and deliver software to different places in very different systems. And that was very nice. But then life took me into a lot of different directions where I got some offer from a small startup in Berkeley and it was totally something different. 

And I accepted the challenge. So it was a startup that was developing some tools for game publishers. So basically those ad monetization and developing as the key libraries or different ways to monetize their game. So it was freemium and premium games. 

And basically my work was pure QA and testing in different ways on using various approaches and different types of testing. So basically, I started learning at that time how to work in a pure, agile environment, how to. Does the software that is being deployed every day to the production, how to test mobile, so it was the beginning basically of mobile Ops, so it was two thousand nine and 10.

And that was something which would start on my different journeys in sorts of development. So this took around four years. I worked in this company and I really grew both technically because I was really well into automation and development in general. And after that, I got an interesting offer from my current company. And then six, to actually join a very small startup at the time was more than four years ago to build the whole cure processes and the team. So it was a very small company at the time.

It was around fifteen engineers in total. In the back end, three mobile engineers and two web engineers and one QA engineer at the time were more like analysts rather than an engineer. So we didn't have any processes, anything. It was pretty much talk testing. It was very small and just trying to justify the idea of digital banking. And that's where my leadership journey started. It didn't have so much experience. Like when I moved to 26, I was having two people that I kind of let us not I was not the manager, but it was more like a mentor.

And I was working in a pure mobile development team. And then I join a complete development company with three cross-functional teams having a lot of responsibilities and inequality. And basically the same time I was learning around leadership and about leadership. And I was at the same time applying the village that I was getting. So it was not like really by the book. A very practical approach and a lot of mistakes. Of course, not out of space and support on how to get things right and how to build successful development teams. So this is like in a very short not show where I spend my last almost 12 years. And today I work as four-point engineering. I can also talk later. What does it mean? 

But in general, I am responsible for a, quote, engineering function in a company which has around 500 people, engineers and other roles working in four different locations worldwide with more than 40 different teams responsible for the quality domain and basically helping and working together with all engineering teams to follow good practices in quality engineering and supporting them, particularly in testing, in delivery, in making sure that the highest quality of the software is built for our customers. So that's like in a nutshell, we can, of course, talk more in detail. 

Jonathon Wright Now, that sounds fantastic. And I think the listeners would really benefit from this kind of learning, the leadership skills you have, what it does leadership look like in this kind of modern scale that Agile kind of view, if lots of teams, lots of deliverables. Now, how did you know? How did you find that transition to such a large scale? 

Milan Kuveljic So I must say that I was not super comfortable at the beginning because it was a lot of unknowns. I had my guides and mentors from my past, like I mentioned, that when I started my software testing journey, I had my mentor. So I was always going back and asking for support. 

I also had my mentors from my previous job at the time, and also I was collecting and observing during all my experience. What were the things that I would always need in my further journeys? So when we talk about leadership, I am in favor of food by example. So if I would need to lead people or manage processes, I would need to not be the smartest person there because that's not the point.

But be the person who can advise, who can show in practice how things work and what are the good or bad things that we can pull or we shouldn't follow. So this was my sort of way to start with it, and I was absolutely not sure if this was the right one, but that's how I approached it. The other thing that helped me. One of them that's a people from the community that influenced a lot how I approach leadership is John Weinberg. I'm sure many of the people from the community know about him and several of his books and blogs that I read, such as General System Thinking and the others helped me to. 

To understand how leadership applies in software development, particularly in software testing and how can I support my teams and people in that way? It was always helpful, to be honest, even when you don't have the good advice or decision to make and ask for help. That's also what leadership is. And what I think it is, it's also good too. 

Need that you make mistakes, and that's also helping people to build trust between you as a leader and them as people who are working with you. So those are the I would say, the main things that help me too. To start my journey in leadership. 

Of course, there are always unpredictable events that happen, like my organization is very demanding things, so we always have a lot of things that are happening. So every day you learn something new that you need some time to spend. Acknowledging it and reflecting.

How does this event affect you as an engineer first, then as a leader, and then as a person as well? On top of everything. And then once you understand that, to apply this further, these are something that that's helped me a lot. In a nutshell, it's leading by example. 

So working together with people, not being on some level that they look at you as someone who is just telling Harwin what to do, but also within Yemen, your hands dirty sometimes and actually do the actual work as well. 

Jonathon Wright I guess, you know, when you talk about, you know, dealing with complex problems and problem-solving and I guess this kind of leads into kind of building a global platform, you know, that must have been a technical challenge as well.

As far as understanding architectures, understanding, you know, what you're going to what testing practices to establish, you know, how did you attack? You attacked that for such a large customer base and, you know, and build those capabilities. 

Milan Kuveljic Yeah, so that's a very interesting challenge and this is one of the talks I'm giving since early this year in international conferences. So I can give a bit of a background of my organization currently. So I mentioned already that there is it's a lot of people working in the Intec organization, which I am part of as well. So when we started, it was a clear idea of what we want to do. We wanted to disrupt the way that banking is done today and how people perform there, how they leave their financial life. And that was always there is given to the main goal.

But that at some point when we started launching products in Europe for REST, we saw that the the the potential to do this is much bigger than just Europe. So then we started exploring how we can actually build a global platform to support a global bank. And it was pretty much the whole year last year where I was very deeply involved in boats like overall engineering side, but also the quality and testing side. So what is different? So you're building a platform that provides or serves you all the products to customers. And sometimes it's important to understand what is the number of customers you have is its thousands, millions. 

Something is also important to understand. Who are your customers as well? For us, from the technical point of view, it was not only about how many customers we have given. So it is because we do we do with the amount of the customers. Things are changing in the background. But more than that, as as the the the product is the banking product, which means that it needs to comply with certain regulations and the laws, the complexity of the go item is higher.

Why? Because you cannot serve your platform from the same place technically. So you need to first differentiate between user data, differentiate between the databases, differentiate between the accounts, et cetera. So you need to have this all in mind when designing. And again, when it comes to testing, it's getting ever more complex because you have, for instance, you have to account for customers, which serves one type of product and has one currency.

Then you have customers from a completely different country, which is to use it, which is serving different products and it's having a completely different currency. And on top of everything is it needs to comply with different to the Relations. So this creates a lot of complexity. And what kind of complexity? So in my talk, I'm presenting the concept of global and local services.

So global service is serving a unique product for customers around the globe while having totally different deployment types, while local service is serving local products for customers in different countries. So this means that you have one service which is deployed in several branches at different times and having totally different tests, but then it has the same version at any point in time. It's just going to different databases, writing different data sets, and it needs to have different. And it's using the same platform to be deployed.

So this is like huge complexity that's needed to revamp how we deal with tests, with institutions, with pipelines, with problems with those issues. So if one pipeline fails, how do we fix it? How do we redeploy will deploy all the pipelines, ect.? And then how do we deal with those data? Do we only test one user? Or do we do the metrics? Different users. So all of these need to be reconsidered. When we started the global forum.

Of course, this is not what one person can sit down in and just writes. How we should do things. It's like collaborate the approach of many, many smart people in the company, which designed and tested a lot of different approaches, how we can make this happen, which resulted basically in launching the global platform in the US mid last year, which brought us to basically to the new markets last year. 

Jonathon Wright And so you talk about region agnostic ecosystems. This first time I've ever heard of this term. What does it actually mean? Does that mean a bit like what you say? The technology is decoupled enough to be agnostic to different currencies. 

And, you know, more sophisticated, like open banking kind of legislations or compliance rules. What kind of what does that mean? 

Milan Kuveljic So that means that there is a service which is providing other services like one service, which talks to downstream services, which provides currency information for different services. 

For instance, one service is responsible for a certain user to provide a dedicated currency, which is either euro or dollar. And then the downstream service, depending on the user type, can then move forward. When I say region agnostic, I mean that we are running on the cloud. So it means that we have a centralized pipeline, which is deploying services to different regions in the cloud. And this is serving both the platform itself, but also compliance. Because once you come to the auditing of your artifacts or your process, you need to be able to extract the information only from one place or only from one pipeline or only from one process where you actually deploy software. So, for instance, the regulations in Europe are different and regulations in us not drastically different, but there are very some different aspects and you need to be able to differentiate between them and provides each of them the same set of reports, for instance. So this is what I mean, region agnostic because in the background things, we are going from a centralized place. Well, when it comes to the end, everything is in different places. This is what I mean by region agnostic. And if you multiply these by not only two regions, then you see that you have a platform which is very complex from one side, but then very, very decentralized at the end of it. 

Jonathon Wright That just sounds incredibly complex to manage. So I know part of your talk. You talk about over the five years, how would they have taken on five million customers, you know? Ensuring the kind of overall quality of the end-user. So that must require, I guess, site reliability. Engineer it's something that's I report to you guys a lot of operations, side of things from a PM. Now, how much focus is on operational excellence? 

Milan Kuveljic So what part is the operational excellence? So there is, of course, the central ability and the operations and I prefer the term the Slack Ops because it's and then some people are saying the SEC quality test product, whatever, Ops. Of course, every function has an operational part and in the cited ability context, things are or shouldn't be different than in other software development. Why? Because in the modern-day operations you're also dealing with managing infrastructure is code.

So basically everything that you do with your infrastructure is in code and it should be treated the same way as any other software. Meaning it should have continuous deployment. It should have all the testing phases. It should show the monitoring rollbacks and all the parts that are also applying to other product development parts. On the other side, the quality aspect is something that also took a lot of Hower and my time to address how we can make this and streamline this process.

And of one of the initiatives and one of the. Well, the journeys that we had in the past few years. It was to move away from quiet assurance. And this is where we come to quality engineering. And my role, which is to move away from assurance, build a lightweight testing phase and more things more into the planning and implementation phase. Whereas in practice, in practice, it means that. In the quality assurance concept, you would have dedicated people who are specialists and they would own this process, meaning that, yes, we are the Agile world, but we still have dedicated roles, were responsible for very special work tasks in the, quote, engineering model.

The responsibility of certain tasks like testing or test automation is distributed to bari's different roles depending on their skill set. So you can have, for instance, your cross-functional team, which is building a product for X amount of customers. You can have roles such as Backend Engineer who is responsible to build microsurgery or API eyes or streams or whatever, but they are also responsible to do the full testing concept for it. Or if we have fronted an engineer who is responsible to build a web interface that will consume the microcircuits, then they are also responsible for testing concept and the delivery from one to one.

Then what is left for specialists in the testing domain, of course, to help the team is capable of doing all these things? And to identify what exactly needs to be done in order to certain competent or software, like just a concept or work to be delivered from one end to another, and to make sure that everyone on the team is comfortable doing this. So this is more of engineering work.

Why? Because you move a lot of things. If it is, somehow people are left moving things to left because you explore and put a lot of focus into the planning phase where you identify all the necessary work to be done and try to use all the engineering capacity to build this in the initial phase. And then when it comes to further testing or delivering or deployment, then you are just verifying what you actually did.

So this is something that we used a lot in order to build a product for five million customers because this evolved into a more efficient and continuous deployment and continuous integration and ultimate, which enable us to iterate faster and build and release more products into in less time, basically. So, for instance, one example is that when we started, let's say not last year, but eight, two years ago, our release cadence on the mobile Ops was, let's say four to six weeks. And this was too long for us because we knew that we were releasing new products every five or six weeks end of last year.

We decreased this to three weeks. And those practices basically enabled us to shorten this period of time. Also on the other side, we were deploying our microservices maybe let's say once every few days, each microcircuit gets a point alive. And today we have each microsurgeons on average as one and a half times deployed per day. So these are the practices that involve the overall process and first make people more comfortable because they know that the process and each part of the process has a great responsibility and they can be comfortable to deploy or these smaller or bigger change to production whenever there is a need for it because there is a process which is controlling this on. On the other side, the teams were more comfortable and capable of doing various different work because they have specialists who are helping them to do this kind of work. So these are the aspects and that's a foundation which helped us to build such a product for five million customers over five years. 

Jonathon Wright Taysir, seriously impressive site for people out there listening. Do you have any kind of suggestions of resources or books that I best mentioned, Jeff, but is there any other kind of go-to materials you'd say, to try and achieve that kind of continuous delivery capability? 

Milan Kuveljic So. Well, there is a bunch of materials there. There are books. Usually, I try to be active in the community. There is a, for instance, dusters IO community where pretty much all the people like or people who are active in the community. But also there are a lot more people from the industry who are there and who can provide a lot of interesting innovations or any kind of advice or learning. But this is something which is following a minister of the test is probably my favorite forum where I like to go and learn what's going on there. Conferences are really, really good. So I'm often in touch with people who are either visiting or organizing the Ministry of tests. And then, of course, Jitter communities superstrong. And there are people who really are active and can provide a lot of. Setting observation facts. Learning new things, etc. So these are my way to to to stay in touch with what's going on outside while still being busy in my current organization. And yeah, following those people is something I would recommend. 

Jonathon Wright So for listeners who'd like to kind of get in touch with you and may pick your brain. And what's the best way to kind of get in touch with you and, you know, subscribed and get a blog like you're doing? The best way to contact your LinkedIn or. 

Milan Kuveljic I have my blog, which is not super active. Now I have plans to go back and write more about my experience. It's called Testing in Space. I'm on Twitter as well. It's my first name. Last name. People can follow. And also, I'm in testers. and also the Slack channel, which I mentioned. Now it has around more than I think its limit is 10000 on Three-tier that 10000 testers and other different people from the community who are collaborating on a daily basis on various different topics are also active there on the leadership side. I'm also in leadership Slack general, which is run running by RANTZ. He's also very popular in the motor community where there is a lot of various different topics that can be discussed and I'm quite often taking part in there as well. Of course, conferences. Now it's a bit tricky to visit them, but I am at least a few times a year. Speaking at a conference and such as agile testing days or smaller events worldwide, both on testing specific topics for those on leadership as well. 

Jonathon Wright

Thanks so much, Milan. 

Jonathon Wright We're going to make sure we add those links in, especially to the Slack channels, because, you know, it's great that people can reach out and maybe pick your brain and around where how far really you've come. This is an amazing accomplishment that you've managed to grow to that kind of size and complexity. So I'm sure people will have a lot of listeners will have a lot of questions. So thanks once again. And yeah, and we'll make sure that we follow up and maybe we can get you to do a blog. 

Milan Kuveljic Of course. Looking forward to it. I will try to park more free time to do this because I really like to show experience and writing is probably the most effective way.