Transforming operations using AI and autonomous systems. Key insights included the need for end-to-end planning and readiness assessment for successful implementation, prioritizing observability and key signals, adopting a step-by-step approach to automation, focusing on employee empowerment and customer satisfaction, understanding both financial and non-financial gains, strategically mitigating risks, and tailoring autonomous solutions to specific industry needs.
Panelists:
Tim Guleri – Managing Director of Sierra Ventures ( Moderator )
Subha Tatavarti – CTO in Wipro
Matthew Duren – Director of Engineering KnowBe4
Shibu Raj – SVP IT
Mohamed Khalid – Jigar Desai, SVP Product and Engineering
Rachit Lohani – CTO and SVP Engineering and Product Paylocity.
00:19 Panel Introduction and Company Scale
05:21 Assessing Infrastructure Readiness for Autonomous Technology
18:02 Adoption of Autonomous Frameworks and the Need for a Framework
19:15 Defining the Framework
20:22 Vertical Stack Differences
21:29 Maturity Levels and Automation
23:08 Autonomous System Tools
27:01 Risk Aversion and Autonomy
28:16 Autonomy at Scale
29:23 Incremental Autonomy Implementation
31:16 Pharmaland’s Autonomous Journey
33:54 Guardrails and Risk Management
36:54 Nonfinancial Gains of Autonomy
38:30 Positive Impacts of Autonomy on Experimentation
39:32 Autonomy and Business Enabler
52:12 Closing Remarks and Q&A
Learn more about Sedai: sedai.io
You can also see the autocon/23 videos on Sedai’s website: sedai.io/autocon23
Follow Tim Guleri on https://www.linkedin.com/in/timguleri/
#sedai #autonomousoptimization #kubernetes #ai #paloaltonetworks #autonomousoperations #autonomousworkloadoptimization #aipowered #sre #sitereliabilityengineering #platformengineering #devops
Uh I know the topic is super exciting I was just telling the panelists they’re very very experienced to talk on the topic of uh transforming Ops using autonomous technology but there are quite a few of us here so we’ll make sure that a fight doesn’t break out when I ask my excellent
Questions we’ll keep it uh keep it uh uh all all nice and and uh and comfy and we also have Mo from GSK I think was going to be beaming in from the East Coast but is there okay wonderful so um I think the first thing I’m going to do
Is have the panelists actually introduce themselves and the company the role real quickly and then also what sort of environment they’re operating in so you get a sense of uh what scale and size is the Ops and then we’ll get into the questions so starting with subba thank
You Tim um so my name is subat tarti I’m the global CTO of VPO uh weo is a global company uh we operate in every continent and every country in the world uh we uh serve 1500 plus customers 25 plus verticals um and I was just looking
At some of our stats um couple of days ago uh in terms of Fleet size We Touch over a million in number of instances we operate for our customers across the globe um we also manage their both public and private uh Cloud environments especially in the retail and uh iot um
And oil and gas so excited to be here wow that’s impressive we should talk uh hi I’m rajet I lead product and technology for pelocity we are a payroll company and we move lots of money get people paid and the goal is to build best possible culture for every company
Who who Ser we serve as our customers we have over 35,000 customers and in terms of technical footprint we have over 600 700 Services powered by 30 40,000 machines thank you my name is shiu Raj I work for a geodist a third party supply chain company out of France uh I’m responsible
For uh platform modernization and platform tools for geodis Americas from north to south including laan um been with the company for 16 years now happy to be here thank you and I also must call out the Shibu was the first customer of sudai so thank you jiger so currently I’m working for
Company that is doing Data Insights uh it’s a startup so we don’t have a large Fleet size but I come from Facebook so we have seen millions of machines and how to manage them so in this talk I’m going to talk a lot more about my Facebook experience as
Well my name is Matt Duran I’m the director of platform engineering at Noor NOA for is the provider of the world’s largest uh security awareness training and simulated fishing platform um in terms of size of our environment we have over 60,000 customer customers uh we run anywhere from two to 4,000 containers in
ECS every day about 350 million Lambda invocations and growing every single month wonderful and I can get more to introduce myself yeah hi my name is Muhammad I am the director Enterprise hosting architect uh within GSK uh clus Smith plan we are a leading biofarma company uh we make medicines both
General medicines speciality medicines and vaccines we span 80,000 80 plus countries and our aim is to posi impact about the health of 2.5 billion people by the end of 2020 2030 so that’s our aim um so that’s me and I’m very excited to be here to talk
About the TSK Journey on cloud and more so so I think one of the benefits you get for the large panel is you get a variety of really amazing perspectives and uh I was looking up you guys know the numbers of course $371 billion spent on cloud growing at
15% uh 40 to 50 cents of every dollar spent in addition to the company spend and just keep the spindle spinning at the optimal um size and scale which is why uh there’s no way to keep this going at a at a level of optimization if you don’t run this automat autonomous
Infrastructure which is the purpose of the panel so I’m going to start by uh uh Zoom like staying at a very high level and then we’ll zoom in into the practical application on how do you actually get started and what’s the what is the Readiness state of the Enterprise
What what does what does it need to be to really adopt and get benefit because not every company when they run into uh the notion of autonomous infrastructure are ready for it so we’re going to start with that uh it’s going to be very interesting uh sort of discussion going
From left to right so um uh on Saturday when we had a conversation I think um uh Shu youu brought up this r idea that uh let’s let’s talk about that as a launch point so I’m going to go right to you with that question and then we’ll just
Have the rest of the panelist jump in so tell us about your perspective on how does anybody that is just coming into the notion of autonomous how should this be thinking about the Readiness equation of their infrastructure I I think in the Keynotes uh spech I will s mentioned
That you know it’s not like a transformation doesn’t come from vacuum you know it has to be thought out uh and and when we started the journey our fleet was purely a old school VM based some are as400 those kinds of system so when we started the journey uh uh you
Know AI processes were there but are we ready to take it and utilize the full benefit of into our infrastructure was the first question so um we take a pause and uh we also have another equation in the bundle that uh where our money will be invested including the human capital
And also what is our Co business we need to do with it so our Co business is in supply chain optimization and their automation is a first class citizen you know you have raw boards start you know picking and packing to everything we do inside the
Warehouse so for a dollar we would like to spend there but since automation was in our DNA when we looked into modernizing our Fleet we know we have to do automation there as well it’s not like our our because then only the customer get the full value at that process when we were
Talking and starting with sedi we quickly realized that we were not mature enough to to adopt and utilize and get the benefit you can invest it because it’s a newer cool technology and talk about it in presentation but that dollar spend is waste because we are the value
So that value uh talking with the you know the people in sedai help us uncover some of the things we are are not ready for and quickly we realize okay we have to invest there either by partnering with other people who are into that or
We have to build ourselves so that was a quick realization for us to make sure that we are ready it doesn’t mean that you have to have five years or six years but we have to do some changes in our thinking to accept the new reality and
Bring the value to the business so that the dollar we spend will earn some contribution margin back to that so the maturity on observability side the signals which are talked about in the keynot are we ready to do we have those signals do we have uh do we know what we
Have and ability to synthesize and use those signals it doesn’t need to be all the signals we need at least we need some vitals uh have that ready then partner with the uh companies like sedai and others then you can bring the automation its benefit and realize that to your
Customers at the same time if you have that equation worked out very well some of the cost things which we talked about in the previous panel will become an easy conversation because sometimes for us the cost is not like pass through but cost can be increased by cost can be
Materialized by uh making sure that the dollar we spent get the benefit via efficiency Improvement the different variables which we talked about in the previous panel so it’s key that we are we are ready to to digest what some of these Technologies are bringing in so the maturity on observability and key
Signals is really key for you to be successful anybody else like to contribute yeah I know we are compressed on time of a long answer no no no it’s all good all the time so I I love the answer and the question and the approach overall let me extend what he just
Shared as people it’s very easy for us to think in terms of framework right we always if we have a guideline a framework it helps us think about what is the journey and where do we want to be the car industry is going to the same thing so they come up with this
Beautiful framework l0 to L5 and it tells you where you are in the terms of maturity right L Zer is you have no driver assistance L1 is you start with assistance and then you get partial assistance conditional assistance full assistance and then autonomous takes a while for you to go
From L zer which is driving in Beed up Hyundai to driving a Tesla which has hopefully someone whoever right that has an fully autonomous system in operations it’s the exact same thing if you walk into an environment at L zero that means every host is handcrafted there is no
Automation you name the host and you go yeah do you know do you remember Alpine or do you remember 10 20 102031 right throwing a random IP that is not you cannot talk about autonomous there so what do you do you invest in technology and you Embrace IAC
Which is infrastructure as code you go to the next level now you have infrastructure as code what do you do you install in or you provide instrumentation with tools like anible control tower so that way you can run commands at scale you go to you graduate
To the next level right which is partial assistance and then to the next level which is where you start to in uh uh observe conditions which is Advanced observability as as you touched on it and then the next level is when you Embrace things like mesh where you’re automatically controlling the system now
You have a fully automated system the next step is autonomous where you leave the system and let it decide what is the best for my customers it takes a while it takes while to go from L1 or L Zer to one to two to three to four and five fifth is
The autonomous state right so these are the steps that are required and each step is has has its own tooling implementation and the metric the success criteria that you would look at more anything to add from your side from GSK no I think I think uh very
Nicely stated uh by the first gentleman I’m sorry uh uh so even for our journey the same thing right it’s a it’s a process for us so you talked about first and foremost the good thing for us uh the good thing from GSK perspective was we adopted Cloud right so so there was
No confusion in our mind that there not API driven so in processor code someone talked about absolutely highly important from code to deployment to runtime to monitoring then from a monitoring perspective very important to us is three things it’s just not performance uh it’s security and then third part is highly importance
Financial because now that we go to the cloud optimize it as people have Journey we a threeyear journey now as people have aded adopting cloud it’s all about cost performance and security so how do we take that together as someone was mentioning the last stage how do we take
That together and do the right right sizing is the biggest challenge we see today uh and hopefully people like s can solve the problem for us and that’s why we are here so so those are the three things so that’s highly important for us yeah I have a perspective I wanted to
Share um and this is like my journey over three different companies I think she covered a lot about uh PayPal and there we were going through transformation right so we had people staring at screens um you know worrying about every machine what’s going on to Automation and and and this autonomous
Systems that we built and we had to take some time because it was not just a technology change it was a culture change where we had to get people along with us and not just abandon them saying hey you don’t know and we’re going to do actually work for you and the journey
Was tough so that was like my uh sort of PayPal Journey on how to to become ready not just from a technology perspective but also from people and culture perspective then I went to Facebook and the mindset was completely different we were doubling in size in terms of
Machines and I’m talking about millions of machines right every year so Readiness was not a word in our dictionary you better be ready because machines are coming um so very different approach where uh culturally we were ready to take on any challenges because we were seeing that growth uh for a
Longer time and then with cesu that’s the startup that I am in um mindset is very different so very first day when we started building system we had autonomous as a principle because we didn’t have enough people to uh build system that can be looked upon by folks
Staring at screens so autonomous system was built as a ground up and something like sedi you can actually start using on day one even for a smaller startup so I think Readiness is very different for which which state of the journey you are in right uh and and you have to behave
Very differently depending upon your needs yeah I think J you touched upon something interesting because it’s exactly what we observe across the board because of the kind of maturity we see across our customer base um the the Readiness um per se and assessment of Readiness is absolutely critical as an
Example one of our largest uh device medical device manufacturers based out of Japan uh we manage their uh Data Center and infrastructure it’s one the customers um introducing sedi would not even be an option for us because right now they are all bare metal they’re sitting in their own data centers and
They’re not even virtualized in some cases actually in most cases so I guess I’m just double clicking on that which is the Readiness part is absolutely critical so that’s a really interesting point that you made now I’m going to pick on Matt and ask him a question
So sounds like some companies just born just to use your you know your your L1 to L5 born on L4 leaning into L5 perhaps and you can just adopt it and you were at scale uh Matt in terms of the lambdas you were using looks like you were
Already L5 as then you had everything internally is that is that am I sort of reaching out too far or would you say where were you on that on that maturation Journey because sitting at the board of sadai I was I was getting updates from uh the team I was quite
Impressed with how quickly you were able to make the decision and deploy and start getting value end to endend with the with the platform yeah you know I think it’s a really good assessment of where we are it’s not something that that just happened to us um it was something that
We had to be very deliberate in order to achieve so I I started at Noble for in 2018 and my boss who’s the SVP of platform engineering started just a year before um he was the first SRE at the company and at that time most of our
Software was running on ac2 instances uh including databases including compute in many cases it was a you know single server that was processing and providing a lot of what we delivered to our customers and our job originally as the SRE team was to clean that up while the
Amazon bill was still four or five figures a month you know 8,000 10,000 something really low like that yeah and if we hadn’t have gone through that Journey when we did it would be significantly harder now that our Amazon commit next year is you know uh more than millions and millions
Of dollars um so I I think yeah I think it’s really it really speaks to you know how easy it was for us to implement sedi because we were strong users of in infrastructure as code we had all worked in different environments and different sized companies before that had varying
You know levels of implementation of I AC lots of experience with you know common tools like like cloud formation we actually chose to use terraform and it’s not even a metric that we track you know how much of our infrastructure is terraformed but it’s high 90s in terms
Of percent so and and then the compute side Lambda and ECS it’s at the 100% level and in the time that it took us to sign up with sadai and you know get from zero to 100% with them uh it was just matter of months because of that got it
So maybe more broadly and this is a question and I could see a show of hands from the room as a affirmation or not the people in this room are very much left of the chasm right so we are the Believers we think autonomous needs to
Happen but there’s a small set of us that really believe it and I think it’s the people in this room that have to sort of force the industry to realize the benefits and move to the right of the chasm and the question is what I’m sensing is that there’s no agreed upon
Framework that talks to the maturation of a of a company where you could say hey I’m L2 right now if I do these three things I’ll be L3 and then at L4 I can start adopting so the first question to the open room is do you believe uh show
Of hands would be a yes that the industry needs some sort of a framework which lets it quickly test where on the journey it is in terms of Readiness to adopt uh autonomous Frameworks okay that’s a super high trade now the second question is whose job is it to Define this framework do
You think it’s sada’s job do you think it’s Gartner and sadai do you think it’s these these great customers that are already starting to benefit from but where does this come from because clearly it’s needed any any and then if you don’t have a mic just uh speak
Loudly I I’ll I’ll repeat but where does this come from because I’d love next year to be able to stand up here and say that we have a framework in place and companies start referring to that framework because it’s all makes it more easy to sell up and down the management
Chain so to maybe I’ll I’ll start first jump um I you have lots of customers to take care of yes um I think it depends um the Frameworks it’s it’s at least so far it’s very hard to standardize on a framework like such given how fragmented
The the the stack as well as a usage and the implications and the and the applications are so so I think and at least from what we have seen so far uh it has to be a combination of that vertical okay there’s another data point there which is every vertical stack is very
Different um this has been a huge Revelation I’ve been here with for two and a half years and um medical device manufacturing has a very different composition of what a stack looks like versus an iot versus a retail versus a manufacturer versus uh a company like ePay uh sorry PayPal versus Walmart Etc
Um so I think it has to be um a generic enough but it won’t then be solving the problem it has to be coming from the customer in consultation with somebody like sedai who has a maturity who has a general understanding of how the system works and also has the collective
Intelligence of implementations across these verticals and perhaps hopefully a company like wepro got slightly different perspective though I agree with I love it I told you if the panel is Big the fight’s about a break up here’s an example just kid it’s not a fight I I I do I I slightly disagree
With there we go now we’re talking this the afteron guys we got to wake up the audience all right let’s go so in technology it’s less about the what it’s more about the how right the hows are pretty standards we don’t have a lot of options
So when you walk into a data center if you’re naming your host with IPS or specific names you know where the maturity is right the next step usually is automate this part once you automate that part you graduate to the next level ways to automate are very limited and
Your key metrics that will come out of it again you know what what that number is right so that is how the framework would be agnostic to what industry you from or what is the outcome that you’re looking at so and then as you go up the stacks the step one usually is
Infrastructure as code you you Embrace that one which is driver assist so you have assistance now right you can look at somebody’s code and go right or wrong the next one is uh partially assistance so something is broken somewhere I should be able to run a command without understanding the knowhow about the
System this is where you introduce tools like anible or control tower or something else right which becomes essentially the brain of your infrastructure and then the next step usually is around more instrumentation where you start to gather more and more input more signals you introduce tools like data dog or uh more observability
Tool stack so that way you can drive better decisions and then after that it’s usually tools like mesh which can route your traffic being a little proactive around what could go wrong I see this host running hot you and I I skipped one step which was around
Scaling up and scaling down you don’t you’re not using traffic you shut down your boxes but you cannot do that in level zero because everything is stateful right and then the the penultimate is around mesh controlling your traffic traffic comes here mesh says oh this is running hot I’ll move
Here without human assistance the last step is autonomous tools like sedi become a catalyst they can help you jump from L3 to L5 without doing all that hard work and that is the magic of the solution right so I completely agree with you my friend except for l0 to L1
L0 to L1 same story no we’ll have our chat offline yeah so JP is there a bar after this okay just cheing you need you wanted me here for a reason yes yes go ahead mat so I’ll definitely join the fight a little bit
And say no nobody in this room is is at is at L zero and I I think the the barrier there is you know even goes back to the introduction of centralized logging um and collection of data from these decentralized systems and you know if you if you really think about it you
Could get from from l0o or L1 to L5 with without you know ever using a tool like sadii and I like the point that you know it’s an accelerant but you know you could you could get to that L5 level with some carefully tailored bash scripting right if if you had this
Algorithmic approach to Automation and to autonomous you could absolutely get there with a way that was very tailored to your environment but I almost want to introduce this idea of of L6 where having a you have this this AI driven system that’s discovering things that you know
Engineers or humans may have never even conceived may have never even thought of um so I don’t I don’t think that you know Noble is is at L6 yet um in some cases we’re definitely not even at L5 but the places where we’re using sedi I
Think are so much more advanced than the places that we’re not and it really feels like almost a a new tier a new level of automation is actually needed to be defined so that’s been a really cool Journey for us this year and you know we’re looking forward to see how
Much stuff we can get to L5 and Beyond you know in the future I’ll just I don’t want to add fuel into the fight but I’ll agree with shba on this one the reason is there are many times we talk about this we talk about institutions which are very
Software technology oriented but for example there is no place in anible for a PLC or a conveyor system I cannot bring up a conveyor system by running a script it’s not going to happen so it depends upon the industry also that and each industry uh if you carefully look
At it there is a story for tools like sedai so that’s what also I the perspective of maturity comes into play that we cannot just Define that maturity or the framework just by looking at a technology Powerhouse like Google’s or the eBay there are other Industries vertical verticals and industries which can
Utilize this yeah so that’s where it comes into Super uh interesting dialogue so I’d like to now switch to the approach and when I talk about the approach I want to ask uh uh Matt starting with you and then we’ll go to more GSK uh and and what is the right
Approach to implement autonomous systems and talk particularly to the risk side of the equation we know all the benefits how did you mitigate risk and this conversation came up on Saturday when we were chatting as well and and after you two go maybe the rest of the panelist
Can uh chime in on how they think about risk mitigation U because I do remember that guy from your presentation uh how do you how do you deal with him yeah I think my biggest answer is if if you’re completely risk averse you’re going to be stuck lower on on a level of
You know autonomous there there’s always going to be some risk that you know you have to accept as an engineering leader or or as an organization and we take those kinds of risks every day and so like you said it was just a matter of making that a low
Risk instead of a high risk something that was small enough that that we were willing to accept it and I think for us a lot of the building blocks for that were sort of already in place you know we had this infrastructure that was well defined by terraform and modules that
Were already centralized so that we knew 90 something percent of all of our compute was being delivered by just a handful of of terraform modules that made it really easy for us to plug into that you know they they were also pulling our latest version of the module
So we didn’t have to go through hundreds or thousands of repos and update to you know a new pinned version of that module or something like that um we were already taking risks by doing that by trying to be more uh closer to the edge
You know in that way um so I think what I would say if you are looking to implement more Automation and more autonomy in what you’re doing find ways where you can approach The Edge find places whether it’s specific services or environments or whatever it may be for you and for your
Industry in vertical where you’re willing to take some kind of risk where you’re willing to you know pull the latest version of a package that’s internally managed that’s internally created um or find some other place where you can Implement you know sit ey or or
Tools like it in a way that if they had problems you know you could roll back quickly um you know you you could you could tolerate a little bit of of an issue or downtime if it were to happen was is one way to mitigate that is to
Start with I don’t know 20% autonomous and then kind of scale your way up because I don’t exactly recall but you went to 100% very quickly yeah we we did once we were ready you know and and we didn’t start at zero and go to 100
Overnight sure um we we sort of Taylor picked a service that we knew would would get some good utilization in in production um it was a service that we knew was going to lead to marketing and and PR releases and and some Buzz um
Even as a beta feature we had you know I think hundreds of customers that were testing and using this feature while we had sedi enabled on it and we had we enabled sedai throughout the entire process of building this new feature so it was a net new feature in the product
It was some new Services behind the scenes we went ahead and while they were in the development Cycles implemented sedai you know early on there for them so Engineers that were working on you know that service team didn’t really know anything was happening um we moved on from there and pretty much
Flipped that flag turned it on for all of our development environments after we had seen a Production Service go through an entire release cycle live in prod for a number of weeks and really just see cost savings and not see any sort of issues so at that point you know once
Once we had seen it running for you know weeks to months in production and in development and had seen nobody asking what happened to this service why did this scale down why did this scale up why did these resources change I Didn’t Do It um you know then we were we felt
Pretty confident to kind of open the floodgates if you will and more how about yourself at uh in farma Land one of the largest farma companies uh in the world how are you thinking about the journey and the implementation so I I think I I I’ll talk about our journey
Right so for us the most important thing is we said we can’t do all these beautiful things if you’re in the data center so Cloud adoption was the highest thing right we we’ve got AER and gcp so that’s how we start adopting right API infrastructure code everything that’s
Stag driven whether research wants to do drug Discovery faster whether you want to e Improvement in supply chain and absolutely completely agreed level two and Below there’s no way hell you can automate okay I completely agree with subra there and then the third part of
Of is how do we sell faster with Market data right so the cloud provider is enough of the API automation from code all the way in but we were not restricted like for example if I take sedi they do a particular job in containers now the number of services we
Turn on within the cloud is human we have 40 plus Services because it’s all over Innovation we are a very data engineer a very data focused company so lot of our data services there so when you talk about risk if you put the appropriate guard rails have appropriate
People people of course who can manage and operate that who understand the technology really well and understand the business really well that’s how these Services were turn on and got accommodated but now today in the cloud whether if you use sedi no sedi there are ways for example if I use Azure
Advisor I’m giving example as an aure advisor it does provide you enough information for you to do manual you can go on and say I can start stop I can do certain things modification I can do a different life cycle of storage right all those can be done and we currently
Our customers do it manually with the right because they understand the applications really well then and and we have built right amount of G we start over with Dev environment then we start with nonpr and then production so far we’ve not done enough but Dev and nonpr
Is where we started playing around and doing these kind of things and as I said our customers are a little bit more mature we have a three-year journey in the cloud hence they already are there we have a chb mechanism so they do see that so the next step in their mind is
How do I save cost How can I become an autonomous system where I can take data from as I said to currently performance that means you have to really understand what the application is doing and second part is how do I really get uh how do I manage that with financial operation
Data finops data which we have different tools how do I bring them together and allow the cust and autonomously do these things is a risk but as I said we are from a risk part we’ll start with Dev that’s my god Ra that’s our god Ra that’s where we start doing these things
And as we mature as we learn we’ll push it to nonpr production that’s a journey from a risk ever’s Journey so we do mitigate risk we don’t put everything in production we start from a Sandbox figure it out to Dev promote it so we do classic promotion before we do anything
On so even autonomous system will follow the same principles put the Right Guard ra put the right thing just like we do for things like AI today open AI engine llms and all that we definitely do that from sandbox and we do so there is way to risk especially we being a highly
Regulated staff risk we don’t take too much risk but we put right security controls automate all the security controls using infrastru code policy code whatever you want to say and try to promote it so but we still haven’t gone to a complete automous Journey hopefully we’ll get there in the next couple of
Years so but again as I told you we we deal with everything we deal with containers we have a big container we deal with virtual machines we deal with storages we deal with every known ETL processes that run in the in the cloud so in our case there is how how does the
Framework work so back to your one more question I have is how does your framework work in all the cloud services how do we do this how do we do an autonomous system is kind of a bit of a challenge for us because everyone does specific things really really well for
Us to in Aton like s does pretty good job on containers they do a fantastic job so now if I do virtual machines if I Migrate how do I do those things so those things have to be thought through also uh from a complete atomous system from a pure infrastructure it’s easy
When people say you can build a stack but your stack is okay as long as your VMS but if your stack is like this horizontal then you got to think through everything in a more ground of fashion so right right any other comments to add
I I wanted to add a couple comments so my experience I think I’m kind of doubled down on guard rails concept um and this was my experience at Facebook that we had lot of Automation and our systems were autonomous and at some point in time somebody was able to push
A change on a network to the entire networ Network and we brought the entire internet like we were disconnected from the internet for several hours right so blast radius with this level of automation is pretty high so to most point you need to have enough guard
Rails and I think I would also add another point which I think uh you mentioned that we need to treat infrastructure code AS code like if you’re developing an application you’re not pushing your code to production without testing so how do you actually bring discipline of development and
Testing to infrastructure code so that there is a uh environment where you can play with these changes before you push it to production you are actually doing release in a controll manner I think these are some of the discipline that you can actually adopt to make sure your
Risk factor is much lower with you know uh especially with the infrastructure changes got it got it I’m going to transition to uh the um I think the PRI panel talked about the financial impact positive Financial impact of going autonomous but there’s the non or the qualitative aspects of going autonomous
And that’s part of the value you you seek to derive uh from the the application of this technology things like you know obviously uh customer sat employe empowerment the triangle that I think sh showed where you know the shiny bit on the top is what you’re spending
Your most time on and the crud at the bottom you’re reducing that so I’d love to hear uh from the panel anybody can start what are some of those nonfinancial gains that you were able to trap and then communicate to the rest of the organization uh which made the basis
For the success the internal success for U and the S salability of that uh once you adopted autonomous I can share yep set the foundation and you can disagree so we are at a precipice when it comes to Innovation especially in the autonomous space we have the right tools
We have the right environment we just need the right actors now we saw this similar story at Netflix around 2013 2014 2015 when the culture in the industry was you are your Dev your Ops Dev does the build Ops would deploy it and maintain the infrastructure Netflix came
Along and said you know what this does not work for me I want to move faster we built systems in there that will help people deploy more and more artifacture production the outcome of that the experimentation went up by 6,000 per. we were doing two three four experiments a
Month that changed to over thousand experiments a day the result of which people got hooked up to Netflix they loved Netflix not because there was something someone really smart sitting behind the screens figuring out what buttons to put where what movies to put there now it was an autonomous system
Making decisions on what can go forward what cannot right so devs felt more comfortable rolling out PRS every single PR was ready for production if it was not ready the system would block it and say nope you’re not ready that was an autonomous autonomous system making decisions we’re seeing the same thing
Happening in production now as companies grow traffic volumes grow it becomes very hard for us to hire more and more people because that’s unjustifiable right if I come to you and say I need 40 more people to run more infrastructure like what are you doing this is not
Effic effic efficient it’s not effective but if you put an autonomous system there that helps you figure out what are the right things to do your customers are happier your people are happier because they don’t have to focus on the mundane task they can focus on more intellectual heavy task more contextual
Driven and on top of that the the dependent teams it frees up all of their time so companies who would start to embrace autonomous now would see more Innovation more disruption they’ll be able to move faster because this is how R&D allocations work there are companies out there with 100 like 100 plus
Millions of in R&D but 80 90% goes towards run the business be you’re spending nothing on Innovation this helps you unlock those dollars divert them back to the actual growth of the business back to the Top Line not just keeping us alive we’ve seen that in the past with
Other things and now it’s happening with operation so it’s it’s extremely exciting I agree with you all right thank God validation is always good okay just because you said no seriously um so what we’ve been doing uh at ppro is um so with Genna it’s going to impact a lot
Of what we do um just to give you a sense of the scale of oper ations we have 250,000 employees globally and a lot of our cost goes into this employee base um in addition to the12 billion we make uh on Services there’s 450 million ARR revenue on from
Platforms so to your point uh we had to create constraints or starve the rtb um and so there is ruthless prioritization on rtb take that SA savings out and invest in our own internal uh core we call that corei platform essentially it’s a gen platform where we’re orchestrating
Across multiple models including some of the models that are being being built internally by our R&D team which are specific around say um I mean obviously some models are better at uh text to voice others are better at images and so on and so forth and we just released
We’re doing Beta release we have more than 5,000 employees and we are driving in addition to the rtb reduction we’re also driving uh more gains on other um use cases starting from HR as an example for us um HR and back I know it’s not about autonom and infrastructure but
It’s related and kind of the principle uh it used to take I mean it’s a people-based business at least it was it was and it continues to be today but in the future hopefully not um and uh most of our cost goes in background checks
And hiring so it used to take 7 to 10 days for us to do background checks and now it’s literally couple of hours so that productivity gains reduces the time to uh onboard and also the total cost of onboarding for employees is an example and that is giving addition savings so
Yes wanted to add a couple points uh first one I think there is people happiness there is employee happiness and I see companies that have adopted um automation deployed autonomous systems I’ve seen employees being happier there they are able to attract good talent because people want to work for companies where they can
Spend time on what they want to do rather than just doing run the business or keeping the lights on so I think it can become a great tool to attract good talent and that’s the reason why Google and Facebook are able to attract the best of the talent because most of the
Work is not mundane work that’s the work you are spending doing creative work to sh’s point where he highlighted this pyramid the second part part is I think the automation can also be business enabler and and I’ve seen it personally many times that some of the complex
Business ask may be solved by automation um we went through that at my uh current startup where there is a company in in in Europe and they’re going through a bunch of regulations and they’re saying we need deployment in London or we are not actually boarding on your on your
Platform and because we had automation we could actually spin up a new instance completely new instance just for them and serve them there so I see that this is just one example but there are many examples where your investment in automation can actually help grow
Business as well so it’s not not C cost saving but I think it’s much more than that and in some cases it can actually grow your business significantly and I’ve seen it at at least a few places where this happen super um we have about
9 minutes to go and I want to switch to some topics that came on on Saturday when we were chatting which were really interesting ideas where the autonomous platform of sadai could go not that the engineering team needs no more more work but uh Benji get your pen ready um these
Guys had some really creative things that they were thinking of so I’ll just open up the floor to uh give ideas on where do you think the technology can go what is that you know 6 L6 you talked about uh what are some of the things
That could come from like the next phase of value right bunch of deployed um gen U powered autonomous systems and and and scale out infrastructure all eyes on you man everyone’s looking at me somebody said this on the call Zoom so you guys should step up well you know
I’m gonna I’m gonna not directly answer that question but just kind of talk about the the approach that I kind of immediately go to when you start thinking about that sort of stuff is what are my cost centers you know and if if I take our company is 100% you know
AWS Cloud native um which you know for the panel up here may be a little unique and we have some things in AWS that cost a ton a ton of money um you know and I I think that Lambda functions and containers are a really really logical place for sedai
To have started you know they they have knobs that can be turned really easily they have sliders that can be you know moved up and down without from what we’ve seen without affecting the customer’s experience or or by making it better or significantly better um and
You start to look at other services in the cloud and you know then even moving to to on premise or to multicloud environments and it becomes a lot more difficult so as a customer I I would love for some solutions to my ever increasing cloudfront spend that just
Goes up up up every time we get more customers um or to you know Aurora or RDS spend or S3 spend where you know you sort of have these unbounded um or provisioned environments that you know oh it could continue to grow as we get more customers or I’ve
Defined a specific you know back-end data store to be this specific size and changing that means downtime for my customers right so once you kind of push the border of the compute pieces in in a cloud uh it becomes very difficult and it it it’s going to require some you
Know I think more Creative Solutions um not all of which are are you know immediately obvious and if if you try and sit and think how would I do that with RDS I don’t know I don’t know uh for for us from a non-technology based organization I would say probably
Taking tools like sadii to the edge for example each of our warehouses uh if you look at by IP addresses each Warehouse has probably 10 thousands of IP addresses dealing with we do on a day-to-day basis so for ex and just to give you an example when a box mve from
One conveyor even a space of a extra centimeter adds money to our that because we want that spacing to be as much as possible but not touching so that the scanners can do it so um a version of or a scale down uh sedi which is only looking at few
Automation signals you know it’s not like uh we just made need to restart some Services before it happens like six hour reboot Cycles we were talking about because if a conveyor goes down uh to bring it back it takes another two three hours so if you can reduce that to the
Edge because every Edge is a mini data center for us yeah how can we do that without the all sophistication we are talking about we just need L2 so that someone you know Sparks the battery again so things will move that will give us more benefit because for US Labor is a huge
Cost if a conveyor goes down the entire employee base which is 4,000 picking and packing will stay put it takes two hours to get back on a minimum wage you multiply that’s a cost right so if you can take a version of sadai to the edge
Because we are seeing value in our data centers and the cloud so if you can take that to the edge you know the proposition become a little bit different front I saw hand go back raised in the back I think I think SES got that so uh no go ahead jer no that’s
Okay go ah okay um so I think one of the areas where we’re at least when we’re orchestrating across llms we’re noticing um the Precision right it’s a it’s a combination of the the scope or the how how wide the applicability is and the need for precision and in use cases like
Sedi and infrastructure you need higher Precision so um one of the trends that we observing is uh training uh Transformer B architecture based or the next next iteration of Transformer architecture based uh models on specific data sets um and this is an area I think Saidi can potentially grow significantly
Which is create llm like or Transformer like models for um the infrastructure space depends on the kinds of data you see so I think there is a massive space there and and and I think others may not be able to place oration there’s actually a related question that
Uh John just handed me directed at I’ll just read it came from an online viewer it said in an llm and gen gen World infrast Stacks are going to be rethought workloads are CPU GPU hybrids what are the autonomous opportunities in this realm yeah that’s a great question I
Think there is a lot to be done um if you look at a typical geni life cycle or even if you don’t take gen but just looking machine learning life cycle there are literally three phases there is one phase of data cleaning and data preparation so I was so happy to see
That sedi is going to handle data platforms because it’s a significant part of cost and there’s a lot of optimization opportunities in just data prep space there’s a second part of it is how do you train the models right so whether you take a gen model which is uh
Llm uh which is available to you on open source or you’re developing your own model model training is super expensive so you’re working on thousands of gpus or you’re kind of using thousands of gpus on cloud super expensive and by the way uh the way we use resources for
Training models is probably a decade behind how we use the production resources all those techniques and optimizations have not been applied on how to optimize GPU usage and then the last phase is how do you serve it right the inference inference is a massive cost cost I think somebody was just
Mentioning that gbt 3.5 versus 4 there’s a different cost I think rri mentioning that and uh there is a massive opportunity to actually trim down the models so that you don’t have to serve this giant models as inference so there is an opportunity which I generally call
It as an ml op space right from data preparation to training the model to serving the model uh and said I I can can become a billion dollar business business by kind of handling this new wave of things that is going on thank you that was great um I think
We’re at time maybe a couple of more minutes so happy to take any more questions from online or uh from the room John any questions do you have anything from online we okay well U I’m sure you sat through a lot of panels and you always
Evaluate them on a 2 by two axis which is content and entertainment so this one pegs it on both thank you very Much thank you Bo there it is bye