The YellowDog Platform Turns Six

This month, the YellowDog Platform turns six. To mark this occasion, CEO, Simon looks back on how the product has developed and evolved over time, to become the award-winning cloud workload management platform it is today.

How it Began

When I joined the company in early 2015, we had a very simple vision: to build a platform that would enable customers to access the computing resources they needed quickly and easily, so they could get their workloads done on time and on budget.

After researching different industries, we decided to initially focus our attention on Media and Entertainment. Why? Firstly, CGI rendering is an embarrassingly parallel workload. This means the work can be easily split into individual segments and spread across thousands of computers, both on-premise and in cloud environments. Secondly, it was apparent that the issues of accessing computing capacity at short notice and at scale was shared amongst animation studios, large and small. Finally, there was a strong community of animation studios and 3D artists in Bristol, so we had some potential customers right on our doorstep.

With our target industry decided upon, we set ourselves the goal of releasing the first version of the YellowDog Platform by the end of October. To do this, we needed to raise some funds. So, we held our first investment round to crowdfund £150k via Seedrs.

At the time the best way of attracting investment was through a video explaining the vision. This was shot at the Bath Innovation Centre and looking back, my performance was incredibly wooden! However, it worked. We raised the money and set about hiring a development team, designing the logic and wireframing a GUI.

The next six months were full of sleepless nights as we worked on Version 1. Unfortunately, we didn’t make our original release date of 30^th October, but we were only delayed by a little over a week. More importantly the YellowDog Platform worked, and our early networking and marketing efforts ensured there were customers ready and waiting to start trialling the first release.

Where We Are Today

Six years down the line, while our vision remains unchanged, our remit has expanded to help customers across multiple industries.

The collection of microservices that make up our Platform enables users to manage workloads across multiple regions, instance types, and cloud providers; automatically provision resources based on specific constraints and preferences; and capture data from the cloud provider’s object storage, all within a secure private network. This sophisticated end-to-end solution means businesses can execute workloads in the cloud at massive scale, with extreme ease.

Then there is the ecosystem that sits alongside YellowDog to run workloads both on-premise and in the cloud. This allows customers to work with YellowDog’s own cloud native scheduler, or orchestrate third-party schedulers. This gives users the benefits of YellowDog’s scaling capabilities, without the need to replace existing on-premise technology.

What’s more, through taking the time to develop an intuitive user interface, customers can easily view, monitor, and coordinate their computing resources, no matter the provider or location. We don’t believe you should have to be an expert in cloud to make the most out of the technology.

What’s to Come?

Although we are proud with how far the technology has come, we are continually looking for new ways to deliver value for our customers.

To do this we are looking to use intelligent “pre-provisioning” to drive efficiencies in the way workloads interact with the resources required for execution. In other words, being smarter with when and where to provision resources in the cloud, paying particular attention to application licenses, data management and execution pipelines. For example, where application licenses are scarce, it is valuable to ensure workloads are tagged with the required licensing before provisioning compute. Another example, commonly used in genomics, is to ensure a workload is arranged in pipelines, to take advantage of different computing infrastructures as tasks progress.

Intelligent pre-provisioning ultimately ensures that compute is not provisioned prematurely, before preceding tasks have completed, to avoid incurring unnecessary costs.

We will also look at showing resource spending in real-time and setting monetary budgets, in addition to the core hour quotas that we have today.