When we first introduced orchestration, we called it, “the programming power to create unified user experiences in the world of async messaging across disparate blockchains and connection protocols.” But it’s not a new concept that emerged with web3. In the context of software, orchestration commonly refers to the automated configuration, coordination, and management of computer systems and software applications. In web2, orchestration is a key part of making sure the applications we use every day like Spotify or Slack are efficient, scalable, and resilient. Without orchestration, these apps wouldn’t work.

Orchestration is about managing interdependent services and processes in a way that streamlines operations and maintains a system’s integrity. To get a better idea how the orchestration tooling being built by Agoric and teams like Skip and Socket can help web3 be ready for millions of users, let's look at some examples of web2 orchestration tech and how it’s used by some of the most successful apps in the world.

Kubernetes 

One of the most prominent examples of software orchestration in practice is Kubernetes. Originally developed by Google and now maintained by the Cloud Native Computing Foundation, Kubernetes is a container orchestration platform designed to automate the deployment, scaling, and operations of application containers across clusters of hosts. It helps manage complex containerized applications and provides tools to deploy applications, scale them as necessary, monitor their health, and manage updates and maintenance. The power of Kubernetes lies in its ability to manage large-scale, distributed systems efficiently and with minimal human intervention.

Used by

Google: As the originator of Kubernetes, Google utilizes this system extensively within its cloud services, offering managed Kubernetes services through Google Kubernetes Engine.

Spotify: The music streaming giant uses Kubernetes to handle over a thousand services and several thousand software deployments daily, ensuring scalability and reliability.

Zalando: The European online fashion retailer uses Kubernetes to orchestrate its container infrastructure, improving its ability to scale its operations seamlessly across multiple regions.

Learn more about Agoric - web3’s native orchestration platform.

Apache Hadoop

Apache Hadoop is an open-source framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. At the core of its architecture is the Hadoop YARN (Yet Another Resource Negotiator), which is responsible for managing computing resources in clusters and using them for scheduling user applications. YARN effectively orchestrates the processing capabilities of these vast data clusters, enabling Hadoop to handle petabytes of data, making it a critical tool for companies dealing with massive amounts of data.

Used by

Yahoo: One of the largest contributors to Hadoop's development, Yahoo uses Hadoop extensively for search, advertising, email, and more, managing one of the world’s largest Hadoop installations.

Facebook: The social media platform uses Hadoop for storing copies of internal logs and data from external applications, processing massive volumes of data efficiently.

X (Twitter): X uses Hadoop as a fundamental component of its data architecture, processing enormous amounts of data generated by posts and interactions.

Terraform

Terraform is an open-source tool that allows programmers to build, change, and version infrastructure, and can transform cloud-based APIs into configuration files. Terraform’s orchestration capabilities allow it to manage the lifecycle of a wide variety of service components across different service providers. This includes initialization, deployment, and updating. What makes Terraform particularly powerful is its ability to manage both existing service providers and custom in-house solutions, providing a consistent workflow for managing and provisioning resources.

Used by

Slack: The popular messaging platform uses Terraform to manage its expansive infrastructure across multiple cloud providers, ensuring consistency in configurations and ease in scaling.

Uber: Uber employs Terraform to handle its vast and dynamic infrastructure needs across its global operations, supporting millions of rides daily.

Pinterest: Pinterest uses Terraform to create, manage, and update hundreds of AWS resources efficiently, which helps in managing the infrastructure for their visual discovery engine.

Jenkins

Jenkins, an open-source automation server, provides hundreds of plugins to support building, deploying, and automating any software project. Jenkins orchestrates continuous integration and continuous delivery (CI/CD) processes to improve the development lifecycle of software projects. By automating elements of each stage of software delivery, Jenkins helps make sure software can always be in a releasable state. The orchestration capabilities of Jenkins enable developers to quickly integrate changes to the project, and see the effects of these changes in real-time, promoting more efficient development practices.

Used by

eBay: eBay uses Jenkins for continuous integration (CI) to automate testing and delivery processes, which helps them manage their large-scale web applications.

Netflix: Netflix has used Jenkins for continuous integration for years, as well as for a variety of automation tasks.

Twitch: The streaming giant Twitch uses Jenkins for automated build and testing, as well as rapid deployment and rollback of platform changes. 

Orchestration and the Future of Web3

Orchestration in software is about more than just automation; it's about making complex systems manageable and efficient. Whether it’s deploying applications with Kubernetes, processing big data with Hadoop, managing infrastructure with Terraform, or automating software development processes with Jenkins, orchestration tools are indispensable in the modern digital landscape. They ensure that as systems grow in complexity, businesses can still deliver services efficiently and reliably. 

The principle of orchestration is even more important for web3. In web3, orchestration will enable decentralized networks to scale, succeed, and thrive despite the myriad challenges faced. Web3 amplifies the need for sophisticated orchestration tools that can manage asynchronous communications and interactions across a fragmented landscape of blockchains and protocols. The work being done at Agoric, along with the teams at Skip and Socket, highlights the critical role orchestration must play in Web3. These platforms are fundamental to ensuring that Web3 can support a user base that rivals that of web2. The ability to manage multiple blockchain interactions smoothly, handle smart contracts efficiently, and maintain system integrity across disparate networks is what will ultimately enable web3 to handle millions of users.

Stay on the cutting edge of web3 development. Subscribe to the Agoric Developer Newsletter for more on chain abstraction, orchestration, and multi-chain design. 

Join us to learn more about orchestration in person around the world, or online at a Welcome to Agoric session!

Want to be the first to try the Orchestration API? Complete the Getting Started tutorial and let us know about your experience on Discord.  Our DevRels are happy to help!