April 17, 2024

How to Migrate Your Self-Hosted Service to Temporal Cloud

Meagan Speare

Meagan Speare

Temporal Cloud offers an improved experience over self-hosted Temporal, such as greater scale, higher availability, lower latency, and consumption-based costs. If you’re currently self-hosting Temporal and considering migrating to Temporal Cloud, you might think a migration sounds daunting. In this post, we’ll provide an overview of what goes into a migration. We recommend watching the webinar recording on this topic if you want to dive deeper.

What we mean when we talk about migrations

With Temporal Cloud, our team manages the Temporal Service and dependencies, while you still manage your Workflow code and Workers (see diagram below). When migrating to Temporal Cloud, a member of our team will guide you through each step.

image-blog-migrate-cloud1

A migration involves routing new executions to start in the Temporal Cloud namespace. It also requires you to resume existing executions in that namespace while gracefully completing them in the self-hosted instance, under load.

As of today, migrations occur at the application-level, not the database-level. There’s no straightforward way to migrate data (e.g. Event Histories) from your self-hosted Temporal Service to your Temporal Cloud Namespace. The good news is this means you don’t have to manage a complex database migration. Instead, migrations typically require a few minimal code changes. If you need to continue accessing Event Histories from your self-hosted Service for reporting or compliance, there are strategies to do so.

An overview of tasks required to migrate

To migrate to Temporal Cloud, here are the key tasks you must complete and considerations you should keep in mind:

  1. Introduce a new Temporal Client to your application. With Temporal Cloud your application must point to a different endpoint. The Temporal Cloud Client connects your new Workflow Executions to Temporal Cloud. Until you fully switch to Temporal Cloud, you can run the old Client and the new Client in tandem. You must also secure that connection, which you can do using API Keys and mTLS.
  2. Decide on a data encryption strategy. Your application will pass Workflow-related data to Temporal Cloud. Many Temporal Cloud customers choose to use a Data Converter, a simple form of middleware, to encrypt this data on the wire.
  3. Choose the lifetime strategy of your Workflow Executions. To migrate Workflow Executions that are open (i.e. in progress), you must decide whether to let them run to completion or whether to migrate them while they’re in progress. The former strategy is more straightforward, because once the Workflow Execution completes, you can simply point it to Temporal Cloud to start again. But oftentimes, customers have migration deadlines. Your Workflow Executions may not complete before your deadline. In these situations, you must refactor your Workflow Code so it passes the state to Temporal Cloud. Our team can provide guidance and help with your code to accomplish this type of migration safely. Ultimately, you should evaluate each Workflow and their individual requirements to decide which strategy is appropriate.
  4. Update code calling your Workflows. Your application has code that calls Workflows; you need to make some minimal changes to this code to route the requests to your Temporal Cloud Namespace.
  5. Create a test and verification plan. Like with any migration, you should plan out how you’ll test your new deployment before turning it live.

So how long does a migration take?

Migrating to Temporal Cloud is generally straightforward. It may take a day or so of creating a plan for each Workflow, to understand whether you can drain them or must make code changes. Most Workflows can be drained and easily restarted on Temporal Cloud. Longer running or mission-critical Workflows that cannot be drained will require more work.

Most importantly, our team will guide you through every step of your migration, for no extra fee. We’ve migrated thousands of Namespaces and will help you successfully migrate yours.

Next steps to migrate

To learn more, we recommend watching the webinar recording on this topic. You can also check out the docs. If you’re considering migrating, contact us to start working with a member of our team, or drop a question into our community Slack.

This post is part of a series about Temporal Cloud. Check out the other posts below: