Back to all
posts
Database branching is amazing. If you branch your code, why don't you branch your data? If you branch your data, why shouldn't you branch your entire production environment?
While building Kardinal , we dove deep into existing techniques for accessing production data safely and efficiently. With Kardinal, we want to enable microservice developers to "build on production" in a sane way. If you want to do that, you need something like copy-on-write for the data layer. The state of the art for that is database branching.
Of all of the database branching solutions we researched, Neon took it the furthest. Neon is a serverless Postgres database that allows you to instantly create branches off of any given table, to enable developers and testers to use production data in realistic testing scenarios.
If you use Neon, you can create "dev" branches off of your production database to test new features on. However, the rest of your development environment is your responsibility: the correct versions of each of the microservices in your stack, the dev API keys for any external services you use, etc.
That's where Kardinal comes in. We built a demo showing how Kardinal's concept of development environments (called flows) seamlessly pair with Neon to enable microservice developers to get button-click dev deployments with production Postgres data.
For our demo, we have a simple voting app using Postgres via Neon as a data layer. Here's what the frontend looks like:
In the Neon admin panel, you can see the data backing our production app. The branch, notable in the upper navbar, is the "main" branch:
Now, imagine we're going to test a new feature. We'll enable users to vote for a third option. We'll test it over production data using Kardinal paired with Neon. First, we'll create a development flow.
~ kardinal flow create voting-app-ui voting-app-ui-dev
In the Kardinal control plane, we can see the state of the cluster once we've created the development flow.
Our development version is deployed alongside our production version, and there is now a Kardinal database sidecar that manages branching in Neon. Traffic from the development version will be routed through the sidecar and to the appropriate branch in Neon.
We can access the development UX and test our feature without impacting production, registering votes across all three options (the first 2 that already existed in production, and the third that we added.
In the Neon admin panel, we can see that the data backing our development version is indeed associated with a new branch - and is not impacting main, the data backing our production application.
When we're done, we can delete the "dev flow" and return the cluster to its normal state, and production will never have been impacted.
For any microservice developer interested in using database branching, or on-demand snapshotting of production data, Kardinal completes the story.
First, Neon (or any other database branching solution) enables you to create on-demand, lightweight branches over your production data.
Then, Kardinal enables you to create on-demand, lightweight "environments" over your production cluster by deploying only the changes that need to be deployed, and safely reusing the rest of the services.
With both combined, the world is your oyster - you can fly at the speed of light with realistic end-to-end development and testing environments.
We're still building towards the first usable beta of Kardinal, which will be licensed under the MIT license. Follow along by joining our waitlist on https://kardinal.dev or checking out the repository at https://github.com/kurtosis-tech/kardinal!
