architecture 10 min read

Data Portability in Enterprise Cloud: my DevFest 2023 New Delhi talk, written up

The written version of my DevFest 2023 New Delhi talk: what enterprise data portability means, data residency, backups vs exports, why a pigeon beats your fibre, and BYOK.

This morning I gave a talk called “Data Portability in Enterprise Cloud” in the Cloud track at GDG DevFest 2023 New Delhi. My slot was 11:05 AM, and I walked in wearing a T-shirt that says “There is no cloud, it’s just someone else’s computer”, which turned out to be a useful prop. More on that below. :P

Some context on how I got there: back in 2019 I went to DevFest in Bengaluru with a badge that said “Attendee” and spent the whole day in other people’s talks. This year I answered the CFP for the New Delhi chapter, got picked for the Cloud track, and announced it on X on 24 September. The topic was an easy pick, because data portability at enterprise scale is pretty much what I spend my days on at work.

The DevFest 2023 New Delhi main stage before the crowd came in, with the event branding on a big LED screen and lighting rigs overhead
The main stage that morning, before anyone was in the seats.

The slides go up with this post, so here’s the written version for folks who couldn’t make it (and for the ones who did but were busy collecting swag during the 11:05 slot).

Before we start: stickers

I opened with the usual whoami (career timeline, homelab, fountain pens, mechanical keyboards, chess, the works) and a quick show of hands to see who was in the room: new grads, folks with under 5 years, folks with 15+.

Then a confession: I had a bag of stickers I wanted to get rid of. So the deal was simple. I’d ask questions through the talk, and the first person to answer correctly gets a random sticker.

Two-panel meme: on the left a shouting kid labelled "People who don't answer", on the right a gigachad with glowing red eyes, a Go gopher and a Java logo, labelled "People who answer correctly!"
The incentive structure, explained for the millennials and gen-z in the room.

Server, data center, cloud

First question of the day: what’s the difference between a server, a data center and the cloud?

Slide with three labels, SERVER, DATA CENTER and CLOUD, under icons of a single server, a rack of servers, and a cloud with three question marks in it
Spot the difference.

The hint was on my T-shirt. For anyone sitting too far back to read it, the next slide was a picture of the same tee.

A black T-shirt with a sad cloud icon and the text "There is no cloud, it's just someone else's computer"

A server is part of a data center, and the cloud is a data center as a service. It’s often accessed remotely and can be distributed across the world, but at the bottom of it all there’s still a rack of machines somewhere. They’re the same picture.

The same three icons and labels, now with "is a part of" between server and data center, "as a service" between data center and cloud, and the cloud icon filled with a server rack
Server is a part of a data center. Cloud is a data center, as a service.

So what does Enterprise mean?

Slide asking "What does Enterprise mean?" with the Star Trek: Enterprise poster covering the middle of the sentence
If you thought of these guys first, congratulations, you are pretty old!

An enterprise is typically made up of a number of entities: divisions, departments, subsidiaries. Google is an enterprise. Your local grocery shop is… a business. Both have data, but only one of them has a legal team asking which country that data lives in.

And data portability? The textbook version is about people: being able to get your own data out of a service and reuse it somewhere else, moving or copying it from one IT environment to another safely, without breaking it along the way. In an enterprise, the “person” is a whole company, and the data is every record, file and log they’ve ever made.

Slide: "It's not just about moving data, but about agility, compliance, and business continuity", with the last three words highlighted

The rest of the talk was about the challenges that come with it. I picked three.

Challenge 1: data residency

Data residency is the physical or geographic location of an organisation’s data. Simple definition, very messy reality.

World map titled "Regulatory drivers include data privacy concerns, nationalism, and the economic value of data", with countries that have data residency laws shaded blue and labelled: GDPR in the EU and UK, the CLOUD Act and CCPA in the US, India's Personal Data Protection Bill, China's Cyber Security Law and more
More and more countries have data residency laws, and the list keeps growing.

More and more countries want their citizens’ data stored inside their borders. The reasons are a mix of privacy, national security and plain economics, and it’s increasingly used as a trade tool too.

Then I showed the region maps for AWS, GCP and Azure, and then the regulation map with the cloud regions superimposed on top of it. They line up suspiciously well.

The regulation map faded out with Google Cloud's region map laid on top of it, showing regions in or near most of the countries with residency laws
Regulation on the bottom, cloud regions on top.

Regulation drives adoption. My example on stage was cars in Delhi and the NGT rules on how old a car is allowed to be. Ironic, but it works. A business may not need distributed cloud computing, but an enterprise almost always does, which is why there’s such a huge demand for public clouds in the first place.

“Can’t we just deploy in each region and call it a day?”

No.

Why? Enterprise scale!

Slide with the starship Enterprise flying across the top and "ENTERPRISE SCALE!" underlined below it
Yes, the Enterprise again.

Spinning up a stack in a new region is the easy part. Getting existing customer data there is what increases cost and complexity massively. For a sense of scale, Uber does about 7.6 billion rides a year, which is around 20 million rides a day. At Atlassian, a single customer’s data can be anything from a few megabytes to tens of terabytes. Now imagine moving all of that into a different region to be residency compliant, while people are using it.

Challenge 2: BRIE

Now that we were talking about scale, here’s the most difficult part of data portability. The slide said BRIE, with a big wedge of cheese, and I asked the room what it stands for.

Green slide: "B.R.I.E. Backup Restore Import Export"

Backup, Restore, Import, Export. Storing data is the easy bit. The hard bit is making sure it’s safe, retrievable and transferable without loss or corruption.

The sticker question here was: what’s the difference between a backup and an export? My take: a backup is for getting you back to a known good state, in the same system, usually in whatever format suits the system. An export is for getting your data out, in a format something else can read. Restore and import are the reverse of each, and import is usually the scarier one, because now you’re taking data you didn’t produce and making it fit your system’s rules, IDs and permissions without breaking anything already there.

At enterprise scale, each of those four needs to be consistent, resumable when it fails halfway, and verifiable at the end. That’s a lot of engineering for what sounds like “copy the data”.

Let’s play a game

This was a thought experiment more than a game. What’s the fastest way to send a friend 3 MB of data?

Just email, right?

What about 3 GB? Maybe a cloud drive. 300 GB? Now we’re talking BitTorrent, or splitting it into batches to parallelise, say 10 chunks of 30 GB each.

And 3 TB?

Slide: "A Pigeon!" with a photo of a grey pigeon on the right

A pigeon!

Slide: "Yes, a Pigeon is Faster* for Data Transfer than Gigabit Fiber Internet." with a footnote "*Upto 600 miles"

I’m only half joking. People have been racing pigeons against networks for a while now. RFC 1149, “IP over Avian Carriers”, was an April Fools’ RFC back in 1990, and in 2009 a company in South Africa actually raced a pigeon carrying a memory card against their internet connection, and the pigeon won. This year Jeff Geerling ran the numbers again for 3 TB of data (Tom’s Hardware wrote it up too): a pigeon still beats a real gigabit connection for distances up to about 600 miles.

Line chart titled "Pigeon vs. Internet - 3 TB data transfer", plotting time in minutes against distance in miles for a pigeon, "Pijeff", real internet and theoretical internet, with the pigeon line staying under the internet line up to around 600 miles
Jeff Geerling’s chart. The pigeon wins up to about 600 miles.

Someone in the audience took a photo right as the internet copy vs pigeon copy slide was up, and later tweeted that the pigeon was news to them. Mission accomplished. 🕊️

Jatin on the DevFest stage, seen from the audience, next to a big screen showing "Internet Copy" as a dotted line from a document to a server and "Pigeon Copy" as a document going onto a USB stick, carried by a bird to a Canadian flag and then into a server
The pigeon slide, from the audience’s side. Photo by an attendee.

Why are we talking about pigeons?

Good question! Next one: what about 3 PB? (That’s 1,000 TB, and 1,000 GB in each of those.) Humans are pretty bad at processing big numbers, and this is where intuition stops helping.

Slide: "After a point, it is faster and cheaper to just "physically" transfer the bits than "digitally"."

That’s the whole point of the pigeon. After a point, it’s faster and cheaper to move the bits physically than over the wire, and the cloud providers know it. AWS will literally send you a truck.

AWS's "How it works" diagram for Snowmobile: the truck arrives at a corporate data center, data is loaded into a 45-foot ruggedised shipping container that holds up to 100 PB, and it's uploaded into Amazon S3
AWS Snowmobile: up to 100 PB per truck.

Google has the Transfer Appliance, a box (up to 300 TB) they ship to you. You plug it in, copy your data onto it, ship it back, and they upload it into a Cloud Storage bucket and wipe the appliance.

Slide with a photo of the Google Cloud Transfer Appliance, a grey rugged case with a handle and wheels, next to its name in green
A pigeon, enterprise edition.

Pretty fascinating, right? For migrations at enterprise scale, “how do we move the bytes” is a real design decision, and sometimes the answer is a courier.

Challenge 3: BYOK

Last one. Another acronym, another sticker.

Green slide with a black key icon: "B.Y.O.K. Bring your own key"

Bring Your Own Key. Encryption is important (another question for the room), and enterprises often want to manage their own encryption keys rather than trust the provider’s. That’s great for security, but managing your own keys means taking on key generation, rotation, backup and the whole lifecycle yourself. That needs dedicated people and expertise.

Slide: "Encrypting data is essential for security, but it can complicate data portability and access!"

Why does it complicate portability? A few reasons from my notes:

  • Performance overhead. Encrypting and decrypting at scale adds latency, and when the customer holds the keys, keeping it fast is harder. A migration can mean decrypting and re-encrypting a lot of data.
  • Audit and logging. With BYOK you need solid logging of who accessed which key and when. In regulated industries this is a must.
  • Disaster recovery. The keys need backups too, and you need to be able to restore them. Lose the key and the data might as well be gone.
  • Access control. Who gets to touch the keys, and how do you make sure nobody else does? In a big organisation with lots of stakeholders, that’s hard.
  • Misconfiguration. Like any security setup, a misconfigured BYOK is a vulnerability. It needs regular reviews.

Put that next to data residency and you can see where it gets fun: the data has to move to a new region, the keys it’s encrypted with belong to the customer, and ideally nobody sees the plaintext in between.

Bonus: zero downtime migration

Before everyone left, I wanted them to ponder an interesting problem.

Slide: "The goal is to transfer data, applications, and services from one environment to another without any disruption to users or loss of service."

Zero downtime migration, especially at enterprise scale on a public cloud, is a complex endeavour. You’re moving data, applications and services from one environment to another, and users shouldn’t notice a thing.

Slide: "Think about the solutioning on a high level and drop it to me in my DM before 4PM. The best answer will win a cool goodie!"
The rules.

Right after the talk I posted the problem again on X, so people who missed the last slide could still have a go. A few things worth thinking about, if you want to try it yourself: how you copy the existing data while new writes keep coming in, how you keep the two sides in sync until cutover (the Outbox Pattern is one piece of that puzzle), how you check the copy is actually correct, and how you roll back if it isn’t.

Wrapping up

Six people sent in some really interesting ideas. Thank you, all of you! It came down to two entries in the final, and they were tied. My speaker notes said “we will do a random gen in case of multiple correct answers”, but we went with something fairer: best of three, stone, paper, scissors! 🪨📄✂️ The winner got the goodie.

Thanks to GDG New Delhi for having me, and to everyone who sat through the pigeons. :)

The full deck is embedded above. If you’re curious what talks looked like for me before this, here’s the time I walked colleagues through my Lord of the Rings homelab back in 2020.

And remember: there is no cloud, it’s just someone else’s computer. ☁️

Data Portability in Enterprise Cloud, slide 1 of 63Data Portability in Enterprise Cloud, slide 2 of 63Data Portability in Enterprise Cloud, slide 3 of 63Data Portability in Enterprise Cloud, slide 4 of 63Data Portability in Enterprise Cloud, slide 5 of 63Data Portability in Enterprise Cloud, slide 6 of 63Data Portability in Enterprise Cloud, slide 7 of 63Data Portability in Enterprise Cloud, slide 8 of 63Data Portability in Enterprise Cloud, slide 9 of 63Data Portability in Enterprise Cloud, slide 10 of 63Data Portability in Enterprise Cloud, slide 11 of 63Data Portability in Enterprise Cloud, slide 12 of 63Data Portability in Enterprise Cloud, slide 13 of 63Data Portability in Enterprise Cloud, slide 14 of 63Data Portability in Enterprise Cloud, slide 15 of 63Data Portability in Enterprise Cloud, slide 16 of 63Data Portability in Enterprise Cloud, slide 17 of 63Data Portability in Enterprise Cloud, slide 18 of 63Data Portability in Enterprise Cloud, slide 19 of 63Data Portability in Enterprise Cloud, slide 20 of 63Data Portability in Enterprise Cloud, slide 21 of 63Data Portability in Enterprise Cloud, slide 22 of 63Data Portability in Enterprise Cloud, slide 23 of 63Data Portability in Enterprise Cloud, slide 24 of 63Data Portability in Enterprise Cloud, slide 25 of 63Data Portability in Enterprise Cloud, slide 26 of 63Data Portability in Enterprise Cloud, slide 27 of 63Data Portability in Enterprise Cloud, slide 28 of 63Data Portability in Enterprise Cloud, slide 29 of 63Data Portability in Enterprise Cloud, slide 30 of 63Data Portability in Enterprise Cloud, slide 31 of 63Data Portability in Enterprise Cloud, slide 32 of 63Data Portability in Enterprise Cloud, slide 33 of 63Data Portability in Enterprise Cloud, slide 34 of 63Data Portability in Enterprise Cloud, slide 35 of 63Data Portability in Enterprise Cloud, slide 36 of 63Data Portability in Enterprise Cloud, slide 37 of 63Data Portability in Enterprise Cloud, slide 38 of 63Data Portability in Enterprise Cloud, slide 39 of 63Data Portability in Enterprise Cloud, slide 40 of 63Data Portability in Enterprise Cloud, slide 41 of 63Data Portability in Enterprise Cloud, slide 42 of 63Data Portability in Enterprise Cloud, slide 43 of 63Data Portability in Enterprise Cloud, slide 44 of 63Data Portability in Enterprise Cloud, slide 45 of 63Data Portability in Enterprise Cloud, slide 46 of 63Data Portability in Enterprise Cloud, slide 47 of 63Data Portability in Enterprise Cloud, slide 48 of 63Data Portability in Enterprise Cloud, slide 49 of 63Data Portability in Enterprise Cloud, slide 50 of 63Data Portability in Enterprise Cloud, slide 51 of 63Data Portability in Enterprise Cloud, slide 52 of 63Data Portability in Enterprise Cloud, slide 53 of 63Data Portability in Enterprise Cloud, slide 54 of 63Data Portability in Enterprise Cloud, slide 55 of 63Data Portability in Enterprise Cloud, slide 56 of 63Data Portability in Enterprise Cloud, slide 57 of 63Data Portability in Enterprise Cloud, slide 58 of 63Data Portability in Enterprise Cloud, slide 59 of 63Data Portability in Enterprise Cloud, slide 60 of 63Data Portability in Enterprise Cloud, slide 61 of 63Data Portability in Enterprise Cloud, slide 62 of 63Data Portability in Enterprise Cloud, slide 63 of 63
1 / 63
The full deck from the talk, all 63 slides. Use the arrows or swipe.

Comments

    Supports `code`, ```blocks```, **bold**, *italics* and links.

    Keep reading