Amazon S3: almost 20 years old, but still very modern

Amazon S3 is the oldest service in the catalogue of Amazon Web Services (AWS). We sat down with Andy Warfield, Distinguished Engineer at AWS, to talk about the 19-year journey of Amazon S3 from simple backup solution to sophisticated data foundation.

Warfield talks about how S3 began as essentially "a storage locker across town" for archival purposes, before customers discovered its REST architecture made it capable of handling massive parallel workloads. This unexpected advantage fueled S3's growth beyond unstructured data into analytics and AI domains. The introduction of columnar formats like Parquet and open table formats like Iceberg transformed what was possible, culminating in AWS's recent launch of S3 Tables.

Besides the general direction Amazon S3 has taken and takes, we also talk about other challenges related to current developments. One of them is how AWS and Amazon S3 handles the bloat that is associated with vectorizing databases for use in AI use cases. This challenge is driving significant innovation at AWS to optimize storage for the AI era.

All in all, Amazon S3 has had to evolve continuously to mirror the broader developments in how businesses approach data - from static archives to dynamic, queryable assets powering real-time decision making.

The final topic we talk about is the future of Amazon S3. Warfield wants it to be "pulled closer to application code" with improved performance and more flexible access methods to ensure customers don't have to choose upfront how they'll use their data.

We are slightly biased of course, but we think this is a very good episode of Techzine Talks, and highly recommend anyone interested in the topic to listen to it. Tune in now!