From 20-21 February, Bojan Sapunov from the Data Team participated in an invite-only, two-day Accelerated Data Lab Workshop intended exclusively for a selected audience consisting of Data Engineers, Cloud Engineers, DBAs, DevOps and Solution Architects from predominantly large enterprises. The sessions deep-dived into the design, development, and architecture of Amazon Aurora with a large focus on how to perform fast migration from the various on-premise databases to the managed cloud DBs (Aurora or RDS).
The session was a valuable opportunity to get hands-on experience through a series of lab exercises led by AWS experts with sample use cases. He got a chance to test Schema Conversion Tool (SCT) and Data Migration Service (DMS) and migrate big databases from MS SQL Server on the stand-alone server to Aurora. He also tried the latest functionalities from Aurora like Aurora Serverless, Backtrack, Autoscaling read replicas, Performance testing, and Insights, Failover, etc and he got a chance to test.
In the end, there was an introduced to the roadmap for Amazon Aurora for 2020.
More detailed agenda covered:
-
Aurora in-depth overview
-
Managed database service compatible with MySql or Postgres.
-
5X the throughput of standard MySQL and 3X the throughput of standard PostgreSQL
-
Scalability
-
High-performance auditing
-
Storage nodes
-
Parallel query processing
-
Availability, cloning, failover
-
Performance: Replicas in multiple AZ and AWS Regions
-
Security: encryption, row-level security, IAM Authentication integration,
-
Serverless option and Data API
-
-
Focus on Schema Conversion Tool (SCT) and Data Migration Service (DMS)
-
Migration options: SQL Server, MySQL, Oracle, PostgreSQL, SAP ASE, Azure DB, DB2, MongoDB
-
Using Schema Conversion Tool
-
Best practices on troubleshooting DMS tasks
-
How Amazon migrated 7500 Oracle DBs with 75PB of data on Aurora, DynamoDB and RDS with little or no downtime and what it meant in terms of performance, infrastructure costs and reduction of database administration overhead.
-
-
Serverless Aurora
-
Auto Scaling, Cloning, Backtrace and Failover
-
Performance tuning and monitoring
-
Aurora Roadmap for 2020:
-
Postgres: Max storage 128/256 TB
-
Postgres: Support for SQL 11
-
Postgres: Major version upgrade Q2
-
Postgres: S3 Export in csv (2020)
-
Postgres: Extract table or schema from snapshot and export in S3 parquet format
-
Postgres: Enhanced Partition Management
-
Postgres: Lambda functions from SQL - Q1
-
Postgres: TimescaleDB extensions
-
Postgres: Background Job and scheduling
-
MySQL: Multimaster single region 4 node support
-
MySQL: Global database
-
MySQL: Cross account cloning
-
MySQL: Active Directory Integration
-
MySQL: Storage tagging
-
MySQL: Zero downtime restart 5.6 and 5.7
-
MySQL: Data Activity Stream
-
MySQL: Support for Sync Lambda
-
MySQL: Instance R5.24XL
-
Serverless: Global database
-
Serverless: Parallel Query expansion
-
Serverless: Backtrack
-
Serverless: Fast DDL, Hash joins
-
Serverless: Long-running transactions
-
Serverless: Improved wake time performance <10s
-