AWS offers many purpose built Databases. Note this is an immense wide topic. Thought of writing this for my own refresher.<\/p>\n\n\n\n
AWS RDS – Relational Database<\/strong>: DynamoDB – NO SQL<\/strong> Amazon Redshift <\/strong>– Dataware House Other options\/flavors are Below are some more purpose built database services options available in AWS<\/p>\n\n\n\n Amazon QLDB <\/strong>– ledger database transparent cryptographically verifiable transaction log. Immutable Amazon Document DB for MongoDB<\/strong> for storage of JSON – fault tolerant, self healing storage that supports automatic data scaling Amazon Neptune<\/strong> – Graph – database option Amazon Keyspaces – AWS Managed Cassandra compatible service<\/strong> Amazon Timestream<\/strong> – Timeseries database. need to analyze billions\/trillions of data Data patterns<\/strong><\/p>\n\n\n\n AWS Database Migrations<\/strong><\/p>\n\n\n\n Consider a source (on prem) –> RDS \/ EC2 migration. <\/p>\n\n\n\n Types of migration<\/strong><\/p>\n\n\n\n Options available for migration<\/strong><\/p>\n\n\n\n for like to like – DMS uses replication instance and keeps the source available\/online. DB engines are compatible<\/p>\n\n\n\n for like to unlike pattern – SCT (Schema conversion tool) + DMS<\/strong> can help Snowball edge:<\/strong> AWS offers many purpose built Databases. Note this is an immense wide topic. Thought of writing this for my own refresher. AWS RDS – Relational Database:– data is actually relation, ACID (atomic, consistent, integrity, durable) compliant– referential integrity– static and unchanging– ubiquitous – easy and available in many flavors. can handle different types of workloads– … <\/p>\n
– data is actually relation, ACID (atomic, consistent, integrity, durable) compliant
– referential integrity
– static and unchanging
– ubiquitous – easy and available in many flavors. can handle different types of workloads
– RDS read replicas for 6\/6 DB engines available
ex: shopper to read history. query from read db instead of main db<\/p>\n\n\n\nAnti pattern: lot of loads can run but are not good for JSon object or if there is no well defined schema<\/code><\/pre>\n\n\n\n
– fully managed, multi region
– multi primary, no SQL
– built in security backup and restore
low latency key baseeed query. fast performance handle throughput maintaining consistency
– ex: product desc<\/p>\n\n\n\nfor Hot data - \n DAX Caching - DAX - items to cache in a matter of microseconds \n reduces response times of eventually consistent read workloads from single digit milliseconds to microseconds\n devs need not modify their app logic\n\n Elasticache for redis and ElasticCache for MemChached<\/code><\/pre>\n\n\n\n
for cold data – for analytics, columnar database services help
columnar storage tech in order to improve I\/O efficiency and parallelized queries across multiple nodes for fast query performance
Analytics, trend monitoring and gaming insights to know what’s ging on in the system
ex: daily report,allow pulling from datalake
use standard SQL.<\/p>\n\n\n\n- AQUA - Advanced query accelerated cache. that does substantial share of data processsing in place on the cache,\n enabling Redshift to run upto 10x faster<\/code><\/pre>\n\n\n\n
financial requirements, doc storage reqruiement or mobile game specific community<\/p>\n\n\n\n
when you need to keep track of financial activity in an organization<\/p>\n\n\n\n
allows customer to scale from 10GB all the way upto 64TB per database cluster<\/p>\n\n\n\n
Propert Graph and W3C’s RDF along with respective query
ex: Games – connectivity between each player
Supports Apache TinkerPop Gremlin and SPARQL<\/p>\n\n\n\n
if customer needs wide column key store –
automatically supports three replicas that can be distributed across different AZs<\/p>\n\n\n\n
ex: user activity log. ex: trivago
automates rollups, retention, tiering and compression of data
ex: from device sensors constantly<\/p>\n\n\n\n
\n\n\n\n-read heavy\n lot of incoming data. more query processing on that. there is a read contention\/querying due to lot of records\n querying over and over again\n Option - materialized view is a database object \n\n-micro service\n avoid sharing data between micro services\n\n- break monolith into small domains which could be microservice or a collection of microservices\n\n- isolate bounded context - part of domain driven design - back your data based on functionality\n\n- Event sourcing and CQRS (command query responsibilty seggregation)\n changig the parts of the app to define which is responsible of reading vs writing\n\n- SAGA Pattern - sequence of events that happen in a Distributed transactions\n orchestrate\/choreograph using orchestrators like step functions to coordinate activities to perform\/trigger a transactional boundary to commit\/rollback across distributed design\n only works when we can embrace eventual consistency<\/code><\/pre>\n\n\n\n
\n\n\n\n
if the datastructure are not same they need to be converted to a compatible type
ex: once-off migration, source to target, consolidate databases, dev and test<\/p>\n\n\n\n
if the bandwidth, infrastructure is not available, snowball edge can be used to copy the data into S3 and can be migrated<\/p>\n","protected":false},"excerpt":{"rendered":"