# Scalable-Software-Architecture **Repository Path**: mirrors_Developer-Y/Scalable-Software-Architecture ## Basic Information - **Project Name**: Scalable-Software-Architecture - **Description**: Collection of tech talks, papers and web links on Distributed Systems, Scalability and System Design. - **Primary Language**: Unknown - **License**: MIT - **Default Branch**: master - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 0 - **Created**: 2022-01-11 - **Last Updated**: 2026-04-05 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README # Scalable-Software-Architecture Collection of tech talks, papers and web links on Distributed Systems, Scalability and System Design. ## Tech Talks #### General Advice on System Design and Scalability * [Lecture - Scalability - Harvard Web Development, David Malan](https://www.youtube.com/watch?v=-W9F__D3oY4) * [Building Software Systems At Google and Lessons Learned](https://www.youtube.com/watch?v=modXC5IWTJI) * [Scalable Internet Architectures - Theo Schlossnagle](https://www.youtube.com/watch?v=2WuT2rdLK5A) * [Seattle Conference on Scalability - Jeff Dean](https://www.youtube.com/watch?v=YFGFvCCFk0g) * [Best Practices for Scaling Web Apps](https://www.youtube.com/watch?v=tQ2V9QSv48M) * [Building a Scalable Architecture for Web Apps](https://www.youtube.com/watch?v=yeyugGZO2qc) * [Web application architecture: The whole stack - Allen Holub](https://www.youtube.com/watch?v=SgEFNRf67Cw) * [Scalable Distributed Design](https://www.youtube.com/watch?v=E9GFA-YkzFY) * [Building Software at Google Scale Tech Talk](https://www.youtube.com/watch?v=2qv3fcXW1mg) * [Seattle Conference on Scalability: Scaling Google for Every User](https://www.youtube.com/watch?v=LT1UFZSbcxE) * [Seattle Conference on Scalability: Lessons In Building Scalable Systems](https://www.youtube.com/watch?v=mS48X9oEar0) * [Workers, Queues, and Cache](https://www.infoq.com/presentations/workers-queues-cache) * [Velocity 2012: Jay Parikh, "Building for a Billion Users"](https://www.youtube.com/watch?v=oodS71YtkGU) * [Building Large Systems at Google](https://www.youtube.com/watch?v=_cltA0W-ehQ) * [3000 images per second - Henna Kerman - @Scale 2016](https://www.youtube.com/watch?v=CEZ6oO1u1u8) * [Scaling to over 1,000,000 requests per second](https://www.youtube.com/watch?v=v-KfhCGW3MA) * [Jeff Dean: "Achieving Rapid Response Times in Large Online Services" Keynote - Velocity 2014](https://www.youtube.com/watch?v=1-3Ahy7Fxsc) * [Getting Things Done at Scale](https://www.infoq.com/presentations/cultural-change-management) * [Scale-oriented Architecture with APIs](https://www.infoq.com/presentations/queues-proxy-microservices-automation) * [You Won't Believe How the Biggest Sites Build Scalable and Resilient Systems!](https://www.infoq.com/presentations/scalable-resilient-systems) * [Scalable Distributed Design](https://www.youtube.com/watch?v=E9GFA-YkzFY) * [Distributed Patterns you should know by Eric Redmond](https://www.youtube.com/watch?v=otWvr8VBRx0) * [GOTO 2012 • Runaway Complexity in Big Data Systems...and a Plan to Stop it • Nathan Marz](https://www.youtube.com/watch?v=ucHjyb6jv08) * [Seattle Conference on Scalability: Abstractions for handling large datasets](https://www.youtube.com/watch?v=oRwFpQKgRps) * [High Performance Web Sites and YSlow](https://www.youtube.com/watch?v=BTHvs3V8DBA) #### Company/Product specific tech talks * [Seattle Conference on Scalability: YouTube Scalability](https://www.youtube.com/watch?v=w5WVu624fY8) * [How to answer design question: How do you design a twitter?](https://www.youtube.com/watch?v=gX8S7b8UYl8) * [Operations at Twitter: Scaling Beyond 100 Million Users](https://www.youtube.com/watch?v=z8LU0Cj6BOU) * [How We've Scaled Dropbox](https://www.youtube.com/watch?v=PE4gwstWhmc) * [Keynote: Twitter's search architecture](https://www.youtube.com/watch?v=AguWva8P_DI) * [Marco Cecconi - "The Architecture of StackOverflow"](https://www.youtube.com/watch?v=t6kM2EM6so4) * [Scalability at YouTube](https://www.youtube.com/watch?v=G-lGCC4KKok) * [Lessons of Scale at Facebook](https://www.youtube.com/watch?v=QCHiNEw73AU) * [Scale at Facebook](https://www.infoq.com/presentations/Scale-at-Facebook) * [Flight Lightning - Scaling Twitter core infrastructure](https://www.youtube.com/watch?v=6OvrFkLSoZ0) * [Scaling Instagram with Mike Krieger](https://www.youtube.com/watch?v=oNA2C1vC8FQ) * [GOTO 2014 • Scaling Pinterest • Marty Weiner](https://www.youtube.com/watch?v=jQNCuD_hxdQ) ([InfoQ link](https://www.infoq.com/presentations/Pinterest)) * [Real-Time Delivery Architecture at Twitter](https://www.youtube.com/watch?v=J5auCY4ajK8) * [OSCON 2014: How Instagram.com Works; Pete Hunt](https://www.youtube.com/watch?v=VkTCL6Nqm6Y) * [Scaling the Data Infrastructure at Instagram](https://www.youtube.com/watch?v=1sPgogJlKWM) * [O'Reilly MySQL CE 2011: Jeremy Cole, "Big and Small Data at @Twitter"](https://www.youtube.com/watch?v=5cKTP36HVgI) * [O'Reilly Webcast: How Pinterest Architected and Built Their Sharded MySQL Datastore](https://www.youtube.com/watch?v=aFJm1YlRr5Q) * [Timelines at Scale (Twitter)](https://www.infoq.com/presentations/Twitter-Timeline-Scalability) * [Architecture at Scale at ESPN](https://www.infoq.com/presentations/Architecture-Scale-ESPN) * [Building Highly-resilient Systems at Pinterest](https://www.infoq.com/presentations/pinterest-resilient-systems) * [Scaling Uber](https://www.infoq.com/presentations/uber-scalability-arch) * [How Zoom works](http://highscalability.com/blog/2020/5/14/a-short-on-how-zoom-works.html) * [Scaling Engineering Culture at Twitter](https://www.youtube.com/watch?v=H9wTuT6yQlc) * [Keynote - Systems at Facebook Scale](https://www.youtube.com/watch?v=dlixGkelP9U) * [Scaling YouTube's Backend: The Vitess Trade-offs](https://www.youtube.com/watch?v=5yDO-tmIoXY) * [Hacker Way: Rethinking Web App Development at Facebook](https://www.youtube.com/watch?v=nYkdrAPrdcw) * [Data Platform Architecture, Evolution, and Philosophy at Netflix](https://www.youtube.com/watch?v=YT-ztW5QNUQ) * [Data Platform Architecture, Evolution, and Philosophy at Netflix](https://www.youtube.com/watch?v=uH8T7JMzloM) * [Structure, Personalization, Scale: A Deep Dive into LinkedIn Search](https://www.infoq.com/presentations/linkedin-search) * [Scaling Foursquare: From Check-ins to Recommendations](https://www.infoq.com/presentations/scale-foursquare) * [How Netflix Leverages Multiple Regions to Increase Availability: An Active-Active Case Study](https://www.infoq.com/presentations/netflix-availability-regions) * [That's 'Billion' with a 'B': Scaling to the Next Level at WhatsApp](https://www.infoq.com/presentations/whatsapp-scalability) * [How Facebook Scales Big Data Systems](https://www.infoq.com/presentations/scale-facebook-big-data) * [Scaling Uber's Real-time Market Platform](https://www.infoq.com/presentations/uber-market-platform) * [Software Development & Architecture @ LinkedIn](https://www.infoq.com/presentations/linkedin-architecture-practices) * [Etsy Search: How We Index and Query 26 Million One-of-a-kind Items](https://www.infoq.com/presentations/etsy-search-ecosystem) * [Scalability Lessons from eBay, Google, and Real-time Games](https://www.infoq.com/presentations/scalability-ebay-google-kixeye) * [How SoundCloud Uses Cassandra](https://www.infoq.com/presentations/soundcloud-cassandra) * [Service Architectures at Scale: Lessons from Google and eBay](https://www.infoq.com/presentations/service-arch-scale-google-ebay) * [Solidifying the Cloud: How Google Backs up the Internet](https://www.infoq.com/presentations/google-cloud-backup-failure) * [Real-Time Systems at Twitter](https://www.infoq.com/presentations/real-time-twitter) * [Serving user intent : Facebook style notifications using HBase and Event streams](https://www.youtube.com/watch?v=hIxXaH0CrVQ) * [Netflix's Distributed Computing Strategies: Optimistic Design for the Eventual Consistency Model](https://www.youtube.com/watch?v=6R1WhWkh6pg) #### Distributed Computing * [Intro to Hadoop and MapReduce (Udacity)](https://www.youtube.com/playlist?list=PLAwxTw4SYaPkXJ6LAV96gH8yxIfGaN3H-) * [Introduction to Hadoop](https://www.youtube.com/watch?v=Pq3OyQO-l3E) * [MapReduce Flow Chart](https://www.youtube.com/watch?v=6OemZEJdMp8) * [Distributed Computing CS 61A UC Berkeley](https://www.youtube.com/watch?v=zGTZ-wwA9O8&list=PL6BsET-8jgYWkCySVpSbOj-MCk6xC0i4N) * [MapReduce CS 61A UC Berkeley](https://www.youtube.com/watch?v=6mABLxxtIlg&list=PL6BsET-8jgYVivQ1AS4UsPR2tyr830OHj) * [Cluster Computing and MapReduce Lecture 1](https://www.youtube.com/watch?v=yjPBkvYh-ss) * [Cluster Computing and MapReduce Lecture 2](https://www.youtube.com/watch?v=-vD6PUdf3Js) * [Introducing Apache Hadoop: The Modern Data Operating System](https://www.youtube.com/watch?v=d2xeNpfzsYI) #### Distributed Database/Large-Scale Storage * [Spanner: Google’s Globally-Distributed Database](https://www.usenix.org/conference/osdi12/technical-sessions/presentation/corbett) ([Youtube link](https://www.youtube.com/watch?v=NthK17nbpYs)) * [Spanner - multi-version, globally- distributed, and synchronously-replicated database](https://www.youtube.com/watch?v=Q7pcMn0_tac) * [BigTable: A Distributed Structured Storage System](https://www.youtube.com/watch?v=2cXBNQClehA) ([slides](http://static.googleusercontent.com/media/research.google.com/en//people/jeff/bigtable-uw-2005.pdf)) * [Large-Scale Low-Latency Storage for the Social Network - Data@Scale](https://www.youtube.com/watch?v=5RfFhMwRAic) * [Structured Data at Box: How We're Building for Scale](https://www.youtube.com/watch?v=KHrw0Xo8tYo) * [F4 - Photo Storage at Facebook](https://www.youtube.com/watch?v=34e_g-Ji_30) * [The Storage Technologies Behind Facebook Messages](https://www.youtube.com/watch?v=cSNGGAKJqwk) * [Cold Storage at Facebook](https://www.youtube.com/watch?v=FGZqToLuwU0) * [Taking Storage for a Ride with Uber](https://www.youtube.com/watch?v=Dg76cNaeB4s) * [Zen: Pinterest's Graph Storage Service - @Scale 2014 - Data](https://www.youtube.com/watch?v=yI0vHfgK6oI) ([With Slides](https://www.infoq.com/presentations/zen-pinterest-graph-storage-service)) * [Storage Systems at a Rapidly Scaling Startup with a Small Team - Data@Scale](https://www.youtube.com/watch?v=bLyv8zKa5DU) * [f4: Facebook's Warm BLOB Storage System](https://www.youtube.com/watch?v=imlP3lxwGgQ) #### Distributed graph processing * [Giraph](https://www.youtube.com/watch?v=xyvs7NQdfZk) * [Apache Giraph Large Scale Graph Processing On Hadoop](https://www.youtube.com/watch?v=uXw2TKHsqvM) * [Processing Over a Billion Edges on Apache Giraph](https://www.youtube.com/watch?v=b5Qmz4zPj-M) * [Graph Search: The Power of Connected Data](https://www.infoq.com/presentations/graph-search-connected-data) * [Using Graph Partitioning in Distributed Systems Design](https://www.youtube.com/watch?v=QHkhyY9atkE) * [Let Me Graph That For You: Building a Graph Database Application](https://www.infoq.com/presentations/graph-db-tools) * [GraphChi: Large-Scale Graph Computation on Just a PC](https://www.usenix.org/conference/osdi12/technical-sessions/presentation/kyrola) * [PowerGraph: Distributed Graph-Parallel Computation on Natural Graphs](https://www.usenix.org/conference/osdi12/technical-sessions/presentation/gonzalez) #### Stream Processing * [Heron: Real-time Stream Data Processing at Twitter](https://www.youtube.com/watch?v=pUaFOuGgmco) * [Samza in LinkedIn: How LinkedIn Processes Billions of Events Everyday in Real-time](https://www.infoq.com/presentations/samza-linkedin-2014) * [Mantis: Netflix's Event Stream Processing System](https://www.infoq.com/presentations/mantis) * [High Throughput Stream Processing with ACID Guarantees](https://www.infoq.com/presentations/acid-stream-processing) * [Martin Kleppmann — Event Sourcing and Stream Processing at Scale](https://www.youtube.com/watch?v=avi-TZI9t2I) * [ETE 2012 - Nathan Marz on Storm](https://www.youtube.com/watch?v=bdps8tE0gYo) * [Cassandra NYC 2011: Nathan Marz - The Storm and Cassandra Realtime Computation Stack](https://www.youtube.com/watch?v=cF8a_FZwULI) #### API Design * [How To Design A Good API and Why it Matters](https://www.youtube.com/watch?v=aAb7hSCtvGw) * [How to Design Great APIs - Parse Developer Day 2013](https://www.youtube.com/watch?v=qCdpTji8nxo) * [Google I/O 2010 - How Google builds APIs](https://www.youtube.com/watch?v=nyu5ZxGUfgs) * [Designing a Beautiful REST+JSON API](https://www.youtube.com/watch?v=5WXYw4J4QOU) #### Web Services and SOA * [Introduction to Service Design and Engineering - University of Trento, Italy](https://www.youtube.com/playlist?list=PLBdajHWwi0JCn87QuFT3e58mekU0-6WUT) * [REST+JSON API Design - Best Practices for Developers](https://www.youtube.com/watch?v=hdSrT4yjS1g) * [What is a Service Oriented Architecture?](https://www.youtube.com/watch?v=nRayJZmj2oY) * [Webinar : Practical SOA for the Solution Architect](https://www.youtube.com/watch?v=1KXKppaOgtY) #### Caching * [Scaling Redis at Twitter](https://www.youtube.com/watch?v=rP9EKvWt0zo) * [Facebook and memcached - Tech Talk](https://www.youtube.com/watch?v=UH7wkvcf0ys) * [Scaling Memcache at Facebook](https://www.youtube.com/watch?v=6phA3IAcEJ8) * [How Netflix and reddit scale to handle massive demand](https://www.youtube.com/watch?v=zVqA7Ogutiw) * [An analysis of Facebook photo caching](https://www.youtube.com/watch?v=ENaQScyvOzY) #### NoSQL * [Introduction to NoSQL • Martin Fowler](https://www.youtube.com/watch?v=qI_g07C_Q5I) * [NoSQL Distilled to an hour by Martin Fowler](https://www.youtube.com/watch?v=ASiU89Gl0F0) * [NoSQL Distilled • Pramod Sadalage](https://www.youtube.com/watch?v=l68zxgZS_wc) * [Tech Talk: Cassandra Data Modeling](https://www.youtube.com/watch?v=tg6eIht-00M) * [NoSQL Explained](https://www.youtube.com/watch?v=Q6gn2kuOTV0) * [Big Data Architecture Patterns](https://www.youtube.com/watch?v=-N9i-YXoQBE) * [Graph Databases Exposed](https://www.youtube.com/watch?v=A44CQ4P7oWw) #### Messaging * [Queue It! What Job Queues Can Do for You!](https://www.infoq.com/presentations/job-queue-gearman-beanstalkd) * [Joydeep Sen Sarma - Messaging architecture at Facebook](https://www.youtube.com/watch?v=5hUmdoMOrpo) * [Messaging at Scale at Instagram](https://www.youtube.com/watch?v=E708csv4XgY) * [Building a Distributed Data Ingestion System with RabbitMQ](https://www.infoq.com/presentations/data-replication-rabbitmq) * [scaling web applications with message queues - Lenz Gschwendtner](https://www.youtube.com/watch?v=aOrGq9yb6og) #### Object Oriented Analysis and Design * [Software Architecture & Design | Udacity](https://www.youtube.com/playlist?list=PLAwxTw4SYaPkMTetlG7xKWaI5ZAZFX8fL) * [OOSE: Software Dev Using UML and Java](https://www.youtube.com/playlist?list=PLJ9pm_Rc9HesnkwKlal_buSIHA-jTZMpO) * [Eric Evans — Tackling Complexity in the Heart of Software](https://www.youtube.com/watch?v=dnUFEg68ESM) * [Domain Driven Design](https://www.youtube.com/watch?v=BPAZwFNxgrg) * [Domain-Driven Design](https://www.youtube.com/watch?v=RNUn2R7TptM) * [How You Can Architect and Develop Enterprise Mission-Critical Applications with Domain-Driven Design]( https://www.youtube.com/watch?v=aieoAWXNjl0) * [DDD: putting the model to work](https://www.infoq.com/presentations/model-to-work-evans) * [Eric Evans on DDD: Strategic Design](https://www.infoq.com/presentations/strategic-design-evans) * [Architecting and Implementing Domain-Driven Design Patterns with Microsoft .NET](https://www.youtube.com/watch?v=DaQCGy_GIPE) * [SOLID Design Patterns in C#](https://www.youtube.com/playlist?list=PL8m4NUhTQU48oiGCSgCP1FiJEcg_xJzyQ) * [Object Oriented Design](https://www.youtube.com/playlist?list=PLGLfVvz_LVvS5P7khyR4xDp7T9lCk9PgE) * [Design Patterns Video Tutorial](https://www.youtube.com/playlist?list=PLF206E906175C7E07) * [Object Oriented Design Interview Question: Design a Car Parking Lot.](https://www.youtube.com/watch?v=2vtT6TBnOAM) * [Google's Clean Code Talks](https://www.youtube.com/playlist?list=PL693EFD059797C21E) * [NYC Tech Talk Series: How Google Backs Up the Internet](https://www.youtube.com/watch?v=eNliOm9NtCM) * [Robert C Martin(Uncle Bob) -Clean Architecture and Design-2012](https://www.youtube.com/watch?v=asLUTiJJqdE) * [Robert C Martin - Clean Architecture and Design](https://www.youtube.com/watch?v=Nsjsiz2A9mg) * [Robert C Martin - The Single Responsibility Principle](https://www.youtube.com/watch?v=Gt0M_OHKhQE) * [Robert C Martin - Clean Architecture](https://www.youtube.com/watch?v=Nltqi7ODZTM) * [The S.O.L.I.D. Principles of OO and Agile Design - by Uncle Bob Martin](https://www.youtube.com/watch?v=t86v3N4OshQ) * [Solid Principles by Uncle Bob Martin](https://www.youtube.com/watch?v=oar-T2KovwE) * [The Principles of Clean Architecture by Uncle Bob Martin](https://www.youtube.com/watch?v=o_TH-Y78tt4) * [Unleash Your Domain - Greg Young](https://vimeo.com/19428577) #### Misc * [Differential Synchronization](https://www.youtube.com/watch?v=S2Hp_1jqpY8) * [Infrastructure at Scale: Apache Kafka, Twitter Storm & Elastic Search (ARC303) | AWS re:Invent 2013](https://www.youtube.com/watch?v=LpNbjXFPyZ0) * [Deepak Agarwal: Recommender Systems - The Art and Science of Matching Items to Users](https://www.youtube.com/watch?v=bRzOBGLCRbc) * [Lecture 12 -Analyzing Big Data with Twitter: Recommender Systems by Alpa Jain](https://www.youtube.com/watch?v=NSscbT7JwxY) * [Transactions across Datacenters](https://www.youtube.com/watch?v=srOgpXECblk) ([Slides](http://snarfed.org/transactions_across_datacenters_io.html)) * [Finding the Needle in the Haystack - or - Troubleshooting Distributed Systems](https://www.infoq.com/presentations/mondemand-troubleshoot-distributed-system) * [Finding the Needle in a Big Data Haystack](https://www.infoq.com/presentations/search-hadoop-data-hub) * [Large scale image processing on the fly in 25ms with Google's first Network Engineer](https://www.youtube.com/watch?v=H_9cH1tiigs) * [Bringing Push Notifications to the Mobile Web](https://www.youtube.com/watch?v=HbmcnjWFGbY) --------------------------------------------------------------- ## Papers #### General * [papers-we-love](https://github.com/papers-we-love/papers-we-love) * [Google Research](http://research.google.com/pubs/papers.html) * [Facebook Research](https://research.facebook.com/publications/) * [MIT PDOS](https://pdos.csail.mit.edu/papers/) * [Distributed Systems Reading List](https://dancres.github.io/Pages/) * [Hints for Computer System Design](http://research.microsoft.com/en-us/um/people/blampson/33-Hints/Acrobat.pdf) * The Little Manual of API Design * [On Designing and Deploying Internet-Scale Services](https://www.usenix.org/event/lisa07/tech/full_papers/hamilton/hamilton.pdf) * [Time, Clocks, and the Ordering of Events in a Distributed System](http://research.microsoft.com/en-us/um/people/lamport/pubs/time-clocks.pdf) * [Above the Clouds: A Berkeley View of Cloud Computing](http://www.eecs.berkeley.edu/Pubs/TechRpts/2009/EECS-2009-28.pdf) * [The Byzantine Generals Problem](http://www.andrew.cmu.edu/course/15-749/READINGS/required/resilience/lamport82.pdf) * [How to Design a Good API and Why it Matters - Google Research](http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/32713.pdf) * Twitter - Automatic Management of Partitioned, Replicated Search Services * [High-Availability at Massive Scale: Building Google’s Data Infrastructure for Ads](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/44686.pdf) * [Making Reliable Distributed Systems in the Presence of Software Errors](http://www.erlang.org/download/armstrong_thesis_2003.pdf) * [Fallacies of Distributed Computing Explained](http://www.rgoarchitects.com/Files/fallacies.pdf) #### Search * [The Anatomy of a Large-Scale Hypertextual Web Search Engine (Google Paper)](http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/334.pdf) (Weblink) * [The PageRank Citation Ranking: Bringing Order to the Web](http://ilpubs.stanford.edu:8090/422/1/1999-66.pdf) * [Web Search for a Planet: The Google Cluster Architecture](http://research.google.com/archive/googlecluster-ieee.pdf) * [Unicorn: A System for Searching the Social Graph](http://db.disi.unitn.eu/pages/VLDBProgram/pdf/industry/p871-curtiss.pdf) ([FB link](https://research.facebook.com/publications/unicorn-a-system-for-searching-the-social-graph/)) #### P2P * [Chord: A scalable peer-to-peer lookup service for Internet applications](https://pdos.csail.mit.edu/papers/chord:sigcomm01/chord_sigcomm.pdf) * [Building peer-to-peer systems with Chord, a distributed lookup service](https://pdos.csail.mit.edu/papers/chord:hotos01/hotos8.pdf) * [Consistent Hashing and Random Trees: Distributed Caching Protocols for Relieving Hot Spots on the World Wide Web](https://www.akamai.com/us/en/multimedia/documents/technical-publication/consistent-hashing-and-random-trees-distributed-caching-protocols-for-relieving-hot-spots-on-the-world-wide-web-technical-publication.pdf) ([CS 168: Consistent Hashing](http://web.stanford.edu/class/cs168/l/l1.pdf) | [Algorithmic Nuggets in Content Delivery](http://www.sigcomm.org/sites/default/files/ccr/papers/2015/July/0000000-0000009.pdf)) * Web Caching with Consistent Hashing ([Web link](http://www8.org/w8-papers/2a-webserver/caching/paper2.html)) * [Pastry: Scalable, decentralized object location and routing for large-scale peer-to-peer systems](https://github.com/papers-we-love/papers-we-love/blob/master/distributed_systems/pastry-scalable-decentralized-object-location-and-routing-for-large-scale-peer-to-peer-systems.pdf) * [Simple Efficient Load Balancing Algorithms for Peer-to-Peer Systems](http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/33339.pdf) #### Distributed Computing * [MapReduce: Simplified Data Processing on Large Clusters](http://research.google.com/archive/mapreduce-osdi04.pdf) * [Resident Distributed Datasets: a Fault-Tolerant Abstraction for In-Memory Cluster Computing (Zahari et al.)](https://www.usenix.org/system/files/conference/nsdi12/nsdi12-final138.pdf) * [Kafka: a Distributed Messaging System for Log Processing](http://research.microsoft.com/en-us/um/people/srikanth/netdb11/netdb11papers/netdb11-final12.pdf) #### Distributed Database/Large-Scale Storage * [Dynamo: Amazon's Highly Available Key-value Datastore](http://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf) * [The Google File System](http://research.google.com/archive/gfs-sosp2003.pdf) * [Bigtable: A Distributed Storage System for Structured Data](http://research.google.com/archive/bigtable-osdi06.pdf) * [Spanner: Google's Globally-Distributed Database - Google Research](http://static.googleusercontent.com/media/research.google.com/en//archive/spanner-osdi2012.pdf) * TAO: Facebook’s Distributed Data Store for the Social Graph * [F1: A Distributed SQL Database That Scales](http://static.googleusercontent.com/media/research.google.com/en/us/pubs/archive/41344.pdf) * [Scuba: Diving into Data at Facebook](http://db.disi.unitn.eu/pages/VLDBProgram/pdf/industry/p767-wiener.pdf) * f4: Facebook’s Warm BLOB Storage System * [Finding a needle in Haystack: Facebook’s photo storage](https://www.usenix.org/legacy/event/osdi10/tech/full_papers/Beaver.pdf) * [Cassandra - A Decentralized Structured Storage System](https://www.cs.cornell.edu/projects/ladis2009/papers/lakshman-ladis2009.pdf) #### Consistency * [Consistency Tradeoffs in Modern Distributed Database System Design](http://cs-www.cs.yale.edu/homes/dna/papers/abadi-pacelc.pdf) * [Paxos Made Live - An Engineering Perspective](http://www.cs.utexas.edu/users/lorenzo/corsi/cs380d/papers/paper2-1.pdf) * [Paxos Made Simple](https://github.com/papers-we-love/papers-we-love/blob/master/distributed_systems/paxos-made-simple.pdf) * [Existential Consistency: Measuring and Understanding Consistency at Facebook](https://fbcdn-dragon-a.akamaihd.net/hphotos-ak-xpt1/t39.2365-6/12057015_1010569855670644_348748003_n/Existential_Consistency_Measuring_and_Understanding_Consistency_at_Facebook.pdf) * [In Search of an Understandable Consensus Algorithm](https://ramcloud.stanford.edu/wiki/download/attachments/11370504/raft.pdf) #### Distributed Graph processing * [SQLGraph: An Efficient Relational-Based Property Graph Store](http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43287.pdf) * [One Trillion Edges: Graph Processing at FacebookScale](http://www.vldb.org/pvldb/vol8/p1804-ching.pdf) * [Pregel: A System for Large-Scale Graph Processing](http://kowshik.github.io/JPregel/pregel_paper.pdf) * [Dremel: Interactive Analysis of Web-Scale Datasets](http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en/us/pubs/archive/36632.pdf) #### Company/Product specific * [Scaling Memcache at Facebook](https://www.usenix.org/system/files/conference/nsdi13/nsdi13-final170_update.pdf) * [Realtime Data Processing at Facebook](https://fbcdn-dragon-a.akamaihd.net/hphotos-ak-xat1/t39.2365-6/13331599_975087972607457_1796386216_n/Realtime_Data_Processing_at_Facebook.pdf) * [Holistic Configuration Management at Facebook](https://fbcdn-dragon-a.akamaihd.net/hphotos-ak-xta1/t39.2365-6/12057212_1623087361291843_1316956191_n/Holistic_Configuration_Management_at_Facebook.pdf) * [The Unified Logging Infrastructure for Data Analytics at Twitter](http://vldb.org/pvldb/vol5/p1771_georgelee_vldb2012.pdf) * [Scaling Big Data Mining Infrastructure: The Twitter Experience](http://www.kdd.org/sites/default/files/issues/14-2-2012-12/V14-02-02-Lin.pdf) * [Large-scale cluster management at Google with Borg](http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43438.pdf) #### Misc * [Differential Synchronization](http://research.google.com/pubs/archive/35605.pdf) * A Low-bandwidth Network File System * [Maglev: A Fast and Reliable Software Network Load Balancer](http://research.google.com/pubs/archive/44824.pdf) * [The Chubby Lock Service for Loosely-Coupled Distributed Systems](http://research.google.com/archive/chubby-osdi06.pdf) * [Everything You Always Wanted to Know About Synchronization but Were Afraid to Ask](http://sigops.org/sosp/sosp13/papers/p33-david.pdf) * [Transactional storage for geo-replicated systems](https://github.com/papers-we-love/papers-we-love/blob/master/datastores/transactional-storage-for-geo-replicated-systems.pdf) * [Highly Available Transactions: Virtues and Limitations](http://www.bailis.org/papers/hat-vldb2014.pdf) * [The Log-Structured Merge-Tree](http://www.cs.umb.edu/~poneil/lsmtree.pdf) -------------------------------------------------------- ## Web Links #### General * https://www.hiredintech.com/ * https://github.com/checkcheckzz/system-design-interview * https://github.com/shashank88/system_design * http://highscalability.com/ ([All Time Favorites](http://highscalability.com/all-time-favorites/)) * http://blog.gainlo.co/index.php/category/system-design-interview-questions/ * http://www.allthingsdistributed.com/archives.html (Back-to-Basics series) * [ Scalability for dummies - Part 1](http://www.lecloud.net/post/7295452622/scalability-for-dummies-part-1-clones) ([Part 2](http://www.lecloud.net/post/7994751381/scalability-for-dummies-part-2-database) | [Part 3](http://www.lecloud.net/post/9246290032/scalability-for-dummies-part-3-cache) | [Part 4](http://www.lecloud.net/post/9699762917/scalability-for-dummies-part-4-asynchronism)) * [Scalable Web Architecture and Distributed Systems](http://aosabook.org/en/distsys.html) * [Introduction to Architecting Systems for Scale](http://lethain.com/introduction-to-architecting-systems-for-scale/) * [Software Architect Roadmap](https://roadmap.sh/software-architect) #### Examples Highscalability.com has wide collection of articles on Scalable architecture. Individual web links will be added below if they are not already highlighted in popular sites like highscalability. * [Trending at Instagram](http://instagram-engineering.tumblr.com/post/122961624217/trending-at-instagram) * [Search Architecture - Instagram](http://instagram-engineering.tumblr.com/post/124162066737/search-architecture) * [THE UBER ENGINEERING TECH STACK, PART I](https://eng.uber.com/tech-stack-part-one/) ([PART II](https://eng.uber.com/tech-stack-part-two/)) * [Processing Payments At Scale](https://engineering.groupon.com/2016/misc/processing-payments-at-scale/) * [Personalized Group Recommendations on Flickr](http://code.flickr.net/2016/09/30/personalized-group-recommendations-on-flickr/) * [Building The LinkedIn Knowledge Graph](https://engineering.linkedin.com/blog/2016/10/building-the-linkedin-knowledge-graph) * [Personal recommendations for the Foursquare homescreen](http://engineering.foursquare.com/2015/08/06/personal-recommendations-for-the-foursquare-homescreen/) -------------------------------------------------- ## Books * [Microsoft Application Architecture Guide, 2nd Edition (Online)](https://msdn.microsoft.com/en-us/library/ff650706.aspx) * [The Architecture of Open Source Applications](http://aosabook.org/en/index.html) * Design Patterns: Elements of Reusable Object-Oriented Software * Head First Design Patterns * Patterns of Enterprise Application Architecture * Domain Driven Design by Eric Evans * Agile Software Development, Principles, Patterns and Practices by Robert Martin --------------------------------------------------