Tech News

Google Provides a Peek into the Architecture of Colossus – Its Storage Foundation

Google Provides a Peek into the Architecture of Colossus – Its Storage Foundation

In a recent post, Google provided a glimpse into the architecture of Colossus. Colossus underpins Google’s scalable storage system, which serves both its Google Cloud offerings and Google’s own globally available services such as YouTube, Google Drive, and Gmail. Five separate components compose Colossus – the client library, curators, metadata database, file servers, and custodians….

Spotify’s Journey to a Unified Codebase for Its Web and Desktop Clients

Spotify’s Journey to a Unified Codebase for Its Web and Desktop Clients

Spotify recently published an account of their journey towards a unified codebase for their web and desktop clients. Spotify’s team was the owner of a web-player and a separate, full-featured desktop client. Due to having to implement many features twice, they were not shipping at the pace they wanted. Now Spotify created one codebase for…

Github Scales Its Rate Limiter Using Redis

Github Scales Its Rate Limiter Using Redis

GitHub engineer Robert Mosolgo yesterday posted a detailed account of how GitHub scaled the GitHub API with a sharded, replicated rate limiter in Redis. GitHub migrated from an older Memcached-based rate limiter to a Redis-based one. According to Mosolgo, the new implementation has improved reliability, fixed issues for clients, and reduced GitHub’s support load. (Read…

Alibaba Cloud Uses Dapr to Support Its Business Growth

Alibaba Cloud Uses Dapr to Support Its Business Growth

In a recent blog post, Sky Ao, a staff engineer at Alibaba Cloud, details how Alibaba Cloud uses the Distributed Application Runtime (Dapr) to support its business growth. As Alibaba’s business rapidly grows while also purchasing other companies, a clear need to support multiple programming languages across varying cloud environments rises. To support this need,…

Dropbox Reveals Atlas – a Managed Service Orchestration Platform

Dropbox Reveals Atlas – a Managed Service Orchestration Platform

In a recent blog post, Dropbox revealed Atlas, a platform whose aim is to provide various benefits of a Service Oriented Architecture while minimizing the operational cost of owning a service. Atlas’ goal is to support small, self-contained functionality, saving product teams the overhead of managing a full-blown service, including capacity planning, alert setup, etc….

Netflix Embraces GraphQL Microservices for Rapid Application Development

Netflix Embraces GraphQL Microservices for Rapid Application Development

Netflix engineering recently published a blog post detailing how Netflix embraced GraphQL microservices for rapid application development. In this post, Dane Avilla, a senior software engineer at Netflix, describes their key learnings in the process and how GraphQL lends itself well for proof-of-concept development. (Read more on InfoQ)

Distributed Application Runtime (Dapr) v1.0 Announced

Distributed Application Runtime (Dapr) v1.0 Announced

The Distributed Application Runtime (Dapr) team announced today that Dapr v1.0 is now available and is considered production-ready. Dapr is an open-source runtime that allows developers to build resilient, microservices-based applications that run on the cloud and edge. With the v1.0 release, developers can deploy Dapr applications to Kubernetes clusters in production scenarios. (Read more…

PayPal Standardizes on Apache Airflow and Apache Goblin for Its Next-Gen Data Movement Platform

PayPal Standardizes on Apache Airflow and Apache Goblin for Its Next-Gen Data Movement Platform

PayPal recently described how it standardized on Apache Airflow and Apache Goblin for implementing its next-gen data movement platform. In a recent blog post, PayPal engineers detail how the existing data movement platform evolved into many tools & platforms in a complex and unmanageable ecosystem and their shift towards a new implementation. (Read more on…

Pinterest Describes an Architecture for Efficient Retrieval of Hierarchical Documents

Pinterest Describes an Architecture for Efficient Retrieval of Hierarchical Documents

In a recent blog post, Pinterest engineers describe how they implemented an efficient two-stage retrieval architecture to retrieve hierarchical documents in a home-grown search engine. They accomplished it by combining index flattening, index normalization, and index denormalization. (Read more on InfoQ)