SEARCH KEYWORD -- CLUSTER


  Redis Cluster and Common Partition Techniques in Distributed Cache

In this post, I will discuss a few common partition techniques in distributed cache. Especially, I will elaborate on my understanding on the use of Redis Cluster. Please understand that at the time of writing, the latest version of Redis is 4.0.10. Many articles on the same topic have a different idea from this post. This is mainly because, those articles are probably outdated. In particular, they may refer to the Redis Cluster implementation in Redis 3. Redis Cluster has been improved...

   REDIS,DISTRIBUTED CACHE,CLOUD COMPUTING     2019-01-17 08:10:37

  Why to opt for Hadoop?

Hadoop is a open source that stores and processes big data. The framework is written in Java for distributed processing and distributed storage of very large data. Hadoop is Scalable. It is a scalable platform because it stores and distributed large amount of data sets to hundreds and thousands of servers that operate in parallel. Traditional database systems cannot process large amount of data. But, hadoop enable business to run applications involving thousands of Terabyte data. Hadoop is ...

       2015-09-22 10:17:43

  Install Kubernetes with minikube and docker on Ubuntu

When someone just wants to play around Kubernetes on its local environment without accessing to Cloud provider resources, one can set Kubernetes up on local environment with minikube with single node mode. This post will provide a simple guideline on how to set up KUbernetes with minikube and docker on Ubuntu. Before all the steps can be started, you may need to first set up kubectl which is a command line tool to operate on Kubernetes resources. Post that, can ensure that docker is installed on...

   KUBERNETES,MINIKUBE,DOCKER,UBUNTU     2021-06-01 05:37:40

  etcd installation and usage

etcd is an open source and highly available distributed key-value storage system and is commonly used in critical data storage and service discovery and registration use cases. It is focusing on: Simple: well-defined, user-facing API (gRPC) Secure: automatic TLS with optional client cert authentication Fast: benchmarked 10,000 writes/sec Reliable: properly distributed using Raft etcd and Redis both support key-value storage and can be set up in distributed systems. Also Redis supporst more key...

   ETCD,TUTORIAL,RAFT,DISTRIBUTED SYSTEM     2021-03-07 03:10:33

  Build Hadoop environment in Linux

Hadoop standalone installation: 1. Install JDK Install JDK with below command: sudo apt-get install sun-java6-jdk Configure Java environment, open /etc/profile, add below contents: export JAVA_HOME = （Java installation directory） export CLASSPATH =".:$JAVA_HOME/lib:$CLASSPATH" export PATH = "$JAVA_HOME/:PATH" Verify installation of Java Type java --version, if it outputs Java version information, then Java is successfully installed. 2. Install SSH Install SSH with below command: sudo ...

   Hadoop.Linux,Configuration     2013-07-31 23:22:27

  Hadoop or Spark: Which One is Better?

What is Hadoop? Hadoop is one of the widely used Apache-based frameworks for big data analysis. It allows distributed processing of large data set over the computer clusters. Its scalable feature leverages the power of one to thousands of system for computing and storage purpose. A complete Hadoop framework comprised of various modules such as: Hadoop Yet Another Resource Negotiator (YARN MapReduce (Distributed processing engine) Hadoop Distributed File System (HDFS) Hadoop Common Thes...

   COMPARISON,HADOOP,SPARK     2018-11-22 07:08:57

  How Kafka achieves high throughput low latency

Kafka is a message streaming system with high throughput and low latency. It is widely adopted in lots of big companies. A well configured Kafka cluster can achieve super high throughput with millions of concurrent writes. How Kafka can achieve this? This post will try to explain some technologies used by Kafka. Page cache + Disk sequential write Every time when Kafka receives a record, it will write it to disk file eventually. But if it writes to disk every time it receives a record, it would ...

   BIG DATA,KAFKA     2019-03-08 09:42:57

  Use of Erlang in WhatsApp

After many years, people are still discussing whether Erlang is a small language. People have all sorts of doubt about it, but with the appearance of WhatsApp, we may change our mind. What is WhatsApp, from its official website: WhatsApp Messenger is a cross-platform mobile messaging app which allows you to exchange messages without having to pay for SMS. The more funny thing is the main developer of WhatsApp Rick Reed worked in Yahoo and SGI before he joined WhatsApp and he didn't have any know...

   Erlang,WhatsApp,Architecture     2013-06-16 03:03:46

  What have been Facebook’s greatest technical accomplishments?

To maintain a large website which gets billions of requests er day and keeps very fast response speed is not an easy task. Many big companies are trying best to improve user experience by adopting different techniques. There is a question on Quora which asks "What have been Facebook’s greatest technical accomplishments?". There is a person who worked in Facebook before provided an answer which helps us understanding how Facebook handles huge amount of traffic each day. Here is the answer f...

   Facebook, Design, Efficiency     2013-01-15 07:32:07

  Kubernetes Authentication & Authorization 101

If we want to build a system with user modules, Authentication and Authorization are something that we can never ignore, though they could be fuzzy to understand. Authentication (from Greek: αὐθεντικÏŒς authentikos, “real, genuine”, from αὐθέντης authentes, “author”) is the act of proving an assertion, such as the identity of a computer s...

   RBAC,AUTHORIZATION,AUTHENTICATION,KUBERNETES     2021-06-05 23:19:18

RECENT