etcd installation and usage

  sonic0002        2021-03-07 03:10:33       3,699        0         

etcd is an open source and highly available distributed key-value storage system and is commonly used in critical data storage and service discovery and registration use cases. It is focusing on:

  • Simple: well-defined, user-facing API (gRPC)
  • Secure: automatic TLS with optional client cert authentication
  • Fast: benchmarked 10,000 writes/sec
  • Reliable: properly distributed using Raft

etcd and Redis both support key-value storage and can be set up in distributed systems. Also Redis supporst more key value types. But they are used in different use cases and have different focus areas given below differences:

  • Redis doesn't provide strong data consistency in distributed environment, also it cannot guarantee one can get the latest data
  • Redis doesn't offer a good watch mechanism for data change
  • etcd ensures strong data consistency but with poorer performance compared to Redis when low latency is needed

Below is a guide on installing etcd and using it.

  1. Go to etcd Github release page and download the available stable release.
  2. Unzip the downloaded package and you would find two binary: etcd and etcdctl
  3. Add the package location to OS Path and then can run below command to verify etcd server cam be started or not.
    [WARNING] Deprecated '--logger=capnslog' flag is set; use '--logger=zap' flag instead
2021-03-07 15:56:07.142971 I | etcdmain: etcd Version: 3.4.15
2021-03-07 15:56:07.826568 I | etcdmain: Git SHA: aa7126864
2021-03-07 15:56:07.831452 I | etcdmain: Go Version: go1.12.17
2021-03-07 15:56:07.831452 I | etcdmain: Go OS/Arch: windows/amd64
2021-03-07 15:56:07.832427 I | etcdmain: setting maximum number of CPUs to 4, total number of available CPUs is 4
2021-03-07 15:56:07.832427 N | etcdmain: failed to detect default host (default host not supported on windows_amd64)
2021-03-07 15:56:07.833404 W | etcdmain: no data-dir provided, using default data-dir ./default.etcd
[WARNING] Deprecated '--logger=capnslog' flag is set; use '--logger=zap' flag instead
2021-03-07 15:56:07.867560 I | embed: name = default
2021-03-07 15:56:07.869512 I | embed: data dir = default.etcd
2021-03-07 15:56:07.869512 I | embed: member dir = default.etcd\member
2021-03-07 15:56:07.869512 I | embed: heartbeat = 100ms
2021-03-07 15:56:07.869512 I | embed: election = 1000ms
2021-03-07 15:56:07.869512 I | embed: snapshot count = 100000
2021-03-07 15:56:07.870492 I | embed: advertise client URLs = http://localhost:2379
2021-03-07 15:56:07.931517 I | wal: releasing file lock to rename "default.etcd\\member\\wal.tmp" to "default.etcd\\member\\wal"
2021-03-07 15:56:07.974514 I | etcdserver: starting member 8e9e05c52164694d in cluster cdf818194e3a8c32
raft2021/03/07 15:56:07 INFO: 8e9e05c52164694d switched to configuration voters=()
raft2021/03/07 15:56:07 INFO: 8e9e05c52164694d became follower at term 0
raft2021/03/07 15:56:07 INFO: newRaft 8e9e05c52164694d [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]
raft2021/03/07 15:56:07 INFO: 8e9e05c52164694d became follower at term 1
raft2021/03/07 15:56:07 INFO: 8e9e05c52164694d switched to configuration voters=(10276657743932975437)
2021-03-07 15:56:07.992597 W | auth: simple token is not cryptographically signed
2021-03-07 15:56:08.081499 I | etcdserver: starting server... [version: 3.4.15, cluster version: to_be_decided]
2021-03-07 15:56:08.088330 E | etcdserver: cannot monitor file descriptor usage (cannot get FDUsage on windows)
2021-03-07 15:56:08.102970 I | etcdserver: 8e9e05c52164694d as single-node; fast-forwarding 9 ticks (election ticks 10)
2021-03-07 15:56:08.105975 I | embed: listening for peers on 127.0.0.1:2380
raft2021/03/07 15:56:08 INFO: 8e9e05c52164694d switched to configuration voters=(10276657743932975437)
2021-03-07 15:56:08.117690 I | etcdserver/membership: added member 8e9e05c52164694d [http://localhost:2380] to cluster cdf818194e3a8c32
raft2021/03/07 15:56:08 INFO: 8e9e05c52164694d is starting a new election at term 1
raft2021/03/07 15:56:08 INFO: 8e9e05c52164694d became candidate at term 2
raft2021/03/07 15:56:08 INFO: 8e9e05c52164694d received MsgVoteResp from 8e9e05c52164694d at term 2
raft2021/03/07 15:56:08 INFO: 8e9e05c52164694d became leader at term 2
raft2021/03/07 15:56:08 INFO: raft.node: 8e9e05c52164694d elected leader 8e9e05c52164694d at term 2
2021-03-07 15:56:08.337163 I | etcdserver: published {Name:default ClientURLs:[http://localhost:2379]} to cluster cdf818194e3a8c32
2021-03-07 15:56:08.353843 I | etcdserver: setting up the initial cluster version to 3.4
2021-03-07 15:56:08.376287 I | embed: ready to serve client requests
2021-03-07 15:56:08.387999 N | embed: serving insecure client requests on 127.0.0.1:2379, this is strongly discouraged!
2021-03-07 15:56:08.414353 N | etcdserver/membership: set the initial cluster version to 3.4
2021-03-07 15:56:08.425087 I | etcdserver/api: enabled capabilities for version 3.4​

    If you see above, it means the server is started successfully.

  4. Now can run the client command etcdctl to verify the version.
    C:\Users\pike6>etcdctl version
etcdctl version: 3.4.15
API version: 3.4​

    It shows the client version and also the API version. 

There are v2 and v3 versions of API supported, ETCDCTL_API can be set to choose which version of API to use. By default the version of API being used would be v3 for release 3.x and above.

Any key created with v2 API cannot be accessed with v3 API and vice versa.

Now given the etcd server is started with command etcd. Let's start a new terminal to run etcdctl to play around with it.

Write key

Since it's a key value pair, so can run below to set key foo and value bar.

C:\Users\pike6>etcdctl put foo bar
OK

Get key

Get a key value pair is also easy, just run etcdctl get.

C:\Users\pike6>etcdctl get foo
foo
bar

If only want to get the value corresponding to a key, can add option --print-value-only.

C:\Users\pike6>etcdctl get foo --print-value-only
bar

If the key doesn't exist, it will print nothing.

C:\Users\pike6>etcdctl get foo1

If wanna get some keys within some range, can run etcdctl get start end. Note, end index is excluded.

C:\Users\pike6>etcdctl put foo1 bar1
OK

C:\Users\pike6>etcdctl put foo2 bar2
OK

C:\Users\pike6>etcdctl put foo3 bar3
OK

C:\Users\pike6>etcdctl get foo1 foo3
foo1
bar1
foo2
bar2

C:\Users\pike6>etcdctl get foo1 foo3 --print-value-only
bar1
bar2

If wanna get all with prefix foo, can run below.

C:\Users\pike6>etcdctl get --prefix foo --print-value-only
bar
bar1
bar2
bar3

If wanna limit the number of keys to return, can add --limit option.

C:\Users\pike6>etcdctl get --prefix --limit=2 foo --print-value-only
bar
bar1

Delete key

etcdctl del can be used to delete a key

C:\Users\pike6>etcdctl del foo
1

The output will be the number of keys deleted.

If wanna delete a range, can do etcdctl del start end.

C:\Users\pike6>etcdctl del foo1 foo3
2

If wanna delete key with prefix, add --prefix option.

C:\Users\pike6>etcdctl del --prefix foo
1

Watch key

Sometimes a key value may have been updated by other client and there might be need to watch the change of a key so that a proper action can be taken by the corresponding client caring about the change. etcdctl watch can be used in such cases.

C:\Users\pike6>etcdctl watch foo
PUT
foo
bar

If wanna watch multiple keys at the same time, can add option -i.

C:\Users\pike6>etcdctl watch -i
watch foo
watch zoo
PUT
foo
bar1
PUT
zoo
test

Grant lease

etcd can have lease set to basically handle expiration of keys. If a key is granted with a lease, it will be deleted once the lease is reached.

A lease can be created by running etcdctl lease grant TTL. And the output will indicate the id of the granted lease.

C:\Users\pike6>etcdctl lease grant 60
lease 694d780bd7fccb0c granted with TTL(60s)

Thereafter a key can be set with the granted lease so that it would be deleted automatically when the TTL reaches.

C:\Users\pike6>etcdctl put --lease=694d780bd7fccb0c foo bar
OK

C:\Users\pike6>etcdctl get foo
foo
bar

C:\Users\pike6>etcdctl get foo
foo
bar

C:\Users\pike6>etcdctl get foo

Revoke lease

Once a lease is granted and if in case wanna revoke the lease, it can be done with etcdctl revoke lease ID. When the lease is revoked, all associated key will be deleted.

C:\Users\pike6>etcdctl lease grant 60
lease 694d780bd7fccb13 granted with TTL(60s)

C:\Users\pike6>etcdctl put --lease=694d780bd7fccb13 foo bar
OK

C:\Users\pike6>etcdctl get foo
foo
bar

C:\Users\pike6>etcdctl lease revoke 694d780bd7fccb13
lease 694d780bd7fccb13 revoked

C:\Users\pike6>etcdctl get foo

Keep lease alive

In case a lease TTL is not enough and wanna keep the key alive for more time, the lease can be renewed by keeping it alive.

C:\Users\pike6>etcdctl lease grant 60
lease 694d780bd7fccb19 granted with TTL(60s)

C:\Users\pike6>etcdctl put --lease=694d780bd7fccb19 foo bar
OK

C:\Users\pike6>etcdctl get foo
foo
bar

C:\Users\pike6>etcdctl lease keep-alive 694d780bd7fccb19
lease 694d780bd7fccb19 keepalived with TTL(60)
lease 694d780bd7fccb19 keepalived with TTL(60)
lease 694d780bd7fccb19 keepalived with TTL(60)
lease 694d780bd7fccb19 keepalived with TTL(60)

Get lease information

Once a lease is granted, can check the lease status with etcdctl lease timetolive.

C:\Users\pike6>etcdctl lease grant 60
lease 694d780bd7fccb1e granted with TTL(60s)

C:\Users\pike6>etcdctl put --lease=694d780bd7fccb1e foo bar
OK

C:\Users\pike6>etcdctl lease timetolive --keys 694d780bd7fccb1e
lease 694d780bd7fccb1e granted with TTL(60s), remaining(40s), attached keys([foo])

A lease can be associated with multiple keys and all keys will be deleted if the lease is expired.

C:\Users\pike6>etcdctl lease grant 600
lease 694d780bd7fccb22 granted with TTL(600s)

C:\Users\pike6>etcdctl put --lease=694d780bd7fccb22 foo bar
OK

C:\Users\pike6>etcdctl put --lease=694d780bd7fccb22 foo1 bar1
OK

C:\Users\pike6>etcdctl lease timetolive --keys 694d780bd7fccb22
lease 694d780bd7fccb22 granted with TTL(600s), remaining(578s), attached keys([foo foo1])

etcd can be used in below common scenarios.

  • Service discovery and registration such as in gRPC micro services
  • Critical data configuration storage such as in Kubernetes clusters
  • Load balancer(mainly for data storage access)
  • Distributed lock
  • Cluster monitoring and distributed system leader election

TUTORIAL  DISTRIBUTED SYSTEM  ETCD  RAFT 

       

  RELATED

  0 COMMENT

No comment for this article.


  RANDOM FUN

Delegation?