夏休みの自由工作(2)
続・Raspberry Pi3でCephを動かす
2. Ceph環境の構築
- 基本はありもののパッケージを利用する
- まずはオブジェクトストレージの検証を第一目標にする
- 手順についてはビットアイルのこちらの記事を多々参考にさせていただきました
2-1. 接続環境
- ズレてたらごめん
+----------------------------------+ |個人用PC(Win10),タブレット機器etc.| +-+--------------------------------+ | +-+-----------------------+ +----------- ... .. . |NECのブロードバンドルータ +--+ The Internet +-+---------------------+-+ +----------- ... .. . | | +-+----------------+ | |自宅サーバ(録画用)| | |Client兼Deployer | | |CentOS7 | | +------------------+ | +-----------------------++ |Netgear スイッチングハブ| +------------------------+ | +----------------------------------------+ +--+※Ceph-node1(MON.0) OSD.0,3(各USB 128GB)| | +----------------------------------------+ | | +----------------------------------------+ +--+※Ceph-node2(MON.1) OSD.1,4(各USB 128GB)| | +----------------------------------------+ | | +----------------------------------------+ +--+※Ceph-node3(MON.2) OSD.2,5(各USB 128GB)| | +----------------------------------------+ | | +--------------------+ +--+※Ceph-node4(MDS.0) | +--------------------+ ※4台共通で Raspberry Pi3/Ubuntu16.04
2-2. CephDeployerのインストール
- Clinetで実行
$ sudo yum install ceph-deploy
- バージョンは
1.5.25-1でした
[ubuntu@comicsong ~]$ rpm -q ceph-deploy ceph-deploy-1.5.25-1.el7.noarch
- 各ceph-nodeに鍵配布してsshでパスなしでのログインを設定
ceph-deployを実施
[ubuntu@comicsong ~]$ ceph-deploy new ceph-node1 ceph-node2 ceph-node3 [ceph_deploy.conf][DEBUG ] found configuration file at: /home/ubuntu/.cephdeploy.conf [ceph_deploy.cli][INFO ] Invoked (1.5.25): /bin/ceph-deploy new ceph-node1 ceph-node2 ceph-node3 [ceph_deploy.new][DEBUG ] Creating new cluster named ceph [ceph_deploy.new][INFO ] making sure passwordless SSH succeeds [ceph-node1][DEBUG ] connected to host: comicsong.jp [ceph-node1][INFO ] Running command: ssh -CT -o BatchMode=yes ceph-node1 [ceph-node1][DEBUG ] connection detected need for sudo [ceph-node1][DEBUG ] connected to host: ceph-node1 [ceph-node1][DEBUG ] detect platform information from remote host [ceph-node1][DEBUG ] detect machine type [ceph-node1][DEBUG ] find the location of an executable [ceph-node1][INFO ] Running command: sudo /bin/ip link show [ceph-node1][INFO ] Running command: sudo /bin/ip addr show [ceph-node1][DEBUG ] IP addresses found: ['192.168.10.7'] [ceph_deploy.new][DEBUG ] Resolving host ceph-node1 [ceph_deploy.new][DEBUG ] Monitor ceph-node1 at 192.168.10.7 [ceph_deploy.new][INFO ] making sure passwordless SSH succeeds [ceph-node2][DEBUG ] connected to host: comicsong.jp [ceph-node2][INFO ] Running command: ssh -CT -o BatchMode=yes ceph-node2 [ceph-node2][DEBUG ] connection detected need for sudo [ceph-node2][DEBUG ] connected to host: ceph-node2 [ceph-node2][DEBUG ] detect platform information from remote host [ceph-node2][DEBUG ] detect machine type [ceph-node2][DEBUG ] find the location of an executable [ceph-node2][INFO ] Running command: sudo /bin/ip link show [ceph-node2][INFO ] Running command: sudo /bin/ip addr show [ceph-node2][DEBUG ] IP addresses found: ['192.168.10.8'] [ceph_deploy.new][DEBUG ] Resolving host ceph-node2 [ceph_deploy.new][DEBUG ] Monitor ceph-node2 at 192.168.10.8 [ceph_deploy.new][INFO ] making sure passwordless SSH succeeds [ceph-node3][DEBUG ] connected to host: comicsong.jp [ceph-node3][INFO ] Running command: ssh -CT -o BatchMode=yes ceph-node3 [ceph-node3][DEBUG ] connection detected need for sudo [ceph-node3][DEBUG ] connected to host: ceph-node3 [ceph-node3][DEBUG ] detect platform information from remote host [ceph-node3][DEBUG ] detect machine type [ceph-node3][DEBUG ] find the location of an executable [ceph-node3][INFO ] Running command: sudo /bin/ip link show [ceph-node3][INFO ] Running command: sudo /bin/ip addr show [ceph-node3][DEBUG ] IP addresses found: ['192.168.10.9'] [ceph_deploy.new][DEBUG ] Resolving host ceph-node3 [ceph_deploy.new][DEBUG ] Monitor ceph-node3 at 192.168.10.9 [ceph_deploy.new][DEBUG ] Monitor initial members are ['ceph-node1', 'ceph-node2', 'ceph-node3'] [ceph_deploy.new][DEBUG ] Monitor addrs are ['192.168.10.7', '192.168.10.8', '192.168.10.9'] [ceph_deploy.new][DEBUG ] Creating a random mon key... [ceph_deploy.new][DEBUG ] Writing monitor keyring to ceph.mon.keyring... [ceph_deploy.new][DEBUG ] Writing initial config to ceph.conf...
- 設定ファイルが作成された
- 何処に作られるのかなと探したがまさかの実行ディレクトリだった
[ubuntu@comicsong ~]$ pwd /home/ubuntu [ubuntu@comicsong ~]$ cat ceph.conf [global] fsid = 37164a6b-7187-4ba7-8b3f-4eccc58a9cf0 mon_initial_members = ceph-node1, ceph-node2, ceph-node3 mon_host = 192.168.10.7,192.168.10.8,192.168.10.9 auth_cluster_required = cephx auth_service_required = cephx auth_client_required = cephx filestore_xattr_use_omap = true
ceph-deploy installを実施
$ ceph-deploy install ceph-node1 ceph-node2 ceph-node3 ceph-node4
ceph-deploy mon create-initialを実施
$ ceph-deploy mon create-initial
- ここで問題発生。対象で
sudo initctl emit ceph-mon cluster=ceph id=ceph-node1を実施する箇所でエラー出る - 実行される各nodeは
initctlじゃなくてsystemdだからね…
[ceph-node1][INFO ] Running command: sudo initctl emit ceph-mon cluster=ceph id=ceph-node1
解決策を調べていたがchangelogを見る限り
1.5.2x台の後半からsystemdへの対応が多い最新の
1.5.34を導入してnewからやり直し
[ubuntu@comicsong ~]$ wget http://download.ceph.com/rpm/el7/noarch/ceph-deploy-1.5.34-0.noarch.rpm . . . `ceph-deploy-1.5.34-0.noarch.rpm' に保存中 100%[=========================================================================>] 288,604 95.8KB/s 時間 2.9s 2016-08-16 16:48:37 (95.8 KB/s) - `ceph-deploy-1.5.34-0.noarch.rpm' へ保存完了 [288604/288604] [ubuntu@comicsong ~]$ rpm --test -Uvh ceph-deploy-1.5.34-0.noarch.rpm 警告: ceph-deploy-1.5.34-0.noarch.rpm: ヘッダー V4 RSA/SHA1 Signature、鍵 ID 460f3994: NOKEY 準備しています... ################################# [100%] [ubuntu@comicsong ~]$ sudo rpm -Uvh ceph-deploy-1.5.34-0.noarch.rpm 警告: ceph-deploy-1.5.34-0.noarch.rpm: ヘッダー V4 RSA/SHA1 Signature、鍵 ID 460f3994: NOKEY 準備しています... ################################# [100%] 更新中 / インストール中... 1:ceph-deploy-1.5.34-0 ################################# [ 50%] 整理中 / 削除中... 2:ceph-deploy-1.5.25-1.el7 ################################# [100%]
- 旧バージョンで作成されたconfを削除しておく
rm ceph.conf
- newからやり直す
$ ceph-deploy new ceph-node1 ceph-node2 ceph-node3 $ ceph-deploy install ceph-node1 ceph-node2 ceph-node3 ceph-node4
- 続く
