启动compression
功能描述:
1.目前从k版本开始支持compression，目前支持zlib、snappy、zstd，注意从ceph version 12.0.0开始才加入了zstd插件。
2.注意配置的时候如果设置的插件不存在或者名称错误，则不启用压缩特性。
3.目前压缩特性只对data_pool这个pool的数据有效。
-
可以实时变更插件类型而不影响之前写入的数据访问。（写入object的时候每个object都有一个内置变量标记当前object使用的压缩插件类型。）
5.compression特性对用户透明，用户是无法感知压缩功能是否存在的。
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 |
root@demo:/home/user# radosgw-admin zone placement modify --rgw-zone=default --placement-id=default-placement --compression=zstd { "id": "eab5746a-26fa-42b4-a5f3-5d887ba846f9", "name": "default", "domain_root": "default.rgw.data.root", "control_pool": "default.rgw.control", "gc_pool": "default.rgw.gc", "lc_pool": "default.rgw.lc", "log_pool": "default.rgw.log", "intent_log_pool": "default.rgw.intent-log", "usage_log_pool": "default.rgw.usage", "user_keys_pool": "default.rgw.users.keys", "user_email_pool": "default.rgw.users.email", "user_swift_pool": "default.rgw.users.swift", "user_uid_pool": "default.rgw.users.uid", "system_key": { "access_key": "", "secret_key": "" }, "placement_pools": [ { "key": "default-placement", "val": { "index_pool": "default.rgw.buckets.index", "data_pool": "default.rgw.buckets.data", "data_extra_pool": "default.rgw.buckets.non-ec", "index_type": 0, "compression": "zstd" #插件类型 } }, { "key": "new", "val": { "index_pool": "new.buckets.index", "data_pool": "new.buckets.data", "data_extra_pool": "new.buckets.extra", "index_type": 0, "compression": "zlib" #插件类型 } } ], "metadata_heap": "", "tier_config": [], "realm_id": "" } |
查看compression配置
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
root@demo:/home/user# radosgw-admin zone placement list --rgw-zone=default [ { "key": "default-placement", "val": { "index_pool": "default.rgw.buckets.index", "data_pool": "default.rgw.buckets.data", "data_extra_pool": "default.rgw.buckets.non-ec", "index_type": 0, "compression": "zstd" #插件类型 } }, { "key": "new", "val": { "index_pool": "new.buckets.index", "data_pool": "new.buckets.data", "data_extra_pool": "new.buckets.extra", "index_type": 0, "compression": "zlib" #插件类型 } } ] |
压缩效果对比
测试使用一个iso镜像文件，对比压缩前后的体检差异
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 |
root@demo:/home/user# radosgw-admin bucket stats --bucket=snappy-test { "bucket": "snappy-test", "pool": "default.rgw.buckets.data", "index_pool": "default.rgw.buckets.index", "id": "0fef9464-bfe0-428b-86b5-b8d51876ff81.4274.1", "marker": "0fef9464-bfe0-428b-86b5-b8d51876ff81.4274.1", "owner": "u-gfn2636", "ver": "0#41", "master_ver": "0#0", "mtime": "2017-03-21 09:44:13.698936", "max_marker": "0#", "usage": { "rgw.main": { "size": 258998272, "size_actual": 248091783 ，#实际文件大小 "size_utilized": 248091783, #压缩后的大小 "size_kb": 252928, "size_kb_actual": 252928, "size_kb_utilized": 242278, "num_objects": 1 }, "rgw.multimeta": { "size": 0, "size_actual": 0, "size_utilized": 0, "size_kb": 0, "size_kb_actual": 0, "size_kb_utilized": 0, "num_objects": 0 } }, "bucket_quota": { "enabled": false, "check_on_raw": false, "max_size": -1, "max_size_kb": 0, "max_objects": -1 } } |
| 插件名称 | 未压缩 | snappy | zstd | zlib |
|---|---|---|---|---|
| 容量大小(kb) | 258998272 | 248091783 | 245851907 | 246105300 |
| 压缩率 | 100% | 95.78% | 94.92% | 95.02% |
总结
不考虑性能和时间的情况下，zstd效果最佳，由于每个插件还有一些压缩测试可以微调，所以测试的结果仅供参考。
附
zstd介绍 https://github.com/facebook/zstd
snappy介绍 https://github.com/google/snappy