首页 产品 技术 查看内容

分布式文件系统:MogileFS 的安装和使用

2014-3-1 01:34| 发布者: tianzc| 查看: 3263| 评论: 0

摘要: 标题:MogileFS for PHP安装和测试(存档)MogileFS是一个perl编写的分布式文件存储应用,目前有一些线上的公司都是以此为基础来构建自己的分布式文件应用,抽时间测试了一下。一、搭建MogileFS关于MogileFS在扶凯的 ...


标题:MogileFS for PHP安装和测试(存档)

 MogileFS是一个perl编写的分布式文件存储应用,目前有一些线上的公司都是以此为基础来构建自己的分布式文件应用,抽时间测试了一下。

一、搭建MogileFS

关于MogileFS在扶凯的网站(http://www.php-oa.com/2010/09/26/perl-mogilefs-1.html)上有很详细的介绍,关于安装搭建我也只是简单描述。

Mysql server:27.17.28.133:3306
MogileFS trackers:27.17.28.133:7777
MogileFS store1:node134  27.17.28.134:7500
MogileFS store2:node135  27.17.28.135:7500

在三台机器上都安装
cpanm MoglieFS::Server
cpanm MoglieFS::Utils

分别配置好
/etc/mogilefs/mogilefs.conf (三台都装)

trackers = 27.17.28.133:7777

/etc/mogilefs/mogilefsd.conf (trackers)

db_dsn = DBI:mysql:MogileFS:host=27.17.28.133
db_user = mogile
db_pass = mogilepw
listen = 27.17.28.133:7777
conf_port = 7777
query_jobs = 10
delete_jobs = 1
replicate_jobs = 5
reaper_jobs = 1

/etc/mogilefs/mogstored.conf (stored)

maxconns = 10000
httplisten = 27.17.28.135:7500
mgmtlisten = 27.17.28.135:7501
docroot=/data/mogdata

配置mysql
CREATE DATABASE MogileFS DEFAULT CHARACTER SET utf8 COLLATE utf8_bin;
grant all on MogileFS.* to ‘mogile’@'%’ identified by ‘mogilepwd’;
FLUSH PRIVILEGES;

初始化tracker
# mogdbsetup –dbhost=27.17.28.133 –dbname=MogileFS –dbuser=mogile –dbpass=mogilepw

添加用户
# useradd mogile

启动trackers
$ mogilefsd -c /etc/mogilefs/mogilefsd.conf –daemon

启动stored
# mogstored -d

二、文件存储

添加存储节点
# mogadm host add node134 –ip=27.17.28.134 –port=7500 –status=alive
# mogadm host add node135 –ip=27.17.28.135 –port=7500 –status=alive
查看节点
# mogadm host list

添加存储设备
node134# mkdir -p /var/mogdata/dev1
# mogadm device add node134 1
查看存储设备
# mogadm device list

添加存储域
# mogadm domain add test.com
查看存储域
# mogadm domain list

添加存储类
存储域内有一个默认的存储类(class)default
# mogadm class add test.com upload_image –mindevcount=2
查看存储类
# mogadm class list

上传一个文件
# mogupload –trackers=27.17.28.133:7777 –domain=test.com –key=’memcached-1.4.13.tar.gz’ –file=’/tmp/memcached-1.4.13.tar.gz’
查看文件信息
# mogfileinfo –domain=test.com –key=’memcached-1.4.13.tar.gz’

三、mogileFS for PHP

下载mogilesFs的php扩展包:http://pecl.php.net/package/mogilefs,和其他php的扩展包一样的安装。
# /data/app/php/bin/phpize
# ./configure –with-php-config=/data/app/php-cgi/bin/php-config
# make & make install
如有neon的报错请安装neon-devel包。

在php.ini中加载拓展
extension=”mogilefs.so”

php中mogileFS模块的使用,在官方文档中的说明了貌似很陈旧,其用法还是直接看扩展包你的README吧。

<?php 
$client = new MogileFs(); 
//链接tracker服务器,指定存储域 
$client--->connect('27.17.28.133', 7777, 'test.com');
//根据key来获取文件信息
//$stat_str=$client->fileInfo("memcached-1.4.13.tar.gz");
//根据key来获取文件保存路径
$stat_str=$client->get("memcached-1.4.13.tar.gz");
//奇怪,用官方的getPaths竟然报错次方法未定义
var_dump($stat_str);
?>


标题:尝试mogilefs for nginx的模块

模块有个简单的文档
http://www.grid.net.ru/nginx/mogilefs.en.html

安装

#wget http://www.grid.net.ru/nginx/download/nginx_mogilefs_module-1.0.4.tar.gz
#tar zxvf nginx_mogilefs_module-1.0.4.tar.gz
#cd nginx-1.0.14
#./configure –prefix=/data/app/nginx –add-module=../nginx_mogilefs_module-1.0.4/ –with-debug
#make & make install

#vim /data/app/nginx/conf/nginx.conf

location /download/ {
                mogilefs_tracker 27.17.28.133:7777;
                mogilefs_domain test1;
                mogilefs_pass {
                        proxy_pass $mogilefs_path;
                        proxy_hide_header Content-Type;
                        proxy_buffering off;
                }
        }

访问 http://nginx/download/mogilefs_key 就可以访问mogiles stored里的文件了,所以,最好是以文件名作为key存入mogilefs。

nginx中的访问控制,流量控制等对这个url同样有效。

关于upload

按照文档nginx.conf如下

location /upload/{
                allow all;

                autoindex       on;
                client_max_body_size    200m;
                client_body_temp_path   /tmp;

                mogilefs_tracker 27.17.28.133:7777;
                mogilefs_domain test1;
                mogilefs_methods PUT DELETE;
                mogilefs_pass {
                        proxy_pass $mogilefs_path;
                        proxy_hide_header Content-Type;
                        proxy_buffering off;
                }
        }

用curl模拟了delete的操作,好像没什么问题,tracker的mysql记录和两台stored的文件都删除了
curl –request DELETE http://27.17.28.134/upload/2222

但是put操作确发生了问题
curl –request PUT –data “/home/ftpuser/memcached-1.4.13.tar.gz” http://27.17.28.134/upload/memcached-1.4.13.tar.gz
tracker的mysql已经写入到tempfile的数据表中了,而且其中一台stored已经写入了文件,但是好像在写入第二台stored的时候出现了问题,一直到curl报错退出
curl: (52) Empty reply from server

追踪了一下nginx的debug日志:
012/05/04 16:11:43 [debug] 31056#0: *10 http upstream request: “/mogstored_spare_352054896/2222?”
2012/05/04 16:11:43 [debug] 31056#0: *10 http upstream process header
2012/05/04 16:11:43 [debug] 31056#0: *10 malloc: 0000000014FA4250:4096
2012/05/04 16:11:43 [debug] 31056#0: *10 recv: fd:7 118 of 4096
2012/05/04 16:11:43 [debug] 31056#0: *10 http proxy status 200 “200 OK”
2012/05/04 16:11:43 [debug] 31056#0: *10 http proxy header: “Content-Type: text/html”
2012/05/04 16:11:43 [debug] 31056#0: *10 http proxy header: “Content-Length: 18″
2012/05/04 16:11:43 [debug] 31056#0: *10 http proxy header: “Server: Perlbal”
2012/05/04 16:11:43 [debug] 31056#0: *10 http proxy header: “Connection: close”
2012/05/04 16:11:43 [debug] 31056#0: *10 http proxy header done
2012/05/04 16:11:43 [debug] 31056#0: *10 finalize http upstream request: 0
2012/05/04 16:11:43 [debug] 31056#0: *10 finalize http proxy request
2012/05/04 16:11:43 [debug] 31056#0: *10 free rr peer 1 0
2012/05/04 16:11:43 [debug] 31056#0: *10 close http upstream connection: 7
2012/05/04 16:11:43 [debug] 31056#0: *10 event timer del: 7: 1336119163692
2012/05/04 16:11:43 [debug] 31056#0: *10 reusable connection: 0
2012/05/04 16:11:43 [debug] 31056#0: *10 http output filter “/mogstored_spare_352054896/2222?”
2012/05/04 16:11:43 [debug] 31056#0: *10 http copy filter: “/mogstored_spare_352054896/2222?”
2012/05/04 16:11:43 [debug] 31056#0: *10 http postpone filter “/mogstored_spare_352054896/2222?” 00007FFF4C412DC0
2012/05/04 16:11:43 [debug] 31056#0: *10 write old buf t:0 f:0 0000000000000000, pos 0000000000000000, size: 0 file: 0, size: 0
2012/05/04 16:11:43 [debug] 31056#0: *10 write new buf t:0 f:0 0000000000000000, pos 0000000000000000, size: 0 file: 0, size: 0
2012/05/04 16:11:43 [debug] 31056#0: *10 http write filter: l:0 f:0 s:0
2012/05/04 16:11:43 [debug] 31056#0: *10 http copy filter: 0 “/mogstored_spare_352054896/2222?”
2012/05/04 16:11:43 [debug] 31056#0: *10 http finalize request: 0, “/mogstored_spare_352054896/2222?” a:1, c:1
2012/05/04 16:11:43 [debug] 31056#0: *10 mogilefs finish phase handler: state=2, status=0
2012/05/04 16:11:43 [debug] 31056#0: *10 http wake parent request: “/upload/2222?”
2012/05/04 16:11:43 [debug] 31056#0: *10 http posted request: “/upload/2222?”
2012/05/04 16:11:43 [debug] 31056#0: *10 content phase: 9
2012/05/04 16:11:43 [debug] 31056#0: *10 mogilefs put handler
2012/05/04 16:11:43 [debug] 31056#0: *10 mogilefs put handler state: 2, status: 0
2012/05/04 16:11:43 [debug] 31056#0: *10 http subrequest “/mogstored_spare_352059552/2222?”
2012/05/04 16:11:43 [debug] 31056#0: *10 http finalize request: -4, “/upload/2222?” a:0, c:1
2012/05/04 16:11:43 [debug] 31056#0: *10 set http keepalive handler
2012/05/04 16:11:43 [debug] 31056#0: *10 http close request
2012/05/04 16:11:43 [debug] 31056#0: *10 http log handler
2012/05/04 16:11:43 [debug] 31056#0: *10 posix_memalign: 0000000014FA5260:4096 @16
2012/05/04 16:11:43 [debug] 31056#0: *10 run cleanup: 0000000014FB45C0
2012/05/04 16:11:43 [debug] 31056#0: *10 file cleanup: fd:4
2012/05/04 16:11:43 [debug] 31056#0: *10 free: 0000000014FA4250
2012/05/04 16:11:43 [debug] 31056#0: *10 free: 0000000014FA1220
2012/05/04 16:11:43 [debug] 31056#0: *10 free: 0000000014F7D370
2012/05/04 16:11:43 [debug] 31056#0: *10 free: 0000000014FB3980, unused: 0
2012/05/04 16:11:43 [debug] 31056#0: *10 free: 0000000014FA0210, unused: 0
2012/05/04 16:11:43 [debug] 31056#0: *10 free: 0000000014FA2230, unused: 2
2012/05/04 16:11:43 [debug] 31056#0: *10 free: 0000000014FA3240, unused: 56
2012/05/04 16:11:43 [debug] 31056#0: *10 free: 0000000014FA5260, unused: 3867
2012/05/04 16:11:43 [debug] 31056#0: *10 event timer add: 3: 65000:1336119168693
2012/05/04 16:11:43 [debug] 31056#0: *10 free: 0000000014F83B80
2012/05/04 16:11:43 [debug] 31056#0: *10 free: 0000000014F84180
2012/05/04 16:11:43 [debug] 31056#0: *10 hc free: 0000000000000000 0
2012/05/04 16:11:43 [debug] 31056#0: *10 hc busy: 0000000000000000 0
2012/05/04 16:11:43 [debug] 31056#0: *10 reusable connection: 1
2012/05/04 16:11:43 [debug] 31056#0: *10 post event 0000000014F862D0
2012/05/04 16:11:43 [debug] 31056#0: timer delta: 1
2012/05/04 16:11:43 [debug] 31056#0: posted events 0000000014F862D0
2012/05/04 16:11:43 [debug] 31056#0: posted event 0000000014F862D0
2012/05/04 16:11:43 [debug] 31056#0: *10 delete posted event 0000000014F862D0
2012/05/04 16:11:43 [debug] 31056#0: *10 http keepalive handler
2012/05/04 16:11:43 [debug] 31056#0: *10 malloc: 0000000014F83B80:1024
2012/05/04 16:11:43 [debug] 31056#0: *10 recv: fd:3 -1 of 1024
2012/05/04 16:11:43 [debug] 31056#0: *10 recv() not ready (11: Resource temporarily unavailable)
2012/05/04 16:11:43 [debug] 31056#0: posted event 0000000000000000
2012/05/04 16:11:43 [debug] 31056#0: worker cycle
2012/05/04 16:11:43 [debug] 31056#0: epoll timer: 65000
2012/05/04 16:12:48 [debug] 31056#0: timer delta: 65001
2012/05/04 16:12:48 [debug] 31056#0: *10 event timer del: 3: 1336119168693
2012/05/04 16:12:48 [debug] 31056#0: *10 http keepalive handler
2012/05/04 16:12:48 [debug] 31056#0: *10 close http connection: 3
2012/05/04 16:12:48 [debug] 31056#0: *10 reusable connection: 0
2012/05/04 16:12:48 [debug] 31056#0: *10 free: 0000000014F83B80
2012/05/04 16:12:48 [debug] 31056#0: *10 free: 0000000000000000
2012/05/04 16:12:48 [debug] 31056#0: *10 free: 0000000014F7B4F0, unused: 8
2012/05/04 16:12:48 [debug] 31056#0: *10 free: 0000000014F84070, unused: 128
2012/05/04 16:12:48 [debug] 31056#0: posted events 0000000000000000
2012/05/04 16:12:48 [debug] 31056#0: worker cycle
2012/05/04 16:12:48 [debug] 31056#0: epoll timer: -1

补:
这个问题已经解决,需要一个小小的patch,打完patch之后重新编译nginx即可。

详情见这个 http://www.ruby-forum.com/topic/217606

diff --git a/ngx_http_mogilefs_module.c b/ngx_http_mogilefs_module.c
index e229f47..a4d249d 100644
--- a/ngx_http_mogilefs_module.c
+++ b/ngx_http_mogilefs_module.c
@@ -483,6 +483,9 @@ ngx_http_mogilefs_put_handler(ngx_http_request_t *r)
         case FETCH:
             spare_location = mgcf->create_close_spare_location;
             ctx->state = CREATE_CLOSE;
+#if defined nginx_version && nginx_version >= 8011
+            r->main->count++;
+#endif
             break;
         case CREATE_CLOSE:
             r->headers_out.content_length_n = 0;

鲜花

握手

雷人

路过

鸡蛋

扫一扫关注最新动态

毒镜头:老镜头、摄影器材资料库、老镜头样片、摄影
爱评测 aipingce.com  
返回顶部