网站首页swoole
在1核CPU,1G内存,2M带宽的vps上实现2000并发无失败写入数据库的程序实现
发布时间：2017-05-25 03:23:22编辑：slayer.hover阅读（6751）
部署swoole服务端，等待接收写数据信号。
使用lua脚本写主页面，使用redis连接池，把要写入的数据lpush推入redis缓存中。
redis数据写入后，用lua脚本触发swoole客户端，发送写数据信号。
swoole服务端接收到信号后，用PHP轮询redis队列，执行数据库写入。
VPS配置 CPU： 1核 内存：1 GB 内存：2Mbps， 用的阿里云最低配，算是垫底的了。
部署环境：
centos7 + openresty(nginx_lua) + redis（连接池) + swoole(php)+ mysql
开发步骤：
AB测压结果
10000请求数，500并发 :
Concurrency Level: 500 Time taken for tests: 19.823 seconds Complete requests: 10000 Failed requests: 0 Write errors: 0 Total transferred: 1670000 bytes HTML transferred: 140000 bytes Requests per second: 504.46 [#/sec] (mean) Time per request: 991.161 [ms] (mean) Time per request: 1.982 [ms] (mean, across all concurrent requests) Transfer rate: 82.27 [Kbytes/sec] received
10000请求数，1000并发 :
Concurrency Level: 1000 Time taken for tests: 24.471 seconds Complete requests: 10000 Failed requests: 0 Write errors: 0 Total transferred: 1670000 bytes HTML transferred: 140000 bytes Requests per second: 408.65 [#/sec] (mean) Time per request: 2447.062 [ms] (mean) Time per request: 2.447 [ms] (mean, across all concurrent requests) Transfer rate: 66.65 [Kbytes/sec] received
10000请求数，2000并发 :
Concurrency Level: 2000 Time taken for tests: 54.464 seconds Complete requests: 10000 Failed requests: 0 Write errors: 0 Total transferred: 1670000 bytes HTML transferred: 140000 bytes Requests per second: 183.61 [#/sec] (mean) Time per request: 10892.885 [ms] (mean) Time per request: 5.446 [ms] (mean, across all concurrent requests) Transfer rate: 29.94 [Kbytes/sec] received
可以看到，在2000并发时，虽然吞吐量和数据传输率下降的厉害，但也并没有失败的请求数。
数据库中也能查到数据完整的写入，缺点就是数据更新并不是实时的, 测压结束后，等swoole更新数据库还得老半天,
开太多task写数据库， 很容易把mysql搞死。
页面高并发，主要用到的是nginx_lua的超快处理能力。
附带上各开发部署的主要代码：
nginx:
server
{
listen 80;
server_name website.com;
index index.html index.php;
root /home/wwwroot/swoole;
location /test {
default_type 'text/html';
content_by_lua_file /usr/local/nginx/conf/lua;
}
}lua:
local function close_redis(red)
if not red then
return
end
local pool_max_idle_time = 10000 --毫秒
local pool_size = 50 --连接池大小
local ok, err = red:set_keepalive(pool_max_idle_time, pool_size)
if not ok then
ngx.say("set keepalive error : ", err)
end
end
ngx.header['Content-Type']="text/html;charset=UTF-8"
local redis = require("resty.redis")
local red = redis:new()
red:set_timeout(1000)
local ip = "127.0.0.1"
local port = 6379
local ok, err = red:connect(ip, port)
if not ok then
ngx.say("connect to redis error : ", err)
return close_redis(red)
end
--获取GET变量
local cjson = require("cjson")
local obj = ngx.req.get_uri_args()
local value= cjson.encode(obj)
ok, err = red:lpush("sql", value)
if not ok then
ngx.say("lpush msg error : ", err)
return close_redis(red)
end
close_redis(red)
ngx.location.capture("/client.php")
ngx.say("执行完毕!")swoole服务端代码
server.php
include_once('DB.php');
class Server
{
private $serv;
private static $db;
private static $cache;
public function __construct() {
$this->serv = new swoole_server("127.0.0.1", 9501);
$this->serv->set(array(
'worker_num' => 4, //一般设置为服务器CPU数的1-4倍
'daemonize' => 1, //以守护进程执行
'max_request' => 20000,
'dispatch_mode' => 1,//进程数据包分配模式 1平均分配，2按FD取摸固定分配，3抢占式分配
'log_file' => "./logs/swoole.log" ,//日志
));
$this->serv->on('Receive',array($this,'onReceive'));//接收到数据时回调此函数
$this->serv->on('WorkerStart',array($this,'onWorkerStart'));
$this->serv->on('WorkerStop',array($this,'onWorkerStop'));
$this->serv->on('ManagerStart', function ($serv) {
global $argv;
swoole_set_process_name("php {$argv[0]}: manager");
});
$this->serv->start();
}
public function onReceive(swoole_server $serv, $fd, $from_id, $data) {
if($data=='shutdown'){
$serv->send($fd, 'SHUTDOWN');
$serv->close($fd);
$serv->shutdown();
}elseif($data=='sql'){
if(self::$cache->exists($data)){
$this->taskQueue($data, $serv->worker_id);
}
}
}
public function taskQueue($data, $worker_id) {
self::$db->ping(); /***判断数据库连接是否正常***/
try{
while($rows= self::$cache->rPop($data)){
$rowsarray =json_decode($row, TRUE);
$result= self::$db->add('test', $rowsarray);
if($result===FALSE)
self::Log(date('Ymd'), "Error on {$worker_id}: ".$rows);
}
}catch(Exception $e){
self::Log('queue', $e->getMessage());
}
}
public function onWorkerStart($serv, $worker_id)
{
global $argv;
if ($worker_id >= 4) {
swoole_set_process_name("php {$argv[0]}: task");
} else {
swoole_set_process_name("php {$argv[0]}: worker");
}
if(self::$db==NULL){
$dbset = array(
'dsn' => 'mysql:host=localhost;dbname=test',
'name' => 'root',
'password' => '123456',
);
self::$db = new DB($dbset);
self::Log(date('Ymd'), "Connect mysql on {$worker_id} at ".date('Y-m-d H:i:s'));
}
if(self::$cache==NULL){
$server=array(
'host'=>'127.0.0.1',
'port'=>6379,
);
self::$cache = new Redis();
self::$cache->connect($server['host'], $server['port']);
self::Log(date('Ymd'), "Connect redis on {$worker_id} at ".date('Y-m-d H:i:s'));
}
}
public function onWorkerStop($serv, $worker_id)
{
if(self::$db!=NULL){
self::$db->close();
self::$db=NULL;
self::Log(date('Ymd'), "Close mysql on {$worker_id} at " . date('Y-m-d H:i:s'));
}
if(self::$cache!=NULL){
self::$cache->close();
self::$cache=NULL;
self::Log(date('Ymd'), "Close redis on {$worker_id} at " . date('Y-m-d H:i:s'));
}
}
public static function Log($file="mylog", $data) {
$file = dirname(__FILE__) . "/logs/" . $file . ".log";
file_put_contents($file, $data."\r\n", FILE_APPEND | LOCK_EX);
}
}客户端代码
client.php
<?php
$client = new swoole_client(SWOOLE_SOCK_TCP);
if (!$client->connect('127.0.0.1', 9501, -1))
{
exit("connect failed. Error: {$client->errCode}\n");
}
$sender=$client->send("sql");
$client->close();
exit;----------------------------redis连接池及lua脚本更新-------------------------------
可以使用redis的upstream配置，在nginx.conf配置文件里
upstream redispool{
server 127.0.0.1:6379;
keepalive 200;
}
server{
location /redis{
internal;
set_unescape_uri $query $arg_query;
redis2_query lpush mykey $query;
redis2_pass redispool;
}
}修改lua脚本为：
ngx.header['Content-Type']="text/html;charset=UTF-8"
--获取GET变量
local cjson= require("cjson")
local obj = ngx.req.get_uri_args()
local value= cjson.encode(obj)
local res = ngx.location.capture("/redis", {
args = {query = value}
})
if res.status ~= 200 then
ngx.log(ngx.DEBUG, "Redis写入失败,capture /redis未返回成功标识.")
end
ngx.location.capture("/client.php")
ngx.say("执行完毕!")
ngx.exit(ngx.HTTP_OK);执行方式和原来一样， 不过可以配置主从redis结构或者使用多个redis服务器来提高并发能力。
评论