Fighting for Dream

Memcache的备忘

• 用memcache保存session的例子，非常简单
   
  CODE:

  1. <?php
  2. $session_save_path = "tcp://$host:$port?persistent=1&weight=2&timeout=2&retry_interval=10, ,tcp://$host:$port ";
  3. ini_set(‘session.save_handler’, ‘memcache’);
  4. ini_set(‘session.save_path’, $session_save_path);
  5. ?>

   

• memcache每一个item上限是1M，注意不要超出上限
• memcache本身并不支持namespace，但是可以通过一些手段模拟出namespace的效果来，见Memcache 中模拟 namespace
• 刚接触memcache的时候，可能会写出这样的代码来
  CODE:

  1. $zhang = $memcache->get(‘key1’);
  2. $li = $memcache->get(‘key2’);
  3. $wang = $memcache->get(‘key3’);

  这种写法实际运行效果是

  • get(key1) – 客户端发出请求 – 服务器端查询 – 客户端获取
  • get(key2) – 客户端 – 服务器端 – 客户端
  • get(key3) – 客户端 – 服务器端 – 客户端
  • …

  如此一来，会有三次客户端和服务器端交互的过程。但是如果用批量查询的方法，就只有一次交互的过程。比如:

   
  CODE:

  1. $all = $memcache->get(array(‘key1’, ‘key2’, ‘key3’));

  这样性能会有一些提升。对于其它程序语言来说，也封装了类似get_multi这样的方法。

• 从数据库从查询获得一个列表，放到memcache里面保存起来是一个不错的主意，但是不要忘记memcache有1m限制。如果列表太大，可以考虑把数据分割开来，然后用key序列来保存这个列表数据，比如event_0_500来保存前500行,用event_0_1000保存500-1000行，在获取的时候可以用前面说的批量get来一次性得到这个列表。
• 经常观察memcached的大小和命中率，方便调整缓存策略

最简便的清空memcache的方法

This is easy right? Can’t you just restart the memcached server? Well yes, but you may cause errors in applications that are already connected to it. You can follow your memcached restart with an application restart, eg for a Ruby on Rails app:

# /etc/init.d/memcached restart && mongrel_rails cluster::restart

Of course if you have more than one application server you have to restart your app on every single one. This would work on an engineyard slice assuming you have the eycap gem installed:

$ cap production memcached:restart
$ cap production mongrel:restart

Restarting your application is not ideal however, you will lose anything cached in memory, cause delays to users trying to access your site, that sort of thing.

So what can be done? The answer is really simple. Assuming a memcached running on the local machine on the default port:

$ echo “flush_all” | nc localhost 11211

Easy!

启用memcached压缩注意事项

在php开发中，开启memcache的数据压缩存储是一件很简单的事情。在多数情况下，压缩数据不仅不会降低程序的执行效率，反倒会因为网络传输的开销降低，带来速度提升。看看最常用的Memcache::set方法:
bool Memcache::set ( string $key , mixed $var [, int $flag [, int $expire ]] )

在这个方法中，将$flag设置为MEMCACHE_COMPRESSED即可启用memcache压缩存储。

这样做有什么弊端？

如果没有做额外判断，每一次写入memcache都会启用压缩，不管数据的大小。对应的，每次获得数据都需要做一次解压缩的操作，这是典型的一刀切手法。实际上在数据很小的情况下，不需要压缩，在这个基础上压缩省不了多少空间。

更好的压缩策略？

好了，我的想法是在数据超过一定大小（比如2k）的情况下，才开启压缩。这个好办，捋起袖子就干，在调用Memcache::set方法之前，首先判断一下数据的大小，一个strlen就搞定了，再简单不过了。

上面的文字都是废话，直接看这段就好。Memcache::setCompressThreshold方法可以包办之前所有的逻辑。

Memcache::setCompressThreshold – Enable automatic compression of large values

bool Memcache::setCompressThreshold ( int $threshold [, float $min_savings ] )

举个例子，下面这段会自动启用压缩策略，当数据大于2k时，以0.2的压缩比进行zlib。

memcache.php stats like apc.php

For a long time I was looking for a nice web interface like the apc.php (comes with the apc’s source) that displays whole nine yards of stats. The only good tool is memcache-tool from the danga guys. It’s quite complete but I guess I’m too lazy to go on the command line.

Anyways, I decided to rip write my own. Totally based on the original apc.php (I even recycled some functions) and apart from completeness, here is a memcache.php that you can get stats and dump from multiple memcache servers.
Here is a screenshot:


And here is the the source code.

As usual, this piece of software comes with the warnings:
– Don’t expect something complete, might have bugs and security problems etc.
– Do not install on a prod environment unless you’re sure!
– Feel free to add stuff to it, I’ll put it to google code or sf.net soon.
– Feel free to request features. (no I’m not planning a backup tool , there is memcache-tool for that)
– Enjoy!

如何监控MemCached的状态

使用MemCached以后，肯定希望知道cache的效果，对于MemCached的一些运行状态进行监控是必要的。MemCached也提供了stats接口输出一些信息，最简单的方式，就是telnet上去输入stats查看：

telnet 127.0.0.1 11211
Trying 127.0.0.1 ...
Connected to memcache_test_host (127.0.0.1 ).
Escape character is '^]'.
stats
STAT pid 7186
STAT uptime 1695
STAT time 1238401344
STAT version 1.2.6
STAT pointer_size 64
STAT rusage_user 0.003999
STAT rusage_system 0.002999
STAT curr_items 1
STAT total_items 54
STAT bytes 135
STAT curr_connections 2
STAT total_connections 111
STAT connection_structures 4
STAT cmd_get 3
STAT cmd_set 54
STAT get_hits 0
STAT get_misses 3
STAT evictions 0
STAT bytes_read 5957
STAT bytes_written 50914
STAT limit_maxbytes 2147483648
STAT threads 1
END

这种方式相当的不方便，所以网上就有各种不同客户端接口写的工具，比如用perl写的这个memcache-tool：

./memcached_tool

Usage: memcached-tool  [mode]

       memcached-tool 10.0.0.5:11211 display    # shows slabs
       memcached-tool 10.0.0.5:11211            # same.  (default is display)
       memcached-tool 10.0.0.5:11211 stats      # shows general stats
       memcached-tool 10.0.0.5:11211 move 7 9   # takes 1MB slab from class #7
                                                # to class #9.

You can only move slabs around once memory is totally allocated, and only
once the target class is full.  (So you can't move from #6 to #9 and #7
to #9 at the same itme, since you'd have to wait for #9 to fill from
the first reassigned page)

$ ./memcached_tool 127.0.0.1:11211 stats

  #127.0.0.1:11211 Field       Value
                   bytes         135
              bytes_read        5964
           bytes_written       51394
                 cmd_get           3
                 cmd_set          54
   connection_structures           4
        curr_connections           3
              curr_items           1
               evictions           0
                get_hits           0
              get_misses           3
          limit_maxbytes  2147483648
                     pid        7186
            pointer_size          64
           rusage_system    0.002999
             rusage_user    0.003999
                 threads           1
                    time  1238401521
       total_connections         112
             total_items          54
                  uptime        1872
                 version       1.2.6

命令行的方式，在批处理调用的时候比较方便。但是在展现方面还是web方式更加直观有效，所以就有了php写的memcache.php，是的，用一次就知道这是我想要的。
 

在PHP使用MemCached

在PHP中使用Memcached，有两种方式，一种是安装PHP的memcache扩展(实际上还有另外一个memcached扩展，是基于比较流行的libmemcached库封装的)，该扩展是用c写的，效率较高，需要在服务器上安装。另外一种则是直接使用客户端的php-memcached-client类库，但是这个我在网上找了半天也没找到一个官方的网站。所以呢，还是装个扩展吧。假设php安装在/home/admin/php目录：

wget http://pecl.php.net/get/memcache-2.2.5.tgz
gzip -d memcache-2.2.5.tgz
tar xvf memcache-2.2.5.tar
cd memcache-2.2.5
/home/admin/php/bin/phpize
Configuring for:
PHP Api Version:         20041225
Zend Module Api No:      20060613
Zend Extension Api No:   220060519

./configure --enable-memcache --with-php-config=/home/admin/php/bin/php-config --with-zlib-dir
Installing shared extensions:     /home/admin/php/lib/php/extensions/no-debug-non-zts-20060613/

注意到最后一行返回的信息，将下面两行添加到/home/admin/php/lib/php.ini

extension_dir = "/home/admin/php/lib/php/extensions/no-debug-non-zts-20060613/"
extension=memcache.so

然后重启web服务器即可。如果安装成功，则通过phpinfo()可以获得该扩展的相关信息：

以上参数都可以在php.ini中进行设置。下面是一段官方网站的php测试代码：

<?php
$memcache = new Memcache;
$memcache->connect('127.0.0.1', 11211) or die ("Could not connect");
$version = $memcache->getVersion();
echo "Server's version: ".$version."\n";
$tmp_object = new stdClass;
$tmp_object->str_attr = 'test';
$tmp_object->int_attr = 123;
$memcache->set('key', $tmp_object, false, 10) or die ("Failed to save data at the server");
echo "Store data in the cache (data will expire in 10 seconds)\n";
$get_result = $memcache->get('key');
echo "Data from the cache:\n";
var_dump($get_result);
?>

运行后输出如下：

Server's version: 1.2.6
Store data in the cache (data will expire in 10 seconds)
Data from the cache: object(stdClass)#3 (2)
{ ["str_attr"]=>  string(4) "test" ["int_attr"]=>  int(123) }

在64位Linux上安装MemCached

在一台64位Linux的机器上安装了MemCached，遇到一个小问题，特记录之。

MemCached使用了libevent，所以必须先安装libevent。安装libevent到/usr/lib

wget http://www.monkey.org/~provos/libevent-1.4.9-stable.tar.gz
gzip -d libevent-1.4.9-stable.tar.gz
tar xvf libevent-1.4.9-stable.tar
cd libevent-1.4.9-stable
./configure --prefix=/usr
make
make install

安装MemCached的到/u01/memcached

wget http://www.danga.com/memcached/dist/memcached-1.2.6.tar.gz
gzip -d memcached-1.2.6.tar.gz
tar xvf memcached-1.2.6.tar
cd memcached-1.2.6
./configure --prefix=/u01/memcached --with-libevent=/usr
make
make install

但是执行memcached命令时出现错误：

#/u01/memcached/bin/memcached -h
/u01/memcached/bin/memcached: error while loading shared libraries: libevent-1.4.so.2:
cannot open shared object file: No such file or directory

一般对于这种依赖的库找不到的情况，在Linux中可以通过设置LD_DEBUG环境变量来获得更多的信息

#LD_DEBUG=help ls
Valid options for the LD_DEBUG environment variable are:

  libs        display library search paths
  reloc       display relocation processing
  files       display progress for input file
  symbols     display symbol table processing
  bindings    display information about symbol binding
  versions    display version dependencies
  all         all previous options combined
  statistics  display relocation statistics
  unused      determined unused DSOs
  help        display this help message and exit

To direct the debugging output into a file instead of standard output
a filename can be specified using the LD_DEBUG_OUTPUT environment variable.

这里由于是库文件依赖有问题，则使用libs参数：

#LD_DEBUG=libs /u01/memcached/bin/memcached -h
     30596:     find library=libevent-1.4.so.2 [0]; searching
     30596:      search cache=/etc/ld.so.cache
     30596:      search path=/lib64/tls/x86_64:/lib64/tls:/lib64/x86_64:/lib64:/usr/lib64/tls/x86_64
/usr/lib64/tls:/usr/lib64/x86_64:/usr/lib64
(system search path)
     30596:       trying file=/lib64/tls/x86_64/libevent-1.4.so.2
     30596:       trying file=/lib64/tls/libevent-1.4.so.2
     30596:       trying file=/lib64/x86_64/libevent-1.4.so.2
     30596:       trying file=/lib64/libevent-1.4.so.2
     30596:       trying file=/usr/lib64/tls/x86_64/libevent-1.4.so.2
     30596:       trying file=/usr/lib64/tls/libevent-1.4.so.2
     30596:       trying file=/usr/lib64/x86_64/libevent-1.4.so.2
     30596:       trying file=/usr/lib64/libevent-1.4.so.2
     30596:
/u01/memcached/bin/memcached: error while loading shared libraries: libevent-1.4.so.2:
cannot open shared object file: No such file or directory

可以看到是在加载/usr/lib64/libevent-1.4.so.2文件时出现了问题，系统中确实是没有该文件的，查找后发现libevent-1.4.so.2存在于/usr/lib目录，这可能是libevent在64位Linux系统上的一个bug吧，没有关系，复制一份或者建一个软链接即可解决问题。

#ln -s /usr/lib/libevent-1.4.so.2 /usr/lib64/libevent-1.4.so.2
#/u01/memcached/bin/memcached -h
memcached 1.2.6
-p
      TCP port number to listen on (default: 11211)
-U
      UDP port number to listen on (default: 0, off)
-s
     unix socket path to listen on (disables network support)
-a
     access mask for unix socket, in octal (default 0700)
-l
  interface to listen on, default is INDRR_ANY
-d            run as a daemon
-r            maximize core file limit
-u
 assume identity of
 (only when run as root)
-m
      max memory to use for items in megabytes, default is 64 MB
-M            return error on memory exhausted (rather than removing items)
-c
      max simultaneous connections, default is 1024
-k            lock down all paged memory.  Note that there is a
              limit on how much memory you may lock.  Trying to
              allocate more than that would fail, so be sure you
              set the limit correctly for the user you started
              the daemon with (not for -u
 user;
              under sh this is done with 'ulimit -S -l NUM_KB').
-v            verbose (print errors/warnings while in event loop)
-vv           very verbose (also print client commands/reponses)
-h            print this help and exit
-i            print memcached and libevent license
-b            run a managed instanced (mnemonic: buckets)
-P
     save PID in
, only used with -d option
-f
   chunk size growth factor, default 1.25
-n
    minimum space allocated for key+value+flags, default 48

启动MemCached，-m表示分配的内存

#/u01/memcached/bin/memcached -d -m 1024 -u admin -l 127.0.0.1 -p 11211