using_queue_daemons.rst 3.85 KB
Newer Older
1 2
Using Queue Daemons
===================
drymer's avatar
drymer committed
3
.. _usando-queue-daemons:
4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51

Queues and Daemons
------------------
Some activities that GNU social needs to do, like broadcast OStatus, SMS, XMPP
messages and TwitterBridge operations, can be 'queued' and done by off-line
bots instead.

Two mechanisms are available to achieve offline operations:

* New embedded OpportunisticQM plugin, which is enabled by default

* Legacy queuedaemon script, which can be enabled via config file.

OpportunisticQM plugin
^^^^^^^^^^^^^^^^^^^^^^
This plugin is enabled by default. It tries its best to do background jobs
during regular HTTP requests, like API or HTML pages calls.

Since queueing system is enabled by default, notices to be broadcasted will be
stored, by default, into DB (table ``queue_item``).

Whenever it has time, OpportunisticQM will try to handle some of them.

This is a good solution whether you:

* have no access to command line (shared hosting)

* do not want to deal with long-running PHP processes

* run a low traffic GNU social instance

In other case, you really should consider enabling the queuedaemon for
performance reasons. Background daemons are necessary anyway if you wish to use
the Instant Messaging features such as communicating via XMPP.

queuedaemon
^^^^^^^^^^^
If you want to use legacy queuedaemon, you must be able to run long-running
offline processes, either on your main Web server or on another server you
control. (Your other server will still need all the above prerequisites, with
the exception of Apache.) Installing on a separate server is probably a good
idea for high-volume sites.

1. You'll need the "CLI" (command-line interface) version of PHP installed on
   whatever server you use.

  Modern PHP versions in some operating systems have disabled functions related
  to forking, which is required for daemons to operate. To make this work, make
drymer's avatar
drymer committed
52
  sure that your ``php-cli`` config (``/etc/php5/cli/php.ini``) does NOT have these
53 54 55
  functions listed under 'disable_functions':

  * pcntl_fork, pcntl_wait, pcntl_wifexited, pcntl_wexitstatus,
drymer's avatar
drymer committed
56
    pcntl_wifsignaled, pcntl_wtermsig
57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76

  Other recommended settings for optimal performance are:

  * ``mysqli.allow_persistent = On``
  * ``mysqli.reconnect = On``

2. If you're using a separate server for queues, install GNU social somewhere
   on the server. You don't need to worry about the ``.htaccess`` file, but
   make sure that your ``config.php`` file is close to, or identical to, your
   Web server's version.

3. In your config.php files (on the server where you run the queue daemon), set
   the following variable::

    $config['queue']['daemon'] = true;

4. On the queues server, run the command ``scripts/startdaemons.sh``.

  This will run the queue handlers:

drymer's avatar
drymer committed
77
  ``queuedaemon.php``
78 79 80
    polls for queued items for inbox processing and pushing out to OStatus,
    SMS, XMPP, etc.

drymer's avatar
drymer committed
81
  ``imdaemon.php``
82 83 84
    if an IM plugin is enabled (like XMPP)

  (plugins)
drymer's avatar
drymer committed
85
    other daemons, like TwitterBridge ones, that you may have enabled
86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104

  These daemons will automatically restart in most cases of failure including
  memory leaks (if a memory_limit is set), but may still die or behave oddly if
  they lose connections to the XMPP or queue servers.

  It may be a good idea to use a daemon-monitoring service, like 'monit', to
  check their status and keep them running.

  All the daemons write their process IDs (pids) to ``/var/run/`` by default.
  This can be useful for starting, stopping, and monitoring the daemons. If you
  are running multiple sites on the same machine, it will be necessary to avoid
  collisions of these PID files by setting a site-specific directory in
  ``config.php``::

    $config['daemon']['piddir'] = __DIR__ . '/../run/';

  It is also possible to use a STOMP server instead of our kind of hacky
  home-grown DB-based queue solution. This is strongly recommended for best
  response time, especially when using XMPP.