1. 24 Jan, 2010 7 commits
  2. 23 Jan, 2010 2 commits
  3. 22 Jan, 2010 29 commits
    • Craig Andrews's avatar
      845f051c
    • Brion Vibber's avatar
      Merge branch 'testing' into 0.9.x · b157fcbb
      Brion Vibber authored
      b157fcbb
    • Brion Vibber's avatar
      a66181c9
    • Brion Vibber's avatar
      Consolidate PuSH publishing ping into a single POST for all feeds, and fix... · 71b3b9ee
      Brion Vibber authored
      Consolidate PuSH publishing ping into a single POST for all feeds, and fix server response (if any on failure) to go to log instead of stdout.
      71b3b9ee
    • Brion Vibber's avatar
      6d055ce0
    • Brion Vibber's avatar
    • Brion Vibber's avatar
      Fix for stuck queue messages: wrap processing in stomp transactions so our... · 6e4cad71
      Brion Vibber authored
      Fix for stuck queue messages: wrap processing in stomp transactions so our lack of an ACK if PHP dies actually triggers redelivery.
      
      Previously, messages once delivered would just get stuck in the queue seemingly forever if they never got ACKed.
      Note this could lead to partial duplication, for instance if the OMB or Twitter queue handlers die after 1/2 of the outgoing sends.
      
      Recommendations:
      * catch exceptions more aggressively within queue handlers (so only PHP fatal errors are likely to kill in the middle)
      * for processing that involves sending to multiple clients, consider a second queue similar to the XMPP output, eg for OMB:
       - first queue gets delivery list and builds message data, enqueueing it for each target address
       - second queue can handle each individual outgoing message (and attempt redelivery etc separately)
      
      This would also protect better against a recurring error preventing delivery in the second part, and could spread out any slow sends over multiple threads.
      6e4cad71
    • Brion Vibber's avatar
      XMPP queued output & initial retooling of DB queue manager to support non-Notice objects. · c7507e7e
      Brion Vibber authored
      Queue handlers for XMPP individual & firehose output now send their XML stanzas
      to another output queue instead of connecting directly to the chat server. This
      lets us have as many general processing threads as we need, while all actual
      XMPP input and output go through a single daemon with a single connection open.
      
      This avoids problems with multiple connected resources:
      * multiple windows shown in some chat clients (psi, gajim, kopete)
      * extra load on server
      * incoming message delivery forwarding issues
      
      Database changes:
      * queue_item drops 'notice_id' in favor of a 'frame' blob.
        This is based on Craig Andrews' work branch to generalize queues to take any
        object, but conservatively leaving out the serialization for now.
        Table updater (preserves any existing queued items) in db/rc3to09.sql
      
      Code changes to watch out for:
      * Queue handlers should now define a handle() method instead of handle_notice()
      * QueueDaemon and XmppDaemon now share common i/o (IoMaster) and respawning
        thread management (RespawningDaemon) infrastructure.
      * The polling XmppConfirmManager has been dropped, as the message is queued
        directly when saving IM settings.
      * Enable $config['queue']['debug_memory'] to output current memory usage at
        each run through the event loop to watch for memory leaks
      
      To do:
      * Adapt XMPP i/o to component connection mode for multi-site support.
      * XMPP input can also be broken out to a queue, which would allow the actual
        notice save etc to be handled by general queue threads.
      * Make sure there are no problems with simply pushing serialized Notice objects
        to queues.
      * Find a way to improve interactive performance of the database-backed queue
        handler; polling is pretty painful to XMPP.
      * Possibly redo the way QueueHandlers are injected into a QueueManager. The
        grouping used to split out the XMPP output queue is a bit awkward.
      
      Conflicts:
      
      	scripts/xmppdaemon.php
      c7507e7e
    • Brion Vibber's avatar
      Fix for stuck queue messages: wrap processing in stomp transactions so our... · 99866a45
      Brion Vibber authored
      Fix for stuck queue messages: wrap processing in stomp transactions so our lack of an ACK if PHP dies actually triggers redelivery.
      
      Previously, messages once delivered would just get stuck in the queue seemingly forever if they never got ACKed.
      Note this could lead to partial duplication, for instance if the OMB or Twitter queue handlers die after 1/2 of the outgoing sends.
      
      Recommendations:
      * catch exceptions more aggressively within queue handlers (so only PHP fatal errors are likely to kill in the middle)
      * for processing that involves sending to multiple clients, consider a second queue similar to the XMPP output, eg for OMB:
       - first queue gets delivery list and builds message data, enqueueing it for each target address
       - second queue can handle each individual outgoing message (and attempt redelivery etc separately)
      
      This would also protect better against a recurring error preventing delivery in the second part, and could spread out any slow sends over multiple threads.
      99866a45
    • Sarven Capadisli's avatar
      8bf2a904
    • Evan Prodromou's avatar
    • Evan Prodromou's avatar
      Merge branch 'testing' into 0.9.x · c8bc598c
      Evan Prodromou authored
      c8bc598c
    • Evan Prodromou's avatar
      Merge branch 'master' into 0.9.x · e666433e
      Evan Prodromou authored
      e666433e
    • Evan Prodromou's avatar
    • Evan Prodromou's avatar
      104d3007
    • Evan Prodromou's avatar
      restructure doc.php for new use · 9f815c96
      Evan Prodromou authored
      9f815c96
    • Evan Prodromou's avatar
      action/doc.php is PHPCS clean · df9b7807
      Evan Prodromou authored
      df9b7807
    • Sarven Capadisli's avatar
    • Sarven Capadisli's avatar
      c9aafe2d
    • Eric Helgeson's avatar
      Fix unqueuemanager to work with new Queue layout pushed in 0e852def · b7940ef3
      Eric Helgeson authored
      "* Queue handlers should now define a handle() method instead of handle_notice()"
      
      And Queue managers should call handle() :)
      b7940ef3
    • Evan Prodromou's avatar
    • Craig Andrews's avatar
      7be5e7e5
    • Craig Andrews's avatar
    • Brion Vibber's avatar
      XMPP queued output & initial retooling of DB queue manager to support non-Notice objects. · 0e852def
      Brion Vibber authored
      Queue handlers for XMPP individual & firehose output now send their XML stanzas
      to another output queue instead of connecting directly to the chat server. This
      lets us have as many general processing threads as we need, while all actual
      XMPP input and output go through a single daemon with a single connection open.
      
      This avoids problems with multiple connected resources:
      * multiple windows shown in some chat clients (psi, gajim, kopete)
      * extra load on server
      * incoming message delivery forwarding issues
      
      Database changes:
      * queue_item drops 'notice_id' in favor of a 'frame' blob.
        This is based on Craig Andrews' work branch to generalize queues to take any
        object, but conservatively leaving out the serialization for now.
        Table updater (preserves any existing queued items) in db/rc3to09.sql
      
      Code changes to watch out for:
      * Queue handlers should now define a handle() method instead of handle_notice()
      * QueueDaemon and XmppDaemon now share common i/o (IoMaster) and respawning
        thread management (RespawningDaemon) infrastructure.
      * The polling XmppConfirmManager has been dropped, as the message is queued
        directly when saving IM settings.
      * Enable $config['queue']['debug_memory'] to output current memory usage at
        each run through the event loop to watch for memory leaks
      
      To do:
      * Adapt XMPP i/o to component connection mode for multi-site support.
      * XMPP input can also be broken out to a queue, which would allow the actual
        notice save etc to be handled by general queue threads.
      * Make sure there are no problems with simply pushing serialized Notice objects
        to queues.
      * Find a way to improve interactive performance of the database-backed queue
        handler; polling is pretty painful to XMPP.
      * Possibly redo the way QueueHandlers are injected into a QueueManager. The
        grouping used to split out the XMPP output queue is a bit awkward.
      0e852def
    • Brion Vibber's avatar
      XMPP queued output & initial retooling of DB queue manager to support non-Notice objects. · 26fdf0c9
      Brion Vibber authored
      Queue handlers for XMPP individual & firehose output now send their XML stanzas
      to another output queue instead of connecting directly to the chat server. This
      lets us have as many general processing threads as we need, while all actual
      XMPP input and output go through a single daemon with a single connection open.
      
      This avoids problems with multiple connected resources:
      * multiple windows shown in some chat clients (psi, gajim, kopete)
      * extra load on server
      * incoming message delivery forwarding issues
      
      Database changes:
      * queue_item drops 'notice_id' in favor of a 'frame' blob.
        This is based on Craig Andrews' work branch to generalize queues to take any
        object, but conservatively leaving out the serialization for now.
        Table updater (preserves any existing queued items) in db/rc3to09.sql
      
      Code changes to watch out for:
      * Queue handlers should now define a handle() method instead of handle_notice()
      * QueueDaemon and XmppDaemon now share common i/o (IoMaster) and respawning
        thread management (RespawningDaemon) infrastructure.
      * The polling XmppConfirmManager has been dropped, as the message is queued
        directly when saving IM settings.
      * Enable $config['queue']['debug_memory'] to output current memory usage at
        each run through the event loop to watch for memory leaks
      
      To do:
      * Adapt XMPP i/o to component connection mode for multi-site support.
      * XMPP input can also be broken out to a queue, which would allow the actual
        notice save etc to be handled by general queue threads.
      * Make sure there are no problems with simply pushing serialized Notice objects
        to queues.
      * Find a way to improve interactive performance of the database-backed queue
        handler; polling is pretty painful to XMPP.
      * Possibly redo the way QueueHandlers are injected into a QueueManager. The
        grouping used to split out the XMPP output queue is a bit awkward.
      26fdf0c9
    • Brion Vibber's avatar
      Merge branch 'testing' into 0.9.x · d412df18
      Brion Vibber authored
      d412df18
    • Brion Vibber's avatar
    • Brion Vibber's avatar
      Merge commit 'origin/testing' into 0.9.x · c9c7bb32
      Brion Vibber authored
      c9c7bb32
    • Brion Vibber's avatar
      Quick hack to avoid breaking with geonames off when there's some old cookie... · 4b0f953c
      Brion Vibber authored
      Quick hack to avoid breaking with geonames off when there's some old cookie state. This code's a little rough and tumble; any breakage halts JS execution and leaves the spinner going and no message submitted.
      4b0f953c
  4. 21 Jan, 2010 2 commits