Development of an internal social media platform with personalised dashboards for students
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

DESCRIPTION.rst 11KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332
  1. .. _kombu-index:
  2. ========================================
  3. kombu - Messaging library for Python
  4. ========================================
  5. :Version: 3.0.37
  6. `Kombu` is a messaging library for Python.
  7. The aim of `Kombu` is to make messaging in Python as easy as possible by
  8. providing an idiomatic high-level interface for the AMQ protocol, and also
  9. provide proven and tested solutions to common messaging problems.
  10. `AMQP`_ is the Advanced Message Queuing Protocol, an open standard protocol
  11. for message orientation, queuing, routing, reliability and security,
  12. for which the `RabbitMQ`_ messaging server is the most popular implementation.
  13. Features
  14. ========
  15. * Allows application authors to support several message server
  16. solutions by using pluggable transports.
  17. * AMQP transport using the `py-amqp`_, `librabbitmq`_, or `qpid-python`_ client libraries.
  18. * High performance AMQP transport written in C - when using `librabbitmq`_
  19. This is automatically enabled if librabbitmq is installed::
  20. $ pip install librabbitmq
  21. * Virtual transports makes it really easy to add support for non-AMQP
  22. transports. There is already built-in support for `Redis`_,
  23. `Beanstalk`_, `Amazon SQS`_, `CouchDB`_, `MongoDB`_, `ZeroMQ`_,
  24. `ZooKeeper`_, `SoftLayer MQ`_ and `Pyro`_.
  25. * You can also use the SQLAlchemy and Django ORM transports to
  26. use a database as the broker.
  27. * In-memory transport for unit testing.
  28. * Supports automatic encoding, serialization and compression of message
  29. payloads.
  30. * Consistent exception handling across transports.
  31. * The ability to ensure that an operation is performed by gracefully
  32. handling connection and channel errors.
  33. * Several annoyances with `amqplib`_ has been fixed, like supporting
  34. timeouts and the ability to wait for events on more than one channel.
  35. * Projects already using `carrot`_ can easily be ported by using
  36. a compatibility layer.
  37. For an introduction to AMQP you should read the article `Rabbits and warrens`_,
  38. and the `Wikipedia article about AMQP`_.
  39. .. _`RabbitMQ`: http://www.rabbitmq.com/
  40. .. _`AMQP`: http://amqp.org
  41. .. _`py-amqp`: http://pypi.python.org/pypi/amqp/
  42. .. _`qpid-python`: http://pypi.python.org/pypi/qpid-python/
  43. .. _`Redis`: http://code.google.com/p/redis/
  44. .. _`Amazon SQS`: http://aws.amazon.com/sqs/
  45. .. _`MongoDB`: http://www.mongodb.org/
  46. .. _`CouchDB`: http://couchdb.apache.org/
  47. .. _`ZeroMQ`: http://zeromq.org/
  48. .. _`Zookeeper`: https://zookeeper.apache.org/
  49. .. _`Beanstalk`: http://kr.github.com/beanstalkd/
  50. .. _`Rabbits and warrens`: http://blogs.digitar.com/jjww/2009/01/rabbits-and-warrens/
  51. .. _`amqplib`: http://barryp.org/software/py-amqplib/
  52. .. _`Wikipedia article about AMQP`: http://en.wikipedia.org/wiki/AMQP
  53. .. _`carrot`: http://pypi.python.org/pypi/carrot/
  54. .. _`librabbitmq`: http://pypi.python.org/pypi/librabbitmq
  55. .. _`Pyro`: http://pythonhosting.org/Pyro
  56. .. _`SoftLayer MQ`: http://www.softlayer.com/services/additional/message-queue
  57. .. _transport-comparison:
  58. Transport Comparison
  59. ====================
  60. +---------------+----------+------------+------------+---------------+
  61. | **Client** | **Type** | **Direct** | **Topic** | **Fanout** |
  62. +---------------+----------+------------+------------+---------------+
  63. | *amqp* | Native | Yes | Yes | Yes |
  64. +---------------+----------+------------+------------+---------------+
  65. | *qpid* | Native | Yes | Yes | Yes |
  66. +---------------+----------+------------+------------+---------------+
  67. | *redis* | Virtual | Yes | Yes | Yes (PUB/SUB) |
  68. +---------------+----------+------------+------------+---------------+
  69. | *mongodb* | Virtual | Yes | Yes | Yes |
  70. +---------------+----------+------------+------------+---------------+
  71. | *beanstalk* | Virtual | Yes | Yes [#f1]_ | No |
  72. +---------------+----------+------------+------------+---------------+
  73. | *SQS* | Virtual | Yes | Yes [#f1]_ | Yes [#f2]_ |
  74. +---------------+----------+------------+------------+---------------+
  75. | *couchdb* | Virtual | Yes | Yes [#f1]_ | No |
  76. +---------------+----------+------------+------------+---------------+
  77. | *zookeeper* | Virtual | Yes | Yes [#f1]_ | No |
  78. +---------------+----------+------------+------------+---------------+
  79. | *in-memory* | Virtual | Yes | Yes [#f1]_ | No |
  80. +---------------+----------+------------+------------+---------------+
  81. | *django* | Virtual | Yes | Yes [#f1]_ | No |
  82. +---------------+----------+------------+------------+---------------+
  83. | *sqlalchemy* | Virtual | Yes | Yes [#f1]_ | No |
  84. +---------------+----------+------------+------------+---------------+
  85. | *SLMQ* | Virtual | Yes | Yes [#f1]_ | No |
  86. +---------------+----------+------------+------------+---------------+
  87. .. [#f1] Declarations only kept in memory, so exchanges/queues
  88. must be declared by all clients that needs them.
  89. .. [#f2] Fanout supported via storing routing tables in SimpleDB.
  90. Disabled by default, but can be enabled by using the
  91. ``supports_fanout`` transport option.
  92. Documentation
  93. -------------
  94. Kombu is using Sphinx, and the latest documentation can be found here:
  95. https://kombu.readthedocs.io/
  96. Quick overview
  97. --------------
  98. ::
  99. from kombu import Connection, Exchange, Queue
  100. media_exchange = Exchange('media', 'direct', durable=True)
  101. video_queue = Queue('video', exchange=media_exchange, routing_key='video')
  102. def process_media(body, message):
  103. print body
  104. message.ack()
  105. # connections
  106. with Connection('amqp://guest:guest@localhost//') as conn:
  107. # produce
  108. producer = conn.Producer(serializer='json')
  109. producer.publish({'name': '/tmp/lolcat1.avi', 'size': 1301013},
  110. exchange=media_exchange, routing_key='video',
  111. declare=[video_queue])
  112. # the declare above, makes sure the video queue is declared
  113. # so that the messages can be delivered.
  114. # It's a best practice in Kombu to have both publishers and
  115. # consumers declare the queue. You can also declare the
  116. # queue manually using:
  117. # video_queue(conn).declare()
  118. # consume
  119. with conn.Consumer(video_queue, callbacks=[process_media]) as consumer:
  120. # Process messages and handle events on all channels
  121. while True:
  122. conn.drain_events()
  123. # Consume from several queues on the same channel:
  124. video_queue = Queue('video', exchange=media_exchange, key='video')
  125. image_queue = Queue('image', exchange=media_exchange, key='image')
  126. with connection.Consumer([video_queue, image_queue],
  127. callbacks=[process_media]) as consumer:
  128. while True:
  129. connection.drain_events()
  130. Or handle channels manually::
  131. with connection.channel() as channel:
  132. producer = Producer(channel, ...)
  133. consumer = Producer(channel)
  134. All objects can be used outside of with statements too,
  135. just remember to close the objects after use::
  136. from kombu import Connection, Consumer, Producer
  137. connection = Connection()
  138. # ...
  139. connection.release()
  140. consumer = Consumer(channel_or_connection, ...)
  141. consumer.register_callback(my_callback)
  142. consumer.consume()
  143. # ....
  144. consumer.cancel()
  145. `Exchange` and `Queue` are simply declarations that can be pickled
  146. and used in configuration files etc.
  147. They also support operations, but to do so they need to be bound
  148. to a channel.
  149. Binding exchanges and queues to a connection will make it use
  150. that connections default channel.
  151. ::
  152. >>> exchange = Exchange('tasks', 'direct')
  153. >>> connection = Connection()
  154. >>> bound_exchange = exchange(connection)
  155. >>> bound_exchange.delete()
  156. # the original exchange is not affected, and stays unbound.
  157. >>> exchange.delete()
  158. raise NotBoundError: Can't call delete on Exchange not bound to
  159. a channel.
  160. Installation
  161. ============
  162. You can install `Kombu` either via the Python Package Index (PyPI)
  163. or from source.
  164. To install using `pip`,::
  165. $ pip install kombu
  166. To install using `easy_install`,::
  167. $ easy_install kombu
  168. If you have downloaded a source tarball you can install it
  169. by doing the following,::
  170. $ python setup.py build
  171. # python setup.py install # as root
  172. Terminology
  173. ===========
  174. There are some concepts you should be familiar with before starting:
  175. * Producers
  176. Producers sends messages to an exchange.
  177. * Exchanges
  178. Messages are sent to exchanges. Exchanges are named and can be
  179. configured to use one of several routing algorithms. The exchange
  180. routes the messages to consumers by matching the routing key in the
  181. message with the routing key the consumer provides when binding to
  182. the exchange.
  183. * Consumers
  184. Consumers declares a queue, binds it to a exchange and receives
  185. messages from it.
  186. * Queues
  187. Queues receive messages sent to exchanges. The queues are declared
  188. by consumers.
  189. * Routing keys
  190. Every message has a routing key. The interpretation of the routing
  191. key depends on the exchange type. There are four default exchange
  192. types defined by the AMQP standard, and vendors can define custom
  193. types (so see your vendors manual for details).
  194. These are the default exchange types defined by AMQP/0.8:
  195. * Direct exchange
  196. Matches if the routing key property of the message and
  197. the `routing_key` attribute of the consumer are identical.
  198. * Fan-out exchange
  199. Always matches, even if the binding does not have a routing
  200. key.
  201. * Topic exchange
  202. Matches the routing key property of the message by a primitive
  203. pattern matching scheme. The message routing key then consists
  204. of words separated by dots (`"."`, like domain names), and
  205. two special characters are available; star (`"*"`) and hash
  206. (`"#"`). The star matches any word, and the hash matches
  207. zero or more words. For example `"*.stock.#"` matches the
  208. routing keys `"usd.stock"` and `"eur.stock.db"` but not
  209. `"stock.nasdaq"`.
  210. Getting Help
  211. ============
  212. Mailing list
  213. ------------
  214. Join the `carrot-users`_ mailing list.
  215. .. _`carrot-users`: http://groups.google.com/group/carrot-users/
  216. Bug tracker
  217. ===========
  218. If you have any suggestions, bug reports or annoyances please report them
  219. to our issue tracker at http://github.com/celery/kombu/issues/
  220. Contributing
  221. ============
  222. Development of `Kombu` happens at Github: http://github.com/celery/kombu
  223. You are highly encouraged to participate in the development. If you don't
  224. like Github (for some reason) you're welcome to send regular patches.
  225. License
  226. =======
  227. This software is licensed under the `New BSD License`. See the `LICENSE`
  228. file in the top distribution directory for the full license text.
  229. .. image:: https://d2weczhvl823v0.cloudfront.net/celery/kombu/trend.png
  230. :alt: Bitdeli badge
  231. :target: https://bitdeli.com/free