Although RQ features the
use_connection() command for convenience, it
is deprecated, since it pollutes the global namespace. Instead, prefer explicit
connection management using the
with Connection(...): context manager, or
pass in Redis connection references to queues directly.
The use of
use_connection is deprecated.
Please don't use `use_connection` in your scripts.
Instead, use explicit connection management.
In development mode, to connect to a default, local Redis server:
from rq import use_connection use_connection()
In production, to connect to a specific Redis server:
from redis import Redis from rq import use_connection redis = Redis('my.host.org', 6789, password='secret') use_connection(redis)
Be aware of the fact that
use_connection pollutes the global namespace. It
also implies that you can only ever use a single connection.
However, the single connection pattern facilitates only those cases where you
connect to a single Redis instance, and where you affect global context (by
replacing the existing connection with the
use_connection() call). You can
only use this pattern when you are in full control of your web stack.
In any other situation, or when you want to use multiple connections, you
Connection contexts or pass connections around explicitly.
Each RQ object instance (queues, workers, jobs) has a
argument that can be passed to the constructor. Using this, you don't need to
use_connection(). Instead, you can create your queues like this:
from rq import Queue from redis import Redis conn1 = Redis('localhost', 6379) conn2 = Redis('remote.host.org', 9836) q1 = Queue('foo', connection=conn1) q2 = Queue('bar', connection=conn2)
Every job that is enqueued on a queue will know what connection it belongs to. The same goes for the workers.
This approach is very precise, but rather verbose, and therefore, tedious.
There is a better approach if you want to use multiple connections, though. Each RQ object instance, upon creation, will use the topmost Redis connection on the RQ connection stack, which is a mechanism to temporarily replace the default connection to be used.
An example will help to understand it:
from rq import Queue, Connection from redis import Redis with Connection(Redis('localhost', 6379)): q1 = Queue('foo') with Connection(Redis('remote.host.org', 9836)): q2 = Queue('bar') q3 = Queue('qux') assert q1.connection != q2.connection assert q2.connection != q3.connection assert q1.connection == q3.connection
You can think of this as if, within the
Connection context, every newly
created RQ object instance will have the
connection argument set implicitly.
Enqueueing a job with
q2 will enqueue it in the second (remote) Redis
backend, even when outside of the connection context.
If your code does not allow you to use a
with statement, for example, if you
want to use this to set up a unit test, you can use the
pop_connection() methods instead of using the context manager.
import unittest from rq import Queue from rq import push_connection, pop_connection class MyTest(unittest.TestCase): def setUp(self): push_connection(Redis()) def tearDown(self): pop_connection() def test_foo(self): """Any queues created here use local Redis.""" q = Queue() ...