Skip to content

Commit 6e2d161

Browse files
yashhyashh
committed
news docs with job and worker class docs.
1 parent 0975d54 commit 6e2d161

File tree

12 files changed

+233
-45
lines changed

12 files changed

+233
-45
lines changed

‎_sources/class.txt‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,3 +4,17 @@ ResQ Classes
44
==========================================
55

66
.. autoclass:: pyres.ResQ
7+
:members:
8+
9+
10+
Job Classes
11+
=================
12+
13+
.. autoclass:: pyres.job.Job
14+
:members:
15+
16+
Worker Classes
17+
=================
18+
19+
.. autoclass:: pyres.worker.Worker
20+
:members:

‎_sources/example.txt‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,10 @@
44

55
Let's take a real wold example of a blog where comments need to be checked for
66
spam check. When the comment is saved in the database, we create a job in the
7-
queue with comment data. Let's take a django model in this case.::
7+
queue with comment data. Let's take a django model in this case.
8+
9+
.. code-block:: python
10+
:linenos:
811

912
class Comment(models.Model):
1013
name = Model.CharField()

‎_sources/install.txt‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,10 +3,10 @@ Installation
33

44
Requirements:
55
--------------
6-
simplejson>=2.0.9
7-
itty>=0.6.2
8-
redis>=0.6.0
9-
pystache>=0.1.0
6+
simplejson>=2.0.9
7+
itty>=0.6.2
8+
redis>=0.6.0
9+
pystache>=0.1.0
1010

1111
Make sure you install these requirements before proceeding.
1212

‎_sources/intro.txt‎

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -6,11 +6,7 @@ message queue. Read_ the blog post from github about how they use resque in
66
production.
77

88
:synopsis: Any job which takes a little while to run can be put on a message
9-
queue. Read our :doc:`Example </example>` implementation of how a PyRes can be used to spam
10-
check comments.
11-
12-
13-
9+
queue. Read our :doc:`Example </example>` implementation of how a PyRes can be used to spam check comments.
1410

1511

1612
.. _resque: http://github.com/defunkt/resque#readme

‎_theme/ADCTheme/README.html‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,9 @@ <h3>Navigation</h3>
2828
<li class="right" style="margin-right: 10px">
2929
<a href="../../genindex.html" title="General Index"
3030
accesskey="I">index</a></li>
31+
<li class="right" >
32+
<a href="../../modindex.html" title="Global Module Index"
33+
accesskey="M">modules</a> |</li>
3134
<li><a href="../../index.html">PyRes v0.4.1 documentation</a> &raquo;</li>
3235
</ul>
3336
</div>
@@ -119,6 +122,9 @@ <h3>Navigation</h3>
119122
<li class="right" style="margin-right: 10px">
120123
<a href="../../genindex.html" title="General Index"
121124
>index</a></li>
125+
<li class="right" >
126+
<a href="../../modindex.html" title="Global Module Index"
127+
>modules</a> |</li>
122128
<li><a href="../../index.html">PyRes v0.4.1 documentation</a> &raquo;</li>
123129
</ul>
124130
</div>

‎class.html‎

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,80 @@ <h3>Navigation</h3>
6262
<span class="go"> # Assuming redis is running on default port with no password</span>
6363
</pre></div>
6464
</div>
65+
<p><strong>r</strong> is a resque object on which we can enqueue tasks.:</p>
66+
<div class="highlight-python"><div class="highlight"><pre><span class="go">&gt;&gt;&gt;&gt; r.enqueue(SomeClass, args)</span>
67+
</pre></div>
68+
</div>
69+
<p>SomeClass can be any python class with <em>perform</em> method and a <em>queue</em>
70+
attribute on it.</p>
71+
<dl class="method">
72+
<dt id="pyres.ResQ.enqueue">
73+
<tt class="descname">enqueue</tt><big>(</big><em>klass</em>, <em>*args</em><big>)</big><a class="headerlink" href="#pyres.ResQ.enqueue" title="Permalink to this definition"></a></dt>
74+
<dd>Enqueue a job into a specific queue. Make sure the class you are passing
75+
has <strong>queue</strong> attribute and a <strong>perform</strong> method on it.</dd></dl>
76+
77+
<dl class="method">
78+
<dt id="pyres.ResQ.info">
79+
<tt class="descname">info</tt><big>(</big><big>)</big><a class="headerlink" href="#pyres.ResQ.info" title="Permalink to this definition"></a></dt>
80+
<dd>Returns a dictionary of the current status of the pending jobs,
81+
processed, no. of queues, no. of workers, no. of failed jobs.</dd></dl>
82+
83+
</dd></dl>
84+
85+
</div>
86+
<div class="section" id="job-classes">
87+
<h1>Job Classes<a class="headerlink" href="#job-classes" title="Permalink to this headline"></a></h1>
88+
<dl class="class">
89+
<dt id="pyres.job.Job">
90+
<em class="property">
91+
class </em><tt class="descclassname">pyres.job.</tt><tt class="descname">Job</tt><big>(</big><em>queue</em>, <em>payload</em>, <em>resq</em>, <em>worker=None</em><big>)</big><a class="headerlink" href="#pyres.job.Job" title="Permalink to this definition"></a></dt>
92+
<dd><p>Every job on the ResQ is a <em>Job</em> object which has queue and payload(all the
93+
args data and when its created etc).</p>
94+
<dl class="method">
95+
<dt id="pyres.job.Job.perform">
96+
<tt class="descname">perform</tt><big>(</big><big>)</big><a class="headerlink" href="#pyres.job.Job.perform" title="Permalink to this definition"></a></dt>
97+
<dd>This method converts payload into args and calls the <strong>perform</strong> method
98+
on the payload class.</dd></dl>
99+
100+
<dl class="classmethod">
101+
<dt id="pyres.job.Job.reserve">
102+
<em class="property">
103+
classmethod </em><tt class="descname">reserve</tt><big>(</big><em>queue</em>, <em>res</em>, <em>worker=None</em><big>)</big><a class="headerlink" href="#pyres.job.Job.reserve" title="Permalink to this definition"></a></dt>
104+
<dd>Reserve a job on the queue. In simple marking this job so that other worker
105+
will not pick it up</dd></dl>
106+
107+
</dd></dl>
108+
109+
</div>
110+
<div class="section" id="worker-classes">
111+
<h1>Worker Classes<a class="headerlink" href="#worker-classes" title="Permalink to this headline"></a></h1>
112+
<dl class="class">
113+
<dt id="pyres.worker.Worker">
114+
<em class="property">
115+
class </em><tt class="descclassname">pyres.worker.</tt><tt class="descname">Worker</tt><big>(</big><em>queues=</em><span class="optional">[</span><span class="optional">]</span>, <em>server='localhost:6379'</em>, <em>password=None</em><big>)</big><a class="headerlink" href="#pyres.worker.Worker" title="Permalink to this definition"></a></dt>
116+
<dd><p>Defines a worker. The <em>pyres_worker</em> script instantiates this Worker class and
117+
pass a comma seperate list of queues to listen on.:</p>
118+
<div class="highlight-python"><div class="highlight"><pre><span class="gp">&gt;&gt;&gt; </span><span class="kn">from</span> <span class="nn">pyres.worker</span> <span class="kn">import</span> <span class="n">Worker</span>
119+
<span class="gp">&gt;&gt;&gt; </span><span class="n">Worker</span><span class="o">.</span><span class="n">run</span><span class="p">([</span><span class="n">queue1</span><span class="p">,</span> <span class="n">queue2</span><span class="p">],</span> <span class="n">server</span><span class="o">=</span><span class="s">&quot;localhost:6379&quot;</span><span class="p">)</span>
120+
</pre></div>
121+
</div>
122+
<dl class="method">
123+
<dt id="pyres.worker.Worker.validate_queues">
124+
<tt class="descname">validate_queues</tt><big>(</big><big>)</big><a class="headerlink" href="#pyres.worker.Worker.validate_queues" title="Permalink to this definition"></a></dt>
125+
<dd>Checks if a worker is given atleast one queue to work on.</dd></dl>
126+
127+
<dl class="method">
128+
<dt id="pyres.worker.Worker.work">
129+
<tt class="descname">work</tt><big>(</big><em>interval=5</em><big>)</big><a class="headerlink" href="#pyres.worker.Worker.work" title="Permalink to this definition"></a></dt>
130+
<dd><p>Invoked by run() method. work() listens on a list of queues and sleeps
131+
for <em>interval</em> time.</p>
132+
<p>default &#8211; 5 secs</p>
133+
<p>Whenever a worker finds a job on the queue it first calls <tt class="docutils literal"><span class="pre">reserve</span></tt> on
134+
that job to make sure other worker won&#8217;t run it, then <em>Forks</em> itself to
135+
work on that job.</p>
136+
<p>Finally process() method actually processes the job.</p>
137+
</dd></dl>
138+
65139
</dd></dl>
66140

67141
</div>
@@ -72,6 +146,13 @@ <h3>Navigation</h3>
72146
</div>
73147
<div class="sphinxsidebar">
74148
<div class="sphinxsidebarwrapper">
149+
<h3><a href="index.html">Table Of Contents</a></h3>
150+
<ul>
151+
<li><a class="reference external" href="">ResQ Classes</a></li>
152+
<li><a class="reference external" href="#job-classes">Job Classes</a></li>
153+
<li><a class="reference external" href="#worker-classes">Worker Classes</a></li>
154+
</ul>
155+
75156
<h4>Previous topic</h4>
76157
<p class="topless"><a href="example.html"
77158
title="previous chapter">EXAMPLE</a></p>

‎example.html‎

Lines changed: 61 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,8 @@
2020
<script type="text/javascript" src="_static/jquery.js"></script>
2121
<script type="text/javascript" src="_static/doctools.js"></script>
2222
<link rel="top" title="PyRes v0.4.1 documentation" href="index.html" />
23-
<link rel="prev" title="Introduction" href="intro.html" />
23+
<link rel="next" title="ResQ Classes" href="class.html" />
24+
<link rel="prev" title="Installation" href="install.html" />
2425
</head>
2526
<body>
2627
<div class="related">
@@ -30,7 +31,13 @@ <h3>Navigation</h3>
3031
<a href="genindex.html" title="General Index"
3132
accesskey="I">index</a></li>
3233
<li class="right" >
33-
<a href="intro.html" title="Introduction"
34+
<a href="modindex.html" title="Global Module Index"
35+
accesskey="M">modules</a> |</li>
36+
<li class="right" >
37+
<a href="class.html" title="ResQ Classes"
38+
accesskey="N">next</a> |</li>
39+
<li class="right" >
40+
<a href="install.html" title="Installation"
3441
accesskey="P">previous</a> |</li>
3542
<li><a href="index.html">PyRes v0.4.1 documentation</a> &raquo;</li>
3643
</ul>
@@ -45,29 +52,48 @@ <h3>Navigation</h3>
4552
<h1>EXAMPLE<a class="headerlink" href="#example" title="Permalink to this headline"></a></h1>
4653
<p>Let&#8217;s take a real wold example of a blog where comments need to be checked for
4754
spam check. When the comment is saved in the database, we create a job in the
48-
queue with comment data. Let&#8217;s take a django model in this case.:</p>
49-
<div class="highlight-python"><div class="highlight"><pre><span class="k">class</span> <span class="nc">Comment</span><span class="p">(</span><span class="n">models</span><span class="o">.</span><span class="n">Model</span><span class="p">):</span>
50-
<span class="n">name</span> <span class="o">=</span> <span class="n">Model</span><span class="o">.</span><span class="n">CharField</span><span class="p">()</span>
51-
<span class="n">email</span> <span class="o">=</span> <span class="n">Model</span><span class="o">.</span><span class="n">EmailField</span><span class="p">()</span>
52-
<span class="n">body</span> <span class="o">=</span> <span class="n">Model</span><span class="o">.</span><span class="n">TextField</span><span class="p">()</span>
53-
<span class="n">spam</span> <span class="o">=</span> <span class="n">Model</span><span class="o">.</span><span class="n">BooleanField</span><span class="p">()</span>
54-
<span class="n">queue</span> <span class="o">=</span> <span class="s">&quot;Spam&quot;</span>
55+
queue with comment data. Let&#8217;s take a django model in this case.</p>
56+
<div class="highlight-python"><table class="highlighttable"><tr><td class="linenos"><pre> 1
57+
2
58+
3
59+
4
60+
5
61+
6
62+
7
63+
8
64+
9
65+
10
66+
11
67+
12
68+
13
69+
14
70+
15
71+
16
72+
17
73+
18
74+
19
75+
20</pre></td><td class="code"><div class="highlight"><pre> <span class="k">class</span> <span class="nc">Comment</span><span class="p">(</span><span class="n">models</span><span class="o">.</span><span class="n">Model</span><span class="p">):</span>
76+
<span class="n">name</span> <span class="o">=</span> <span class="n">Model</span><span class="o">.</span><span class="n">CharField</span><span class="p">()</span>
77+
<span class="n">email</span> <span class="o">=</span> <span class="n">Model</span><span class="o">.</span><span class="n">EmailField</span><span class="p">()</span>
78+
<span class="n">body</span> <span class="o">=</span> <span class="n">Model</span><span class="o">.</span><span class="n">TextField</span><span class="p">()</span>
79+
<span class="n">spam</span> <span class="o">=</span> <span class="n">Model</span><span class="o">.</span><span class="n">BooleanField</span><span class="p">()</span>
80+
<span class="n">queue</span> <span class="o">=</span> <span class="s">&quot;Spam&quot;</span>
5581

56-
<span class="nd">@staticmethod</span>
57-
<span class="k">def</span> <span class="nf">perform</span><span class="p">(</span><span class="n">comment_id</span><span class="p">):</span>
58-
<span class="n">comment</span> <span class="o">=</span> <span class="n">Comment</span><span class="o">.</span><span class="n">objects</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">pk</span><span class="o">=</span><span class="n">comment_id</span><span class="p">)</span>
59-
<span class="n">params</span> <span class="o">=</span> <span class="p">{</span><span class="s">&quot;comment_author_email&quot;</span><span class="p">:</span> <span class="n">comment</span><span class="o">.</span><span class="n">user</span><span class="o">.</span><span class="n">email</span><span class="p">,</span>
60-
<span class="s">&quot;comment_content&quot;</span><span class="p">:</span> <span class="n">comment</span><span class="o">.</span><span class="n">body</span><span class="p">,</span>
61-
<span class="s">&quot;comment_author_name&quot;</span><span class="p">:</span> <span class="n">comment</span><span class="o">.</span><span class="n">user</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
62-
<span class="s">&quot;request_ip&quot;</span><span class="p">:</span> <span class="n">comment</span><span class="o">.</span><span class="n">author_ip</span><span class="p">}</span>
63-
<span class="n">x</span> <span class="o">=</span> <span class="n">urllib</span><span class="o">.</span><span class="n">urlopen</span><span class="p">(</span><span class="s">&quot;http://apikey.rest.akismet.com/1.1/comment-check&quot;</span><span class="p">,</span> <span class="n">params</span><span class="p">)</span>
64-
<span class="k">if</span> <span class="n">x</span> <span class="o">==</span> <span class="s">&quot;true&quot;</span><span class="p">:</span>
65-
<span class="n">comment</span><span class="o">.</span><span class="n">spam</span> <span class="o">=</span> <span class="bp">True</span>
66-
<span class="k">else</span><span class="p">:</span>
67-
<span class="n">comment</span><span class="o">.</span><span class="n">spam</span> <span class="o">=</span> <span class="bp">False</span>
68-
<span class="n">comment</span><span class="o">.</span><span class="n">save</span><span class="p">()</span>
82+
<span class="nd">@staticmethod</span>
83+
<span class="k">def</span> <span class="nf">perform</span><span class="p">(</span><span class="n">comment_id</span><span class="p">):</span>
84+
<span class="n">comment</span> <span class="o">=</span> <span class="n">Comment</span><span class="o">.</span><span class="n">objects</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">pk</span><span class="o">=</span><span class="n">comment_id</span><span class="p">)</span>
85+
<span class="n">params</span> <span class="o">=</span> <span class="p">{</span><span class="s">&quot;comment_author_email&quot;</span><span class="p">:</span> <span class="n">comment</span><span class="o">.</span><span class="n">user</span><span class="o">.</span><span class="n">email</span><span class="p">,</span>
86+
<span class="s">&quot;comment_content&quot;</span><span class="p">:</span> <span class="n">comment</span><span class="o">.</span><span class="n">body</span><span class="p">,</span>
87+
<span class="s">&quot;comment_author_name&quot;</span><span class="p">:</span> <span class="n">comment</span><span class="o">.</span><span class="n">user</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
88+
<span class="s">&quot;request_ip&quot;</span><span class="p">:</span> <span class="n">comment</span><span class="o">.</span><span class="n">author_ip</span><span class="p">}</span>
89+
<span class="n">x</span> <span class="o">=</span> <span class="n">urllib</span><span class="o">.</span><span class="n">urlopen</span><span class="p">(</span><span class="s">&quot;http://apikey.rest.akismet.com/1.1/comment-check&quot;</span><span class="p">,</span> <span class="n">params</span><span class="p">)</span>
90+
<span class="k">if</span> <span class="n">x</span> <span class="o">==</span> <span class="s">&quot;true&quot;</span><span class="p">:</span>
91+
<span class="n">comment</span><span class="o">.</span><span class="n">spam</span> <span class="o">=</span> <span class="bp">True</span>
92+
<span class="k">else</span><span class="p">:</span>
93+
<span class="n">comment</span><span class="o">.</span><span class="n">spam</span> <span class="o">=</span> <span class="bp">False</span>
94+
<span class="n">comment</span><span class="o">.</span><span class="n">save</span><span class="p">()</span>
6995
</pre></div>
70-
</div>
96+
</td></tr></table></div>
7197
<p>You can convert your existing class to be compatible with Pyres. All you need
7298
to do is add a &#64;queue&#64; variable and define a &#64;perform&#64; method on the class.</p>
7399
<p>To insert a job into the queue you need to do something like this::</p>
@@ -90,8 +116,11 @@ <h1>EXAMPLE<a class="headerlink" href="#example" title="Permalink to this headli
90116
<div class="sphinxsidebar">
91117
<div class="sphinxsidebarwrapper">
92118
<h4>Previous topic</h4>
93-
<p class="topless"><a href="intro.html"
94-
title="previous chapter">Introduction</a></p>
119+
<p class="topless"><a href="install.html"
120+
title="previous chapter">Installation</a></p>
121+
<h4>Next topic</h4>
122+
<p class="topless"><a href="class.html"
123+
title="next chapter">ResQ Classes</a></p>
95124
<h3>This Page</h3>
96125
<ul class="this-page-menu">
97126
<li><a href="_sources/example.txt"
@@ -121,7 +150,13 @@ <h3>Navigation</h3>
121150
<a href="genindex.html" title="General Index"
122151
>index</a></li>
123152
<li class="right" >
124-
<a href="intro.html" title="Introduction"
153+
<a href="modindex.html" title="Global Module Index"
154+
>modules</a> |</li>
155+
<li class="right" >
156+
<a href="class.html" title="ResQ Classes"
157+
>next</a> |</li>
158+
<li class="right" >
159+
<a href="install.html" title="Installation"
125160
>previous</a> |</li>
126161
<li><a href="index.html">PyRes v0.4.1 documentation</a> &raquo;</li>
127162
</ul>

0 commit comments

Comments
 (0)