In this blog post, we explore Job Queues Queue_Job in Odoo 18 Tutorial with a focus on mastering asynchronous job queues. In our tutorial, we cover installation, configuration, and usage of the Job Queue module. We include key code examples and practical tips that demonstrate how to postpone method calls, delegate work to background jobs, and build robust delayable workflows. Throughout this post, you will notice key phrases—such as, queue_job, Job Queue, and Odoo 18—integrated within the content to ensure a comprehensive learning experience.
Introduction to Asynchronous Job Queues
When you work on enterprise applications with Odoo, you often need to execute tasks in the background without affecting the user experience. Therefore, integrating a Job Queue helps postpone heavy method calls and execute them asynchronously. In this guide, we illustrate how the queue_job module allows you to delegate time-consuming work to a dedicated Jobrunner. Consequently, jobs run in their own transaction and use PostgreSQL’s efficiency with the NOTIFY mechanism.
Transitioning smoothly into our discussion, we explain that the queue_job addon provides several features like job priorities, retries, and channels that segregate heavy tasks from lighter ones. Additionally, you will learn to install and configure your queue_job module and integrate it into your own custom models in Odoo 18.
Understanding the Job Queue Module in Odoo 18
What Is a Job Queue?
A job queue organizes tasks that will run asynchronously. By using with_delay(), you postpone method calls so that the work executes later. This design not only improves performance by executing background processes in parallel but also prevents the user interface from slowing down. For example, our module captures the method and its arguments and stores them as a delayed job.
Consider the following example:
from odoo import models, api, _
import logging
_logger = logging.getLogger(__name__)
class MyModel(models.Model):
_name = 'my.model'
def my_method(self, a, k=None):
_logger.info('Executed with a: %s and k: %s', a, k)
In this snippet, when you call a button method in another model, you can enqueue the call to my_method using its delay mechanism:
class MyOtherModel(models.Model):
_name = 'my.other.model'
def button_do_stuff(self):
self.env['my.model'].with_delay().my_method('a', k=2)
This code enqueues the job by capturing the method and its arguments. Subsequently, the Jobrunner picks it up from the queue_job system and executes it asynchronously.
Key Features and Benefits
Transitioning to our next section, the following benefits shape the value of the module:
- Asynchronous Execution: Jobs run in the background and do not block user interactions.
- Retries and Priorities: You control the number of job retries and their execution priority.
- Channel Management: Channels segregate jobs, ensuring that heavyweight tasks do not slow down lighter ones.
- Efficient Jobrunner: Thanks to integration with PostgreSQL’s NOTIFY, job processing remains highly efficient.
For more details on asynchronous job queues, please check out the official Odoo documentation.
Installation and Configuration of Queue_Job Module
Installing Queue_Job
To install the queue_job module, first ensure that you have the requests library installed. Then, add the module to your Odoo instance by specifying it in your configuration file. Transitioning smoothly into the installation steps, follow the instructions below.
- Download the Module:
Retrieve the queue_job module from the Odoo app store or GitHub repository. - Update your Odoo Configuration File:
Edit your configuration file (e.g.,odoo.conf) to include the module and specify the channels configuration. For instance:[options] workers = 6 server_wide_modules = web,queue_job
[queue_job]
channels = root:2
This configuration instructs Odoo to load the queue_job module and use two channels under the root.
Start Odoo:
Run Odoo with multiple workers to ensure that jobs are processed asynchronously. Use the following command:
./odoo-bin --load=web,queue_job --workers=6
With this step, you guarantee that queue_job reuses normal Odoo workers to process the enqueued jobs.
Configuring Environment Variables
Transitioning to configuration nuances, you can set environment variables to adjust behavior. For instance, to override default channel values, use:
export ODOO_QUEUE_JOB_CHANNELS="root:4"
export ODOO_QUEUE_JOB_PORT=8069
These settings grant you flexibility in managing how many jobs run concurrently and when they should start processing.
Verifying the Installation
Once installed, inspect your log file for messages from the Jobrunner. You should see entries similar to:
... INFO ... queue_job.jobrunner.runner: starting
... INFO ... queue_job.jobrunner.runner: queue job runner ready for db <dbname>
These messages confirm that your queue_job system is up and running, ensuring that your jobs will be executed as expected.
How to Use the Queue_Job Module in Your Custom Models
Creating Asynchronous Methods
Now that you have installed and configured the module, you can create asynchronous methods in your models. Transitioning with clarity, the following example demonstrates how to use with_delay():
class SaleOrder(models.Model):
_name = 'sale.order'
def action_confirm(self):
self.write({'state': 'confirmed'})
# Enqueue the method call to send a confirmation email asynchronously
self.with_delay(priority=10).send_confirmation_email()
return True
def send_confirmation_email(self):
# Actual code to send email
_logger.info("Confirmation email sent for order %s", self.id)
In this code, the confirmation email is sent asynchronously after updating the record. The call to with_delay() ensures the method runs in the background via the queue_job module.
Enqueuing Job Options and Customizing Delays
Transitioning to more advanced usage, you can fine-tune how jobs are enqueued by setting properties such as priority, estimated time of arrival (ETA), and maximum retries. Here is an example that enqueues a job with custom options:
class Invoice(models.Model):
_name = 'account.invoice'
def process_invoice(self):
# Process invoice immediately
self.mark_as_processed()
# Delay the generation of a PDF asynchronously
self.with_delay(priority=5, eta='2025-02-14 10:00:00', max_retries=3).generate_pdf()
return True
def generate_pdf(self):
# Code to generate PDF
_logger.info("Generating PDF for invoice %s", self.id)
This snippet shows that you can define parameters directly in the delay call. Transition words such as “immediately” and “then” help guide readers through the logical sequence of operations.
Building Delayable Job Chains
Moreover, you can chain several jobs together to form a workflow graph. For example, if you want to produce thumbnails and then process other tasks in sequence, you might write:
from odoo.addons.queue_job.delay import group, chain
class Product(models.Model):
_name = 'product.template'
def button_process_images(self):
group_a = group(self.delayable().generate_thumbnail((50, 50)),
self.delayable().generate_thumbnail((100, 100)))
group_b = group(self.delayable().optimize_image(), self.delayable().store_image())
chain(group_a, group_b).delay()
self.write({"state": "processed"})
return True
def generate_thumbnail(self, size):
_logger.info("Thumbnail generated with size %s for product %s", size, self.id)
def optimize_image(self):
_logger.info("Image optimized for product %s", self.id)
def store_image(self):
_logger.info("Image stored for product %s", self.id)
This example demonstrates chaining with both group and chain primitives. Transition words like “furthermore” and “consequently” enhance the clarity of these advanced patterns.
Advanced Features and Best Practices
Testing Jobs and Trapping Enqueued Jobs
To guarantee that your jobs run as intended in production, you must test them effectively. Transitioning toward testing best practices, you can use the trap_jobs() context manager to verify that jobs are enqueued correctly without executing them immediately. Consider the following test code:
from odoo.addons.queue_job.tests.common import trap_jobs
def test_invoice_processing(self):
with trap_jobs() as trap:
result = self.env['account.invoice'].process_invoice()
expected_eta = '2025-02-14 10:00:00'
# Verify exactly one job is enqueued for generating a PDF
trap.assert_jobs_count(1, only=self.env['account.invoice'].generate_pdf)
trap.assert_enqueued_job(
self.env['account.invoice'].generate_pdf,
args=(),
kwargs={},
properties={'priority': 5, 'eta': expected_eta, 'max_retries': 3}
)
self.assertTrue(result)
This test confirms that your delayed method was added to the queue with the expected parameters and provides you with an opportunity to simulate asynchronous behavior. Additionally, you can perform the job synchronously by calling trap.perform_enqueued_jobs() within the test block.
Handling Dependencies and Retry Patterns
When jobs depend on other jobs, use the .on_done() pattern. Transitioning further, the following code shows how to chain dependent jobs:
def chain_dependent_jobs(self):
self.ensure_one()
job1 = self.delayable().generate_thumbnail((60, 60))
job2 = self.delayable().optimize_image()
job3 = self.delayable().store_image()
# Chain the jobs so that job2 starts after job1 finishes, and job3 after job2
job1.on_done(job2.on_done(job3)).delay()
self.write({"state": "queued"})
return True
This method ensures that each step is executed sequentially only after the previous job completes successfully. Transition by saying, “first, this happens; then, that occurs.”
Retrying Jobs upon Failure
The queue_job module automatically retries failed jobs based on defined patterns. Transitioning to error handling, you can implement custom retry patterns without complications. For example, when a job fails, the system waits 10 seconds for the first few retries and increases waiting time for subsequent attempts. This behavior helps maintain data integrity and improves the robustness of your application.
Tips, Tricks, and Additional Best Practices
Efficient Use of Channels
Channels allow you to manage multiple job pipelines simultaneously. Transitioning effortlessly into channel segmentation, you can restrict heavy jobs (such as bulk data processing) to run sequentially while running lighter tasks in parallel. In your configuration, you can set channels like so:
[queue_job]
channels = root:4,sale:2
This configuration ensures that the heavy-duty jobs are processed one at a time on the sale channel while normal jobs run parallel on the root channel.
Monitoring and Garbage Collection
Always monitor your queue_job dashboard to track performance. Transitioning further, you should schedule a periodic garbage collector to requeue stale jobs that remain in the “started” or “enqueued” state if the server crashes. For instance, run an SQL statement similar to:
UPDATE queue_job SET state='pending' WHERE state IN ('started', 'enqueued');
This SQL snippet resets stale jobs. By doing so, you maintain a healthy job queue and ensure that new jobs can be processed without delay.
Debug Logging for Queue Jobs
For debugging purposes, enabling debug logging for the queue job module is essential. Transition by noting that you can add the command line parameter:
./odoo-bin --log-handler=odoo.addons.queue_job:DEBUG
This command activates detailed logs that help you trace job execution steps and diagnose issues promptly.
Ensure Idempotency in Your Jobs
Idempotency is critical when designing asynchronous jobs. Transition by emphasizing that your jobs must produce the same result even when executed multiple times. This practice prevents data corruption due to retries and ensures that your workflow is robust.
Final Thoughts on Using Queue_Job in Odoo 18
In summary, the queue_job module in Odoo 18 equips you with a powerful mechanism for delegating tasks asynchronously. You now know how to install, configure, and use the module by leveraging its robust features such as delaying method calls, chaining dependent jobs, and setting custom job options. Additionally, you learned best practices for testing, managing job dependencies, and ensuring that your jobs are idempotent.
By following this tutorial and using the code examples provided, you can integrate a seamless background job processing system into your Odoo applications. Transitioning to a production environment, incorporate these principles to optimize performance and deliver a smooth user experience.
For more tutorials and detailed documentation, please visit the Odoo website. Moreover, feel free to explore additional posts on our blog that cover advanced techniques and common troubleshooting tips for the queue_job module.
Detailed Code Explanation and Implementation Walkthrough
Code Structure and Explanation
Below is a recap of the core concepts explained above. The following code snippet demonstrates a complete example of enqueuing an asynchronous job in a custom model:
from odoo import models, fields, api, _
import logging
_logger = logging.getLogger(__name__)
class MyModel(models.Model):
_name = 'my.model'
name = fields.Char("Name")
def my_method(self, a, k=None):
# Active method that logs its input values
_logger.info('Executed with a: %s and k: %s', a, k)
class MyOtherModel(models.Model):
_name = 'my.other.model'
state = fields.Selection([
('draft', 'Draft'),
('done', 'Done')
], default='draft')
def button_do_stuff(self):
# Writing immediate update in the current transaction
self.write({'state': 'done'})
# Enqueuing the job to run asynchronously via with_delay()
self.env['my.model'].with_delay(priority=10).my_method('a', k=2)
return True
Explanation:
- Model Definition:
BothMyModelandMyOtherModelare defined with the necessary fields and methods. - Method
my_method:
This method logs its parameters and is meant to be executed asynchronously. - Enqueuing the Job:
Inbutton_do_stuff(), we update the state field immediately and enqueuemy_methodto run in the background usingwith_delay(). We pass additional parameters likepriorityto optimize job execution. - Logging:
We actively log messages to monitor the job’s behavior in real time.
Testing the Enqueued Job
For testing, we use the trap_jobs() context manager as explained earlier:
from odoo.addons.queue_job.tests.common import trap_jobs
def test_button_do_stuff(self):
with trap_jobs() as trap:
result = self.env['my.other.model'].button_do_stuff()
trap.assert_jobs_count(1, only=self.env['my.model'].my_method)
trap.assert_enqueued_job(
self.env['my.model'].my_method,
args=("a",),
kwargs={'k': 2},
properties={'priority': 10}
)
self.assertTrue(result)
Explanation:
- Job Trapping:
We capture enqueued jobs using thetrap_jobs()context. - Verification:
We assert that exactly one job is enqueued and that it carries the specified arguments and properties. - Test Outcome:
We confirm that the method returns the correct state and delves into job verification.
Additional Configuration Code
To ensure your job runner remains robust, add the following environment variables in your odoo.conf or export them in your shell:
[queue_job]
channels = root:2,sale:2
This configuration supports multiple channels such as root and sale so that different jobs can run concurrently according to their workload.
Conclusion
To wrap up, this tutorial has taken you through the key aspects of using queue_job in Odoo 18. We explained how to install and configure the module, enqueue jobs with delay, chain asynchronous tasks, and test these background processes effectively. By following the steps outlined and referring to our comprehensive code examples, you ensure that your jobs run efficiently and reliably.
We trust that this detailed guide improves your understanding of asynchronous processing in Odoo 18. Transition confidently into adopting these practices in your own projects. For further reading and advanced tutorials, feel free to visit the Odoo official website and follow our blog for continuous updates.
Happy coding with queue_job in Odoo 18, and may your job queue always optimize your business processes!
Discover more from teguhteja.id
Subscribe to get the latest posts sent to your email.