Database connection leak while using threads in Django

Oftentimes we need to call a third party API in our web application. The recommended way would be using a task runner and task queue like Celery and RabbitMQ.

But sometimes, in a small project it is overkill to use Celery. Especially for a small traffic web site. On the other hand, I don’t want to call external API directly because it will block the request and user may have to wait, and that is never a good thing, small traffic or not.

So I decided to use different thread to call API. And it worked for my purpose.

Everything was fine until I write the unit tests. When I run the tests, an error was thrown at at the end, saying the test database cannot be dropped because there was another user using it.

Usually Django will automatically close the database connections when they exceeds maximum age defined or are not usable any longer.

There was a database access using Django model object in the new thread. After a quick search on google, I found out that the database connection were not closed even after the threads execution finished. The connections need to be manually closed.

By placing the above code at the end of the thread function body solved the problem.

On a separate note, if you are using uWSGI in deployment, you need to add

--enable-threads

argument to enable multi threading.

 

 

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s