Celery Vs Aws lambda for scheduled task

Adarsha Regmi
1 min readJun 21, 2024

--

I have researched throughly in the internet as I required a suggestion to choose between aws lambda to run the task or use celery to perform the task. It is quite amazing as for my use case lambda was a better choice.
If you are interested lets dive in. ::

Lets dive in only if are interested otherwise you can scroll

1) Amazon lambda service:

>> serverless compute service that runs code in response to events and automatically manages compute resources

Pros:

  • no need to handle servers
  • automatic scaling with respect to demand
  • pay as you go
  • aws friendly , event driven

Cons:

  • slow latency for first task
  • 15 minutes max exec time

use cases :

  • real time stock price updates
  • process tak, small, quick like data transformation, notification triggers.

2) Celery

open source , distributed task queue system that allows to run async tasks and manage task scheduling.

Pros:

  • flexible [works with message brokers likes rabbitMQ, Redis, ]
  • task with longer running execution time
  • full control over task scheduling, retries

Cons:

  • Require managing and scaling of infrastructure
  • more complex setup

Suitable for:

  • periodic data fetch, update
  • complex workflows requiring task chaining and coordination

--

--