Celery is such garbage to run/maintain at any sort of scale. Very excited for this. Rq/temporal also seem to solve this well.

Anyone here done the migration off of celery to another thing? Any wisdom?

A customer of mine has two projects. One running on their own hardware, Django + Celery. The other one running on AWS EC2, Django alone.

In the first one we use Celery to run some jobs that may last from a few seconds to some minutes. In the other one we create a new VM and make it run the job and we make it self destroy on job termination. The communication is over a shared database and SQS queues.

We have periodic problems with celery: workers losing connection with RabbitMQ, Celery itself getting stuck, gevent issues maybe caused by C libraries but we can't be sure (we use prefork for some workers but not for everything)

We had no problems with EC2 VMs. By the way, we use VirtualBox to simulate EC2 locally: a Python class encapsulates the API to start the VMs and does it with boto3 in production and with VBoxManage in development.

What I don't understand is: it's always Linux, amd64, RabbitMQ but my other customer using Rails and Sidekiq has no problems and they run many more jobs. There is something in the concurrency stack inside Celery that is too fragile.

Celery with the Redis backend seemed pretty solid.

RabbitMQ and friends are just a pain to use.

Migrated Celery to Argo Workflows. No wisdom as it was straightforward. You lose a lot startup speed though, so it's not a drop-in replacement and is only a good choice for long-running workflows. Celery was easier than Argo Workflows. Celery is really easy to get started with. I like Airflow the best, but it's closer to Argo Workflows in terms of more long-lived workflows. I hope to try Hatchet soon. I've read Temporal is even harder to manage.

Can share the sentiment, had to work with celery years ago, and the maintenance/footguns exceeded the expectations. The codebase and docs are also a bit messy, it's a huge project used and contributed by many so it's understandable I guess. Anyway, Argo if you are in K8S, something else if you aren't. And if you are a startup and need speed, just go with something like procrastinate.

I remember working at a startup in the early 2010's that used celery for all of its background job infrastructure. There were several million tasks run daily across dozens of servers. Celery would regularly hang. Queues would pile up. We had some crazy scripts that would restart celery, detect and kill hanging processes, etc. Fun times.

We switched from Celery to Temporal. Temporal is such a great piece of distributed system.

What were the problems you had with Celery?

They for sure nerfed it within the last ~3 weeks. There's a measurable difference in quality.

They actually just had a bug fix and it seems like it recently got a lot better in the last week or so

Chonkie is great software. Congrats on the launch! Has been a pleasure to use so far.

Thank you :)

OpenAi has a version called Codex that has support. It's lacking in a few features like MCP right now and the TUI isn't there yet, but interestingly they are building a Rust version (it's all open source) that seems to include MCP support and looks significantly higher quality. I'd bet within the next few weeks there will be a high quality claude code alternative.

Congrats on the V2 launch. Does Plandex support MCP? Will take it for a test drive tonight.

Thanks! It doesn't support MCP yet, but it has some MCP-like features built-in. For example, it can launch a browser, pull in console logs or errors, and send them to model for debugging (either step-by-step or fully automated).

Agree. My wish-list is:

1. Non JS based. I've noticed a ton of random bugs/oddities in Claude Code, and now Codex with UI flickering, scaling, user input issues, etc, all from what I believe of trying to do React stuff and writing half-baked LLM produced JS in a CLI application. Using a more appropriate language that is better for CLIs I think would help a lot here (Go or Rust for eg).

2. Customized model selection (eg. OpenRouter, etc).

3. Full MCP support.

What's the gold standard paid offering that does this?

Not a paid solution, but great for testing models yourself: AWS bedrock.

Wonky documentation (definitely released too early), but imo the best model agnostic diy solution out there.

I much prefer the standalone design to being editor integrated.

The cost is absurd (compared to other LLM providers these days). I asked 3 questions and the cost was ~0.77c.

I do like how this is implemented as a bash tool and not an editor replacement though. Never leaving Vim! :P

That 0.77 can save hours of work though, fighting with or being misdirected by other LLM. And, relative to hourly rate, or a cup of coffee, it's incredibly insignificant, if just used for the heavy questions.

My LLM client can switch between whatever models, mid conversation. So I'll have a question or two in the more expensive, then drop down to the cheaper for explanations/questions that help me understand. Rewind time, then hit the more expensive models with relevant prompts.

At the edges, it really ends up being "this is the only model that can do this".

After playing with it for a few days, I agree the cost is easily worth the $$. It's almost in a different playing field of "normal" LLM usage.. and more so in the "good productivity tools" realm.

Yep, my experience as well. It's just not worth it.

It burns through tokens like crazy on a small code base https://i.imgur.com/16GCxiy.png