The key to a durable workflow is making each step idempotent. Then you don't have to worry about those things. You just run the failed step again. If it already worked the first time, it's a no-op.
For example, stripe lets you include an idempotency key with your request. If you try to make a charge again with the same key, it ignores you. A DE framework like DBOS will automatically generate the idempotency key for you.
But you're correct, if you can't make the operation idempotent, then you have to handle that yourself.
I kind of feel that using examples where idempotent keys are implemented in a SAAS in a engine is side stepping the issue not because NIH (it’s the right thing to do) but it glosses over the complexity of implementing idempotency for the part you need to yourself , and I’ll bet most people have a kinda promise to be idempotent unless edge case
Ah I don't know if I would agree with that. Temporal does a lot of stuff; we just don't happen to need most of it and it's really heavyweight on the database side (running low 500 or so workflows/second of their own 'hello world' style echo benchmark translates to 100k database ops/second..
DBOS is tied to Postgres, right? That wouldn't scale anywhere near where we need either.
Sadly there aren't many shortcuts in this space and pretending there are seems a bit hip at the moment. In the end, mostly everyone who can afford to solve such problems are gonna end up writing their own systems for this.
> DBOS is tied to Postgres, right? That wouldn't scale anywhere near where we need either.
I would challenge that assumption. We have 50 years of experience scaling Postgres. It can scale pretty far, and then you can shard it for even more. Or you can use one of the new flavors of Postgres compatible database that has unlimited horizontal scaling.
> In the end, mostly everyone who can afford to solve such problems are gonna end up writing their own systems for this.
Hard disagree (granted, I'm the CEO of one of the companies selling a solution in this space). If done right with a good DX and lightweight enough, ideally everyone will use DE by default, and will use one of the frameworks provided. Most likely one of the new style frameworks that you see in this blog post and that DBOS uses, that don't use an external coordinator and black box binary with a shim.
DBOS uses in process coordination with a pure language library, which makes it far more performant with a lot less hardware. It's not an apples to apples comparison.
For example, stripe lets you include an idempotency key with your request. If you try to make a charge again with the same key, it ignores you. A DE framework like DBOS will automatically generate the idempotency key for you.
But you're correct, if you can't make the operation idempotent, then you have to handle that yourself.