Since its a tool itself, I dont see the benefit of relying on Anthropic for this. if anything it now becomes vendor lock in.

Correct, I wouldn't use it myself as it's a trivial addition to your implementation. Personally I keep all my work in this space as provider agnostic as I can. When the bubble eventually pops there will be victims, and you don't want a stack that's hard coded to one of the casualties.

They can post-train the model on usage of their specific tool along with the specific prompt they're using.

LLMs generalize obviously, but I also wouldn't be shocked if it performs better than a "normal" implementation.

These meta features are nice, but I feel they create new issues. Like debugging. Since this tool search feature is completely opaque, the wrong tool might not get selected. Then you'll have to figure out if it was the search, and if it was how you can push the right tool to the top.

People talk about an AI bubble. I think this is the real bubble.

Not really because the money involved is relatively small. The bubble is where people are using D8s to push square kilometers of dirt around for data centers that need new nuclear power plants built, to house millions of obsolete Nvidia GPUs that need new fabs constructed to make, using yet more D8s..

(D8s apparently refers to a specific Caterpillar-brand bulldozer, not some kubernetes takeoff.)

I think the prompt is probably at fault here. You can use LLMs for object segmentation and they do fairly well, less than 1% seems too low.

The cross-tile challenges were quite robust - every model struggled with them, and we tried with several iterations of the prompt. I'm sure you could improve with specialized systems, but the models out-of-the-box definitely struggle with segmentation

Yeah, our team of two gets two interns a semester. We cannot convert them to full time as there is no position open. Complete hiring freeze since 2022.

We paused hiring fresh grads, but still hire interns, and those who prove themselves get full-time offers. We've found internships to be a great pipeline to great hires over the years.

We've had several candidates with completed bachelor's degrees apply for internships, prove themselves, and get full-time jobs that way. This "back door" job hiring pathway might work elsewhere as well.

Same here. We pay interns pretty well and we invest a lot in them during their internship. It doesn't make sense for us (and I imagine others) to take in interns and then not hire the good ones. That's the entire reason we do internships to start with.

I wonder if the reason AI is better at these diagnostics, is because the amount of time it spends with the patient is unbounded. Whereas a doctor is always restricted by the amount of time they have with the patient.

I don't think we can say it's "better" based on a bunch of anecdotes, especially when they're coming exclusively from people who are more intelligent, educated, and AI-literate than most of the population. But it is true that doctors are far more rushed than they used to be, disallowed from providing the attentiveness they'd like or ought to give to each patient. And knowledge and skill vary across doctors.

It's an imperfect situation for sure, but I'd like to see more data.

That was my observation as well. To be fair their business is to sell a hosted version, they’re under no obligation to release a truly self hosted version.

There is some intense FOMO right now. I work for a large SAAS company and our guidelines went from no AI to "Use AI for everything everywhere". This does not come from a position of understanding (the people in charge are the same), but rather a deep fear that we could fall behind. Its not rooted in tangible metrics.

One is a machine the other one is not. People have to stop comparing LLMs to humans. Would you hold a car to human standards?

The machine just needs to be coded to run stuff (as shown in this very post). My coworkers can’t be coded to follow procedures and still submit PRs failing basic checks, sadly.

A self driving car, yes.

Because you also need proper access controls. In many cases database access is too low level, you need to bring it up a layer or two to know who can access what. Even more so when you want to do more than read data.